Files
xiaoyin-gateway/scripts/sync_models.py
T
Jidekin 83ecea482f fix(sync): OpenCode 模型改用官方 models 接口,修复文档改版导致的连续两天 exit=2
官方把文档的 `## Endpoints` 表改成「看 console 页 / 用 models 接口」,
HTML 里不再有带 opencode.ai/ 地址的模型行,fetch_opencode() 的 <tr>/<td>
解析拿不到任何行 → 自动化日报「模型清单同步」整块失败:

  [2026-11-17 23:02:18] exit=2  RuntimeError: OpenCode 文档没解析出 Endpoints 表
  [2026-11-18 23:02:29] exit=2  同上

改为优先读官方 models 接口(实测无需鉴权):
  GET https://opencode.ai/inference/go/v1/models   (zen/go/v1/models 同源同内容)

每条自带 id/name/type/api{chat-completions,responses,messages,generate-content}/
recommended_api/limit{context,output}/cost.text.*.price/go.limited_regions,
据此推导协议线、免费标记与地区限制;HTML 表解析降级为兜底(接口不可用才走)。

首次落盘:
  - 删除 opencode/mimo-v2.5、mimo-v2.5-pro(上游已被 V2.6 取代)
  - 上下文 hy4-flash-free → 1048576/65536;qwen4-max → 1000000/262144
  - 跳过 gemini-4-argon(仅 generate-content)、beam-free 与 muse-spark-1.2/1.3(地区限制)

验证:落盘后 28 个模型、/healthz 200、hot_reload 自动生效未重启、重跑「无漂移」;
sync_onemin.py 同步通过(49 接口 / 43 配置 / 跳过重名 6)。

顺带把运行时的 data/ban_history.jsonl 加进 .gitignore(含被封 IP,属隐私)。
2026-11-19 08:09:28 +08:00

1422 lines
70 KiB
Python
Executable File
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
#!/usr/bin/env python3
"""模型清单同步:把 data/gateway.yaml 的模型清单和官方文档对齐。
用途:每天定时抓官方文档,与网关配置比对并报告漂移;带 --apply 时**直接落盘同步**。
(老板 2026-09-24 定:文档新增的加进去、下线的删掉、参数变了就更新,网关要和文档同步。)
数据源(都是官方页面,没有第三方):
· Dots AI https://dots.ai/platform/docs(第三十七轮纳入;SPA,正文在 JS chunk 里)
· OpenCode Go https://opencode.ai/v2/docs/console/go/
末尾 `## Endpoints` 表 → 模型 id + 官方协议线(chat/completions | messages | responses)
价格表 → 免费模型识别(全 Free 的那些)
· Agnes https://www.agnes-ai.cn/zh-Hans/docs/pricing (Mintlify 的 .md 版)
文本 / 图片 / 视频 三段 → 模型 id + 现价(取「现价」列,不是划掉的刊例价)
落盘边界(--apply):
· 文档新增的模型 → 插到该 provider 模型列表末尾,带「自动同步」注释
· 文档已下线的模型 → 删除(连它的 pricing 条目一起删)
· 价格 / 免费标记变化 → 就地更新
· 协议线只加不砍:文档给的新线会补进去,网关逐条实测出来的更宽的线不动
· 视频默认模型 media.video_model_name → **不自动换**:只保证它不被自动删(被 media 引用
的 id 永不自动删),并在报告里列出文档的视频清单当候选。文档对视频模型没有
「哪个最新 / 最好」的机器可读标记,自动切会直接改变线上视频行为,所以换默认由人点
· 被别名 / media 引用的 id → **永不自动删**,只在报告里点名(删了会打断功能)
· PROTECTED 保护名单**当前为空**:2026-09-24 老板拍板,文档表没收录的
deepseek-flash / omen-alpha 照文档删(原先怕的是文档收录滞后,老板选择以上游文档为准)
写盘走网关自带的 write_config(备份 + 原子替换 + 权限 600),落盘前后回读校验,
校验不过就中止、不写盘。改的是**文本层**,手写注释不会被整份重渲染冲掉。
输出:默认人读报告;--json 给程序用。
退出码:0 = 无漂移(或已同步成功);1 = 有漂移但没落盘(只报告模式);2 = 抓取/解析/写盘失败。
"""
from __future__ import annotations
import argparse
import json
import re
import sys
import time
import urllib.request
from pathlib import Path
import yaml
ROOT = Path(__file__).resolve().parent.parent
DEFAULT_CFG = ROOT / "data" / "gateway.yaml"
UA = "api-gateway-model-sync/1.0 (+ops; contact admin)"
OC_DOC_URL = "https://opencode.ai/v2/docs/console/go/"
AGNES_PRICING_URL = "https://www.agnes-ai.cn/zh-Hans/docs/pricing.md"
# Agnes 文档段标题 → 网关 models[].type
AGNES_SECTION_TYPE = {"文本模型": "text", "图片模型": "image", "视频模型": "video"}
# 同步的两个供应商 → 各自「文档里的模型清单」从哪来
SYNC_PROVIDERS = ("opencode", "agnes", "dots")
# 永不自动删的名单:官方文档表**收录滞后**,实测上游还在服务的模型。
# 删了 = 白丢一个能用的模型,所以列在这里;每次报告都会点名,要删说一声就删。
PROTECTED: dict[str, dict[str, str]] = {
# 2026-09-24 老板拍板「删了」:文档表没收录的 deepseek-flash / omen-alpha 已照文档删除,
# 名单清空(机制保留 —— 以后要临时护住某个 id 在这里登记即可;被别名 / media 引用的 id
# 本来就有独立保护,不依赖这张表)。
"opencode": {},
}
def _get(url: str, timeout: int = 30) -> str:
req = urllib.request.Request(url, headers={"User-Agent": UA, "Accept": "*/*"})
with urllib.request.urlopen(req, timeout=timeout) as resp:
return resp.read().decode("utf-8", "replace")
def _strip_tags(s: str) -> str:
s = re.sub(r"<[^>]+>", "", s)
return s.replace("`", "").strip()
def _norm(s: str) -> str:
"""把展示名归一成可比较的键(去括号后缀/空格/连字符)。"""
s = re.sub(r"\(.*?\)", "", s)
return re.sub(r"[^a-z0-9]", "", s.lower())
# ── OpenCode Go ────────────────────────────────────────────────
# 2026-10-21 文档改版:Endpoints 表里的地址从
# https://opencode.ai/zen/go/v1/<path> (旧)
# 改成按供应商分线:
# https://opencode.ai/inference/go/openai/v1/chat/completions
# https://opencode.ai/inference/go/openai/v1/responses
# https://opencode.ai/inference/go/anthropic/v1/messages
# https://opencode.ai/inference/go/google/v1beta/models/<id> (Google 原生,网关不支持)
# 网关只认三条线,所以按「路径末尾」认协议线,认不出的(Google 原生)跳过并报告。
OC_PROTOCOLS = ("chat/completions", "messages", "responses")
OC_SKIPPED: list[str] = []
# 2026-10-21:Go 页脚注里**写明了地区限制**,之前的解析只认表格、把脚注漏了 ——
# "Beam Free: Free for a limited time. Not available in Angola, Belarus, Cambodia,
# China (including Hong Kong and Macau), Cuba, the Democratic Republic of the Congo,
# Equatorial Guinea, Iran, Iraq, Laos, Libya, Nigeria, North Korea, Russia, Syria,
# Turkmenistan, Uz..."
# "Muse Spark 1.x Contributor: ... Availability is limited to regions permitted by
# Meta's Geographic Use Policy."
# 本机出口在中国大陆,实测这三个模型都是 403 RegionError「This model is not available
# in your country」(zen 与 inference 两条地址都试过,换地址没用;key 本身正常,
# 同批新模型 minimax-m3.1-flash-free / step-5-preview 都是 200)。
# 口径:**跳过不落盘**——不新增;网关里已有的按"文档已不收录"处理(删掉)。
# 以后换到境外出口、或文档解除限制,把 id 从这里移走即可恢复同步。
OC_REGION_BLOCKED = {
"beam-free",
"muse-spark-1.3-contributor",
"muse-spark-1.2-contributor",
}
OC_REGION_SKIPPED: list[str] = []
# ── 2026-11-19 文档二次改版:Endpoints 表不再逐条列 URL ──────────────
# 官方把 `## Endpoints` 改成了「去看 console 页 / 用 models 接口」,HTML 里
# 已经没有 `opencode.ai/` 的模型行 → 连续两天 exit=2「没解析出 Endpoints 表」。
# 现在官方 models 接口直接给出机器可读的元数据(**无需鉴权**,实测不带 token 也 200):
# GET https://opencode.ai/inference/go/v1/models (zen/go/v1/models 同源同内容)
# 每条模型带:
# id / name / type
# api: {chat-completions|responses|messages|generate-content: {base_url, sdk}}
# recommended_api: 官方推荐线
# limit: {context, output}
# cost.text.input.price / cost.text.output.price (都为 0 → 免费)
# go.limited_regions: true → 对部分地区不开放(实测正是这三个 contributor / beam-free)
# 口径:**以 models 接口为准**,HTML 表解析只作兜底;接口也挂了才报错。
OC_MODELS_API = "https://opencode.ai/inference/go/v1/models"
OC_MODELS_API_FALLBACK = "https://opencode.ai/zen/go/v1/models"
# models 接口的 api 键 → 网关协议线(网关只支持这三条线)
OC_API_TO_PROTO = {
"chat-completions": "chat/completions",
"responses": "responses",
"messages": "messages",
}
def _oc_protocol_from_api(model: dict) -> tuple[str | None, list[str]]:
"""从 models 接口的 api / recommended_api 选网关协议线。
返回 (协议线, 文档声明的全部 api 键);认不出(例如只有 generate-content)返回 (None, keys)。
"""
keys = list((model.get("api") or {}).keys())
supported = [OC_API_TO_PROTO[k] for k in keys if k in OC_API_TO_PROTO]
if not supported:
return None, keys
rec = model.get("recommended_api")
if rec in OC_API_TO_PROTO and OC_API_TO_PROTO[rec] in supported:
return OC_API_TO_PROTO[rec], keys
for proto in OC_PROTOCOLS: # 没有推荐或推荐不支持 → chat/completions 优先
if proto in supported:
return proto, keys
return supported[0], keys
def _oc_is_free(model: dict) -> bool:
"""cost.text 的 input / output 单价都为 0 → 免费。"""
price = (model.get("cost") or {}).get("text") or {}
return ((price.get("input") or {}).get("price") == 0
and (price.get("output") or {}).get("price") == 0)
def fetch_opencode() -> dict[str, dict]:
"""返回 {model_id: {name, protocol, free}}。
优先走官方 models 接口(稳定、机器可读);接口不可用时回退到旧的 HTML 表解析。
"""
payload = None
err: Exception | None = None
for url in (OC_MODELS_API, OC_MODELS_API_FALLBACK):
try:
payload = json.loads(_get(url, timeout=60))
if isinstance(payload, dict) and payload.get("data"):
break
payload = None
except Exception as exc: # noqa: BLE001
err = exc
if not payload:
print(f"[warn] OpenCode models 接口不可用({err}),回退 HTML 解析", file=sys.stderr)
return _fetch_opencode_html()
models: dict[str, dict] = {}
skipped: list[str] = []
region_skipped: list[str] = []
for m in payload["data"]:
mid = m.get("id")
if not mid or (m.get("type") or "chat") != "chat":
continue
if (m.get("go") or {}).get("limited_regions") or mid in OC_REGION_BLOCKED:
region_skipped.append(mid)
continue
proto, keys = _oc_protocol_from_api(m)
if proto is None:
skipped.append(f"{mid}({'/'.join(keys) or '无 api 字段'})")
continue
models[mid] = {"name": m.get("name") or mid,
"protocol": proto,
"free": _oc_is_free(m)}
if not models:
raise RuntimeError("OpenCode models 接口返回 0 个可用模型(接口可能改版)")
OC_SKIPPED[:] = sorted(skipped)
OC_REGION_SKIPPED[:] = sorted(region_skipped)
return models
def _oc_protocol(url: str) -> str | None:
"""文档 Endpoint → 网关协议线;认不出返回 None。"""
path = url.split("?", 1)[0].rstrip("/")
for proto in OC_PROTOCOLS:
if path.endswith("/" + proto):
return proto
return None
def _fetch_opencode_html() -> dict[str, dict]:
"""【兜底】旧版 HTML 表解析。
2026-11-19 起官方 `## Endpoints` 表已不再逐条列 URL(内容搬到 console 页 +
models 接口),这里正常不会走到;保留只为页面万一改回去时还能用。
"""
html = _get(OC_DOC_URL)
models: dict[str, dict] = {}
skipped: list[str] = []
region_skipped: list[str] = []
for row in re.findall(r"<tr>(.*?)</tr>", html, re.S):
cells = [_strip_tags(c) for c in re.findall(r"<td[^>]*>(.*?)</td>", row, re.S)]
if len(cells) < 4 or "opencode.ai/" not in cells[2]:
continue
proto = _oc_protocol(cells[2])
if proto is None:
skipped.append(cells[1])
continue
if cells[1] in OC_REGION_BLOCKED:
region_skipped.append(cells[1])
continue
models[cells[1]] = {"name": cells[0], "protocol": proto, "free": False}
if not models:
raise RuntimeError("OpenCode 文档没解析出 Endpoints 表(页面可能改版)")
OC_SKIPPED[:] = sorted(skipped)
OC_REGION_SKIPPED[:] = sorted(region_skipped)
# 免费模型:价格表里 Input/Output 都是 Free 的那些(名字对 id 做归一匹配)
by_norm = {_norm(mid): mid for mid in models}
for row in re.findall(r"<tr>(.*?)</tr>", html, re.S):
cells = [_strip_tags(c) for c in re.findall(r"<td[^>]*>(.*?)</td>", row, re.S)]
if len(cells) >= 3 and cells[1].lower() == "free":
mid = by_norm.get(_norm(cells[0]))
if mid:
models[mid]["free"] = True
return models
# ── models.dev(上下文窗口 / 最大输出)──────────────────────────
# 第三十三轮(2026-09-24 老板要求:接口加入上下文参数):
# OpenCode 官方文档(Zen / Go / v2)**都不写**上下文长度,`/zen/go/v1/models`
# 也只回 id/object/created/owned_by(实测:单模型详情 404、include 参数被忽略)。
# 唯一可用的机器可读来源是 models.dev(OpenCode 团队维护的模型目录),其
# `opencode-go` / `opencode` 两个供应商的每条模型都带 `limit.context` / `limit.output`。
# 口径(老板拍板):
# · 只用于**更新网关已有模型**的这两个字段,不拿它当增删依据
# (增删仍以官方 Go 文档表为准 —— models.dev 收录更多,含 omen-alpha 等文档未收的);
# · 同名模型在两个供应商下不一致时(如 minimax-m3:Go 1M / Zen 512K)取**保守下限**
# (宁可早压缩,不能超窗口);
# · models.dev 挂了/改版 → 只跳过上下文同步,不影响增删判定(返回 None)。
MD_DEV_URL = "https://models.dev/api.json"
def fetch_models_dev() -> dict[str, dict] | None:
"""返回 {model_id: {"context": int, "output": int}}(两供应商合并取保守下限)。
拉不到就返回 None —— 调用方据此跳过上下文同步,不当成漂移。
"""
try:
raw = _get(MD_DEV_URL, timeout=60)
data = json.loads(raw)
except Exception as exc: # noqa: BLE001
print(f"[warn] models.dev 拉取失败,跳过上下文同步:{exc}", file=sys.stderr)
return None
merged: dict[str, dict] = {}
for prov in ("opencode-go", "opencode"):
for mid, m in ((data.get(prov) or {}).get("models") or {}).items():
lim = (m or {}).get("limit") or {}
ctx, out = lim.get("context"), lim.get("output")
if not ctx:
continue
cur = merged.get(mid) or {"context": ctx, "output": out or 0}
cur["context"] = min(cur["context"], ctx) # 保守下限
if out:
cur["output"] = min(cur["output"] or out, out) if cur["output"] else out
merged[mid] = cur
if not merged:
print("[warn] models.dev 解析出 0 个模型,跳过上下文同步", file=sys.stderr)
return None
return merged
# ── Agnes ──────────────────────────────────────────────────────
_MODEL_ID_RE = re.compile(r"[a-z][a-z0-9.\-]*[0-9][a-z0-9.\-]*")
def _price_of(cell: str) -> float | None:
m = re.search(r"¥([0-9.]+)", cell)
return float(m.group(1)) if m else None
# 免费状态判定用的严格措辞(2026-09-24 老板要求 Agnes 也标免费 / 限时免费)
_AGNES_LIMITED_RE = re.compile(r"限时免费")
_AGNES_FREE_RE = re.compile(r"(当前免费|当前均免费|均免费)")
_AGNES_BULLET_RE = re.compile(r"^\s*[*\-]\s+")
_AGNES_TICKED_RE = re.compile(r"`([a-z][a-z0-9.\-]*)`")
def _agnes_free_map(md: str, known: set[str]) -> dict[str, object]:
"""从 Agnes 文档的「### 当前优惠」小节判定每个模型的免费状态。
返回 `{model_id: True | "limited"}`,**只收录文档明说免费的**;没提的模型不在结果里
(= 按收费处理,前端不打标)。
判定口径(按 bullet / 段落**切块**,块内用**严格措辞**):
① 块里出现「限时免费」 → "limited"(限时免费)
② 块里出现「当前免费 / 当前均免费 / 均免费」 → True(免费)
③ 其它(含「按…计费」「刊例价」等) → 不算免费
为什么不能全文扫「免费」:视频 `agnes-video-2.5` 那条 bullet 写着「输入图片前 5 张免费」——
那是计费公式里的免费额度说明,不是模型免费。所以必须按块 + 严格措辞,不能关键词泛扫。
另外块内提 id 必须取**反引号里的完整 id**:`agnes-video-2.5` 是 `agnes-video-2.5-flash`
的子串,用字符串包含会把两者搞混。
"""
out: dict[str, object] = {}
# 逐个「### 当前优惠」小节(正文到下一个标题为止)
for m in re.finditer(r"^#{3,4}\s*当前优惠\s*$", md, flags=re.M):
body = md[m.end():]
nxt = re.search(r"^#{2,4}\s+\S", body, flags=re.M)
if nxt:
body = body[:nxt.start()]
chunk: list[str] = []
for ln in body.splitlines() + [""]:
if _AGNES_BULLET_RE.match(ln) or not ln.strip():
if chunk: # 块结束 → 判定
text = " ".join(chunk)
state: object = ("limited" if _AGNES_LIMITED_RE.search(text)
else True if _AGNES_FREE_RE.search(text) else None)
if state is not None:
for mid in _AGNES_TICKED_RE.findall(text):
if mid in known:
out[mid] = state
chunk = []
if _AGNES_BULLET_RE.match(ln):
chunk = [ln]
else:
chunk.append(ln)
return out
def fetch_agnes() -> dict[str, dict]:
"""返回 {model_id: {section, type, input, output, free}}(input/output = 现价,¥/M,仅文本模型有)。
页面是 JSX 风格的 HTML 表:模型列有的带 rowSpan(一行跨多行计费项)、有的单行自带,
所以按 `<tr>` 块解析,**第一格是模型 id 就认模型行,否则归到上一个模型**。
`free`(免费标注)另从「### 当前优惠」小节判定,见 `_agnes_free_map`。
"""
md = _get(AGNES_PRICING_URL)
out: dict[str, dict] = {}
# 按 ## / ### 标题切块,只在「文本/图片/视频模型」三块里找
parts = re.split(r"^(#{2,3}\s+.+?)\s*$", md, flags=re.M)
for i in range(1, len(parts) - 1, 2):
title = parts[i].lstrip("#").strip()
body = parts[i + 1]
if title not in AGNES_SECTION_TYPE:
continue
cur: str | None = None
for row in re.findall(r"<tr>(.*?)</tr>", body, re.S):
cells = [_strip_tags(c) for c in re.findall(r"<td[^>]*>(.*?)</td>", row, re.S)]
if not cells:
continue
first = cells[0]
if _MODEL_ID_RE.fullmatch(first):
cur = first
out.setdefault(cur, {"section": title, "type": AGNES_SECTION_TYPE[title],
"input": None, "output": None, "free": None})
continue
if not cur:
continue
price = _price_of(cells[-1]) if len(cells) >= 2 else None
if price is None:
continue
if first.startswith("输入 Token"):
out[cur]["input"] = price
elif first.startswith("输出 Token"):
out[cur]["output"] = price
if not out:
raise RuntimeError("Agnes 定价页没解析出模型(页面可能改版)")
for mid, state in _agnes_free_map(md, set(out)).items():
out[mid]["free"] = state
return out
# ── Agnes 上下文窗口(模型详情页「限制与价格」表)──────────────────
# 第三十三轮(2026-09-24): Agnes 官方**每个文本模型页**都有一张「限制与价格」表,
# 写明 `上下文窗口`(如 `512K` / `1M tokens`)与 `最大输出`(如 `65,536 Token`)。
# 定价页没有这两项 —— 所以要去逐个模型页抓。图片/视频模型页无此表 → 抓不到 = 留空。
AGNES_MODEL_PAGES = {
"agnes-3.0-flash": "https://www.agnes-ai.cn/zh-Hans/docs/agnes-30-flash",
"agnes-2.5-flash": "https://www.agnes-ai.cn/zh-Hans/docs/agnes-25-flash",
"agnes-2.5-pro": "https://www.agnes-ai.cn/zh-Hans/docs/agnes-25-pro",
"agnes-2.5-pro-beta": "https://www.agnes-ai.cn/zh-Hans/docs/agnes-25-pro-beta",
}
def _parse_tokens(cell: str) -> int | None:
"""`512K` / `1M tokens` / `65,536 Token` / `65536` → 整数 token 数。"""
s = _strip_tags(cell).lower().replace(",", "").replace(" ", "")
m = re.search(r"([0-9.]+)\s*([km])?", s)
if not m:
return None
val = float(m.group(1))
unit = m.group(2)
if unit == "k":
val *= 1000
elif unit == "m":
val *= 1_000_000
return int(val)
def fetch_agnes_context() -> dict[str, dict]:
"""逐个 Agnes 文本模型页抓 `上下文窗口` / `最大输出` → {id: {context, output}}。
单页抓失败只跳过该模型(不抛)—— 上下文同步是增量能力,不该拖垮整个同步任务。
"""
out: dict[str, dict] = {}
for mid, url in AGNES_MODEL_PAGES.items():
try:
html = _get(url)
except Exception as exc: # noqa: BLE001
print(f"[warn] Agnes 模型页抓取失败 {mid}: {exc}", file=sys.stderr)
continue
ctx = out_tok = None
for row in re.findall(r"<tr>(.*?)</tr>", html, re.S):
cells = [_strip_tags(c) for c in re.findall(r"<t[dh][^>]*>(.*?)</t[dh]>", row, re.S)]
if len(cells) < 2:
continue
label = cells[0].replace(" ", "")
if "上下文窗口" in label:
ctx = _parse_tokens(cells[1])
elif "最大输出" in label:
out_tok = _parse_tokens(cells[1])
if ctx:
out[mid] = {"context": ctx, "output": out_tok or 0}
if not out:
print("[warn] Agnes 模型页一个上下文都没解析出来(页面可能改版)", file=sys.stderr)
return out
# ── 比对 ───────────────────────────────────────────────────────
# ── Dots AI(第三十七轮,老板要求纳入每日同步)──────────────────────
DOTS_DOCS_URL = "https://dots.ai/platform/docs"
def _dots_doc_text() -> str:
"""取 Dots 文档正文(两步发现)。
坑:`/platform/docs` 是 SPA —— **正文不在 HTML 里**。实测模型定价表、上下文窗口、
「免费体验」说明都是**随前端 JS chunk 一起发布**的。所以:
① GET 文档页 → 找主包 `assets/index-*.js`
② GET 主包 → 找文档页自己的 chunk `assets/ApiDocsPage-*.js`
(名字带内容 hash,每次发版都变 → **不能写死**)
"""
html = _get(DOTS_DOCS_URL)
m = re.search(r"/platform/assets/(index-[A-Za-z0-9_.-]+\.js)", html)
if not m:
raise RuntimeError("文档页里没找到主包(页面结构可能变了)")
main_js = _get(f"https://dots.ai/platform/assets/{m.group(1)}")
m2 = re.search(r"assets/(ApiDocsPage-[A-Za-z0-9_.-]+\.js)", main_js)
if not m2:
raise RuntimeError("主包里没找到文档页 chunk(构建产物结构可能变了)")
return _get(f"https://dots.ai/platform/assets/{m2.group(1)}")
def fetch_dots() -> dict[str, dict]:
"""抓 Dots 的模型清单(口径与 Agnes 一致:以文档里的**模型定价表**为准)。
· 模型行形如 `model:"dots3-note-prev",input:"$0.14",output:"$0.28"`(目前只一个模型);
按出现次数全收,将来加模型也不用改代码。
· `free` 取文档的「免费体验」横幅 —— **注意它是"当前免费试用"**(原文:「当前免费体验,
暂不收费。恢复收费前将通过官网及 API 页面提前告知」):横幅在 = 免费;
横幅没了 = 按表上报价收费 → 同步会把 free 改成 false(这正是要盯的漂移)。
· **不写定价**:文档给的是 **USD**(`$0.14/$0.28`),而网关 `pricing.per_model` 是
**人民币 ¥/M**(Agnes 那套口径)—— 换算要汇率,不能猜;报价只放进报告里给人看。
· 上下文窗口从正文抓:`<id> 的上下文窗口上限为 524,288 Token(512K)`。
"""
js = _dots_doc_text()
free_now = "免费体验" in js
out: dict[str, dict] = {}
for mid, inp, outp in re.findall(
r'model:\s*"([^"]+)"\s*,\s*input:\s*"([^"]*)"\s*,\s*output:\s*"([^"]*)"', js):
out[mid] = {"name": mid, "free": free_now, "input_usd": inp, "output_usd": outp}
for m in re.finditer(
r"([A-Za-z0-9][A-Za-z0-9._-]*)\s*的上下文窗口上限为\s*([\d,]+)\s*Token", js):
mid = m.group(1)
if mid in out:
out[mid]["context"] = int(m.group(2).replace(",", ""))
if not out:
raise RuntimeError("文档里没解析出任何模型(页面结构可能变了)")
return out
def _references(cfg: dict) -> dict[str, str]:
"""被别名 / media 指向的模型 id → 引用原因。删这些会直接打断功能,故永不自动删。"""
refs: dict[str, str] = {}
for name, alias in (cfg.get("aliases") or {}).items():
target = alias.get("target") if isinstance(alias, dict) else None
if target:
refs[target] = f"别名 {name} 的目标"
for prov, pcfg in (cfg.get("providers") or {}).items():
for key, val in ((pcfg or {}).get("media") or {}).items():
if isinstance(val, str) and val:
refs[val] = f"{prov}.media.{key}"
return refs
def _cfg_free_state(m: dict) -> object:
"""配置里一个模型的免费状态:True(免费)/ "limited"(限时免费)/ None(没标)。"""
if m.get("free_limited"):
return "limited"
return True if m.get("free") else None
def _split_removed(prov: str, cur: dict, doc: dict, refs: dict) -> tuple[list[str], dict[str, str]]:
"""「网关有、文档没收录」的模型 → (该删的, 该保的 {id: 原因})。
文档没收录未必等于上游下线(文档表有收录滞后),所以:被别名 / media 引用的 id
一律不删(删了会打断功能),另留一张 PROTECTED 名单可临时护住某个 id。
2026-09-24 起 PROTECTED 为空 —— 老板拍板以上游文档为准(deepseek-flash /
omen-alpha 这两个文档未收录的已照文档删掉)。
"""
guard = PROTECTED.get(prov, {})
drop: list[str] = []
keep: dict[str, str] = {}
for mid in sorted(set(cur) - set(doc)):
if mid in guard:
keep[mid] = f"保护名单({guard[mid]})"
elif mid in refs:
keep[mid] = f"被引用:{refs[mid]} —— 删了会打断功能"
else:
drop.append(mid)
return drop, keep
def diff(cfg: dict, oc_doc: dict, agnes_doc: dict,
md_dev: dict[str, dict] | None = None,
ag_ctx: dict[str, dict] | None = None,
dots_doc: dict[str, dict] | None = None) -> dict:
providers = cfg.get("providers", {})
pricing = (cfg.get("pricing") or {}).get("per_model") or {}
refs = _references(cfg)
report: dict[str, dict] = {}
# OpenCode
oc_cur = {m["id"]: m for m in providers.get("opencode", {}).get("models", [])}
proto_drift = []
for mid, meta in sorted(oc_doc.items()):
if mid not in oc_cur:
continue
lines = oc_cur[mid].get("protocols") or ["chat/completions"]
if meta["protocol"] not in lines:
proto_drift.append({"id": mid, "doc": meta["protocol"], "cfg": lines})
free_drift = [{"id": mid, "doc_free": meta["free"], "cfg_free": bool(oc_cur[mid].get("free"))}
for mid, meta in sorted(oc_doc.items())
if mid in oc_cur and meta["free"] != bool(oc_cur[mid].get("free"))]
oc_drop, oc_keep = _split_removed("opencode", oc_cur, oc_doc, refs)
# 第三十三轮: 上下文窗口漂移(models.dev 现值 vs 网关现值)。
# 只比对**网关已有模型**(不拿 models.dev 当增删依据);md_dev 为 None(拉取失败)
# 时留空 → 不产生漂移、不写盘。
oc_ctx_drift = []
for mid, m in sorted(oc_cur.items()):
want = (md_dev or {}).get(mid)
if not want or not want.get("context"):
continue
have_ctx = int(m.get("context_window") or 0)
have_out = int(m.get("max_output_tokens") or 0)
if have_ctx != want["context"] or (want.get("output") and have_out != want["output"]):
oc_ctx_drift.append({"id": mid, "cfg_ctx": have_ctx, "doc_ctx": want["context"],
"cfg_out": have_out, "doc_out": want.get("output") or have_out})
report["opencode"] = {
"doc_count": len(oc_doc), "cfg_count": len(oc_cur),
"added": [{"id": k, "name": v["name"], "protocol": v["protocol"], "free": v["free"]}
for k, v in sorted(oc_doc.items()) if k not in oc_cur],
"removed": oc_drop,
"protected": oc_keep,
"protocol_drift": proto_drift,
"free_drift": free_drift,
"ctx_drift": oc_ctx_drift,
"skipped": list(OC_SKIPPED),
"region_skipped": list(OC_REGION_SKIPPED),
}
# Agnes
ag_cur = {m["id"]: m for m in providers.get("agnes", {}).get("models", [])}
price_drift = []
for mid, meta in sorted(agnes_doc.items()):
if meta["input"] is None: # 图片/视频模型文档没给单价
continue
want = {"input": meta["input"], "output": meta["output"]}
have = pricing.get(mid)
if have != want:
price_drift.append({"id": mid, "cfg": have, "doc": want})
ag_drop, ag_keep = _split_removed("agnes", ag_cur, agnes_doc, refs)
# 第三十三轮: Agnes 上下文漂移(模型详情页 vs 网关现值),同 opencode 口径。
ag_ctx_drift = []
for mid, want in sorted((ag_ctx or {}).items()):
m = ag_cur.get(mid)
if not m or not want.get("context"):
continue
have_ctx = int(m.get("context_window") or 0)
have_out = int(m.get("max_output_tokens") or 0)
if have_ctx != want["context"] or (want.get("output") and have_out != want["output"]):
ag_ctx_drift.append({"id": mid, "cfg_ctx": have_ctx, "doc_ctx": want["context"],
"cfg_out": have_out, "doc_out": want.get("output") or have_out})
report["agnes"] = {
"doc_count": len(agnes_doc), "cfg_count": len(ag_cur),
"ctx_drift": ag_ctx_drift,
"added": [{"id": k, "section": v["section"], "type": v["type"],
"display_name": pretty_name(k), "input": v["input"], "output": v["output"]}
for k, v in sorted(agnes_doc.items()) if k not in ag_cur],
"removed": ag_drop,
"protected": ag_keep,
"price_drift": price_drift,
}
# 视频默认模型(media.video_model_name):同步**只保它不死 + 把它摆在报告里**,
# 不自动换。文档对三个视频模型没有任何「哪个最新 / 最好」的机器可读标记
# (v2.0 当前免费、2.5 按输出分辨率计费、2.5-flash 限时免费),自动挑一个切上去
# 会直接改变线上视频行为。换默认由老板点模型名,改一行配置。
# 免费标注漂移(2026-09-24 老板要求:Agnes 也要标免费 / 限时免费)——
# 口径见 _agnes_free_map;这里只比「文档明说免费」的那批,文档没提的不打标。
ag_free_drift = [{"id": mid, "cfg": _cfg_free_state(ag_cur[mid]), "doc": meta["free"]}
for mid, meta in sorted(agnes_doc.items())
if mid in ag_cur and _cfg_free_state(ag_cur[mid]) != meta["free"]]
vid_cfg = ((providers.get("agnes") or {}).get("media") or {}).get("video_model_name")
report["agnes"]["free_drift"] = ag_free_drift
# Dots AI(第三十七轮):模型清单 / 免费试用状态 / 上下文窗口三项。
# 抓取失败(dots_doc 为 None)时留空 —— 不当成漂移,只在报告里说明。
dots_doc = dots_doc or {}
dots_cur = {m["id"]: m for m in providers.get("dots", {}).get("models", [])}
dots_drop, dots_keep = _split_removed("dots", dots_cur, dots_doc, refs)
dots_free_drift = [{"id": mid, "doc_free": bool(meta["free"]),
"cfg_free": bool(dots_cur[mid].get("free"))}
for mid, meta in sorted(dots_doc.items())
if mid in dots_cur and bool(meta["free"]) != bool(dots_cur[mid].get("free"))]
dots_ctx_drift = []
for mid, meta in sorted(dots_doc.items()):
m = dots_cur.get(mid)
if not m or not meta.get("context"):
continue
have_ctx = int(m.get("context_window") or 0)
if have_ctx != meta["context"]:
# 官方没给「最大输出」上限 → 保持网关现值,只比上下文
have_out = int(m.get("max_output_tokens") or 0)
dots_ctx_drift.append({"id": mid, "cfg_ctx": have_ctx, "doc_ctx": meta["context"],
"cfg_out": have_out, "doc_out": have_out})
report["dots"] = {
"doc_count": len(dots_doc), "cfg_count": len(dots_cur),
"fetched": bool(dots_doc),
"free_trial": any(v.get("free") for v in dots_doc.values()),
"usd_price": {k: f"{v.get('input_usd')}/{v.get('output_usd')}"
for k, v in sorted(dots_doc.items())},
"added": [{"id": k, "name": v["name"], "free": v["free"]} for k, v in sorted(dots_doc.items())
if k not in dots_cur],
"removed": dots_drop, "protected": dots_keep,
"free_drift": dots_free_drift, "ctx_drift": dots_ctx_drift,
}
report["agnes"]["video_default"] = {
"cfg": vid_cfg,
"in_doc": bool(vid_cfg) and vid_cfg in agnes_doc,
"doc_videos": sorted(k for k, v in agnes_doc.items() if v["section"] == "视频模型"),
}
return report
def pretty_name(mid: str) -> str:
"""模型 id → 展示名(文档没给展示名时用):agnes-image-2.0-flash → Agnes Image 2.0 Flash"""
return " ".join(p[:1].upper() + p[1:] for p in mid.split("-"))
def render(report: dict) -> str:
lines: list[str] = []
oc, ag = report["opencode"], report["agnes"]
lines.append(f"== OpenCode Go == 文档 {oc['doc_count']} 个 / 网关 {oc['cfg_count']} 个")
if oc["added"]:
lines.append(" 新增(文档有、网关没有)→ 落盘补进配置:")
lines += [f" + {m['id']} [{m['protocol']}] {m['name']}" for m in oc["added"]]
if oc["removed"]:
lines.append(" 下线(文档已不收录)→ 落盘删除:")
lines += [f" - {m}" for m in oc["removed"]]
if oc.get("protected"):
lines.append(" 文档未收录但**受保护不删**(要删请说一声):")
lines += [f" · {m} —— {why}" for m, why in oc["protected"].items()]
if oc["protocol_drift"]:
lines.append(" 协议线漂移(文档的线网关没有 → 落盘时补上,现有线不砍):")
lines += [f" ~ {d['id']} 文档={d['doc']} 网关={','.join(d['cfg'])}" for d in oc["protocol_drift"]]
if oc["free_drift"]:
lines.append(" 免费标记漂移 → 落盘时更新:")
lines += [f" ~ {d['id']} 文档={d['doc_free']} 网关={d['cfg_free']}" for d in oc["free_drift"]]
if oc.get("ctx_drift"):
lines.append(" 上下文窗口漂移(models.dev vs 网关)→ 落盘更新:")
lines += [f" ~ {d['id']} 网关={d['cfg_ctx']:,}/{d['cfg_out']:,}"
f" 来源={d['doc_ctx']:,}/{d['doc_out']:,}" for d in oc["ctx_drift"]]
if not any(oc[k] for k in ("added", "removed", "protocol_drift", "free_drift",
"ctx_drift")):
lines.append(" 无漂移 ✓")
if oc.get("skipped"):
lines.append(" 跳过(文档端点不是三条网关协议线,网关不支持):")
lines += [f" · {m}" for m in oc["skipped"]]
if oc.get("region_skipped"):
lines.append(" 跳过(文档/实测:对中国大陆出口不可用,RegionError):")
lines += [f" · {m}" for m in oc["region_skipped"]]
lines.append("")
lines.append(f"== Agnes == 文档 {ag['doc_count']} 个 / 网关 {ag['cfg_count']} 个")
if ag["added"]:
lines.append(" 新增(文档有、网关没有)→ 落盘补进配置:")
lines += [f" + {m['id']} ({m['section']} / {m['type']})" for m in ag["added"]]
if ag["removed"]:
lines.append(" 下线(文档已不收录)→ 落盘删除:")
lines += [f" - {m}" for m in ag["removed"]]
if ag.get("protected"):
lines.append(" 文档未收录但**受保护不删**(要删请说一声):")
lines += [f" · {m} —— {why}" for m, why in ag["protected"].items()]
# 新增模型的价格已随「新增」块一起落盘,这里不再重复列为定价漂移
added_ids = {a["id"] for a in ag["added"]}
real_drift = [d for d in ag["price_drift"] if d["id"] not in added_ids]
if real_drift:
lines.append(" 定价漂移(网关 pricing.per_model vs 文档现价,¥/M)→ 落盘更新:")
for d in real_drift:
lines.append(f" ~ {d['id']} 网关={d['cfg']} 文档={d['doc']}")
if ag.get("free_drift"):
lines.append(" 免费标注漂移(文档「当前优惠」vs 网关 free / free_limited)→ 落盘更新:")
label = {True: "免费", "limited": "限时免费", None: "(无标注)"}
lines += [f" ~ {d['id']} 网关={label.get(d['cfg'], d['cfg'])}"
f" 文档={label.get(d['doc'], d['doc'])}" for d in ag["free_drift"]]
if ag.get("ctx_drift"):
lines.append(" 上下文窗口漂移(Agnes 模型页 vs 网关)→ 落盘更新:")
lines += [f" ~ {d['id']} 网关={d['cfg_ctx']:,}/{d['cfg_out']:,}"
f" 文档={d['doc_ctx']:,}/{d['doc_out']:,}" for d in ag["ctx_drift"]]
vd = ag.get("video_default") or {}
if vd.get("cfg"):
mark = ("✓ 文档里还在" if vd.get("in_doc")
else "✗ 文档里已没有 → 需要指定新默认(当前值受保护,不会自动删)")
lines.append(f" 视频默认模型 media.video_model_name:{vd['cfg']} —— {mark}")
lines.append(f" 文档视频清单:{', '.join(vd.get('doc_videos') or []) or '(无)'}"
" ← 换默认由老板点模型名,改一行配置(不自动挑)")
if not (any(ag[k] for k in ("added", "removed", "free_drift")) or real_drift):
lines.append(" 无漂移 ✓")
lines.append("")
dt = report["dots"]
if not dt.get("fetched"):
lines.append("== Dots AI == ⚠ 文档抓取失败(SPA 正文在 JS chunk 里,结构可能变了)"
"→ 本次跳过 Dots 的比对")
else:
lines.append(f"== Dots AI == 文档 {dt['doc_count']} 个 / 网关 {dt['cfg_count']} 个"
+ (" (文档标「免费体验」= 当前免费试用)" if dt.get("free_trial") else ""))
if dt["added"]:
lines.append(" 新增(文档有、网关没有)→ 落盘补进配置:")
lines += [f" + {m['id']}" for m in dt["added"]]
if dt["removed"]:
lines.append(" 下线(文档已不收录)→ 落盘删除:")
lines += [f" - {m}" for m in dt["removed"]]
if dt.get("protected"):
lines.append(" 文档未收录但**受保护不删**(要删请说一声):")
lines += [f" · {m} —— {why}" for m, why in dt["protected"].items()]
if dt.get("free_drift"):
lines.append(" 免费状态漂移 → 落盘更新(试用结束会在这里现形):")
lines += [f" ~ {d['id']} 文档={'免费试用' if d['doc_free'] else '按报价收费'}"
f" 网关={'免费' if d['cfg_free'] else '收费'}" for d in dt["free_drift"]]
if dt.get("ctx_drift"):
lines.append(" 上下文窗口漂移(文档正文 vs 网关)→ 落盘更新:")
lines += [f" ~ {d['id']} 网关={d['cfg_ctx']:,} 文档={d['doc_ctx']:,}"
for d in dt["ctx_drift"]]
if dt.get("usd_price"):
# 文档报价是 USD,网关 pricing 是 ¥/M → 不自动换算(要汇率、不能猜),只列出来给人看
lines.append(" 文档报价(USD/M,仅展示、不写入 pricing):"
+ "、".join(f"{k} {v}" for k, v in dt["usd_price"].items()))
if not any(dt[k] for k in ("added", "removed", "free_drift", "ctx_drift")):
lines.append(" 无漂移 ✓")
return "\n".join(lines)
_DRIFT_KEYS = ("added", "removed", "protocol_drift", "free_drift", "price_drift",
"ctx_drift")
def has_drift(report: dict) -> bool:
"""有没有需要落盘的差异。
`protected`(文档没收录但受保护不删的那些)**故意不算**漂移 —— 它们永远不会被
自动改,算进去等于每天推一条同样的提示,看久了就没人看了。
"""
return any(report[p].get(k) for p in report for k in _DRIFT_KEYS)
# ── 落盘(--apply):文本级增 / 删 / 改 ─────────────────────────
# 为什么不整份重渲染:data/gateway.yaml 里有大量**逐条实测**的手写注释(哪条线实测通了、
# 哪个模型上游有文档没收录……),整份 dump 会把它们全冲掉。所以这里只动需要动的行。
_MODEL_ID_LINE = re.compile(r'^ - id:\s*"?(?P<id>[^"\s]+)"?\s*$')
_SECTION_LINE = re.compile(r"^(?: [A-Za-z_][A-Za-z0-9_]*:|[A-Za-z_])")
_PRICING_ANY = re.compile(r"^ (?P<mid>[A-Za-z0-9._\-]+):")
_PRICING_LINE = re.compile(
r"^(?P<ind> )(?P<mid>[A-Za-z0-9._\-]+):(?P<pad>\s+)"
r"\{ input: (?P<in>[0-9.]+), output: (?P<out>[0-9.]+) \}(?P<tail>.*)$"
)
PRICE_COL = 26 # per_model 里 `{` 的起始列(跟现网对齐风格一致)
def _find_model_blocks(lines: list[str], provider: str) -> dict[str, tuple[int, int]]:
"""定位 <provider>.models 下每个模型块的行区间 [start, end)(半开)。
一个块 = 从 ` - id: "x"` 到下一个模型块 / 下一个段(2 空格缩进的 key)/
文件尾。块尾的空行不算进块里(留给下一段)。
"""
def _find(pattern: str, lo: int) -> int | None:
for i in range(lo, len(lines)):
if re.fullmatch(pattern, lines[i]):
return i
return None
p = _find(rf" {re.escape(provider)}:\s*", 0)
if p is None:
return {}
m = _find(r" models:\s*", p + 1)
if m is None:
return {}
starts: dict[str, int] = {}
order: list[str] = []
stop = len(lines)
for i in range(m + 1, len(lines)):
if _SECTION_LINE.match(lines[i]):
stop = i
break
hit = _MODEL_ID_LINE.match(lines[i])
if hit:
starts[hit.group("id")] = i
order.append(hit.group("id"))
blocks: dict[str, tuple[int, int]] = {}
sec_end = stop # 供应商段结束:下一个 2 空格 key 或文件尾
for n, mid in enumerate(order):
start = starts[mid]
nxt = starts[order[n + 1]] if n + 1 < len(order) else sec_end
blocks[mid] = (start, _block_end(lines, start, nxt))
return blocks
# 「块头」注释:分组分隔线(`# ── …`)和同步自己写的「官方文档新增」标记 —— 这些行属于
# **下一个块**(本文件惯例:分组标题写在组里第一个模型上方)。
# 2026-09-24 实测踩到过:删 deepseek-flash 时把 "# ── Anthropic Messages 线" 一起删了。
# 同一注释组里落在第一行「块头」**之前**的行(例如同步写的 "# 文档现价 …" 尾注)仍算
# 上一个块,跟着块一起删/留。
_BLOCK_HEADER_RE = re.compile(r"^\s*#\s*(?:──|\d{4}-\d{2}-\d{2}\s*自动同步)")
def _block_end(lines: list[str], start: int, stop: int) -> int:
"""块尾收口:去掉尾空行,并把紧贴下一个块的「块头注释」让给下一个块(见上)。"""
end = stop
while end > start and not lines[end - 1].strip():
end -= 1
k = stop - 1
while k > start and lines[k].strip().startswith("#"):
k -= 1
for i in range(k + 1, stop): # 紧贴下一个 `- id:` 的注释组
if _BLOCK_HEADER_RE.match(lines[i]):
return i if i < end else end
return end
def _pricing_span(lines: list[str]) -> tuple[int, int] | None:
"""定位 pricing.per_model 段内条目行的区间 [start, end)(半开)。"""
def _find(pattern: str, lo: int) -> int | None:
for i in range(lo, len(lines)):
if re.fullmatch(pattern, lines[i]):
return i
return None
p = _find(r"pricing:\s*", 0)
if p is None:
return None
m = _find(r" per_model:\s*", p + 1)
if m is None:
return None
end = m + 1
for i in range(m + 1, len(lines)):
if lines[i].strip() and not lines[i].startswith(" "):
break
end = i + 1
# 段尾空行不算进段里:补价目要贴在最后一条价目后面,不能掉到空行下面去
while end > m + 1 and not lines[end - 1].strip():
end -= 1
return m + 1, end
def _insert_after(seg: list[str], pattern: str, newline: str) -> list[str]:
"""把一行插到 seg 里第一条匹配 pattern 的行之后(找不到就原样返回)。"""
for i, ln in enumerate(seg):
if re.match(pattern, ln):
return seg[:i + 1] + [newline] + seg[i + 1:]
return seg
def _render_model_block(provider: str, mid: str, meta: dict, date: str) -> list[str]:
"""渲染一个新模型块(缩进跟现网一致:`- id:` 6 空格、字段 8 空格)。"""
out = [f" # {date} 自动同步:官方文档新增(scripts/sync_models.py)\n",
f' - id: "{mid}"\n',
f' display_name: "{meta.get("display_name") or pretty_name(mid)}"\n',
f' type: "{meta.get("type") or "text"}"\n']
if provider == "opencode":
out.append(" # 协议线取自官方 Endpoints 表,未逐线实测(现网其它模型是实测过的)\n")
out.append(f' protocols: ["{meta.get("protocol") or "chat/completions"}"]\n')
if meta.get("free"):
out.append(" free: true # 官方价格表标 Free\n")
elif meta.get("input") is not None:
out.append(f" # 文档现价 ¥{meta['input']:g}/M 输入、¥{meta['output']:g}/M 输出\n")
if provider == "agnes" and meta.get("free"):
limited = meta["free"] == "limited"
out.append(f" free: true # {date} 自动同步:文档「当前优惠」写当前"
f"{'限时' if limited else ''}免费\n")
if limited:
out.append(" free_limited: true # 限时免费;文档改口径后同步会自动撤掉这行\n")
return out
def _add_protocol(seg: list[str], proto: str, date: str) -> list[str]:
"""把文档声明的一条协议线补进模型块(**只加不砍**)。"""
for i, ln in enumerate(seg):
m = re.match(r"^(\s*protocols:\s*)\[(.*?)\](.*)$", ln)
if not m:
continue
items = [x.strip().strip("\"'") for x in m.group(2).split(",") if x.strip()]
if proto in items:
return seg
items.append(proto)
tail = m.group(3)
if "#" not in tail:
tail = f" # {date} 自动同步:文档声明的线(现网其它线是实测出来的)"
body = ", ".join(json.dumps(x, ensure_ascii=False) for x in items)
return seg[:i] + [f"{m.group(1)}[{body}]{tail}\n"] + seg[i + 1:]
return _insert_after(seg, r"^\s*type:", f' protocols: ["{proto}"]\n')
def _set_free(seg: list[str], want: bool, date: str) -> list[str]:
"""更新免费标记(文档价格表说全 Free 的模型)。原注释保留,只追加同步说明。
幂等(第三十七轮补):**同一条说明不重复追加** —— 否则同一份 plan 重复应用会把
注释滚成「原注释;说明;说明」。不同日期的说明仍会各记一条(那是变更历史,要留)。
"""
note = f"{date} 自动同步:文档价格表{'标 Free' if want else '已不标 Free'}"
for i, ln in enumerate(seg):
if re.match(r"^\s*free:\s*", ln):
comment = ln.split("#", 1)[1].strip() if "#" in ln else ""
if note not in comment:
comment = f"{comment};{note}" if comment else note
return seg[:i] + [f" free: {'true' if want else 'false'} # {comment}\n"] + seg[i + 1:]
if want:
return _insert_after(seg, r"^\s*type:", f" free: true # {note}\n")
return seg
# Agnes 的免费标注行(撤标注时要整行删掉,注释是行内的,跟着走)
_AGNES_FREE_LINE_RE = re.compile(r"^\s*(free|free_limited):\s*")
_CTX_KEY_RE = re.compile(r"^\s*(context_window|max_output_tokens):")
def _set_context(seg: list[str], ctx: int, out_tok: int, date: str) -> list[str]:
"""块内设置 context_window / max_output_tokens(第三十三轮)。
· 已有该行 → 只改值,保留行尾注释;
· 没有 → 插在 `type:` 行之后(与手工插入的位置一致)。
只动这两行,其余原文(含手写注释)不动。
"""
want = {"context_window": ctx, "max_output_tokens": out_tok}
seen: set[str] = set()
for i, ln in enumerate(seg):
m = _CTX_KEY_RE.match(ln)
if not m:
continue
key = m.group(1)
seen.add(key)
tail = ln.split("#", 1)[1] if "#" in ln else f" # 同步 {date}(scripts/sync_models.py)"
seg[i] = f" {key}: {want[key]} #{tail}"
missing = [k for k in want if k not in seen]
if missing:
for i, ln in enumerate(seg):
if re.match(r'^\s{8}type:\s*"', ln):
extra = [f" {k}: {want[k]} # 同步 {date}(scripts/sync_models.py)\n"
for k in missing]
seg[i + 1:i + 1] = extra
break
return seg
def _set_agnes_free(seg: list[str], want: object, date: str) -> list[str]:
"""按文档写 Agnes 模型的免费标注(2026-09-24 老板要求 Agnes 也要标)。
want = `True`(免费)/ `"limited"`(限时免费)/ `None`(文档没说免费 → 撤掉标注)。
写法:`free: true`,限时再加一行 `free_limited: true`(前端据此显示「限时免费」)。
撤标注时**整行删掉**(不留 `free: false` 垃圾行 —— Agnes 十个模型里大多数不免费,
留 false 会把配置撑起来);原行内的手写注释也随行删除,报告里会说明这次撤了哪些。
"""
out = [ln for ln in seg if not _AGNES_FREE_LINE_RE.match(ln)]
if want is None:
return out
limited = want == "limited"
note = (f"{date} 自动同步:文档「当前优惠」写当前{'限时' if limited else ''}免费"
"(scripts/sync_models.py)")
extra = [f" free: true # {note}\n"]
if limited:
extra.append(" free_limited: true # 限时免费;文档改口径后同步会自动撤掉这行\n")
for i, ln in enumerate(out):
if re.match(r"^\s*type:\s*", ln):
return out[:i + 1] + extra + out[i + 1:]
for i, ln in enumerate(out):
if re.match(r"^\s*-\s*id:\s*", ln):
return out[:i + 1] + extra + out[i + 1:]
return out
def build_plan(report: dict, cfg: dict, oc_doc: dict, agnes_doc: dict,
dots_doc: dict | None = None) -> dict:
"""把报告翻成可执行动作(纯计算,不碰磁盘)。"""
docs = {"opencode": oc_doc, "agnes": agnes_doc}
pricing = (cfg.get("pricing") or {}).get("per_model") or {}
plan: dict = {"add": {}, "remove": {}, "price_set": {}, "price_add": {},
"proto_add": {}, "free_set": {}, "agnes_free": {}, "ctx_set": {},
"dots_free": {}}
for prov in SYNC_PROVIDERS:
plan["add"][prov] = {m["id"]: m for m in report[prov]["added"]}
plan["remove"][prov] = list(report[prov]["removed"])
plan["proto_add"] = {d["id"]: d["doc"] for d in report["opencode"]["protocol_drift"]}
plan["free_set"] = {d["id"]: d["doc_free"] for d in report["opencode"]["free_drift"]}
# Agnes 的免费标注(True / "limited" / None)单独一条 plan,走 _set_agnes_free
plan["agnes_free"] = {d["id"]: d["doc"] for d in report["agnes"]["free_drift"]}
# 第三十三轮: 上下文窗口 / 最大输出(models.dev + Agnes 文档)→ 块内改值
# Dots 的免费标记(布尔):试用结束 / 恢复免费都在这里落到配置
plan["dots_free"] = {d["id"]: d["doc_free"] for d in (report.get("dots") or {}).get("free_drift", [])}
plan["ctx_set"] = {d["id"]: {"context": d["doc_ctx"], "output": d["doc_out"]}
for d in (report["opencode"]["ctx_drift"] + report["agnes"]["ctx_drift"]
+ (report.get("dots") or {}).get("ctx_drift", []))}
for mid, meta in sorted(agnes_doc.items()):
if meta["input"] is None: # 图片/视频模型文档没给单价
continue
want = {"input": meta["input"], "output": meta["output"]}
if mid in pricing:
if pricing[mid] != want:
plan["price_set"][mid] = {"old": pricing[mid], "new": want}
else:
plan["price_add"][mid] = want
return plan
def plan_is_empty(plan: dict) -> bool:
"""plan 里是不是真的一条动作都没有。
坑:`plan["add"] / plan["remove"]` 是按 provider 分桶的两层 dict,一个模型都没变时
也是 `{"opencode": {}, "agnes": {}}` —— 真值。早先这里直接 `any(plan[k] ...)` 判断,
结果「无漂移」也走写盘分支:每天白重写一次配置(还把上一份备份覆盖掉),
报告也会误报「已写盘:…N 字节」。所以必须先下钻到内层桶再判。
"""
for prov in SYNC_PROVIDERS:
if plan["add"].get(prov) or plan["remove"].get(prov):
return False
return not any(plan.get(k) for k in
("price_set", "price_add", "proto_add", "free_set", "agnes_free",
"ctx_set", "dots_free"))
def render_plan(plan: dict, date: str) -> str:
lines = [f"-- 落盘动作({date})--"]
for prov in SYNC_PROVIDERS:
for mid in sorted(plan["add"].get(prov) or {}):
lines.append(f" 新增 {prov}/{mid}")
for mid in plan["remove"].get(prov) or []:
lines.append(f" 删除 {prov}/{mid}")
for mid, proto in sorted(plan["proto_add"].items()):
lines.append(f" 补协议线 opencode/{mid} += {proto}")
for mid, want in sorted(plan["free_set"].items()):
lines.append(f" 免费标记 opencode/{mid} → {want}")
_flabel = {True: "免费", "limited": "限时免费", None: "撤掉标注"}
for mid, want in sorted(plan.get("agnes_free", {}).items()):
lines.append(f" 免费标注 agnes/{mid} → {_flabel.get(want, want)}")
for mid, want in sorted(plan.get("dots_free", {}).items()):
lines.append(f" 免费标记 dots/{mid} → {'免费试用' if want else '按报价收费'}")
for mid, spec in sorted(plan.get("ctx_set", {}).items()):
lines.append(f" 上下文 {mid} → {spec['context']:,} 输入 / {spec['output']:,} 输出")
for mid, spec in sorted(plan["price_set"].items()):
lines.append(f" 改价 {mid} {spec['old']} → {spec['new']}")
for mid, want in sorted(plan["price_add"].items()):
lines.append(f" 补价 {mid} → {want}")
if plan_is_empty(plan):
lines.append(" (无:网关已与文档一致)")
return "\n".join(lines)
def apply_plan(text: str, plan: dict, date: str) -> str:
"""按 plan 改配置文本。只动该动的行,其余(含注释)原样保留。"""
lines = text.splitlines(keepends=True)
# 1) 删除模型块(行号从后往前删,避免下标漂移)
cuts: list[tuple[int, int]] = []
for prov in SYNC_PROVIDERS:
blocks = _find_model_blocks(lines, prov)
for mid in plan["remove"].get(prov) or []:
if mid in blocks:
cuts.append(blocks[mid])
for s, e in sorted(cuts, reverse=True):
del lines[s:e]
# 2) 新增模型块(插在该 provider 最后一个模型块之后)
for prov in SYNC_PROVIDERS:
add = plan["add"].get(prov) or {}
if not add:
continue
blocks = _find_model_blocks(lines, prov)
if not blocks:
continue
at = max(e for _, e in blocks.values())
block: list[str] = []
for mid, meta in sorted(add.items()):
block += _render_model_block(prov, mid, meta, date)
lines[at:at] = block
# 3) 块内改:补协议线 / 免费标记(按块起点倒序处理,插入不影响未处理块的下标)
touched: dict[str, set[str]] = {}
for mid in plan["proto_add"]:
touched.setdefault("opencode", set()).add(mid)
for mid in plan["free_set"]:
touched.setdefault("opencode", set()).add(mid)
for prov, mids in touched.items():
blocks = _find_model_blocks(lines, prov)
for mid in sorted(mids, key=lambda m: blocks.get(m, (0, 0))[0], reverse=True):
if mid not in blocks:
continue
s, e = blocks[mid]
seg = lines[s:e]
if mid in plan["proto_add"]:
seg = _add_protocol(seg, plan["proto_add"][mid], date)
if mid in plan["free_set"]:
seg = _set_free(seg, plan["free_set"][mid], date)
lines[s:e] = seg
# 3b) Agnes 的免费标注(免费 / 限时免费 / 撤标注)——同一套倒序处理,
# 字段插在块的 type: 之后,撤标注整行删。放在 3) 之后重取块表,避免下标串位。
if plan.get("agnes_free"):
blocks = _find_model_blocks(lines, "agnes")
for mid in sorted(plan["agnes_free"], key=lambda m: blocks.get(m, (0, 0))[0], reverse=True):
if mid not in blocks:
continue
s, e = blocks[mid]
lines[s:e] = _set_agnes_free(lines[s:e], plan["agnes_free"][mid], date)
# 3d) Dots 的免费标记(布尔 True/False,第三十七轮)——与 3b 同一套倒序处理。
# `free: true` 是「免费试用」;试用结束文档横幅会消失 → 这里把它落成 false。
if plan.get("dots_free"):
blocks = _find_model_blocks(lines, "dots")
for mid in sorted(plan["dots_free"], key=lambda m: blocks.get(m, (0, 0))[0], reverse=True):
if mid not in blocks:
continue
s0, e0 = blocks[mid]
lines[s0:e0] = _set_free(lines[s0:e0], plan["dots_free"][mid], date)
# 3c) 上下文窗口 / 最大输出(第三十三轮)——同一套倒序处理,字段插在块的 type: 之后。
if plan.get("ctx_set"):
for prov in SYNC_PROVIDERS:
blocks = _find_model_blocks(lines, prov)
mids = [m for m in plan["ctx_set"] if m in blocks]
for mid in sorted(mids, key=lambda m: blocks[m][0], reverse=True):
s, e = blocks[mid]
spec = plan["ctx_set"][mid]
lines[s:e] = _set_context(lines[s:e], spec["context"], spec["output"], date)
# 4) 定价:改价 / 补价 / 删掉被删模型的价
gone = {m for prov in SYNC_PROVIDERS for m in plan["remove"].get(prov) or []}
span = _pricing_span(lines)
if span and gone:
s, e = span
keep = [ln for ln in lines[s:e] if not (m := _PRICING_ANY.match(ln)) or m.group("mid") not in gone]
lines[s:e] = keep
span = _pricing_span(lines)
if span:
s, e = span
for i in range(s, e):
m = _PRICING_LINE.match(lines[i])
if not m or m.group("mid") not in plan["price_set"]:
continue
mid = m.group("mid")
old, new = plan["price_set"][mid]["old"], plan["price_set"][mid]["new"]
tail = m.group("tail")
if "#" not in tail:
tail = (f" # 同步 {date}(原 {old['input']:g}/{old['output']:g};"
f"文档现价 ¥{new['input']:g}/M 输入、¥{new['output']:g}/M 输出)")
lines[i] = (f"{m.group('ind')}{mid}:{m.group('pad')}"
f"{{ input: {new['input']:g}, output: {new['output']:g} }}{tail}\n")
if plan["price_add"]:
fresh = []
for mid, want in sorted(plan["price_add"].items()):
key = f"{mid}:"
pad = " " * max(1, PRICE_COL - 4 - len(key))
fresh.append(f" {key}{pad}{{ input: {want['input']:g}, output: {want['output']:g} }}"
f" # {date} 自动同步(文档现价)\n")
lines[e:e] = fresh
return "".join(lines)
def verify(new_text: str, cfg_before: dict, plan: dict) -> list[str]:
"""回读校验:不通过就**不写盘**。宁可这次不同步,也不能把配置写坏。"""
problems: list[str] = []
try:
cfg2 = yaml.safe_load(new_text)
except Exception as e: # noqa: BLE001
return [f"YAML 解析失败:{type(e).__name__}: {e}"]
if not isinstance(cfg2, dict):
return ["YAML 顶层不是映射"]
for prov in SYNC_PROVIDERS:
before = {m["id"] for m in (cfg_before.get("providers", {}).get(prov) or {}).get("models", [])}
after = [m["id"] for m in (cfg2.get("providers", {}).get(prov) or {}).get("models", [])]
if len(after) != len(set(after)):
problems.append(f"{prov}: 模型 id 出现重复")
expect = (before - set(plan["remove"].get(prov) or [])) | set(plan["add"].get(prov) or {})
if set(after) != expect:
problems.append(
f"{prov}: 模型集合不符 期望 {len(expect)} 个 / 实际 {len(set(after))} 个"
f"(多 {sorted(set(after) - expect)} / 少 {sorted(expect - set(after))})")
# 无关 provider 一个都不能碰
for prov, pcfg in (cfg_before.get("providers") or {}).items():
if prov in SYNC_PROVIDERS:
continue
b = [m["id"] for m in (pcfg or {}).get("models", [])]
a = [m["id"] for m in ((cfg2.get("providers", {}).get(prov) or {}).get("models", []))]
if b != a:
problems.append(f"{prov}: 与本次同步无关,却被改动了")
# 块内改的字段真写进去了
oc2 = {m["id"]: m for m in (cfg2.get("providers", {}).get("opencode") or {}).get("models", [])}
for mid, proto in plan["proto_add"].items():
lines = oc2.get(mid, {}).get("protocols") or []
if proto not in lines:
problems.append(f"协议线 {mid} 没补上(现有 {lines})")
for mid, want in plan["free_set"].items():
if bool(oc2.get(mid, {}).get("free")) != bool(want):
problems.append(f"免费标记 {mid} 没改成 {want}")
# Agnes 的免费标注:True(免费)/ "limited"(限时免费)/ None(该没有标注)
ag2 = {m["id"]: m for m in (cfg2.get("providers", {}).get("agnes") or {}).get("models", [])}
# 第三十三轮: 上下文窗口 / 最大输出真写进去了
dots2 = {m["id"]: m for m in (cfg2.get("providers", {}).get("dots") or {}).get("models", [])}
for mid, want in plan.get("dots_free", {}).items():
if bool(dots2.get(mid, {}).get("free")) != bool(want):
problems.append(f"免费标记 dots/{mid} 没改成 {want}")
for prov, table in (("opencode", oc2), ("agnes", ag2), ("dots", dots2)):
for mid, spec in plan.get("ctx_set", {}).items():
m = table.get(mid)
if m is None:
continue # 该模型属于另一个 provider
got_ctx = int(m.get("context_window") or 0)
got_out = int(m.get("max_output_tokens") or 0)
if got_ctx != spec["context"] or got_out != spec["output"]:
problems.append(
f"上下文 {prov}/{mid} 没写成 {spec['context']}/{spec['output']}"
f"(实际 {got_ctx}/{got_out})")
for mid, want in plan.get("agnes_free", {}).items():
got = _cfg_free_state(ag2.get(mid, {}))
if got != want:
problems.append(
f"免费标注 {mid} 没写成 {{True: '免费', 'limited': '限时免费', None: '无标注'}}"
f"[{want!r}](现 {got!r})")
# 定价
pr2 = ((cfg2.get("pricing") or {}).get("per_model") or {})
for mid, spec in list(plan["price_set"].items()):
got = pr2.get(mid)
want = spec["new"]
if not got or float(got["input"]) != want["input"] or float(got["output"]) != want["output"]:
problems.append(f"定价 {mid} 没写成 {want}(现 {got})")
for mid, want in plan["price_add"].items():
got = pr2.get(mid)
if not got or float(got["input"]) != want["input"] or float(got["output"]) != want["output"]:
problems.append(f"定价 {mid} 没补上(现 {got})")
gone = {m for prov in SYNC_PROVIDERS for m in plan["remove"].get(prov) or []}
for mid in gone:
if mid in pr2:
problems.append(f"被删模型 {mid} 的定价条目还在")
# 别名目标不能因为同步消失
ids_now = {m["id"] for prov in SYNC_PROVIDERS
for m in (cfg2.get("providers", {}).get(prov) or {}).get("models", [])}
for name, alias in (cfg_before.get("aliases") or {}).items():
target = alias.get("target") if isinstance(alias, dict) else None
if target and target not in ids_now:
problems.append(f"别名 {name} 的目标 {target} 同步后不存在了")
return problems
def _write_config_text(path: Path, text: str) -> dict:
"""写盘:复用网关自己的 write_config(备份 + 原子替换 + 权限 600)。
延迟导入:默认只读路径不碰 app 包,脚本在裸 python 下也能跑起来。
"""
sys.path.insert(0, str(ROOT))
from app.settings import write_config # noqa: PLC0415
return write_config(path, text)
def _append_log(path: str, text: str, code: int) -> None:
"""把报告追加到日志文件(定时任务用,避免报告随 console 过期丢失)。
只追加不覆盖;父目录不存在就建;写日志失败不抛异常 —— 不能因为日志问题
让一次同步任务整体失败(报告还是会打到 stdout)。
"""
try:
p = Path(path)
if p.parent and not p.parent.exists():
p.parent.mkdir(parents=True, exist_ok=True)
stamp = time.strftime("%Y-%m-%d %H:%M:%S")
with p.open("a", encoding="utf-8") as fh:
fh.write(f"\n{'=' * 72}\n[{stamp}] exit={code}\n{text}\n")
except OSError as e: # noqa: BLE001
print(f"写日志失败(不影响本次结果):{type(e).__name__}: {e}", file=sys.stderr)
def main() -> int:
ap = argparse.ArgumentParser(description="模型清单同步(默认只报告,--apply 才写盘)")
ap.add_argument("--config", default=str(DEFAULT_CFG), help="gateway.yaml 路径")
ap.add_argument("--json", action="store_true", help="输出 JSON")
ap.add_argument("--log", default=None,
help="把报告追加写入该文件(带时间戳)。定时任务用,避免报告丢失")
ap.add_argument("--apply", action="store_true",
help="真的写盘同步(新增 / 删除 / 改参数)。默认只报告;写盘前回读校验,"
"校验不过就中止、不落盘")
ap.add_argument("--date", default=None, help="报告里用的日期(默认当天;仅测试用)")
args = ap.parse_args()
date = args.date or time.strftime("%Y-%m-%d")
cfg_path = Path(args.config)
try:
oc_doc = fetch_opencode()
agnes_doc = fetch_agnes()
except Exception as e: # noqa: BLE001
msg = f"抓取/解析失败:{type(e).__name__}: {e}"
print(msg, file=sys.stderr)
if args.log:
_append_log(args.log, msg, 2)
return 2
try:
text_before = cfg_path.read_text(encoding="utf-8")
cfg = yaml.safe_load(text_before)
except Exception as e: # noqa: BLE001
msg = f"读配置失败:{type(e).__name__}: {e}"
print(msg, file=sys.stderr)
if args.log:
_append_log(args.log, msg, 2)
return 2
# 第三十三轮: 上下文来源(拉不到 = None/空 dict,只跳过上下文同步,不产生漂移)
md_dev = fetch_models_dev()
ag_ctx = fetch_agnes_context()
# Dots 的文档是 SPA + 带 hash 的 JS chunk,抓取链路比另外两家脆 —— 所以**失败不致命**:
# 置空 → diff 里跳过 Dots,报告里明写「抓取失败」,其余两家照常同步。
dots_doc: dict = {}
try:
dots_doc = fetch_dots()
except Exception as e: # noqa: BLE001
print(f"[warn] Dots 文档抓取失败,本次跳过 Dots 比对:{type(e).__name__}: {e}",
file=sys.stderr)
report = diff(cfg, oc_doc, agnes_doc, md_dev, ag_ctx, dots_doc)
plan = build_plan(report, cfg, oc_doc, agnes_doc, dots_doc)
drift = has_drift(report)
code = 1 if drift else 0
out = [json.dumps(report, ensure_ascii=False, indent=2) if args.json else render(report),
render_plan(plan, date)]
if args.apply:
if plan_is_empty(plan):
out.append("→ 无需落盘:网关已与文档一致 ✓")
code = 0
else:
new_text = apply_plan(text_before, plan, date)
problems = verify(new_text, cfg, plan)
if problems:
out.append("→ 回读校验不通过,**未写盘**:")
out += [f" ✗ {p}" for p in problems]
code = 2
else:
info = _write_config_text(cfg_path, new_text)
out.append(f"→ 已写盘:{info['path']}({info['bytes']} 字节,备份 {info['backup']})")
out.append(" server.hot_reload 开着,15 秒内自动生效,无需重启")
code = 0
elif drift:
out.append("→ 只报告模式,未写盘(加 --apply 才同步)")
text = "\n".join(out)
print(text)
if args.log:
_append_log(args.log, text, code)
return code
if __name__ == "__main__":
sys.exit(main())