feat(agents): 图片附件 + 修两个真 bug(版本 2.5.0 → 2.5.1)
老板要求「图片附件功能加上,其他的先不做」。
QwenPaw 侧格式(实测确认后再动手):{"type":"image","image_url":"<data-URI>"
—— 塞纯红 64×64 PNG 的 data URI,模型正确答出「红色」。
改动:
- app/adapters/qwenpaw_engine.py:放行图片内容块,并同时支持两种写法 ——
AgentScope 原生 image 与 OpenAI 风格 image_url(前端发的是后者);
新增 _image_url();缺 url 的块跳过;audio/video/file 仍忽略
- views/Agents.vue:对话输入区加附件按钮(pickImage 复用现有浏览器内压缩,最多 4 张)
+ 待发缩略图可删 + 消息气泡内渲染图片;允许「只发图不带字」
- lib/agents.ts:content 放宽为 string | any[]
- i18n 新增 4 键,中英对称 879/879
修掉两个真 bug(本轮才暴露):
1. image_url 写法被忽略 —— 分支只认 type=="image",而前端发 image_url
→ 图片会静默丢失(功能等于没做)。测试用例抓出来的。
2. /agents/{slug}/chat 500 —— 上轮加计量打标时 log.info(..., rounds) 写在
rounds = ... 之前,UnboundLocalError → 端点 500。已加回归用例
test_agent_chat_endpoint_source_order 固化(这类声明顺序错误跑不出来)。
上线实测:两张图(红+蓝)→ 模型答「红色,蓝色」顺序正确,system_suffix 同时生效,
usage wjd/24112 tokens/缓存 2048;浏览器路径也验证过(上传→缩略图→发送→渲染→无 JS 错误)。
已知引擎侧行为(非网关问题):QwenPaw SSE 只有 text 事件、无独立 reasoning 通道,
模型有时把思考写进正文;且上下文会污染视觉判断(同图脏上下文答「白色」、干净会话答「蓝色」)。
测试 218 → 219 全过。
This commit is contained in:
1 parent
c0706d1846
commit
c62be67aa8
9 files changed
+230
-16
No files matched your search
@@ -5967,6 +5967,78 @@ def test_agents_routes_installed():
|
||||
assert need in paths, f"缺少端点 {need}"
|
||||
|
||||
|
||||
def test_qwenpaw_engine_image_content_passthrough():
|
||||
"""图片附件:内容块必须原样转成 AgentScope 的 image 形状(第三十九轮)。
|
||||
|
||||
QwenPaw 侧的格式已实测确认:`{"type":"image","image_url":"<data-URI 或 URL>"}`
|
||||
—— 塞一张纯红 64×64 PNG 的 data URI,模型正确答出"红色"。
|
||||
"""
|
||||
from app.adapters.qwenpaw_engine import build_body, _image_url
|
||||
|
||||
uri = "data:image/png;base64,iVBORw0KGgoAAAANSUhEUg=="
|
||||
# AgentScope 原生扁平写法
|
||||
b = build_body([{"role": "user", "content": [
|
||||
{"type": "text", "text": "这是什么颜色?"},
|
||||
{"type": "image", "image_url": uri},
|
||||
]}], "s1")
|
||||
parts = b["input"][0]["content"]
|
||||
assert parts[0] == {"type": "text", "text": "这是什么颜色?"}
|
||||
assert parts[1] == {"type": "image", "image_url": uri}
|
||||
|
||||
# OpenAI 风格嵌套写法也要认(前端 Chat 页发的是这个形状)
|
||||
b2 = build_body([{"role": "user", "content": [
|
||||
{"type": "image_url", "image_url": {"url": uri}},
|
||||
]}], "s2")
|
||||
assert b2["input"][0]["content"][0]["image_url"] == uri
|
||||
|
||||
# 只发图不带字 → content 里只有图片块
|
||||
b3 = build_body([{"role": "user", "content": [{"type": "image", "image_url": uri}]}], "s3")
|
||||
assert len(b3["input"][0]["content"]) == 1
|
||||
assert b3["input"][0]["content"][0]["type"] == "image"
|
||||
|
||||
# 缺 url 的图片块被跳过(不产生空块)
|
||||
b4 = build_body([{"role": "user", "content": [
|
||||
{"type": "text", "text": "x"}, {"type": "image"},
|
||||
]}], "s4")
|
||||
assert len(b4["input"][0]["content"]) == 1
|
||||
|
||||
# 不支持的媒体类型(audio/视频/文件)忽略,不炸
|
||||
b5 = build_body([{"role": "user", "content": [
|
||||
{"type": "text", "text": "x"}, {"type": "audio", "data": "..."},
|
||||
]}], "s5")
|
||||
assert len(b5["input"][0]["content"]) == 1
|
||||
|
||||
# _image_url 的两种取值
|
||||
assert _image_url({"image_url": uri}) == uri
|
||||
assert _image_url({"image_url": {"url": uri}}) == uri
|
||||
assert _image_url({"image_url": {}}) == ""
|
||||
assert _image_url({}) == ""
|
||||
|
||||
# 多图 + 文字混合顺序保持
|
||||
b6 = build_body([{"role": "user", "content": [
|
||||
{"type": "image", "image_url": "u1"},
|
||||
{"type": "text", "text": "t"},
|
||||
{"type": "image", "image_url": "u2"},
|
||||
]}], "s6")
|
||||
got = b6["input"][0]["content"]
|
||||
assert [p["type"] for p in got] == ["image", "text", "image"]
|
||||
|
||||
def test_agent_chat_endpoint_source_order():
|
||||
"""回归:agents_chat 里不能在 rounds 赋值**之前**引用它。
|
||||
|
||||
第三十九轮实测踩到 —— 打标日志被写在 `rounds = ...` 之前,
|
||||
`log.info(..., rounds)` 直接 UnboundLocalError → 整个 /agents/{slug}/chat 返回 500。
|
||||
这类「声明顺序」错误跑不出来,只能靠读代码或真机调用发现,所以固化成断言。
|
||||
"""
|
||||
import inspect
|
||||
import app.agents_routes as ar
|
||||
|
||||
src = inspect.getsource(ar.install)
|
||||
i_assign = src.find("rounds = len(")
|
||||
i_use = src.find('log.info("agent chat:')
|
||||
assert i_assign != -1 and i_use != -1, "端点源码结构变了,请更新本用例"
|
||||
assert i_assign < i_use, "rounds 必须在打标日志之前赋值(否则 UnboundLocalError → 500)"
|
||||
|
||||
# 直接运行(非 pytest)—— 必须留在文件末尾,见上方 `test_image_request_goes_to_requested_model`
|
||||
# 之前的旁注:运行器只会收集执行到这一行时已经存在的 globals。
|
||||
if __name__ == "__main__":
|
||||
|
||||
Reference in new issue
Block a user