Files
knowledge-adapter-service/src/knowledge_adapter/content_budget.py
T
姚章浩 693cd952cd feat(asset): 清理被截断的结尾图片引用,修复整单 502
仓库首次入库(此前仅存在于本地工作区与测试服构建产物中,未上传)。

改动(src/knowledge_adapter/asset_content_processor.py):
- 新增 _TRUNCATED_TRAILING_MARKDOWN_RESOURCE_IMAGE 规则,清除 chunk 末尾被截断的
  ![](resource://xxxx 残片(WeKnora 分块在图片引用中间截断时产生,旧逻辑无法匹配完整引用,
  残留 resource:// 触发 ContentProcessingError,导致整单 502)
- 回归测试 tests/test_asset_content_processor.py::
  test_truncated_trailing_markdown_resource_image_is_removed

验证证据:
- 测试服 120.76.217.51,镜像 knowledge-adapter-service:reviewed-test-aeab4b5fca2d
  (release aeab4b5fca2d8661):KB d015d05c-25c8-4d63-b2ab-e8590d3c2dfb 5 个查询全部
  http=200、records=5、残留 resource:// = 0;KB b9dacd62 无退化;容器日志无 error/exception
- 本地:pytest 141 项全通过、ruff 全通过
- 该文件与 258c31e0fe07fd08 源码的差异经逐文件 md5 比对,为唯一变更文件
2026-09-16 10:01:27 +08:00

26 lines
929 B
Python

import re
TRUNCATION_MARKER = "[内容已由 Knowledge Adapter 截断]"
_MARKDOWN_LINK_PATTERN = re.compile(r"!?\[[^\]\n]*\]\([^\)\n]*\)")
_FENCED_CODE_PATTERN = re.compile(r"```[\s\S]*?(?:```|\Z)")
def _safe_cutoff(content: str, cutoff: int) -> int:
protected_starts = [
match.start()
for pattern in (_MARKDOWN_LINK_PATTERN, _FENCED_CODE_PATTERN)
for match in pattern.finditer(content)
if match.start() < cutoff < match.end()
]
return min(protected_starts, default=cutoff)
def truncate_content(content: str, *, max_chars: int) -> str:
if len(content) <= max_chars:
return content
if max_chars < len(TRUNCATION_MARKER):
raise ValueError("max_chars is too small for the truncation marker")
prefix_chars = max_chars - len(TRUNCATION_MARKER)
prefix_chars = _safe_cutoff(content, prefix_chars)
return f"{content[:prefix_chars]}{TRUNCATION_MARKER}"