feat(compose): align env vars with code + fix helm GraphRAG bug (#2164)

* feat(compose): align env vars in docker-compose and .env.example with code

docker-compose.yml / docker-compose.dev.yml
- app.environment: remove deprecated vars Go app no longer reads
  (CRYPTO_MASTER_KEY, CRYPTO_SALT, TENANT_AES_KEY, ENABLE_GRAPH_RAG);
  add missing vars the code actually consumes (OSS/S3/TOS/COS completion,
  OpenSearch, Tencent VectorDB, Milvus credentials, Redis TLS/namespace,
  OIDC, audit/invitation/quota, LLM tuning, doc timeouts, etc.)
- docreader.environment: complete PDF tuning knobs, SSRF/LOG_LEVEL/gRPC
  params, GRPC_MTLS_REQUIRE_CLIENT_CERT; fix DOCREADER_PDF_JPEG_QUALITY
  default 90 -> 85 to match code (config.py)
- mcp.environment: add MCP_SERVER_AUTH_TOKEN (required by HTTP transport),
  WEKNORA_CHAT_TIMEOUT, WEKNORA_VERIFY_SSL, MCP_ALLOWED_UPLOAD_DIRS
- dev compose docreader.environment kept symmetric with prod

.env.example
- reorganize into A-J sections (deploy / data / retrieval / models /
  parsing / auth / agent / integrations / observability / security)
- remove the four deprecated vars, document why
- add missing vars, fix NEO4J_URI to bolt:// (matches compose + single-node),
  fix DORIS_COMPAT_MODE typo, complete docreader PDF tuning defaults

Co-Authored-By: Claude <noreply@anthropic.com>

* fix(helm): inject NEO4J_ENABLE and drop deprecated TENANT_AES_KEY

helm graph bug fix
- NEO4J_ENABLE is the sole graph switch the Go app reads (since v0.1.6,
  ENABLE_GRAPH_RAG is no longer read). helm never injected NEO4J_ENABLE,
  so GraphRAG was silently broken: setting neo4j.enabled=true gave the
  app NEO4J_URI/USERNAME/PASSWORD but not the switch, and the UI rejected
  Node Extractor config with "请正确配置环境变量NEO4J_ENABLE".
- app.yaml: inject NEO4J_ENABLE=true inside the neo4j.enabled block;
  remove the dead ENABLE_GRAPH_RAG injection
- NOTES.txt: update GraphRAG instructions (just enable neo4j, no need
  to set the deprecated ENABLE_GRAPH_RAG)
- values.yaml: drop ENABLE_GRAPH_RAG, fix neo4j section comment

TENANT_AES_KEY cleanup (deprecated since v0.4.0, encryption now uses
SYSTEM_AES_KEY exclusively; Go app no longer reads it)
- secrets.yaml: stop generating/looking-up/writing TENANT_AES_KEY,
  fix stale comment about tenants.api_key rotation
- app.yaml: remove TENANT_AES_KEY secretKeyRef
- values.yaml: drop tenantAesKey, fix existingSecret key list + comments
- cloud-image/firstboot.sh: stop generating/writing/printing TENANT_AES
- cloud-image/README.md: drop TENANT_AES_KEY from credential list

Co-Authored-By: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>
This commit is contained in:
Greenplumwine
2026-07-21 15:55:52 +08:00
committed by GitHub
parent 150c07368b
commit fa7ae075ee
9 changed files with 926 additions and 782 deletions
+61 -9
View File
@@ -247,26 +247,78 @@ services:
volumes:
- docreader-tmp-dev:/tmp/docreader
environment:
# ========== 基础 ==========
- TZ=${TZ:-Asia/Shanghai}
- LOG_LEVEL=${LOG_LEVEL:-}
- DOCREADER_IMAGE_OUTPUT_DIR=/tmp/docreader
- DOCREADER_PDF_FORCE_SCANNED=${DOCREADER_PDF_FORCE_SCANNED:-false}
- MAX_FILE_SIZE_MB=${MAX_FILE_SIZE_MB:-}
# docreader 对外 HTTP 抓取的 SSRF 白名单(逗号分隔;与 app 共用同名变量)
- SSRF_WHITELIST=${SSRF_WHITELIST:-}
- SSRF_WHITELIST_EXTRA=${SSRF_WHITELIST_EXTRA:-}
# ========== gRPC 服务端 TLS / 认证 ==========
- GRPC_TLS_ENABLED=${GRPC_TLS_ENABLED:-false}
- GRPC_TLS_CERT=${GRPC_TLS_CERT:-}
- GRPC_TLS_KEY=${GRPC_TLS_KEY:-}
- GRPC_TLS_CA=${GRPC_TLS_CA:-}
- GRPC_TLS_SERVER_NAME=${GRPC_TLS_SERVER_NAME:-}
# true 时强制要求客户端证书(mTLS)
- GRPC_MTLS_REQUIRE_CLIENT_CERT=${GRPC_MTLS_REQUIRE_CLIENT_CERT:-false}
- GRPC_AUTH_TOKEN=${GRPC_AUTH_TOKEN:-}
# ========== gRPC 服务参数 ==========
- DOCREADER_GRPC_MAX_WORKERS=${DOCREADER_GRPC_MAX_WORKERS:-4}
- DOCREADER_GRPC_MAX_FILE_SIZE_MB=${DOCREADER_GRPC_MAX_FILE_SIZE_MB:-${MAX_FILE_SIZE_MB:-50}}
- DOCREADER_GRPC_PORT=${DOCREADER_GRPC_PORT:-50051}
# ========== 解析通用 ==========
- DOCREADER_DOCX_MAX_PAGES=${DOCREADER_DOCX_MAX_PAGES:-0}
- DOCREADER_MARKITDOWN_MAX_WORKERS=${DOCREADER_MARKITDOWN_MAX_WORKERS:-1}
# 出站 HTTP/HTTPS 代理(抓取网页 / 远程图片时)
- DOCREADER_EXTERNAL_HTTP_PROXY=${DOCREADER_EXTERNAL_HTTP_PROXY:-}
- DOCREADER_EXTERNAL_HTTPS_PROXY=${DOCREADER_EXTERNAL_HTTPS_PROXY:-}
# ========== OpenDataLoader hybrid ==========
- DOCREADER_ODL_MAX_WORKERS=${DOCREADER_ODL_MAX_WORKERS:-1}
- DOCREADER_ODL_HYBRID=${DOCREADER_ODL_HYBRID:-off}
- DOCREADER_ODL_HYBRID_URL=${DOCREADER_ODL_HYBRID_URL:-http://odl-hybrid:5002}
- DOCREADER_ODL_HYBRID_MODE=${DOCREADER_ODL_HYBRID_MODE:-auto}
- DOCREADER_ODL_HYBRID_FALLBACK=${DOCREADER_ODL_HYBRID_FALLBACK:-false}
- DOCREADER_ODL_MARKDOWN_WITH_HTML=${DOCREADER_ODL_MARKDOWN_WITH_HTML:-false}
- DOCREADER_MARKITDOWN_MAX_WORKERS=${DOCREADER_MARKITDOWN_MAX_WORKERS:-1}
# ========== PDF 渲染 ==========
- DOCREADER_PDF_RENDER_MAX_WORKERS=${DOCREADER_PDF_RENDER_MAX_WORKERS:-1}
# 单个 PDF 内部扫描页渲染并行进程数(pdfium 非线程安全;默认随 CPU min(4,cpu))
- DOCREADER_PDF_RENDER_PARALLELISM=${DOCREADER_PDF_RENDER_PARALLELISM:-4}
- DOCREADER_PDF_RENDER_DPI=${DOCREADER_PDF_RENDER_DPI:-200}
- DOCREADER_PDF_JPEG_QUALITY=${DOCREADER_PDF_JPEG_QUALITY:-90}
- GRPC_TLS_ENABLED=${GRPC_TLS_ENABLED:-false}
- GRPC_TLS_CERT=${GRPC_TLS_CERT:-}
- GRPC_TLS_KEY=${GRPC_TLS_KEY:-}
- GRPC_TLS_CA=${GRPC_TLS_CA:-}
- GRPC_TLS_SERVER_NAME=${GRPC_TLS_SERVER_NAME:-}
- GRPC_AUTH_TOKEN=${GRPC_AUTH_TOKEN:-}
# JPEG 质量(代码默认 85;compose 此前误写 90,已对齐)
- DOCREADER_PDF_JPEG_QUALITY=${DOCREADER_PDF_JPEG_QUALITY:-85}
# 渲染页图最大长边像素(防超大页撑爆 gRPC 消息上限;密集中文建议 >=1600)
- DOCREADER_PDF_RENDER_MAX_EDGE=${DOCREADER_PDF_RENDER_MAX_EDGE:-2000}
# 强制 PDF 全页按扫描件解析(文本层差但可 OCR 时;速度/成本上升)
- DOCREADER_PDF_FORCE_SCANNED=${DOCREADER_PDF_FORCE_SCANNED:-false}
# ========== PDF 版式 / 文本调优 ==========
# glyph 间距超过 median 字宽的多少倍时插空格(默认 0.4)
- DOCREADER_PDF_WORD_GAP_WIDTH_RATIO=${DOCREADER_PDF_WORD_GAP_WIDTH_RATIO:-0.4}
# 丢弃窄边栏(arXiv 侧栏)的页宽比例阈值(默认 0.12)
- DOCREADER_PDF_MARGIN_COL_WIDTH_RATIO=${DOCREADER_PDF_MARGIN_COL_WIDTH_RATIO:-0.12}
- DOCREADER_PDF_MIN_HEADING_LINE_CHARS=${DOCREADER_PDF_MIN_HEADING_LINE_CHARS:-8}
# 清理 U+FFFE/软连字符、剥离矢量图轴文字、图表区按 JPEG 渲染
- DOCREADER_PDF_SANITIZE_TEXT=${DOCREADER_PDF_SANITIZE_TEXT:-true}
- DOCREADER_PDF_STRIP_CHART_DEBRIS=${DOCREADER_PDF_STRIP_CHART_DEBRIS:-true}
- DOCREADER_PDF_RENDER_VECTOR_FIGURES=${DOCREADER_PDF_RENDER_VECTOR_FIGURES:-true}
- DOCREADER_PDF_DETECT_HEADINGS=${DOCREADER_PDF_DETECT_HEADINGS:-}
- DOCREADER_PDF_LAYOUT_ORDERING=${DOCREADER_PDF_LAYOUT_ORDERING:-}
- DOCREADER_PDF_FILTER_HIDDEN_TEXT=${DOCREADER_PDF_FILTER_HIDDEN_TEXT:-}
# 扫描件判定:图片面积占比 >= 此值视为扫描页(默认留空走代码默认)
- DOCREADER_PDF_SCAN_IMAGE_RATIO=${DOCREADER_PDF_SCAN_IMAGE_RATIO:-}
- DOCREADER_PDF_SCAN_MIN_CHARS=${DOCREADER_PDF_SCAN_MIN_CHARS:-}
# 嵌入图片抽取
- DOCREADER_PDF_EXTRACT_EMBEDDED_IMAGES=${DOCREADER_PDF_EXTRACT_EMBEDDED_IMAGES:-}
- DOCREADER_PDF_EMBED_MIN_PIXELS=${DOCREADER_PDF_EMBED_MIN_PIXELS:-}
- DOCREADER_PDF_EMBED_MIN_AREA_RATIO=${DOCREADER_PDF_EMBED_MIN_AREA_RATIO:-}
- DOCREADER_PDF_EMBED_REPEAT_PAGE_FRAC=${DOCREADER_PDF_EMBED_REPEAT_PAGE_FRAC:-}
- DOCREADER_PDF_EMBED_MAX_IMAGES=${DOCREADER_PDF_EMBED_MAX_IMAGES:-}
# 图表区识别
- DOCREADER_PDF_MIN_CHART_REGION_AREA=${DOCREADER_PDF_MIN_CHART_REGION_AREA:-}
- DOCREADER_PDF_MIN_CHART_REGION_CHARS=${DOCREADER_PDF_MIN_CHART_REGION_CHARS:-}
- DOCREADER_PDF_MAX_CHART_REGION_AREA=${DOCREADER_PDF_MAX_CHART_REGION_AREA:-}
- DOCREADER_PDF_MAX_FIGURE_HEIGHT_RATIO=${DOCREADER_PDF_MAX_FIGURE_HEIGHT_RATIO:-}
healthcheck:
test: ["CMD", "grpc_health_probe", "-addr=localhost:50051"]
interval: 30s