姚章浩
693cd952cd
仓库首次入库（此前仅存在于本地工作区与测试服构建产物中，未上传）。 改动（src/knowledge_adapter/asset_content_processor.py）： - 新增 _TRUNCATED_TRAILING_MARKDOWN_RESOURCE_IMAGE 规则，清除 chunk 末尾被截断的 ![](resource://xxxx 残片（WeKnora 分块在图片引用中间截断时产生，旧逻辑无法匹配完整引用， 残留 resource:// 触发 ContentProcessingError，导致整单 502） - 回归测试 tests/test_asset_content_processor.py:: test_truncated_trailing_markdown_resource_image_is_removed 验证证据： - 测试服 120.76.217.51，镜像 knowledge-adapter-service:reviewed-test-aeab4b5fca2d （release aeab4b5fca2d8661）：KB d015d05c-25c8-4d63-b2ab-e8590d3c2dfb 5 个查询全部 http=200、records=5、残留 resource:// = 0；KB b9dacd62 无退化；容器日志无 error/exception - 本地：pytest 141 项全通过、ruff 全通过 - 该文件与 258c31e0fe07fd08 源码的差异经逐文件 md5 比对，为唯一变更文件
145 lines
6.9 KiB
Markdown
145 lines
6.9 KiB
Markdown
# Test Deployment and Rollback
|
|
|
|
> **TEST ENVIRONMENT ONLY**
|
|
>
|
|
> Production deployment is prohibited until Slice 9 real E2E evidence and the audit report are reviewed and explicitly accepted. This document is a prepared operator runbook, not authorization to execute it.
|
|
|
|
## Delivered files
|
|
|
|
```text
|
|
Dockerfile
|
|
deploy/docker-compose.adapter.yml
|
|
deploy/adapter.env.example
|
|
deploy/nginx/knowledge-adapter.conf
|
|
deploy/ram/knowledge-adapter-oss-read-policy.json
|
|
deploy/sql/weknora_readonly_grants.sql
|
|
migrations/0001_asset_mapping.up.sql
|
|
migrations/0001_asset_mapping.down.sql
|
|
```
|
|
|
|
## Prerequisites
|
|
|
|
- Dedicated test OSS AccessKey pair; test and production identities must be different.
|
|
- Private OSS bucket in the configured region.
|
|
- Existing private WeKnora API endpoint.
|
|
- Dedicated Adapter PostgreSQL database/login.
|
|
- Different dedicated WeKnora read-only PostgreSQL login.
|
|
- Host Nginx and a test TLS certificate for the reviewed public hostname.
|
|
- Docker Compose operated by an authorized human; this repository does not modify Docker daemon permissions.
|
|
|
|
## 1. Prepare non-secret configuration
|
|
|
|
Copy `deploy/adapter.env.example` outside Git, for example to `/opt/knowledge-adapter/adapter.env`, and replace every placeholder. Do not add raw API keys or database URLs.
|
|
|
|
`DIFY_KEY_HASH_CURRENT` is the SHA-256 digest of the active Dify key. `DIFY_KEY_HASH_NEXT` remains empty until a dual-slot rotation.
|
|
|
|
Review these invariants:
|
|
|
|
- `ASSET_PUBLIC_BASE_URL` exactly matches the HTTPS Nginx hostname.
|
|
- `WEKNORA_NETWORK_NAME` names an existing external Docker network and `WEKNORA_BASE_URL` resolves the existing WeKnora App service on it; the test baseline uses `weknora071full-network` and `http://app:8080`.
|
|
- `OSS_CREDENTIAL_MODE=static_ak`; `OSS_REGION` and `OSS_BUCKET` match the test resources.
|
|
- `OSS_ACCESS_KEY_ID_HOST_FILE` and `OSS_ACCESS_KEY_SECRET_HOST_FILE` point to the two protected host Secret files.
|
|
- the KB allowlist contains only approved UUIDs;
|
|
- all response budgets match the approved values.
|
|
|
|
## 2. Prepare Secret files
|
|
|
|
Create an untracked `/opt/knowledge-adapter/secrets` directory. Populate each file through an approved secret manager or interactive editor; never put a secret in a command argument or shell history.
|
|
|
|
Required files:
|
|
|
|
```text
|
|
weknora_api_key
|
|
asset_database_url
|
|
weknora_readonly_database_url
|
|
oss_access_key_id
|
|
oss_access_key_secret
|
|
```
|
|
|
|
Set directory mode `0700`. Set each file owner to UID/GID `10001:10001` and mode `0400`. Confirm each file is non-empty without printing its contents. The two database DSNs must identify different accounts and databases. Never place either OSS value in `adapter.env`.
|
|
|
|
## 3. Provision database permissions
|
|
|
|
A DBA reviews and applies `migrations/0001_asset_mapping.up.sql` only to the dedicated Adapter database. Separately, a WeKnora DBA reviews and applies `deploy/sql/weknora_readonly_grants.sql` to the dedicated read-only role.
|
|
|
|
The application does not run migration automatically. Before migration:
|
|
|
|
1. take a database-native backup;
|
|
2. verify the backup catalog is readable;
|
|
3. record the current schema state;
|
|
4. apply the migration in the test database;
|
|
5. verify table/index constraints and role privileges.
|
|
|
|
The down migration drops Adapter mapping state and is destructive. Do not use it as routine rollback after data exists.
|
|
|
|
## 4. Review the OSS static AK identity
|
|
|
|
For the approved test window, the existing test-only AK may be used from the two Secret files. Before production, create a separate RAM user and AccessKey, replace `${OSS_BUCKET}` and `${WEKNORA_TENANT_ID}` in `deploy/ram/knowledge-adapter-oss-read-policy.json`, and confirm the only Action is `oss:GetObject` and the only Resource is the intended production object prefix. Do not reuse the test AK or place AccessKey values in environment variables.
|
|
|
|
## 5. Render configuration
|
|
|
|
The required read-only render gate is `docker compose config`; it must pass before any build or runtime action.
|
|
|
|
From the repository root, an authorized operator can perform a read-only render:
|
|
|
|
```text
|
|
docker compose --env-file /opt/knowledge-adapter/adapter.env \
|
|
-f deploy/docker-compose.adapter.yml config --quiet
|
|
```
|
|
|
|
Then inspect the rendered service for image/build source, loopback port, UID 10001, Secret mount targets, resource limits, and healthcheck. Rendering must not print Secret file contents.
|
|
|
|
## 6. Build and start the test candidate
|
|
|
|
Build only after source tests, Ruff, JSON validation, and Compose rendering pass. Use an immutable reviewed image tag rather than `latest`. Starting/recreating the candidate requires a separate operator approval because it changes runtime state.
|
|
|
|
The prepared Compose file contains only the Adapter; it does not create or replace PostgreSQL, Redis, WeKnora, Nginx, or OSS resources.
|
|
|
|
## 7. Install Nginx test configuration
|
|
|
|
Review `deploy/nginx/knowledge-adapter.conf` and replace the example hostname and certificate paths. Validate the complete host Nginx configuration before reload. The Adapter upstream remains `127.0.0.1:18080`.
|
|
|
|
Nginx rules intentionally:
|
|
|
|
- reject cleartext HTTP;
|
|
- expose only `/retrieval`, `/i/{UUID}`, and local `/healthz`;
|
|
- limit methods, body size, and request rate;
|
|
- sanitize `X-Request-ID`;
|
|
- disable access logs;
|
|
- pass 302 `Location` without logging it.
|
|
|
|
Installing or reloading host Nginx is outside this repository and requires explicit operator approval.
|
|
|
|
## 8. Test-environment acceptance
|
|
|
|
Do not print credentials or the 302 Location value. Record only sanitized results:
|
|
|
|
1. container health becomes healthy and `GET /healthz` returns 200;
|
|
2. no-auth and wrong-key `/retrieval` return 401;
|
|
3. allowlisted retrieval returns valid Dify records;
|
|
4. non-allowlisted KB returns fixed 400;
|
|
5. rewritten content has no internal locator, private endpoint, local path, object key, or signature;
|
|
6. valid `/i/{UUID}` returns 302 with `no-store` and a V4 URL whose expiry class is 600 seconds;
|
|
7. direct anonymous OSS object GET is denied;
|
|
8. following the 302 retrieves the expected test object;
|
|
9. a later visit to the same stable UUID produces a fresh temporary URL;
|
|
10. missing/disabled/malformed image IDs do not invoke OSS signing;
|
|
11. structured logs correlate request IDs without prohibited values;
|
|
12. CPU/RAM, database pool usage, upstream latency, and error rates are recorded.
|
|
|
|
The full real E2E belongs to Slice 9.
|
|
|
|
## Rollback
|
|
|
|
Application rollback is image/config rollback, not database destruction:
|
|
|
|
1. stop routing new test traffic at Nginx;
|
|
2. preserve sanitized logs and current rendered configuration;
|
|
3. restore the previously reviewed image tag and non-secret env file;
|
|
4. restore prior Secret files only when credential rotation caused the failure;
|
|
5. recreate only the Adapter service;
|
|
6. verify `/healthz`, authentication, retrieval, and stable image redirects;
|
|
7. restore Nginx routing.
|
|
|
|
Do not run `down -v`, prune volumes, drop `asset_mapping`, revoke the active AK before rollback is proven, or restore over a database without a separately reviewed recovery plan and verified backup.
|