Commit Graph

3 Commits

Author SHA1 Message Date
wizardchen dc634e863a feat(crypto): AES-GCM at-rest for MCP and DataSource credentials with graceful key-rotation handling
Two encryption gaps and one fail-loud regression, fixed together because
they are inseparable in practice.

At-rest encryption gaps
-----------------------
Before this change, only Tenant.APIKey, Tenant.Credentials.WeKnoraCloud.AppSecret,
Model.Parameters.APIKey/AppSecret, and WebSearchProviderParameters.APIKey
were encrypted with SYSTEM_AES_KEY. MCP and DataSource were missed:

- MCPAuthConfig.Value/Scan went through bare json.Marshal/Unmarshal, so
  api_key and token were written to mcp_services.auth_config (json) in
  plaintext regardless of whether SYSTEM_AES_KEY was set.
- DataSource.Config (types.JSON) is a byte passthrough, so connector
  secrets (Notion token, GitHub PAT, Feishu app_secret, ...) inside the
  Credentials map were stored in plaintext too.

Now both follow the ModelParameters / WebSearchProviderParameters pattern.
Encryption operates on a local copy to avoid mutating the caller's
in-memory struct (otherwise the next read of the same *MCPAuthConfig
would surface ciphertext as if it were the API key). For DataSource the
ToJSON ↔ ParseConfig pair is the only application boundary that
materializes credentials, so encrypting there is sufficient and avoids
having to fork the generic types.JSON.

Backward compatibility
----------------------
DecryptStoredSecret returns rows without the enc:v1: prefix verbatim, so
historical plaintext records keep working. Next write re-saves them
encrypted. No migration step.

Graceful Scan-time degradation
------------------------------
The strict "fail loud on key rotation" behaviour of DecryptStoredSecret
turned into a footgun once it ran from GORM Scan: a single broken row
caused ListModels / ListMCPServices / ListDataSources to return zero
results, hiding all other rows and giving the operator no way to find
which credential needed re-entering.

New utils.DecryptStoredSecretLenient: same logic, but on failure returns
("", false). The strict variant is kept for hot paths that are about to
actually USE the credential (so a rotated key surfaces loudly there
rather than silently degrading to "missing api key").

All Scan paths switch to lenient + per-failure log line:
  ModelParameters.Scan, WebSearchProviderParameters.Scan,
  MCPAuthConfig.Scan, DataSource.ParseConfig, CredentialsConfig.Scan.

On lenient failure the field blanks out (never ciphertext), the row
still loads, the DTO reports configured=false so the UI shows "not
configured", and any upstream call attempted with the empty value fails
with a clear "missing api key" rather than sending ciphertext as the key.

Tenant.AfterFind (Tenant.APIKey) intentionally stays strict — it is the
primary HTTP auth credential and a silent empty value would change auth
semantics ambiguously.
2026-05-17 15:27:52 +08:00
wizardchen 3b23713c54 fix(crypto): fail loudly when encrypted DB fields cannot be decrypted
GORM Scan / AfterFind hooks for every AES-encrypted column followed
the same lenient pattern:

    if decrypted, err := utils.DecryptAESGCM(c.APIKey, key); err == nil {
        c.APIKey = decrypted
    }

When SYSTEM_AES_KEY was missing, rotated, or the wrong length the
decryption error was swallowed and the in-memory struct kept the raw
"enc:v1:..." ciphertext. The application then happily forwarded the
ciphertext upstream as the actual API key / password, surfacing as
401/403/SignatureDoesNotMatch from third-party vendors. Worse, a
ciphertext snippet of a customer credential was leaking into the
external provider's request logs.

Introduce utils.DecryptStoredSecret that:
  - returns "" / legacy plaintext untouched (no false positives for
    pre-encryption rows);
  - returns ErrEncryptedDataMissingKey when the value carries the
    enc:v1: prefix but no AES key is configured;
  - propagates any GCM auth-tag failure from a rotated key.

Wire it into the five Scan / AfterFind sites that currently swallow
the error:
  - Tenant.AfterFind (tenants.api_key)
  - CredentialsConfig.Scan (tenants.we_knora_cloud.app_secret)
  - ModelParameters.Scan (models.parameters.api_key, .app_secret)
  - ConnectionConfig.Scan (vector_store_connections.password, .api_key)
  - WebSearchProviderParameters.Scan (web_search_providers.api_key)

The error is wrapped with the originating column so the failure is
diagnosable in the logs. Operators must restore the previous
SYSTEM_AES_KEY (or rotate the affected secrets) instead of receiving
silent vendor 401s.

Add table-driven tests covering empty input, legacy plaintext,
round-trip, missing key, wrong-length key, and rotated key. The
strict path explicitly asserts that ciphertext does NOT leak in the
returned plaintext on any error path.
2026-04-30 15:17:56 +08:00
AndyYang 6c69de2df1 feat(security): add AES-256-GCM encryption for API keys at rest
- Add crypto utility (internal/utils/crypto.go) with AES-256-GCM encrypt/decrypt
  using SYSTEM_AES_KEY env var, with "enc:v1:" prefix for versioned ciphertext
- Encrypt tenant API key via GORM BeforeSave/AfterFind hooks and manual
  encryption in CreateTenant/UpdateAPIKey (db.Updates bypasses hooks)
- Encrypt model API key in ModelParameters Value/Scan (driver.Valuer)
- Widen api_key column from varchar(64) to varchar(256) across all DB dialects
  (MySQL, ParadeDB, SQLite) and add versioned migration 000018
- Propagate SYSTEM_AES_KEY through docker-compose, Helm secrets and values
- Fix migration 000017 PL/pgSQL dollar-quoting syntax ($ -> $$)
2026-03-09 10:35:07 +08:00