fix(storage): merge tos/s3/oss/ks3 tenant configs in buildStorageConfig (#1117)
knowledgeService.buildStorageConfig had hardcoded switches in two places that
only handled local/minio/cos. Knowledge bases configured for tos, s3, oss, or
ks3 reached the docreader with a DocParserStorageConfig containing only the
Provider tag — bucket, endpoint, and credentials were silently dropped, and
extraction stalled or fetched from the wrong location.
- Tenant-merge switch: add tos/s3/oss/ks3 cases mapping
StorageEngineConfig.{TOS,S3,OSS,KS3} fields onto DocParserStorageConfig.
Same shape as the existing minio/cos branches; AccessKey -> AccessKeyID,
SecretKey -> SecretAccessKey, no AppID.
- Legacy compat switch: list tos/s3/oss/ks3 alongside local with hasKBFull=false
so fall-through to the tenant-merge path is intentional rather than an
unrecognised provider sliding past the switch. The legacy `cos_config` column
predates these providers and they should never resolve via that path.
- Doc drift: types/custom_agent.go ImageStorageProvider comment and
handler/system.go StorageEngineStatusItem.Name comment now list all 7
providers (matched against types.StorageEngineConfig and ParseProviderScheme).
Tests (knowledge_storage_config_test.go, +268 LOC, 17 cases):
- TestBuildStorageConfig_TenantMergeAllProviders pins per-provider field
population for all 7 providers; the tos/s3/oss/ks3 sub-tests are the
regression guard for #1117.
- TestBuildStorageConfig_LegacyPathOnlyForCOSAndMinIO pins that legacy
StorageConfig data is only consumed for cos/minio and skipped for the
others (intentional fall-through).
- TestBuildStorageConfig_NoTenantFallsThroughToEmpty pins the no-tenant case.
Out of scope (called out in the linked issue):
- The systemic refactor (single source-of-truth provider registry) — left as
a follow-up; this PR keeps changes local to the buggy code path.
- initialization.go validateStorageEngineConfig multimodal validation gap —
the InitializationRequest struct does not carry tos/s3/oss/ks3 fields, so
fixing it requires API surface additions; can ship separately.
Closes #1117 (the buildStorageConfig switches; remaining items deferred).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -7642,6 +7642,10 @@ func (s *knowledgeService) buildStorageConfig(ctx context.Context, kb *types.Kno
|
||||
}
|
||||
|
||||
// Backward compatibility: if legacy cos_config has full params for the chosen provider, use them.
|
||||
// Note: legacy StorageConfig predates tos/s3/oss/ks3, so those providers always
|
||||
// resolve via the tenant-merge path below. Listing them here keeps the fall-through
|
||||
// intentional (instead of an unrecognised provider silently sliding past the switch).
|
||||
// See issue #1117: provider enum was missing tos/s3/oss in this switch.
|
||||
sc := &kb.StorageConfig
|
||||
hasKBFull := false
|
||||
switch provider {
|
||||
@@ -7649,7 +7653,7 @@ func (s *knowledgeService) buildStorageConfig(ctx context.Context, kb *types.Kno
|
||||
hasKBFull = sc.SecretID != "" && sc.BucketName != ""
|
||||
case "minio":
|
||||
hasKBFull = sc.BucketName != ""
|
||||
case "local":
|
||||
case "local", "tos", "s3", "oss", "ks3":
|
||||
hasKBFull = false
|
||||
}
|
||||
|
||||
@@ -7678,6 +7682,10 @@ func (s *knowledgeService) buildStorageConfig(ctx context.Context, kb *types.Kno
|
||||
provider = strings.ToLower(strings.TrimSpace(sec.DefaultProvider))
|
||||
out.Provider = strings.ToUpper(provider)
|
||||
}
|
||||
// Provider list must match types.StorageEngineConfig + ParseProviderScheme.
|
||||
// Missing a case here causes DocParserStorageConfig to be returned with only
|
||||
// Provider set — bucket/endpoint/credentials are silently dropped, and the
|
||||
// docreader then fails or fetches from the wrong location. See issue #1117.
|
||||
switch provider {
|
||||
case "local":
|
||||
if sec.Local != nil {
|
||||
@@ -7706,6 +7714,42 @@ func (s *knowledgeService) buildStorageConfig(ctx context.Context, kb *types.Kno
|
||||
out.AppID = sec.COS.AppID
|
||||
out.PathPrefix = sec.COS.PathPrefix
|
||||
}
|
||||
case "tos":
|
||||
if sec.TOS != nil {
|
||||
out.Endpoint = sec.TOS.Endpoint
|
||||
out.Region = sec.TOS.Region
|
||||
out.AccessKeyID = sec.TOS.AccessKey
|
||||
out.SecretAccessKey = sec.TOS.SecretKey
|
||||
out.BucketName = sec.TOS.BucketName
|
||||
out.PathPrefix = sec.TOS.PathPrefix
|
||||
}
|
||||
case "s3":
|
||||
if sec.S3 != nil {
|
||||
out.Endpoint = sec.S3.Endpoint
|
||||
out.Region = sec.S3.Region
|
||||
out.AccessKeyID = sec.S3.AccessKey
|
||||
out.SecretAccessKey = sec.S3.SecretKey
|
||||
out.BucketName = sec.S3.BucketName
|
||||
out.PathPrefix = sec.S3.PathPrefix
|
||||
}
|
||||
case "oss":
|
||||
if sec.OSS != nil {
|
||||
out.Endpoint = sec.OSS.Endpoint
|
||||
out.Region = sec.OSS.Region
|
||||
out.AccessKeyID = sec.OSS.AccessKey
|
||||
out.SecretAccessKey = sec.OSS.SecretKey
|
||||
out.BucketName = sec.OSS.BucketName
|
||||
out.PathPrefix = sec.OSS.PathPrefix
|
||||
}
|
||||
case "ks3":
|
||||
if sec.KS3 != nil {
|
||||
out.Endpoint = sec.KS3.Endpoint
|
||||
out.Region = sec.KS3.Region
|
||||
out.AccessKeyID = sec.KS3.AccessKey
|
||||
out.SecretAccessKey = sec.KS3.SecretKey
|
||||
out.BucketName = sec.KS3.BucketName
|
||||
out.PathPrefix = sec.KS3.PathPrefix
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
Reference in New Issue
Block a user