Files
Enterprise-Knowledge-Base/internal/application/service/retriever/factory.go
T
ochan.kwon 7478e3cddb refactor(retriever): introduce factory for KB-scoped retrieve engine resolution
Extracts the 25 repeated NewCompositeRetrieveEngine call sites across
seven services into two factory functions with tenant-ownership
verification: CreateRetrieveEngineForKB (synchronous) and
CreateRetrieveEngineFromPayload (async task handlers). Promotes
GetByStoreID from concrete-only to interfaces.RetrieveEngineRegistry.
Extends KBDeletePayload / IndexDeletePayload with an omitempty
VectorStoreID snapshot.

Design highlights
- Factory verifies tenant ownership of the resolved store
  (defense-in-depth) so a gap in PR3 validation or a tampered Asynq
  payload cannot cross tenants. Cross-tenant attempts return
  ErrVectorStoreForbidden with a structured log entry.
- Sentinel errors (ErrTenantInfoMissing, ErrVectorStoreNotFound,
  ErrVectorStoreForbidden) let async handlers classify non-retryable
  failures as asynq.SkipRetry.
- nil and empty-string pointers for vectorStoreID normalize to
  "unbound" so callers never send an empty UUID to GetByStoreID.
- resolveBoundEngine constructs engineInfos directly (with
  slices.Clone of Support()) instead of going through
  NewCompositeRetrieveEngine. The latter only reads from the
  byEngineType map and cannot reach DB stores in byStoreID -- multiple
  stores can share the same engine type. Unbound fallbacks still reuse
  NewCompositeRetrieveEngine so the tenant effective-engines path is
  byte-for-byte unchanged.
- DeleteKnowledgeBase now reads the KB before soft-delete so the
  enqueued KBDeletePayload carries a VectorStoreID snapshot; once the
  row is soft-deleted GORM's default scope hides it. Cost: +1 SELECT
  per KB-delete request (a rare admin operation).
- enqueueIndexDeleteTask gains a vectorStoreID *string parameter that
  the single caller (tag.go) populates from the owning KB. The worker
  (ProcessIndexDelete) validates ownership via the factory.
- cleanupKnowledgeResources loads the KB so bound-store cleanup routes
  correctly. If the load fails it warns and falls back to tenant
  effective engines; orphan vectors become observable in logs instead
  of disappearing silently.
- DeleteKnowledges (batch) stays on the tenant effective-engines path
  -- batches can span stores and multi-store fan-out is PR4 scope.
  Noted inline.

Merge safety

All knowledge bases have VectorStoreID = NULL at merge time (introduced
nullable in PR1, Tencent/WeKnora#994). Every factory call falls back to
NewCompositeRetrieveEngine(registry, tenantInfo.GetEffectiveEngines()),
which is the existing code path for these KBs. Async payloads from
before this change decode to VectorStoreID = nil (omitempty) and take
the same fallback.

Testing

- go build ./...
- go vet ./...
- go test -race -count=1 ./internal/application/service/retriever/...
  covers unbound (nil/empty), bound, cross-tenant, unregistered store,
  ownership infra error, legacy payload shapes (missing field/null/
  empty string/nil), tampered payload, and parallel invocation.
- grep -r NewCompositeRetrieveEngine internal/application/service
  returns zero results outside factory.go/composite.go.

Refs: Part of Tencent/WeKnora#993. Depends on Tencent/WeKnora#994.
2026-05-14 15:53:58 +08:00

171 lines
6.7 KiB
Go

package retriever
import (
"context"
"errors"
"slices"
"github.com/Tencent/WeKnora/internal/logger"
"github.com/Tencent/WeKnora/internal/types"
"github.com/Tencent/WeKnora/internal/types/interfaces"
)
// Sentinel errors returned by factory functions. Callers may use errors.Is to
// classify. User-facing responses MUST wrap or replace these with generic
// messages — the sentinels intentionally omit store UUIDs to avoid enumeration
// leaks. Structured logs inside the factory record the tenant/store IDs.
var (
// ErrTenantInfoMissing is returned when the factory needs a tenant from
// context (synchronous, unbound KB path) and none is present.
ErrTenantInfoMissing = errors.New("tenant info not found in context")
// ErrVectorStoreNotFound is returned when the store ID is not registered
// (or an internal lookup for ownership failed). Async workers should treat
// this as non-retryable.
ErrVectorStoreNotFound = errors.New("vector store not available")
// ErrVectorStoreForbidden is returned when the resolved store is not
// owned by the given tenant. This guards against cross-tenant access
// in case the upstream validation layer has a gap. Async workers should
// treat this as non-retryable.
ErrVectorStoreForbidden = errors.New("vector store access denied")
)
// TenantStoreOwnership abstracts the lookup used by factory functions to
// verify that a given vector store ID is owned by the given tenant ID.
//
// Production implementations wrap the VectorStoreRepository; tests inject
// in-memory fakes so they can cover the ownership branches without touching
// a database.
type TenantStoreOwnership interface {
// StoreOwnedBy reports whether the store with the given ID is owned
// by the given tenant. When the store does not exist, it returns
// (false, nil). Errors are reserved for infrastructure failures such as
// database connectivity issues.
StoreOwnedBy(ctx context.Context, storeID string, tenantID uint64) (bool, error)
}
// CreateRetrieveEngineForKB returns a CompositeRetrieveEngine resolved from
// a KB's VectorStore binding.
//
// Resolution rules:
//
// - vectorStoreID == nil || *vectorStoreID == "" →
// falls back to the tenant's effective engines (env-store flow driven
// by RETRIEVE_DRIVER). TenantInfo is read from ctx.
// - otherwise →
// 1) ownership.StoreOwnedBy(*storeID, tenantID) must return true;
// cross-tenant attempts yield ErrVectorStoreForbidden.
// 2) registry.GetByStoreID(*storeID) must succeed;
// unregistered stores yield ErrVectorStoreNotFound.
// 3) the single engine is wrapped by NewCompositeRetrieveEngine so
// that its Support()-based retriever-type matching is preserved.
//
// Use this for 23 synchronous call sites across the application services.
// Async task handlers that cannot rely on ctx-based TenantInfo (currently:
// ProcessKBDeleteTask, ProcessIndexDelete) must use
// CreateRetrieveEngineFromPayload instead.
func CreateRetrieveEngineForKB(
ctx context.Context,
registry interfaces.RetrieveEngineRegistry,
ownership TenantStoreOwnership,
tenantID uint64,
vectorStoreID *string,
) (*CompositeRetrieveEngine, error) {
// Normalize nil and empty-string pointer to "unbound" so that callers
// cannot accidentally route an empty UUID into GetByStoreID.
if vectorStoreID == nil || *vectorStoreID == "" {
tenantInfo, ok := types.TenantInfoFromContext(ctx)
if !ok {
return nil, ErrTenantInfoMissing
}
return NewCompositeRetrieveEngine(registry, tenantInfo.GetEffectiveEngines())
}
return resolveBoundEngine(ctx, registry, ownership, tenantID, *vectorStoreID)
}
// CreateRetrieveEngineFromPayload is the async-task variant. It does not
// read TenantInfo from ctx because async handlers do not populate it.
// Instead, tenantID is passed explicitly from the deserialized payload and
// is verified against the store's tenant when vectorStoreID is non-empty.
//
// Tasks enqueued before vectorStoreID was added to the payload decode it as
// nil and transparently fall back to the pre-serialized effectiveEngines
// path — no in-flight task is lost across upgrades.
func CreateRetrieveEngineFromPayload(
ctx context.Context,
registry interfaces.RetrieveEngineRegistry,
ownership TenantStoreOwnership,
tenantID uint64,
effectiveEngines []types.RetrieverEngineParams,
vectorStoreID *string,
) (*CompositeRetrieveEngine, error) {
if vectorStoreID == nil || *vectorStoreID == "" {
return NewCompositeRetrieveEngine(registry, effectiveEngines)
}
return resolveBoundEngine(ctx, registry, ownership, tenantID, *vectorStoreID)
}
// resolveBoundEngine is the shared ownership-verified lookup path used by
// both CreateRetrieveEngineForKB and CreateRetrieveEngineFromPayload. It
// returns sentinel errors so that handlers can classify them (for example,
// async workers convert Forbidden/NotFound into asynq.SkipRetry).
func resolveBoundEngine(
ctx context.Context,
registry interfaces.RetrieveEngineRegistry,
ownership TenantStoreOwnership,
tenantID uint64,
storeID string,
) (*CompositeRetrieveEngine, error) {
owned, err := ownership.StoreOwnedBy(ctx, storeID, tenantID)
if err != nil {
// Infrastructure failure — record the raw error for operators but
// do not leak internals to the caller.
logger.ErrorWithFields(ctx, err, map[string]interface{}{
"tenant_id": tenantID,
"store_id": storeID,
"reason": "ownership lookup failed",
})
return nil, ErrVectorStoreNotFound
}
if !owned {
// Cross-tenant attempt (or the store has been deleted in the
// meantime). Log with WARN so that audits can surface probing.
logger.Warnf(ctx,
"[retriever.factory] cross-tenant store access attempted: tenant=%d store=%s",
tenantID, storeID)
return nil, ErrVectorStoreForbidden
}
svc, err := registry.GetByStoreID(storeID)
if err != nil {
logger.ErrorWithFields(ctx, err, map[string]interface{}{
"tenant_id": tenantID,
"store_id": storeID,
"reason": "store not registered",
})
return nil, ErrVectorStoreNotFound
}
// Build the composite directly from the resolved service.
//
// We cannot delegate to NewCompositeRetrieveEngine here because that
// function resolves engines through registry.GetRetrieveEngineService,
// which reads from the byEngineType map (env stores). DB stores live
// in the byStoreID map and are not reachable via engine type alone —
// multiple stores can share the same engine type.
//
// Semantics: a KB bound to a DB store uses every retriever type that
// store supports. This intentionally overrides the tenant-level
// effective-engines filter, because binding a KB to a specific store
// is an explicit opt-out of tenant-default routing.
return &CompositeRetrieveEngine{
engineInfos: []*engineInfo{{
retrieveEngine: svc,
retrieverType: slices.Clone(svc.Support()),
}},
}, nil
}