feat(retriever): add OpenSearch type prep ahead of Phase 3 driver
Lays type-system groundwork for the upcoming OpenSearch k-NN driver (Phase 3 PR 2, see Tencent/WeKnora#1440), with strict feature-gate: this PR ships only inert constants, schema extensions, and an unreachable normalizer case. No path activates an OpenSearch VectorStore yet — creation continues to fail with "not a valid engine type" until the activation switch lands in Phase 3 PR 3. Changes: - OpenSearchRetrieverEngineType constant ("opensearch") in internal/types/retriever.go. Not added to validEngineTypes / GetVectorStoreTypes / retrieverEngineMapping yet (gated). - ConnectionConfig.InsecureSkipVerify (bool, default false) in internal/types/vectorstore.go, placed inside the // Common section because it is a cross-driver TLS option. Distinct from the Qdrant-specific UseTLS, which enables TLS on gRPC — InsecureSkipVerify only controls verification of an already-TLS HTTPS connection. AES-GCM Value/Scan round-trips the field as plaintext (it travels alongside the encrypted Password / APIKey but is not sensitive itself). - VectorStoreFieldInfo gains four optional fields: Immutable (bool), Min/Max (*float64), Enum ([]string). All omitempty so existing UI schema entries serialize identically. The fields will drive the UI in Phase 3 PR 3 (read-only on Edit + range/enum constraints). - Six new AuditAction constants under vector_store.* and opensearch.* namespaces. Definitions only — emission lands in Phase 3 PR 3. - EngineAwareNormalizer.Normalize is restructured to group engines by the effective score range observed by the normalizer (not the theoretical raw cosine range): Range [-1, 1] (raw cosine, mapped via (score + 1) / 2): - Milvus (COSINE metric mode). Milvus docs explicitly state the COSINE metric range is [-1, 1]. Range [0, 1] (passthrough — already on the target scale): - Elasticsearch v8 / ElasticFaiss. The driver issues a cosineSimilarity(...) script_score script, and Lucene rejects negative final scores ("Final relevance scores from the script_score query cannot be negative" — ES docs); the effective range observed by the normalizer is therefore [0, 1] for IR-normalized embeddings. ES was previously grouped with Milvus and over-corrected via (score + 1) / 2, which inflated every ES result by 50% in mixed-engine RRF fusion. - OpenSearch. The k-NN plugin's SpaceType.COSINESIMIL.scoreTranslation maps the underlying Lucene/Faiss distance (1 - cosine) to (1 + cosine) / 2 ∈ [0, 1] before the score reaches us. Source: github.com/opensearch-project/k-NN at src/main/java/org/opensearch/knn/index/SpaceType.java (COSINESIMIL enum, scoreTranslation method). - Weaviate. The driver requests `certainty`, defined by Weaviate as (2 - distance) / 2 = (1 + cosine) / 2, intrinsically [0, 1]. - Postgres pgvector / SQLite sqlite-vec / Qdrant / TencentVectorDB / Doris. The driver computes (1 - cosine_distance) or normalized inner_product, whose theoretical range is [-1, 1]. The IR-normalized positive-component unit vectors WeKnora targets (BGE / OpenAI text-embedding-3 / Cohere / sentence-transformers) keep the observed range in [0, 1]; negative-cosine embedding models would silently clamp to 0 downstream — explicitly documented as the IR-normalization caveat in the struct godoc. Dead enum references (ElasticFaiss, Infinity) are flagged in the godoc with a pointer to internal/types/vectorstore.go's existing "legacy/experimental, no standalone deployable instance" annotation. Their case labels are kept for switch exhaustiveness. Test coverage: - OpenSearch constant wire value + collision check against existing 10 engines + gated invariant (NOT in validEngineTypes). - ConnectionConfig.InsecureSkipVerify backward-compat (missing JSON field deserializes as false) + round-trip + AES-GCM coexistence with encrypted Password / APIKey. - VectorStoreFieldInfo omitempty preservation + new-field serialization + *float64 pointer distinction (min=0 vs nil). - AuditAction dot-namespace convention enforcement, prefix invariants for vector_store.* and opensearch.*, no wire-string collisions, exact wire-value pins for the six new constants. - EngineAwareNormalizer: - CosineRange retains its (score + 1) / 2 coverage but now only for Milvus. - UnitInterval now covers the full passthrough group (ES / ElasticFaiss / OpenSearch / Weaviate / Postgres / SQLite / Qdrant / Infinity / TencentVectorDB / Doris). - New TestEngineAwareNormalizer_ElasticsearchCosinePassthrough is an explicit regression guard for the score-range correction: cos=0.5 maps to 0.5 (not (0.5 + 1) / 2 = 0.75 as an earlier draft assumed). - OpenSearch passthrough across (0 / 0.5 / 0.75 / 1) + engine drift (1.0001) + defensive negative + ±Inf / NaN edges + keyword passthrough + nil-ctx safety. Backward compatibility is preserved at every layer: - All new struct fields are omitempty / pointer-tagged so existing rows and existing wire formats remain unchanged. - Normalizer's new OpenSearch case is unreachable until the driver lands. The ES regrouping changes the post-normalization value for every ES vector search result (a correctness fix, not a feature) — ES vector retrieval is currently the only production path affected. - AuditAction constants emit no audit_log rows in this PR. - engine_type=opensearch VectorStore creation still rejected (gated).
This commit is contained in:
@@ -31,14 +31,73 @@ type ScoreNormalizer interface {
|
||||
// are semantically comparable across engines.
|
||||
//
|
||||
// Source formulas (verified against repository implementations on
|
||||
// upstream/main 3214e3d9):
|
||||
// upstream/main 3214e3d9, OpenSearch 2.17 / 3.5 docs, the k-NN plugin's
|
||||
// SpaceType.java source, the Tencent VectorDB Go SDK base_document.go,
|
||||
// pgvector / sqlite-vec / Qdrant / Weaviate docs, and the Lucene
|
||||
// script_score non-negative invariant). Engines are grouped by the
|
||||
// effective score range observed at the normalizer's input:
|
||||
//
|
||||
// - Elasticsearch v8 / ElasticFaiss / Milvus (cosine mode):
|
||||
// raw cosine in [-1, 1]
|
||||
// - Postgres pgvector: (1 - cosine_distance) in [0, 1]
|
||||
// - SQLite sqlite-vec: (1 - cosine_distance) in [0, 1]
|
||||
// - Weaviate: raw score in [0, 1]
|
||||
// - Qdrant / Infinity / TencentVectorDB / Doris: raw score in [0, 1]
|
||||
// Range [-1, 1] (raw cosine, normalized via (score + 1) / 2):
|
||||
// - Milvus (COSINE metric mode). Milvus docs explicitly state the
|
||||
// COSINE metric "has a range of [-1, 1]".
|
||||
//
|
||||
// Range [0, 1] (passthrough — already on the target scale):
|
||||
// - Elasticsearch v8 — driver issues `cosineSimilarity(query, 'embedding')`
|
||||
// as a script_score script. Lucene rejects negative final scores
|
||||
// ("Final relevance scores from the script_score query cannot be
|
||||
// negative") so the effective range observed here is [0, 1] for
|
||||
// IR-normalized embeddings (see IR-normalization note below). If
|
||||
// a future driver swap inserts `1.0 + cosineSimilarity(...)` (the
|
||||
// pattern recommended by ES docs) this group will need to move out.
|
||||
// - OpenSearch — the k-NN plugin's SpaceType.COSINESIMIL.scoreTranslation
|
||||
// applies Math.max((2.0F - rawScore) / 2.0F, 0.0F) where rawScore is
|
||||
// the distance (1 - cosine) from Lucene/Faiss, so the score returned
|
||||
// to the client is (1 + cosine) / 2 ∈ [0, 1]. The OpenSearch driver
|
||||
// itself ships in a later Phase 3 PR (see #1440); this branch is
|
||||
// unreachable from production retrieval paths until then, but the
|
||||
// unit tests pin the formula in advance. Source:
|
||||
// github.com/opensearch-project/k-NN at
|
||||
// src/main/java/org/opensearch/knn/index/SpaceType.java (COSINESIMIL
|
||||
// enum, scoreTranslation method).
|
||||
// - Weaviate — driver requests `certainty`, defined by Weaviate as
|
||||
// (2 - distance) / 2 = (1 + cosine) / 2, intrinsically ∈ [0, 1].
|
||||
// - Postgres pgvector — driver computes `(1 - distance) as score`
|
||||
// where `<=>` is cosine distance ∈ [0, 2]; the theoretical range
|
||||
// is therefore [-1, 1] but IR-normalized embeddings (see below)
|
||||
// keep the observed range in [0, 1].
|
||||
// - SQLite sqlite-vec — driver computes `1 - v.Distance` where the
|
||||
// vec0 `distance_metric=cosine` returns distance ∈ [0, 2] (per
|
||||
// sqlite-vec docs: "0 for identical, 2 for opposite"). Same IR
|
||||
// caveat as pgvector.
|
||||
// - Qdrant — Distance.Cosine; Qdrant normalizes vectors at insert
|
||||
// time and the score returned by ScoredPoint.score is the dot
|
||||
// product of the normalized vectors = raw cosine ∈ [-1, 1] in
|
||||
// theory, [0, 1] under the IR caveat.
|
||||
// - TencentVectorDB — MetricType COSINE; per the SDK godoc
|
||||
// (tcvectordb base_document.go): "COSINE: return when score >=
|
||||
// radius, value range [-1, 1]." Same IR caveat.
|
||||
// - Doris — driver runs `inner_product_approximate` against
|
||||
// L2-normalized embeddings (or legacy `(1 - cosine_distance_approximate)`),
|
||||
// which equals raw cosine ∈ [-1, 1]. Same IR caveat.
|
||||
//
|
||||
// IR-normalization caveat (applies to pgvector, sqlite-vec, Qdrant,
|
||||
// TencentVectorDB, Doris): modern RAG embeddings are L2-normalized
|
||||
// positive-component unit vectors (sentence-transformers, BGE, OpenAI
|
||||
// text-embedding-3, Cohere, E5, etc.) that empirically keep cosine in
|
||||
// [0, 1]. Theoretical [-1, 1] inputs would clamp to 0 below — silent
|
||||
// information loss only for embedding models that produce negative
|
||||
// cosines (rare in IR workloads but possible for centered word2vec,
|
||||
// GloVe, or non-IR embeddings). If a future use case violates this,
|
||||
// the affected engines should move into the [-1, 1] group.
|
||||
//
|
||||
// Dead enum references (kept as RetrieverEngineType constants but
|
||||
// without a driver implementation — see internal/types/vectorstore.go
|
||||
// near the GetVectorStoreTypes definition, where they are flagged
|
||||
// "legacy/experimental, no standalone deployable instance"):
|
||||
// - InfinityRetrieverEngineType
|
||||
// - ElasticFaissRetrieverEngineType
|
||||
// Their case labels below route to clamp01(score) defensively, but
|
||||
// production code never returns these engine types.
|
||||
//
|
||||
// Unknown engines clamp to [0, 1]; the fan-out caller emits a single WARN
|
||||
// per request via warnIfUnknownEngine so Normalize itself stays lock-free
|
||||
@@ -62,21 +121,35 @@ func (EngineAwareNormalizer) Normalize(
|
||||
}
|
||||
|
||||
switch engineType {
|
||||
case types.MilvusRetrieverEngineType:
|
||||
// Raw cosine in [-1, 1] → [0, 1]. Milvus is the only engine in
|
||||
// this codebase whose driver surfaces the raw cosine signed range
|
||||
// to the normalizer. Pass through clamp01 once more so that a
|
||||
// misbehaving engine returning 1.0000002 does not leak past the
|
||||
// [0, 1] envelope (caller sorts by score afterwards).
|
||||
return clamp01((score + 1) / 2)
|
||||
case types.ElasticsearchRetrieverEngineType,
|
||||
types.ElasticFaissRetrieverEngineType,
|
||||
types.MilvusRetrieverEngineType:
|
||||
// Raw cosine in [-1, 1] → [0, 1]. Pass through clamp01 once more so
|
||||
// that a misbehaving engine returning 1.0000002 does not leak past
|
||||
// the [0, 1] envelope (caller sorts by score afterwards).
|
||||
return clamp01((score + 1) / 2)
|
||||
case types.PostgresRetrieverEngineType,
|
||||
types.QdrantRetrieverEngineType,
|
||||
types.OpenSearchRetrieverEngineType,
|
||||
types.WeaviateRetrieverEngineType,
|
||||
types.PostgresRetrieverEngineType,
|
||||
types.SQLiteRetrieverEngineType,
|
||||
types.QdrantRetrieverEngineType,
|
||||
types.InfinityRetrieverEngineType,
|
||||
types.TencentVectorDBRetrieverEngineType,
|
||||
types.DorisRetrieverEngineType:
|
||||
// Already in [0, 1] by repository contract.
|
||||
// Already in [0, 1] when the value reaches us. See struct godoc
|
||||
// above for the per-engine derivation: Elasticsearch's Lucene
|
||||
// script_score non-negative invariant; OpenSearch's k-NN plugin
|
||||
// SpaceType.COSINESIMIL.scoreTranslation pre-translation;
|
||||
// Weaviate's certainty intrinsic; and the IR-normalization
|
||||
// caveat covering pgvector / sqlite-vec / Qdrant /
|
||||
// TencentVectorDB / Doris (theoretical [-1, 1] but observed
|
||||
// [0, 1] for L2-normalized positive-component IR embeddings).
|
||||
//
|
||||
// ElasticFaiss and Infinity are dead enum references — their
|
||||
// case labels are kept so the switch is exhaustive over the
|
||||
// declared engine constants, but no driver returns these.
|
||||
return clamp01(score)
|
||||
default:
|
||||
// Unknown engine. Clamp defensively; the caller emits WARN with ctx.
|
||||
|
||||
@@ -29,6 +29,14 @@ func TestEngineAwareNormalizer_KeywordPassthrough(t *testing.T) {
|
||||
func TestEngineAwareNormalizer_CosineRange(t *testing.T) {
|
||||
t.Parallel()
|
||||
n := EngineAwareNormalizer{}
|
||||
// Only Milvus surfaces the raw cosine signed range [-1, 1] to the
|
||||
// normalizer in this codebase. Elasticsearch v8 used to be in this
|
||||
// group, but the driver issues a script_score cosineSimilarity script
|
||||
// and Lucene rejects negative final scores ("Final relevance scores
|
||||
// from the script_score query cannot be negative" — ES docs), so the
|
||||
// score observed by the normalizer is already in [0, 1]; ES is now
|
||||
// part of the passthrough group below. ElasticFaiss is a dead enum
|
||||
// (no driver) and follows ES.
|
||||
cases := []struct {
|
||||
score float64
|
||||
want float64
|
||||
@@ -43,8 +51,6 @@ func TestEngineAwareNormalizer_CosineRange(t *testing.T) {
|
||||
{1.5, 1.0},
|
||||
}
|
||||
for _, engine := range []types.RetrieverEngineType{
|
||||
types.ElasticsearchRetrieverEngineType,
|
||||
types.ElasticFaissRetrieverEngineType,
|
||||
types.MilvusRetrieverEngineType,
|
||||
} {
|
||||
for _, tc := range cases {
|
||||
@@ -61,6 +67,22 @@ func TestEngineAwareNormalizer_CosineRange(t *testing.T) {
|
||||
func TestEngineAwareNormalizer_UnitInterval(t *testing.T) {
|
||||
t.Parallel()
|
||||
n := EngineAwareNormalizer{}
|
||||
// All engines whose effective score arriving at the normalizer is
|
||||
// already in [0, 1]:
|
||||
// - Elasticsearch v8 / ElasticFaiss — Lucene script_score's
|
||||
// non-negative invariant truncates the theoretical [-1, 1]
|
||||
// cosineSimilarity output to [0, 1].
|
||||
// - OpenSearch — the k-NN plugin's
|
||||
// SpaceType.COSINESIMIL.scoreTranslation pre-maps to (1 + cos)/2.
|
||||
// - Weaviate — driver requests certainty, intrinsically in [0, 1].
|
||||
// - Postgres pgvector / SQLite sqlite-vec / Qdrant /
|
||||
// TencentVectorDB / Doris — theoretically [-1, 1] but the
|
||||
// IR-normalized embeddings WeKnora targets (BGE / OpenAI /
|
||||
// Cohere / sentence-transformers) keep the observed range
|
||||
// in [0, 1]; the silent floor at clamp01(<0) → 0 is the
|
||||
// defense for embedding models that violate that assumption.
|
||||
// - Infinity — dead enum reference; case label kept for switch
|
||||
// exhaustiveness, never reached in production.
|
||||
cases := []struct {
|
||||
score float64
|
||||
want float64
|
||||
@@ -73,10 +95,13 @@ func TestEngineAwareNormalizer_UnitInterval(t *testing.T) {
|
||||
{1.5, 1},
|
||||
}
|
||||
for _, engine := range []types.RetrieverEngineType{
|
||||
types.PostgresRetrieverEngineType,
|
||||
types.QdrantRetrieverEngineType,
|
||||
types.ElasticsearchRetrieverEngineType,
|
||||
types.ElasticFaissRetrieverEngineType,
|
||||
types.OpenSearchRetrieverEngineType,
|
||||
types.WeaviateRetrieverEngineType,
|
||||
types.PostgresRetrieverEngineType,
|
||||
types.SQLiteRetrieverEngineType,
|
||||
types.QdrantRetrieverEngineType,
|
||||
types.InfinityRetrieverEngineType,
|
||||
types.TencentVectorDBRetrieverEngineType,
|
||||
types.DorisRetrieverEngineType,
|
||||
@@ -92,6 +117,125 @@ func TestEngineAwareNormalizer_UnitInterval(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
// TestEngineAwareNormalizer_ElasticsearchCosinePassthrough is an explicit
|
||||
// regression guard for the score-range correction landed in this PR. ES's
|
||||
// driver returns the raw output of the `cosineSimilarity(...)` script_score
|
||||
// script. Lucene rejects negative final scores (per the ES documentation:
|
||||
// "Final relevance scores from the script_score query cannot be negative.
|
||||
// To support certain search optimizations, Lucene requires scores be
|
||||
// positive or 0"), so for IR-normalized embeddings the effective range
|
||||
// observed at the normalizer is already in [0, 1]. ES therefore belongs
|
||||
// in the passthrough group and the cosine=0.5 case must map to 0.5 (not
|
||||
// (0.5 + 1) / 2 = 0.75 as an earlier draft assumed).
|
||||
func TestEngineAwareNormalizer_ElasticsearchCosinePassthrough(t *testing.T) {
|
||||
t.Parallel()
|
||||
n := EngineAwareNormalizer{}
|
||||
cases := []struct {
|
||||
name string
|
||||
score float64
|
||||
want float64
|
||||
}{
|
||||
{"cos=0 orthogonal IR-clamped to 0", 0, 0},
|
||||
{"cos=0.25 IR-typical low passes through", 0.25, 0.25},
|
||||
{"cos=0.5 IR-typical mid passes through (not 0.75)", 0.5, 0.5},
|
||||
{"cos=0.75 IR-typical high passes through", 0.75, 0.75},
|
||||
{"cos=1 identical passes through as 1", 1, 1},
|
||||
// Lucene's non-negative invariant means the normalizer should
|
||||
// not receive <0 from a sane driver, but the clamp guards the
|
||||
// downstream sort comparator either way.
|
||||
{"negative defensive clamps to 0", -0.5, 0},
|
||||
// Engine drift past 1 clamps to 1.
|
||||
{"drift past 1 clamps to 1", 1.0001, 1},
|
||||
// Float-edge cases handled by clamp01.
|
||||
{"+Inf clamps to 1", math.Inf(1), 1},
|
||||
{"-Inf clamps to 0", math.Inf(-1), 0},
|
||||
{"NaN clamps to 0", math.NaN(), 0},
|
||||
}
|
||||
for _, tc := range cases {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
got := n.Normalize(context.Background(), tc.score,
|
||||
types.VectorRetrieverType, types.ElasticsearchRetrieverEngineType)
|
||||
if math.Abs(got-tc.want) > 1e-9 {
|
||||
t.Fatalf("elasticsearch score=%v: want %v, got %v",
|
||||
tc.score, tc.want, got)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestEngineAwareNormalizer_OpenSearchCosinePassthrough(t *testing.T) {
|
||||
t.Parallel()
|
||||
n := EngineAwareNormalizer{}
|
||||
// OpenSearch k-NN cosinesimil scores arrive already mapped to [0, 1]
|
||||
// by the plugin's SpaceType.COSINESIMIL.scoreTranslation function
|
||||
// (rawScore = 1 - cosine; output = max((2 - rawScore) / 2, 0) =
|
||||
// (1 + cosine) / 2). The normalizer therefore passes the value through
|
||||
// (clamping only to guard against engine drift and NaN/Inf inputs that
|
||||
// could otherwise break the slices.SortFunc strict-weak-ordering
|
||||
// invariant downstream).
|
||||
// Subtests run serially (no inner t.Parallel) — consistent with
|
||||
// the other table-driven tests in this file.
|
||||
cases := []struct {
|
||||
name string
|
||||
score float64
|
||||
want float64
|
||||
}{
|
||||
// Basic mapping (k-NN-translated _score in [0, 1])
|
||||
{"score 0 (cos=-1) passes through as 0", 0, 0},
|
||||
{"score 0.5 (cos=0, orthogonal) passes through", 0.5, 0.5},
|
||||
{"score 0.75 (cos=0.5, IR-typical mid) passes through", 0.75, 0.75},
|
||||
{"score 1 (cos=1, identical) passes through as 1", 1, 1},
|
||||
// Engine drift past [0, 1] envelope clamps.
|
||||
{"score 1.0001 clamps to 1", 1.0001, 1},
|
||||
// Negative defensive: cosinesimil's translation has a max(., 0)
|
||||
// floor so <0 should never reach us, but clamp guards the
|
||||
// comparator-ordering invariant downstream.
|
||||
{"score -0.5 clamps to 0", -0.5, 0},
|
||||
// Float-edge cases handled by clamp01.
|
||||
{"+Inf clamps to 1", math.Inf(1), 1},
|
||||
{"-Inf clamps to 0", math.Inf(-1), 0},
|
||||
{"NaN clamps to 0", math.NaN(), 0},
|
||||
}
|
||||
for _, tc := range cases {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
got := n.Normalize(context.Background(), tc.score,
|
||||
types.VectorRetrieverType, types.OpenSearchRetrieverEngineType)
|
||||
if math.Abs(got-tc.want) > 1e-9 {
|
||||
t.Fatalf("opensearch score=%v: want %v, got %v",
|
||||
tc.score, tc.want, got)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestEngineAwareNormalizer_OpenSearchKeywordPassthrough(t *testing.T) {
|
||||
t.Parallel()
|
||||
n := EngineAwareNormalizer{}
|
||||
// BM25 keyword scores on OpenSearch are unbounded positive raw
|
||||
// values — they MUST pass through unchanged so the service-layer
|
||||
// RRF rank-based fusion handles the scale-mixed combination.
|
||||
got := n.Normalize(context.Background(), 12.7,
|
||||
types.KeywordsRetrieverType, types.OpenSearchRetrieverEngineType)
|
||||
if got != 12.7 {
|
||||
t.Fatalf("opensearch keyword passthrough: want 12.7, got %v", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestEngineAwareNormalizer_OpenSearchNilCtx_DoesNotPanic(t *testing.T) {
|
||||
// nil ctx must not panic. The OpenSearch case is IO-free by
|
||||
// contract — it never reads from ctx and never logs.
|
||||
t.Parallel()
|
||||
defer func() {
|
||||
if r := recover(); r != nil {
|
||||
t.Fatalf("OpenSearch Normalize panicked on nil ctx: %v", r)
|
||||
}
|
||||
}()
|
||||
_ = EngineAwareNormalizer{}.Normalize(
|
||||
nil, 1.0, types.VectorRetrieverType,
|
||||
types.OpenSearchRetrieverEngineType,
|
||||
)
|
||||
}
|
||||
|
||||
func TestEngineAwareNormalizer_Unknown_ClampsAndDoesNotPanic(t *testing.T) {
|
||||
t.Parallel()
|
||||
n := EngineAwareNormalizer{}
|
||||
|
||||
@@ -50,6 +50,45 @@ const (
|
||||
// AuditActionInvitationExpired fires when the lazy sweep transitions
|
||||
// an overdue pending row to expired. Actor is empty (system).
|
||||
AuditActionInvitationExpired AuditAction = "rbac.invitation_expired"
|
||||
|
||||
// VectorStore lifecycle actions. Emitted by VectorStoreService.
|
||||
// Cover both env-store-derived (__env_*) and DB store create /
|
||||
// update / delete paths. Details payload identifies the store_id
|
||||
// and the changed key set; secret values (Password, APIKey,
|
||||
// connection_config encrypted blob) MUST NOT appear in details.
|
||||
|
||||
// AuditActionVectorStoreCreated fires when a new VectorStore row
|
||||
// is committed to the DB (Phase 1 CRUD path). Actor is the tenant
|
||||
// user; resource is the VectorStore.
|
||||
AuditActionVectorStoreCreated AuditAction = "vector_store.created"
|
||||
// AuditActionVectorStoreUpdated fires on UPDATE of any VectorStore
|
||||
// mutable field. Details payload carries the changed key set
|
||||
// (never the secret values themselves — only the field names).
|
||||
AuditActionVectorStoreUpdated AuditAction = "vector_store.updated"
|
||||
// AuditActionVectorStoreDeleted fires when a VectorStore is
|
||||
// (soft-)deleted. Phase 2's delete guard already prevents deletion
|
||||
// of stores with bound KBs; the audit row records the actor and
|
||||
// the store_id for forensic traceability.
|
||||
AuditActionVectorStoreDeleted AuditAction = "vector_store.deleted"
|
||||
|
||||
// OpenSearch-specific actions emitted by the driver shipped in
|
||||
// Phase 3. The OpenSearch index is a derived resource of the
|
||||
// VectorStore — these events capture cluster-side side effects
|
||||
// (PUT /<index>, DELETE /<index>, POST /_reindex) that operators
|
||||
// may need to correlate with VectorStore lifecycle events.
|
||||
|
||||
// AuditActionOpenSearchIndexCreated fires when the OpenSearch
|
||||
// driver lazily creates a per-dimension index (the first time a
|
||||
// KB with a given embedding dim binds to the store). Details
|
||||
// payload: index name, alias name, dimension.
|
||||
AuditActionOpenSearchIndexCreated AuditAction = "opensearch.index_created"
|
||||
// AuditActionOpenSearchIndexDeleted fires when the OpenSearch
|
||||
// driver drops an index (e.g. cascade from VectorStore delete).
|
||||
AuditActionOpenSearchIndexDeleted AuditAction = "opensearch.index_deleted"
|
||||
// AuditActionOpenSearchReindexExecuted fires when CopyIndices
|
||||
// initiates a _reindex (sync or async). Details payload: source
|
||||
// KB id, target KB id, sync-or-async, doc count if known.
|
||||
AuditActionOpenSearchReindexExecuted AuditAction = "opensearch.reindex_executed"
|
||||
)
|
||||
|
||||
// AuditOutcome distinguishes successful mutations from middleware-level
|
||||
|
||||
@@ -0,0 +1,135 @@
|
||||
package types
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/stretchr/testify/assert"
|
||||
)
|
||||
|
||||
// TestAuditAction_DotNamespaceConvention ensures every AuditAction
|
||||
// constant follows the dot-namespaced `<area>.<event>` convention
|
||||
// documented at the top of audit_log.go. Future PRs that add actions
|
||||
// must keep this invariant so audit-log consumers can prefix-filter
|
||||
// by area.
|
||||
func TestAuditAction_DotNamespaceConvention(t *testing.T) {
|
||||
all := []AuditAction{
|
||||
// RBAC namespace (Phase 2 PR 5 / #1427)
|
||||
AuditActionMemberAdded,
|
||||
AuditActionMemberRemoved,
|
||||
AuditActionMemberRoleChanged,
|
||||
AuditActionMemberLeft,
|
||||
AuditActionAccessDenied,
|
||||
AuditActionInvitationSent,
|
||||
AuditActionInvitationAccepted,
|
||||
AuditActionInvitationDeclined,
|
||||
AuditActionInvitationRevoked,
|
||||
AuditActionInvitationExpired,
|
||||
// VectorStore namespace (Phase 3 PR 1 / #1440)
|
||||
AuditActionVectorStoreCreated,
|
||||
AuditActionVectorStoreUpdated,
|
||||
AuditActionVectorStoreDeleted,
|
||||
// OpenSearch namespace (Phase 3 PR 1 / #1440)
|
||||
AuditActionOpenSearchIndexCreated,
|
||||
AuditActionOpenSearchIndexDeleted,
|
||||
AuditActionOpenSearchReindexExecuted,
|
||||
}
|
||||
for _, a := range all {
|
||||
s := string(a)
|
||||
area, event, ok := strings.Cut(s, ".")
|
||||
assert.True(t, ok,
|
||||
"action %q must contain exactly one dot separator", s)
|
||||
assert.NotEmpty(t, area, "action %q has empty area", s)
|
||||
assert.NotEmpty(t, event, "action %q has empty event", s)
|
||||
}
|
||||
}
|
||||
|
||||
// TestAuditAction_VectorStoreNamespacePrefix pins the three
|
||||
// vector_store.* actions added in Phase 3 PR 1 to their shared area
|
||||
// prefix. The prefix is the contract by which an audit-log reader
|
||||
// can subscribe to "all VectorStore lifecycle events" with a single
|
||||
// filter.
|
||||
func TestAuditAction_VectorStoreNamespacePrefix(t *testing.T) {
|
||||
cases := []AuditAction{
|
||||
AuditActionVectorStoreCreated,
|
||||
AuditActionVectorStoreUpdated,
|
||||
AuditActionVectorStoreDeleted,
|
||||
}
|
||||
for _, a := range cases {
|
||||
assert.True(t,
|
||||
strings.HasPrefix(string(a), "vector_store."),
|
||||
"expected %q to start with 'vector_store.'", a,
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
// TestAuditAction_OpenSearchNamespacePrefix pins the three
|
||||
// opensearch.* actions added in Phase 3 PR 1 to their shared area
|
||||
// prefix. OpenSearch index lifecycle is a driver-specific concern
|
||||
// distinct from VectorStore lifecycle — both can co-occur for the
|
||||
// same logical operation (e.g. CreateVectorStore + IndexCreated).
|
||||
func TestAuditAction_OpenSearchNamespacePrefix(t *testing.T) {
|
||||
cases := []AuditAction{
|
||||
AuditActionOpenSearchIndexCreated,
|
||||
AuditActionOpenSearchIndexDeleted,
|
||||
AuditActionOpenSearchReindexExecuted,
|
||||
}
|
||||
for _, a := range cases {
|
||||
assert.True(t,
|
||||
strings.HasPrefix(string(a), "opensearch."),
|
||||
"expected %q to start with 'opensearch.'", a,
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
// TestAuditAction_NoCollisionsAcrossNamespaces ensures no two
|
||||
// AuditAction constants share the same wire string. A duplicate would
|
||||
// silently merge two logical events into one entry in the audit-log
|
||||
// UI, defeating the dot-namespace convention.
|
||||
func TestAuditAction_NoCollisionsAcrossNamespaces(t *testing.T) {
|
||||
seen := make(map[AuditAction]string)
|
||||
register := func(name string, a AuditAction) {
|
||||
t.Helper()
|
||||
if prev, exists := seen[a]; exists {
|
||||
t.Fatalf("collision: %s and %s both map to %q", prev, name, a)
|
||||
}
|
||||
seen[a] = name
|
||||
}
|
||||
register("AuditActionMemberAdded", AuditActionMemberAdded)
|
||||
register("AuditActionMemberRemoved", AuditActionMemberRemoved)
|
||||
register("AuditActionMemberRoleChanged", AuditActionMemberRoleChanged)
|
||||
register("AuditActionMemberLeft", AuditActionMemberLeft)
|
||||
register("AuditActionAccessDenied", AuditActionAccessDenied)
|
||||
register("AuditActionInvitationSent", AuditActionInvitationSent)
|
||||
register("AuditActionInvitationAccepted", AuditActionInvitationAccepted)
|
||||
register("AuditActionInvitationDeclined", AuditActionInvitationDeclined)
|
||||
register("AuditActionInvitationRevoked", AuditActionInvitationRevoked)
|
||||
register("AuditActionInvitationExpired", AuditActionInvitationExpired)
|
||||
register("AuditActionVectorStoreCreated", AuditActionVectorStoreCreated)
|
||||
register("AuditActionVectorStoreUpdated", AuditActionVectorStoreUpdated)
|
||||
register("AuditActionVectorStoreDeleted", AuditActionVectorStoreDeleted)
|
||||
register("AuditActionOpenSearchIndexCreated", AuditActionOpenSearchIndexCreated)
|
||||
register("AuditActionOpenSearchIndexDeleted", AuditActionOpenSearchIndexDeleted)
|
||||
register("AuditActionOpenSearchReindexExecuted", AuditActionOpenSearchReindexExecuted)
|
||||
}
|
||||
|
||||
// TestAuditAction_Phase3WireValues pins the exact wire strings for
|
||||
// the six new Phase 3 actions. The wire strings are the public
|
||||
// contract for audit-log consumers; changing them is a breaking
|
||||
// change.
|
||||
func TestAuditAction_Phase3WireValues(t *testing.T) {
|
||||
cases := []struct {
|
||||
constant AuditAction
|
||||
wire string
|
||||
}{
|
||||
{AuditActionVectorStoreCreated, "vector_store.created"},
|
||||
{AuditActionVectorStoreUpdated, "vector_store.updated"},
|
||||
{AuditActionVectorStoreDeleted, "vector_store.deleted"},
|
||||
{AuditActionOpenSearchIndexCreated, "opensearch.index_created"},
|
||||
{AuditActionOpenSearchIndexDeleted, "opensearch.index_deleted"},
|
||||
{AuditActionOpenSearchReindexExecuted, "opensearch.reindex_executed"},
|
||||
}
|
||||
for _, c := range cases {
|
||||
assert.Equal(t, c.wire, string(c.constant))
|
||||
}
|
||||
}
|
||||
@@ -15,6 +15,15 @@ const (
|
||||
DorisRetrieverEngineType RetrieverEngineType = "doris"
|
||||
SQLiteRetrieverEngineType RetrieverEngineType = "sqlite"
|
||||
TencentVectorDBRetrieverEngineType RetrieverEngineType = "tencent_vectordb"
|
||||
// OpenSearchRetrieverEngineType identifies the OpenSearch k-NN driver
|
||||
// introduced in Phase 3 (see issue tracker). The driver itself ships
|
||||
// in a subsequent PR; this constant exists so that
|
||||
// EngineAwareNormalizer can register the OpenSearch case ahead of the
|
||||
// driver landing and the AuditAction constants can reference it
|
||||
// without forward dependencies. Wire value "opensearch" mirrors the
|
||||
// official product name and matches the value used in
|
||||
// retrieverEngineMapping / GetVectorStoreTypes once activation lands.
|
||||
OpenSearchRetrieverEngineType RetrieverEngineType = "opensearch"
|
||||
)
|
||||
|
||||
// RetrieverType represents the type of retriever
|
||||
|
||||
@@ -119,6 +119,16 @@ type ConnectionConfig struct {
|
||||
Username string `yaml:"username" json:"username,omitempty"`
|
||||
Password string `yaml:"password" json:"password,omitempty"` // AES-GCM encrypted
|
||||
APIKey string `yaml:"api_key" json:"api_key,omitempty"` // AES-GCM encrypted
|
||||
// InsecureSkipVerify disables TLS certificate verification when
|
||||
// talking to the backing store over HTTPS. Defaults to false
|
||||
// (secure). Set to true ONLY for self-signed development clusters;
|
||||
// production deployments should provide trusted certificates via
|
||||
// the system CA pool. Cross-driver applicable but currently only
|
||||
// the OpenSearch driver (Phase 3) reads this field. Note: this
|
||||
// differs from the Qdrant-specific UseTLS below, which *enables*
|
||||
// TLS on gRPC connections — InsecureSkipVerify only controls
|
||||
// *verification* of an already-TLS connection.
|
||||
InsecureSkipVerify bool `yaml:"insecure_skip_verify" json:"insecure_skip_verify,omitempty"`
|
||||
// Qdrant
|
||||
Host string `yaml:"host" json:"host,omitempty"`
|
||||
Port int `yaml:"port" json:"port,omitempty"`
|
||||
@@ -590,7 +600,11 @@ type VectorStoreTypeInfo struct {
|
||||
IndexFields []VectorStoreFieldInfo `json:"index_fields,omitempty"`
|
||||
}
|
||||
|
||||
// VectorStoreFieldInfo describes a single configuration field.
|
||||
// VectorStoreFieldInfo describes a single configuration field exposed
|
||||
// by /api/v1/vector-stores/types for the registration UI. The optional
|
||||
// validation hints (`Immutable`, `Min`, `Max`, `Enum`) are used by both
|
||||
// the frontend (to disable / constrain inputs) and the backend
|
||||
// (defense-in-depth validation in the service layer).
|
||||
type VectorStoreFieldInfo struct {
|
||||
Name string `json:"name"`
|
||||
Type string `json:"type"` // "string", "number", "boolean"
|
||||
@@ -598,6 +612,24 @@ type VectorStoreFieldInfo struct {
|
||||
Sensitive bool `json:"sensitive,omitempty"`
|
||||
Default any `json:"default,omitempty"`
|
||||
Description string `json:"description,omitempty"`
|
||||
|
||||
// Immutable marks a field whose value cannot be changed after the
|
||||
// VectorStore is first created. The UI shows the input as read-only
|
||||
// in edit mode; the backend rejects modification attempts. Used by
|
||||
// engines whose underlying index structure is fixed at create time
|
||||
// (e.g. OpenSearch's HNSW engine / M / ef_construction).
|
||||
Immutable bool `json:"immutable,omitempty"`
|
||||
|
||||
// Min / Max set inclusive bounds for "number"-typed fields. nil
|
||||
// means no bound on that side. Frontend uses these to constrain
|
||||
// inputs; backend re-validates as defense-in-depth.
|
||||
Min *float64 `json:"min,omitempty"`
|
||||
Max *float64 `json:"max,omitempty"`
|
||||
|
||||
// Enum constrains the allowed string values. Empty means no
|
||||
// constraint. Used by fields whose value space is closed (e.g.
|
||||
// OpenSearch's knn_engine ∈ {"lucene", "faiss"}).
|
||||
Enum []string `json:"enum,omitempty"`
|
||||
}
|
||||
|
||||
// GetVectorStoreTypes returns metadata for all supported engine types.
|
||||
|
||||
@@ -600,6 +600,22 @@ func TestConnectionConfig_MaskSensitiveFields(t *testing.T) {
|
||||
assert.Equal(t, "secret-pass", c.Password)
|
||||
assert.Equal(t, "sk-api-key", c.APIKey)
|
||||
})
|
||||
|
||||
t.Run("preserves insecure_skip_verify as a visible operator-facing knob", func(t *testing.T) {
|
||||
// InsecureSkipVerify is deliberately NOT redacted: operators
|
||||
// must be able to see whether TLS verification is disabled.
|
||||
// This pin prevents a well-meaning future change from quietly
|
||||
// masking the flag and hiding a misconfiguration.
|
||||
c := ConnectionConfig{
|
||||
Addr: "https://os:9200",
|
||||
Password: "secret-pass",
|
||||
InsecureSkipVerify: true,
|
||||
}
|
||||
masked := c.MaskSensitiveFields()
|
||||
assert.Equal(t, RedactedSecretPlaceholder, masked.Password)
|
||||
assert.True(t, masked.InsecureSkipVerify,
|
||||
"InsecureSkipVerify must remain visible after masking")
|
||||
})
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
@@ -1063,10 +1079,10 @@ func TestIndexConfig_ScalabilityFieldsRoundTrip(t *testing.T) {
|
||||
// read-path consistency for the two engines that are only meaningful as env
|
||||
// stores. They must:
|
||||
//
|
||||
// 1. Be rejected by Validate() so POST /vector-stores returns a 4xx
|
||||
// instead of silently persisting a row that has no separation effect.
|
||||
// 2. Be absent from GetVectorStoreTypes() so the UI dropdown doesn't
|
||||
// offer them as a choice.
|
||||
// 1. Be rejected by Validate() so POST /vector-stores returns a 4xx
|
||||
// instead of silently persisting a row that has no separation effect.
|
||||
// 2. Be absent from GetVectorStoreTypes() so the UI dropdown doesn't
|
||||
// offer them as a choice.
|
||||
//
|
||||
// Both checks live here so a future change that re-introduces one path
|
||||
// (e.g., adds Postgres back to validEngineTypes for some niche case)
|
||||
@@ -1105,3 +1121,293 @@ func TestVectorStore_PostgresSqliteNotRegisterable(t *testing.T) {
|
||||
"sqlite must not appear in the UI dropdown — env-store only")
|
||||
})
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Phase 3 PR1 additions: ConnectionConfig.InsecureSkipVerify backward-compat
|
||||
// and VectorStoreFieldInfo schema extensions (Immutable / Min / Max / Enum)
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
// TestConnectionConfig_InsecureSkipVerify_BackwardCompat ensures that
|
||||
// ConnectionConfigs persisted before Phase 3 deserialize cleanly: the
|
||||
// missing JSON field maps to false (Go zero-value). This is the
|
||||
// foundational backward-compat guarantee for the schema extension.
|
||||
func TestConnectionConfig_InsecureSkipVerify_BackwardCompat(t *testing.T) {
|
||||
t.Run("missing field defaults to false", func(t *testing.T) {
|
||||
legacy := `{"addr":"https://es:9200","username":"u","password":"p"}`
|
||||
var cfg ConnectionConfig
|
||||
require.NoError(t, json.Unmarshal([]byte(legacy), &cfg))
|
||||
assert.False(t, cfg.InsecureSkipVerify)
|
||||
})
|
||||
|
||||
t.Run("explicit false deserializes correctly", func(t *testing.T) {
|
||||
raw := `{"addr":"https://es:9200","insecure_skip_verify":false}`
|
||||
var cfg ConnectionConfig
|
||||
require.NoError(t, json.Unmarshal([]byte(raw), &cfg))
|
||||
assert.False(t, cfg.InsecureSkipVerify)
|
||||
})
|
||||
|
||||
t.Run("explicit true deserializes correctly", func(t *testing.T) {
|
||||
raw := `{"addr":"https://os:9200","insecure_skip_verify":true}`
|
||||
var cfg ConnectionConfig
|
||||
require.NoError(t, json.Unmarshal([]byte(raw), &cfg))
|
||||
assert.True(t, cfg.InsecureSkipVerify)
|
||||
})
|
||||
}
|
||||
|
||||
// TestConnectionConfig_InsecureSkipVerify_RoundTrip verifies that the
|
||||
// new field serializes correctly when set, and is omitted (omitempty)
|
||||
// when false — so existing entries do not gain a new wire-format key
|
||||
// after the Phase 3 schema extension lands.
|
||||
func TestConnectionConfig_InsecureSkipVerify_RoundTrip(t *testing.T) {
|
||||
t.Run("true serializes the field", func(t *testing.T) {
|
||||
cfg := ConnectionConfig{Addr: "https://os:9200", InsecureSkipVerify: true}
|
||||
out, err := json.Marshal(cfg)
|
||||
require.NoError(t, err)
|
||||
assert.Contains(t, string(out), `"insecure_skip_verify":true`)
|
||||
})
|
||||
|
||||
t.Run("false omits the field via omitempty", func(t *testing.T) {
|
||||
cfg := ConnectionConfig{Addr: "https://os:9200", InsecureSkipVerify: false}
|
||||
out, err := json.Marshal(cfg)
|
||||
require.NoError(t, err)
|
||||
assert.NotContains(t, string(out), `"insecure_skip_verify"`)
|
||||
})
|
||||
}
|
||||
|
||||
// TestConnectionConfig_AESGCMRoundTrip_PreservesInsecureSkipVerify
|
||||
// ensures the AES-GCM Value/Scan path round-trips the new field
|
||||
// without corruption. The field is stored as plaintext (no AES-GCM)
|
||||
// but travels alongside encrypted Password / APIKey.
|
||||
func TestConnectionConfig_AESGCMRoundTrip_PreservesInsecureSkipVerify(t *testing.T) {
|
||||
t.Setenv("SYSTEM_AES_KEY", testAESKey)
|
||||
|
||||
original := ConnectionConfig{
|
||||
Addr: "https://os:9200",
|
||||
Username: "admin",
|
||||
Password: "secret-pass",
|
||||
APIKey: "sk-api-key",
|
||||
InsecureSkipVerify: true,
|
||||
}
|
||||
|
||||
// Value — encrypt
|
||||
raw, err := original.Value()
|
||||
require.NoError(t, err)
|
||||
|
||||
// Verify the serialized JSON has the plaintext flag alongside the
|
||||
// encrypted secret fields.
|
||||
var intermediate map[string]interface{}
|
||||
require.NoError(t, json.Unmarshal(raw.([]byte), &intermediate))
|
||||
assert.True(t, strings.HasPrefix(intermediate["password"].(string), "enc:v1:"),
|
||||
"password should still be encrypted")
|
||||
assert.Equal(t, true, intermediate["insecure_skip_verify"],
|
||||
"insecure_skip_verify should be plaintext bool")
|
||||
|
||||
// Scan — decrypt secrets and preserve the flag
|
||||
var scanned ConnectionConfig
|
||||
require.NoError(t, scanned.Scan(raw.([]byte)))
|
||||
assert.Equal(t, "secret-pass", scanned.Password)
|
||||
assert.Equal(t, "sk-api-key", scanned.APIKey)
|
||||
assert.True(t, scanned.InsecureSkipVerify,
|
||||
"InsecureSkipVerify should round-trip through Value/Scan")
|
||||
}
|
||||
|
||||
// TestVectorStoreFieldInfo_OmitemptyPreservesWire confirms that the
|
||||
// four new optional fields (Immutable / Min / Max / Enum) added in
|
||||
// Phase 3 PR 1 are omitted from JSON when unset, so existing
|
||||
// VectorStoreFieldInfo entries (without these fields) serialize
|
||||
// identically before and after the schema extension.
|
||||
func TestVectorStoreFieldInfo_OmitemptyPreservesWire(t *testing.T) {
|
||||
legacy := VectorStoreFieldInfo{
|
||||
Name: "addr",
|
||||
Type: "string",
|
||||
Required: true,
|
||||
Description: "URL",
|
||||
Default: "http://localhost:9200",
|
||||
}
|
||||
out, err := json.Marshal(legacy)
|
||||
require.NoError(t, err)
|
||||
// Phase 3 new fields must all be omitted from the wire format.
|
||||
assert.NotContains(t, string(out), `"immutable"`)
|
||||
assert.NotContains(t, string(out), `"min"`)
|
||||
assert.NotContains(t, string(out), `"max"`)
|
||||
assert.NotContains(t, string(out), `"enum"`)
|
||||
}
|
||||
|
||||
// TestVectorStoreFieldInfo_NewFieldsSerialize verifies the new fields
|
||||
// appear in JSON when set — exercised by the OpenSearch IndexFields
|
||||
// entry that lands in a later Phase 3 PR.
|
||||
func TestVectorStoreFieldInfo_NewFieldsSerialize(t *testing.T) {
|
||||
t.Run("immutable", func(t *testing.T) {
|
||||
f := VectorStoreFieldInfo{Name: "knn_engine", Type: "string", Immutable: true}
|
||||
out, err := json.Marshal(f)
|
||||
require.NoError(t, err)
|
||||
assert.Contains(t, string(out), `"immutable":true`)
|
||||
})
|
||||
|
||||
t.Run("min and max", func(t *testing.T) {
|
||||
minV, maxV := float64(4), float64(64)
|
||||
f := VectorStoreFieldInfo{Name: "hnsw_m", Type: "number", Min: &minV, Max: &maxV}
|
||||
out, err := json.Marshal(f)
|
||||
require.NoError(t, err)
|
||||
assert.Contains(t, string(out), `"min":4`)
|
||||
assert.Contains(t, string(out), `"max":64`)
|
||||
})
|
||||
|
||||
t.Run("enum", func(t *testing.T) {
|
||||
f := VectorStoreFieldInfo{
|
||||
Name: "knn_engine", Type: "string",
|
||||
Enum: []string{"lucene", "faiss"},
|
||||
}
|
||||
out, err := json.Marshal(f)
|
||||
require.NoError(t, err)
|
||||
assert.Contains(t, string(out), `"enum":["lucene","faiss"]`)
|
||||
})
|
||||
|
||||
t.Run("empty enum omits via omitempty", func(t *testing.T) {
|
||||
f := VectorStoreFieldInfo{Name: "addr", Type: "string", Enum: []string{}}
|
||||
out, err := json.Marshal(f)
|
||||
require.NoError(t, err)
|
||||
assert.NotContains(t, string(out), `"enum"`)
|
||||
})
|
||||
}
|
||||
|
||||
// TestVectorStoreFieldInfo_RoundTrip ensures deserialization back into
|
||||
// the struct works for all new fields, including the *float64 pointer
|
||||
// distinction (nil vs explicit 0).
|
||||
func TestVectorStoreFieldInfo_RoundTrip(t *testing.T) {
|
||||
t.Run("min=0 deserializes as &0, not nil", func(t *testing.T) {
|
||||
raw := `{"name":"replicas","type":"number","min":0,"max":5}`
|
||||
var f VectorStoreFieldInfo
|
||||
require.NoError(t, json.Unmarshal([]byte(raw), &f))
|
||||
require.NotNil(t, f.Min)
|
||||
assert.Equal(t, float64(0), *f.Min)
|
||||
require.NotNil(t, f.Max)
|
||||
assert.Equal(t, float64(5), *f.Max)
|
||||
})
|
||||
|
||||
t.Run("missing min/max deserialize as nil", func(t *testing.T) {
|
||||
raw := `{"name":"addr","type":"string"}`
|
||||
var f VectorStoreFieldInfo
|
||||
require.NoError(t, json.Unmarshal([]byte(raw), &f))
|
||||
assert.Nil(t, f.Min)
|
||||
assert.Nil(t, f.Max)
|
||||
})
|
||||
|
||||
t.Run("min and max are independent pointers", func(t *testing.T) {
|
||||
// Setting only Min must leave Max nil (and vice versa). A
|
||||
// refactor that accidentally aliases the two via a shared
|
||||
// local would still pass the previous two subtests because
|
||||
// they always set both ends.
|
||||
raw := `{"name":"upper_only","type":"number","max":10}`
|
||||
var f VectorStoreFieldInfo
|
||||
require.NoError(t, json.Unmarshal([]byte(raw), &f))
|
||||
assert.Nil(t, f.Min)
|
||||
require.NotNil(t, f.Max)
|
||||
assert.Equal(t, float64(10), *f.Max)
|
||||
})
|
||||
|
||||
t.Run("nil enum omits via omitempty", func(t *testing.T) {
|
||||
// `nil` and `[]string{}` both round-trip through omitempty
|
||||
// identically — pin both shapes so a future Go encoder
|
||||
// behavior change cannot regress the wire format.
|
||||
f := VectorStoreFieldInfo{Name: "addr", Type: "string"} // Enum nil
|
||||
out, err := json.Marshal(f)
|
||||
require.NoError(t, err)
|
||||
assert.NotContains(t, string(out), `"enum"`)
|
||||
})
|
||||
}
|
||||
|
||||
// TestOpenSearchRetrieverEngineType_StringValue pins the wire string
|
||||
// to the official product name. The constant exists in PR 1 so the
|
||||
// EngineAwareNormalizer case and AuditAction constants can reference
|
||||
// it; the driver itself lands in a later PR.
|
||||
func TestOpenSearchRetrieverEngineType_StringValue(t *testing.T) {
|
||||
assert.Equal(t,
|
||||
RetrieverEngineType("opensearch"),
|
||||
OpenSearchRetrieverEngineType,
|
||||
)
|
||||
}
|
||||
|
||||
// TestOpenSearchRetrieverEngineType_DistinctFromExisting ensures the
|
||||
// new wire value does not collide with any of the 10 existing engine
|
||||
// types. A collision would silently route requests to the wrong
|
||||
// engine after the activation switch lands.
|
||||
func TestOpenSearchRetrieverEngineType_DistinctFromExisting(t *testing.T) {
|
||||
existing := []RetrieverEngineType{
|
||||
PostgresRetrieverEngineType,
|
||||
ElasticsearchRetrieverEngineType,
|
||||
InfinityRetrieverEngineType,
|
||||
ElasticFaissRetrieverEngineType,
|
||||
QdrantRetrieverEngineType,
|
||||
MilvusRetrieverEngineType,
|
||||
WeaviateRetrieverEngineType,
|
||||
DorisRetrieverEngineType,
|
||||
SQLiteRetrieverEngineType,
|
||||
TencentVectorDBRetrieverEngineType,
|
||||
}
|
||||
for _, e := range existing {
|
||||
assert.NotEqual(t, e, OpenSearchRetrieverEngineType,
|
||||
"OpenSearch wire value must not collide with %s", e)
|
||||
}
|
||||
}
|
||||
|
||||
// TestOpenSearchRetrieverEngineType_NotInValidEngineTypes verifies
|
||||
// that PR 1 does NOT add the new engine type to validEngineTypes —
|
||||
// this is the gate that keeps OpenSearch VectorStore registration
|
||||
// rejected until activation lands in a later PR.
|
||||
func TestOpenSearchRetrieverEngineType_NotInValidEngineTypes(t *testing.T) {
|
||||
assert.False(t, IsValidEngineType(OpenSearchRetrieverEngineType),
|
||||
"OpenSearch must remain invalid for VectorStore registration "+
|
||||
"until activation lands in a later PR (gated activation)")
|
||||
}
|
||||
|
||||
// The next three tests are defense-in-depth companions to
|
||||
// TestOpenSearchRetrieverEngineType_NotInValidEngineTypes. Each pins
|
||||
// a separate activation surface that must remain closed in PR 1 (and
|
||||
// in PR 2). Activation lands together with a coordinated flip in
|
||||
// PR 3, at which point each of these `assert.False` / `assert.NotContains`
|
||||
// / `assert.Nil` lines flips to its positive counterpart in the same
|
||||
// diff — the test suite becomes the activation checklist.
|
||||
|
||||
// TestRetrieverEngineMapping_OpenSearchNotRegistered pins that
|
||||
// `retrieverEngineMapping` does not have an `"opensearch"` key. Without
|
||||
// this entry, setting `RETRIEVE_DRIVER=opensearch` is a silent no-op
|
||||
// (GetDefaultRetrieverEngines drops the unknown driver from its loop
|
||||
// at tenant.go).
|
||||
func TestRetrieverEngineMapping_OpenSearchNotRegistered(t *testing.T) {
|
||||
mapping := GetRetrieverEngineMapping()
|
||||
_, ok := mapping["opensearch"]
|
||||
assert.False(t, ok,
|
||||
"retrieverEngineMapping must not register opensearch until "+
|
||||
"activation lands in a later PR (gated activation)")
|
||||
}
|
||||
|
||||
// TestGetVectorStoreTypes_OmitsOpenSearch pins that the
|
||||
// /api/v1/vector-stores/types response does NOT list opensearch.
|
||||
// Without this entry, the UI dropdown cannot offer OpenSearch as a
|
||||
// store type even if a frontend renderer accidentally tries.
|
||||
func TestGetVectorStoreTypes_OmitsOpenSearch(t *testing.T) {
|
||||
listed := GetVectorStoreTypes()
|
||||
for _, info := range listed {
|
||||
assert.NotEqual(t, "opensearch", info.Type,
|
||||
"GetVectorStoreTypes must not surface opensearch until "+
|
||||
"activation lands in a later PR (gated activation)")
|
||||
}
|
||||
}
|
||||
|
||||
// TestBuildEnvVectorStores_OpenSearchSkipped pins that
|
||||
// `BuildEnvVectorStores("opensearch", lookup)` returns nil — the
|
||||
// `default:` arm of `buildEnvStoreForDriver`. Without an explicit
|
||||
// `case "opensearch":` arm, container.go cannot synthesize an env
|
||||
// store for the driver, completing the third lock on the activation
|
||||
// chain.
|
||||
func TestBuildEnvVectorStores_OpenSearchSkipped(t *testing.T) {
|
||||
stores := BuildEnvVectorStores("opensearch", mockEnvLookup(map[string]string{
|
||||
"OPENSEARCH_ADDR": "https://os:9200",
|
||||
"OPENSEARCH_USERNAME": "admin",
|
||||
}))
|
||||
assert.Empty(t, stores,
|
||||
"BuildEnvVectorStores must not synthesize an env store for "+
|
||||
"opensearch until activation lands in a later PR (gated "+
|
||||
"activation)")
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user