refactor(cli): delete envelope infrastructure, errors to stderr
Removes the entire envelope machinery now that every success path
emits bare JSON:
- cli/internal/format/envelope.go (Envelope, Success, Failure,
SuccessWithRisk, WriteEnvelope, Meta, Notice, UpdateNotice,
VersionSkewNotice, Risk, RiskLevel, ErrorBody) + tests.
- cli/internal/format/filter.go envelope-specific helpers
(WriteEnvelopeFiltered, marshalEnvelope, applyFieldFilter,
filterDataPayload, filterObjectData); the reusable
filterArrayItems / filterObjectKeys / writeJQ stay for bare.go.
- cli/internal/cmdutil/exporter.go + tests (envelope-only).
- cli/internal/cmdutil/PrintErrorEnvelope + ToErrorBody +
operationRiskOf + Error.OperationRisk field + OperationRisk struct.
Error path: all errors now go to stderr via cmdutil.PrintError in
`code: message\nhint: ...` form, regardless of --json. Stdout stays
empty (or holds the partial-success the command already wrote) so
downstream `--json | jq` pipelines never have to filter error shapes
out of the success stream. Typed exit codes (3 auth.* / 4
resource.not_found / 5 input.* / 6 server.rate_limited / 7 server.*
+ network.* / 10 input.confirmation_required) carry the failure
class for agents that branch on it.
Acceptance contract:
- envelope_test.go → wire_test.go (TestEnvelopeGolden → TestWireGolden).
- testdata/envelopes/ → testdata/wire/.
- Error-path cases assert the typed code substring on stderr.
- Orphan whoami.*.json goldens deleted.
AGENTS.md + README.md rewritten for the bare-data contract:
- Drop envelope schema section + dry-run rule.
- Document bare JSON on stdout + `code: msg\nhint: …` on stderr.
- ADR-3 reframed around bare data and why error separation matters
for `--json | jq` pipelines.
WriteJSONFiltered short-circuits to WriteJSON when both filters are
empty (skip the marshal-buffer round-trip for the common case).
Final review pass:
- Fix wire-contract bug: `--json id,name` (space form) is broken by
pflag's NoOptDefVal; AGENTS.md / README.md / SetAgentHelp + the
field-discovery help text all switched to `--json=id,name`.
- Fix `weknora api --jq` silently ignored: api.go now routes through
WriteJSONFiltered with jopts.JQ.
- AGENTS.md: drop the false claim that `auth logout` honors `-y`
(logout is local-only with no ConfirmDestructive guard); list the
actual destructive commands instead.
- Rewrite cli/acceptance/e2e/e2e_test.go for the bare-data wire shape
(was still parsing `out["data"]` / `env["ok"]`).
- Add `JSONOptions.Emit(w, v)` helper; collapse ~33 repeated
`format.WriteJSONFiltered(iostreams.IO.Out, X, jopts.Fields,
jopts.JQ)` sites to `jopts.Emit(iostreams.IO.Out, X)` — drops the
format import from 22 cmd/* files.
- Delete single-caller `cmdutil.MustRequireFlag`; inline as
`_ = cmd.MarkFlagRequired(...)` everywhere.
- Add `_ = cmd.MarkFlagRequired("name")` to `kb create`; it was the
only write command relying on runtime --name validation while
`context add` already used the cobra-level mark.
- `context use`: register `--json` / `--jq` (was always emitting JSON
unconditionally with no human path and no flag — diverged from
every other write command); human mode now prints
`✓ Switched context to X (was Y)`.
- Replace per-package `confirmPrompter` / `scriptedConfirm` /
`errPrompter` test doubles with `testutil.ConfirmPrompter`.
- Rename `chatService` → `ChatService` (export to match siblings
`ListService` / `ViewService`); rename `printUploadSuccess` →
`renderUploadSuccess` (siblings use `render*`).
- `defaultHint(CodeResourceNotFound)`: drop the hardcoded
"list available with `weknora kb list`" — misleading on agent /
doc / session 404. Replaced with "verify the resource ID and try
again".
- Strip stale `v0.2/v0.3` / "envelope" / "v0.0/v0.1 supports only"
historical tags from production comments and a few test
descriptions.
This commit is contained in:
@@ -21,7 +21,7 @@ import (
|
||||
// TestMain pins the doctor credential-storage outcome for the whole suite.
|
||||
// Otherwise the check probes the real OS keyring, which differs between
|
||||
// macOS dev machines (Keychain present → ok) and Linux CI runners without
|
||||
// libsecret (file fallback → warn), making golden envelopes host-dependent.
|
||||
// libsecret (file fallback → warn), making golden outputs host-dependent.
|
||||
// MemStore is neither *FileStore nor a real keyring, so the doctor's
|
||||
// type-switch hits the StatusOK branch.
|
||||
func TestMain(m *testing.M) {
|
||||
@@ -62,13 +62,9 @@ func newTestFactory(t *testing.T, mockServer *httptest.Server, mockClient *sdk.C
|
||||
// runCmd executes the root command in-process and returns captured stdout/stderr.
|
||||
// Replaces iostreams.IO singleton via SetForTest (auto-restored in t.Cleanup).
|
||||
//
|
||||
// Mirrors cmd.Execute() carefully: callers expect the same envelope-printing
|
||||
// behavior the real entrypoint provides. The helper (a) wires the cobra Out /
|
||||
// Err sinks to the same buffers it returns (the `version` leaf and any future
|
||||
// command using c.OutOrStdout would otherwise leak to os.Stdout), and (b)
|
||||
// re-runs the error-envelope path so failure cases produce the JSON envelope
|
||||
// the contract test compares against. Without (b), every error scenario's
|
||||
// golden would be empty.
|
||||
// Mirrors cmd.Execute(): wires the cobra Out / Err sinks to the same buffers
|
||||
// it returns, and re-runs cmdutil.PrintError on stderr for failure cases so
|
||||
// the contract assertion sees the typed `code: message\nhint: ...` line.
|
||||
func runCmd(t *testing.T, f *cmdutil.Factory, args ...string) (stdout, stderr string, exitCode int) {
|
||||
t.Helper()
|
||||
out, errBuf := iostreams.SetForTest(t)
|
||||
@@ -77,31 +73,25 @@ func runCmd(t *testing.T, f *cmdutil.Factory, args ...string) (stdout, stderr st
|
||||
root.SetContext(context.Background())
|
||||
root.SetOut(out)
|
||||
root.SetErr(errBuf)
|
||||
leaf, err := root.ExecuteC()
|
||||
_, err := root.ExecuteC()
|
||||
if err != nil {
|
||||
err = cmd.MapCobraError(err)
|
||||
if cmd.WantsJSONOutput(leaf) {
|
||||
cmdutil.PrintErrorEnvelope(iostreams.IO.Out, err)
|
||||
} else {
|
||||
cmdutil.PrintError(iostreams.IO.Err, err)
|
||||
}
|
||||
cmdutil.PrintError(iostreams.IO.Err, err)
|
||||
}
|
||||
return out.String(), errBuf.String(), cmdutil.ExitCode(err)
|
||||
}
|
||||
|
||||
// assertGolden compares got against the JSON golden file at path.
|
||||
// With -update, writes got to path. Normalizes _meta.request_id to "<id>"
|
||||
// before compare (only field known unstable in v0.0).
|
||||
// With -update, writes got to path.
|
||||
//
|
||||
// CRLF normalization: Windows checkouts with the default core.autocrlf=true
|
||||
// turn LF in tracked text files into CRLF on disk. The command output is
|
||||
// always LF, so byte-equal would fail despite identical content. .gitattributes
|
||||
// is the primary defense (forcing LF on testdata/**/*.json), but we also
|
||||
// strip CR here so a misconfigured contributor checkout doesn't break the
|
||||
// suite locally before they push.
|
||||
// always LF, so byte-equal would fail despite identical content.
|
||||
// .gitattributes is the primary defense (forcing LF on testdata/**/*.json),
|
||||
// but we also strip CR here so a misconfigured contributor checkout doesn't
|
||||
// break the suite locally before they push.
|
||||
func assertGolden(t *testing.T, got []byte, path string) {
|
||||
t.Helper()
|
||||
got = normalizeEnvelope(got)
|
||||
if *update {
|
||||
if err := os.MkdirAll(filepath.Dir(path), 0755); err != nil {
|
||||
t.Fatalf("mkdir testdata: %v", err)
|
||||
@@ -118,7 +108,7 @@ func assertGolden(t *testing.T, got []byte, path string) {
|
||||
want = stripCR(want)
|
||||
got = stripCR(got)
|
||||
if !bytes.Equal(want, got) {
|
||||
t.Errorf("envelope mismatch for %s\nwant:\n%s\ngot:\n%s", path, want, got)
|
||||
t.Errorf("stdout mismatch for %s\nwant:\n%s\ngot:\n%s", path, want, got)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -127,10 +117,3 @@ func assertGolden(t *testing.T, got []byte, path string) {
|
||||
func stripCR(b []byte) []byte {
|
||||
return bytes.ReplaceAll(b, []byte{'\r'}, nil)
|
||||
}
|
||||
|
||||
// normalizeEnvelope replaces unstable fields with placeholders for stable diff.
|
||||
// Currently no-op (v0.0 commands don't set _meta.request_id, so output is stable).
|
||||
// Hook for future fields.
|
||||
func normalizeEnvelope(b []byte) []byte {
|
||||
return b
|
||||
}
|
||||
|
||||
@@ -1,39 +1,34 @@
|
||||
// cli/acceptance/contract/envelope_test.go
|
||||
// cli/acceptance/contract/wire_test.go
|
||||
//
|
||||
// Envelope contract test (PR-8 Task 18). Drives root cobra in-process for
|
||||
// each scenario, captures stdout, and compares against a JSON golden file
|
||||
// in cli/acceptance/testdata/envelopes/.
|
||||
// Wire contract test. Drives root cobra in-process for each scenario,
|
||||
// captures stdout + stderr, and asserts:
|
||||
//
|
||||
// Spec §4.1 lists 19 envelope scenarios. Implemented count: 16.
|
||||
// - stdout matches a JSON golden in cli/acceptance/testdata/wire/
|
||||
// - on wantErr cases, stderr contains the expected typed error code
|
||||
//
|
||||
// Cases dropped (with reason):
|
||||
// - doctor.success — non-offline path emits unstable
|
||||
// timing ("reachable in 2ms").
|
||||
// Unit tests in cli/cmd/doctor
|
||||
// cover the all-pass shape;
|
||||
// doctor.success_offline is the
|
||||
// deterministic sibling kept here.
|
||||
// - auth_login.success — requires stdin pipe
|
||||
// (--with-token) + keyring-aware
|
||||
// Secrets store; helpers_test
|
||||
// (PR-6) does not yet expose a
|
||||
// stdin hook. Deferred to the e2e harness.
|
||||
// - auth_login.error_auth_unauthenticated — same setup as above; deferred
|
||||
// together.
|
||||
// - context_use.error_local_context_not_found — `context use` has no --json
|
||||
// flag in v0.1, so error path
|
||||
// renders plain stderr. Pinning
|
||||
// its envelope shape needs either
|
||||
// a --json flag added to the leaf
|
||||
// or a global --json. Deferred
|
||||
// until that lands; the success
|
||||
// case is golden-pinned (writes
|
||||
// envelope unconditionally).
|
||||
// Successful cases produce bare JSON on stdout (no envelope wrapper);
|
||||
// failure cases produce empty stdout (or, for `doctor`, the data object
|
||||
// the command writes before returning SilentError) and a `code: msg`
|
||||
// line on stderr.
|
||||
//
|
||||
// All cases use leaf-positioned --json (e.g. `version --json`) instead of the
|
||||
// `--json version` form sketched in the spec. v0.0–v0.1 implements --json as a
|
||||
// per-leaf flag, not a global persistent flag — root-level --json is detected
|
||||
// only as an error-envelope fallback (see argsRequestJSON in cmd/root.go).
|
||||
// Cases intentionally omitted (with reason):
|
||||
// - doctor.success — non-offline path emits
|
||||
// unstable timing
|
||||
// ("reachable in 2ms").
|
||||
// Unit tests in cli/cmd/doctor
|
||||
// cover the all-pass shape;
|
||||
// doctor.success_offline is
|
||||
// the deterministic sibling
|
||||
// kept here.
|
||||
// - auth_login.success — requires stdin pipe
|
||||
// (--with-token) + keyring-
|
||||
// aware Secrets store; the
|
||||
// helper does not yet expose
|
||||
// a stdin hook.
|
||||
// - auth_login.error_auth_unauthenticated — same setup as above.
|
||||
//
|
||||
// All cases use leaf-positioned --json (e.g. `version --json`). --json is a
|
||||
// per-leaf flag, not a global persistent flag.
|
||||
package contract_test
|
||||
|
||||
import (
|
||||
@@ -49,23 +44,28 @@ import (
|
||||
sdk "github.com/Tencent/WeKnora/client"
|
||||
)
|
||||
|
||||
// envelopeCase declares one row in the contract matrix. Optional fields:
|
||||
// server — mock /api/v1/* endpoints; nil means no network needed.
|
||||
// preConfig — seed config.yaml under the per-test XDG_CONFIG_HOME (set by
|
||||
// newTestFactory); use for cases like context use that read
|
||||
// local state without an SDK round-trip.
|
||||
// wantErr — true means the run is expected to exit non-zero.
|
||||
type envelopeCase struct {
|
||||
name string
|
||||
args []string
|
||||
server http.HandlerFunc
|
||||
preConfig func(t *testing.T)
|
||||
wantErr bool
|
||||
// wireCase declares one row in the contract matrix. Optional fields:
|
||||
// server — mock /api/v1/* endpoints; nil means no network.
|
||||
// preConfig — seed config.yaml under the per-test XDG_CONFIG_HOME
|
||||
// (set by newTestFactory); use for cases like
|
||||
// `context use` that read local state without an
|
||||
// SDK round-trip.
|
||||
// wantErr — non-zero exit expected.
|
||||
// wantStderrSubstring — stderr must contain this substring (typically the
|
||||
// typed error code, e.g. "auth.unauthenticated").
|
||||
// Only meaningful when wantErr=true.
|
||||
type wireCase struct {
|
||||
name string
|
||||
args []string
|
||||
server http.HandlerFunc
|
||||
preConfig func(t *testing.T)
|
||||
wantErr bool
|
||||
wantStderrSubstring string
|
||||
}
|
||||
|
||||
// envelopeCases enumerates every contract scenario whose envelope is golden-
|
||||
// pinned. Order is illustrative (matches spec §4.1 mostly), not load-bearing.
|
||||
var envelopeCases = []envelopeCase{
|
||||
// wireCases enumerates every contract scenario whose stdout is golden-pinned.
|
||||
// Order is illustrative, not load-bearing.
|
||||
var wireCases = []wireCase{
|
||||
// 1. version.success — pure local; no client touched.
|
||||
{
|
||||
name: "version.success",
|
||||
@@ -82,9 +82,9 @@ var envelopeCases = []envelopeCase{
|
||||
|
||||
// 3. doctor.error_network — base_url returns 500 → ping fail → cascade
|
||||
// skip on auth_credential + server_version. credential_storage still
|
||||
// runs (independent). v0.2 contract: any check=fail flips envelope.ok
|
||||
// to false and exits 1 (RunE returns SilentError so the data envelope
|
||||
// written by emit() is preserved as the only stdout content).
|
||||
// runs (independent). Contract: any check=fail bumps summary.failed
|
||||
// and RunE returns SilentError → exit 1 with the data object
|
||||
// written by emit() as the only stdout content.
|
||||
{
|
||||
name: "doctor.error_network",
|
||||
args: []string{"doctor", "--json"},
|
||||
@@ -104,10 +104,11 @@ var envelopeCases = []envelopeCase{
|
||||
server: kbListEmpty,
|
||||
},
|
||||
{
|
||||
name: "kb_list.error_auth_forbidden",
|
||||
args: []string{"kb", "list", "--json"},
|
||||
server: always403,
|
||||
wantErr: true,
|
||||
name: "kb_list.error_auth_forbidden",
|
||||
args: []string{"kb", "list", "--json"},
|
||||
server: always403,
|
||||
wantErr: true,
|
||||
wantStderrSubstring: "auth.forbidden",
|
||||
},
|
||||
{
|
||||
name: "kb_view.success",
|
||||
@@ -115,16 +116,17 @@ var envelopeCases = []envelopeCase{
|
||||
server: kbGetOne,
|
||||
},
|
||||
{
|
||||
name: "kb_view.error_resource_not_found",
|
||||
args: []string{"kb", "view", "missing", "--json"},
|
||||
server: always404,
|
||||
wantErr: true,
|
||||
name: "kb_view.error_resource_not_found",
|
||||
args: []string{"kb", "view", "missing", "--json"},
|
||||
server: always404,
|
||||
wantErr: true,
|
||||
wantStderrSubstring: "resource.not_found",
|
||||
},
|
||||
|
||||
// 8. context use — pure local I/O against config.yaml.
|
||||
{
|
||||
name: "context_use.success",
|
||||
args: []string{"context", "use", "production"},
|
||||
args: []string{"context", "use", "production", "--json"},
|
||||
preConfig: func(t *testing.T) {
|
||||
cfg := &config.Config{
|
||||
CurrentContext: "staging",
|
||||
@@ -147,10 +149,11 @@ var envelopeCases = []envelopeCase{
|
||||
server: whoamiOK,
|
||||
},
|
||||
{
|
||||
name: "auth_status.error_auth_unauthenticated",
|
||||
args: []string{"auth", "status", "--json"},
|
||||
server: always401,
|
||||
wantErr: true,
|
||||
name: "auth_status.error_auth_unauthenticated",
|
||||
args: []string{"auth", "status", "--json"},
|
||||
server: always401,
|
||||
wantErr: true,
|
||||
wantStderrSubstring: "auth.unauthenticated",
|
||||
},
|
||||
|
||||
// 11-13. search chunks — verb-noun shape (gh search parity), positional query, --kb required.
|
||||
@@ -163,26 +166,28 @@ var envelopeCases = []envelopeCase{
|
||||
server: searchTwoResults,
|
||||
},
|
||||
{
|
||||
name: "search.error_resource_not_found",
|
||||
args: []string{"search", "chunks", "query", "--kb=eeeeeeee-eeee-4eee-8eee-eeeeeeeeeeee", "--json"},
|
||||
server: always404,
|
||||
wantErr: true,
|
||||
name: "search.error_resource_not_found",
|
||||
args: []string{"search", "chunks", "query", "--kb=eeeeeeee-eeee-4eee-8eee-eeeeeeeeeeee", "--json"},
|
||||
server: always404,
|
||||
wantErr: true,
|
||||
wantStderrSubstring: "resource.not_found",
|
||||
},
|
||||
{
|
||||
// --no-vector + --no-keyword is the input.invalid case; the KB UUID
|
||||
// is just there to satisfy MarkFlagRequired so validation runs deep
|
||||
// enough to hit the mutex-channel check.
|
||||
name: "search.error_input_invalid",
|
||||
args: []string{"search", "chunks", "query", "--kb=11111111-1111-4111-8111-111111111111", "--no-vector", "--no-keyword", "--json"},
|
||||
wantErr: true,
|
||||
name: "search.error_input_invalid",
|
||||
args: []string{"search", "chunks", "query", "--kb=11111111-1111-4111-8111-111111111111", "--no-vector", "--no-keyword", "--json"},
|
||||
wantErr: true,
|
||||
wantStderrSubstring: "input.invalid_argument",
|
||||
},
|
||||
}
|
||||
|
||||
// TestEnvelopeGolden is the matrix-runner. Cases are sequential (the
|
||||
// iostreams singleton swap inside helpers.runCmd is package-global; t.Parallel
|
||||
// is contractually forbidden — see helpers_test.go SetForTest comment).
|
||||
func TestEnvelopeGolden(t *testing.T) {
|
||||
for _, tc := range envelopeCases {
|
||||
// TestWireGolden is the matrix-runner. Cases are sequential (the
|
||||
// iostreams singleton swap inside helpers.runCmd is package-global;
|
||||
// t.Parallel is contractually forbidden — see helpers_test.go).
|
||||
func TestWireGolden(t *testing.T) {
|
||||
for _, tc := range wireCases {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
var ts *httptest.Server
|
||||
var mockClient *sdk.Client
|
||||
@@ -202,7 +207,10 @@ func TestEnvelopeGolden(t *testing.T) {
|
||||
if !tc.wantErr && exit != 0 {
|
||||
t.Errorf("unexpected non-zero exit %d; stdout=%q stderr=%q", exit, stdout, stderr)
|
||||
}
|
||||
path := filepath.Join("..", "testdata", "envelopes", tc.name+".json")
|
||||
if tc.wantStderrSubstring != "" && !strings.Contains(stderr, tc.wantStderrSubstring) {
|
||||
t.Errorf("stderr missing %q; got %q", tc.wantStderrSubstring, stderr)
|
||||
}
|
||||
path := filepath.Join("..", "testdata", "wire", tc.name+".json")
|
||||
assertGolden(t, []byte(stdout), path)
|
||||
})
|
||||
}
|
||||
@@ -31,7 +31,7 @@ import (
|
||||
|
||||
// TestRAGFullLoop walks the demo MVP path: link a context, create a KB,
|
||||
// upload a doc, wait for indexing, search it, then chat against it. Each
|
||||
// step parses the CLI's JSON envelope to extract IDs for the next step —
|
||||
// step parses the CLI's bare JSON to extract IDs for the next step —
|
||||
// validating both functional behavior and wire-contract stability.
|
||||
func TestRAGFullLoop(t *testing.T) {
|
||||
host := mustEnv(t, "WEKNORA_E2E_HOST")
|
||||
@@ -45,75 +45,69 @@ func TestRAGFullLoop(t *testing.T) {
|
||||
env := append(os.Environ(),
|
||||
"XDG_CONFIG_HOME="+xdg,
|
||||
"XDG_CACHE_HOME="+filepath.Join(xdg, "cache"),
|
||||
// SDK debug off — explicit so the CI run isn't noisy. C1 SDK silence
|
||||
// makes this redundant in practice but the explicit flag documents
|
||||
// intent.
|
||||
// SDK debug off — explicit so the CI run isn't noisy.
|
||||
"WEKNORA_SDK_DEBUG=",
|
||||
)
|
||||
|
||||
// 1. kb create
|
||||
// 1. kb create → bare KnowledgeBase object
|
||||
kbName := prefix + fmt.Sprintf("%d", time.Now().UnixNano())
|
||||
createOut := runJSON(t, bin, env, "kb", "create", "--name", kbName, "--json")
|
||||
kbData, ok := createOut["data"].(map[string]any)
|
||||
if !ok {
|
||||
t.Fatalf("kb create envelope: data not an object: %v", createOut)
|
||||
var created struct {
|
||||
ID string `json:"id"`
|
||||
Name string `json:"name"`
|
||||
}
|
||||
kbID, _ := kbData["id"].(string)
|
||||
if kbID == "" {
|
||||
t.Fatalf("kb create returned no id: %v", createOut)
|
||||
runJSONInto(t, bin, env, &created, "kb", "create", "--name", kbName, "--json")
|
||||
if created.ID == "" {
|
||||
t.Fatalf("kb create returned no id")
|
||||
}
|
||||
t.Logf("created KB: %s (%s)", kbID, kbName)
|
||||
t.Logf("created KB: %s (%s)", created.ID, kbName)
|
||||
|
||||
t.Cleanup(func() {
|
||||
// Best-effort cleanup; a 404 means the KB was already gone.
|
||||
out, err := run(bin, env, "kb", "delete", kbID, "-y", "--json")
|
||||
out, err := run(bin, env, "kb", "delete", created.ID, "-y", "--json")
|
||||
if err != nil {
|
||||
t.Logf("cleanup kb delete: %v\n%s", err, out)
|
||||
}
|
||||
})
|
||||
|
||||
// 2. doc upload
|
||||
// 2. doc upload → bare Knowledge object
|
||||
docPath := writeSampleDoc(t)
|
||||
uploadOut := runJSON(t, bin, env, "doc", "upload", docPath, "--kb", kbID, "--json")
|
||||
docData, _ := uploadOut["data"].(map[string]any)
|
||||
docID, _ := docData["id"].(string)
|
||||
if docID == "" {
|
||||
t.Fatalf("doc upload returned no id: %v", uploadOut)
|
||||
var uploaded struct {
|
||||
ID string `json:"id"`
|
||||
}
|
||||
t.Logf("uploaded doc: %s", docID)
|
||||
runJSONInto(t, bin, env, &uploaded, "doc", "upload", docPath, "--kb", created.ID, "--json")
|
||||
if uploaded.ID == "" {
|
||||
t.Fatalf("doc upload returned no id")
|
||||
}
|
||||
t.Logf("uploaded doc: %s", uploaded.ID)
|
||||
|
||||
// 3. poll until indexing finishes (status changes from "pending" / "processing" to "ready" / similar)
|
||||
waitDocReady(t, bin, env, kbID, docID, 90*time.Second)
|
||||
waitDocReady(t, bin, env, created.ID, uploaded.ID, 90*time.Second)
|
||||
|
||||
// 4. search — verify retrieval returns chunks
|
||||
searchOut := runJSON(t, bin, env, "search", "chunks", "sample", "--kb", kbID, "--limit", "5", "--json")
|
||||
searchData, _ := searchOut["data"].(map[string]any)
|
||||
results, _ := searchData["results"].([]any)
|
||||
// 4. search chunks → bare []SearchResult
|
||||
var results []map[string]any
|
||||
runJSONInto(t, bin, env, &results, "search", "chunks", "sample", "--kb", created.ID, "--limit", "5", "--json")
|
||||
if len(results) == 0 {
|
||||
t.Fatalf("search returned no results: %v", searchOut)
|
||||
t.Fatalf("search returned no results")
|
||||
}
|
||||
t.Logf("search returned %d results", len(results))
|
||||
|
||||
// 5. chat — verify LLM answer + references in --json + --no-stream mode
|
||||
// (--no-stream forces accumulator path; --json gates envelope output)
|
||||
chatOut := runJSON(t, bin, env, "chat", "summarize the document briefly", "--kb", kbID, "--no-stream", "--json")
|
||||
chatData, _ := chatOut["data"].(map[string]any)
|
||||
answer, _ := chatData["answer"].(string)
|
||||
if strings.TrimSpace(answer) == "" {
|
||||
t.Fatalf("chat returned empty answer: %v", chatOut)
|
||||
// 5. chat --no-stream --json → bare {answer, references, ...} object
|
||||
var chat struct {
|
||||
Answer string `json:"answer"`
|
||||
References []map[string]any `json:"references"`
|
||||
}
|
||||
refs, _ := chatData["references"].([]any)
|
||||
t.Logf("chat answer (%d chars), %d references", len(answer), len(refs))
|
||||
if len(refs) == 0 {
|
||||
runJSONInto(t, bin, env, &chat, "chat", "summarize the document briefly", "--kb", created.ID, "--no-stream", "--json")
|
||||
if strings.TrimSpace(chat.Answer) == "" {
|
||||
t.Fatalf("chat returned empty answer")
|
||||
}
|
||||
t.Logf("chat answer (%d chars), %d references", len(chat.Answer), len(chat.References))
|
||||
if len(chat.References) == 0 {
|
||||
// Soft warning — some servers may not surface references for every
|
||||
// question, but the demo flow is supposed to.
|
||||
t.Logf("warning: chat returned 0 references (server may have a different config)")
|
||||
}
|
||||
}
|
||||
|
||||
// mustEnv reads an env var and skips the test if missing — keeps the
|
||||
// suite friendly to community contributors who clone the repo without
|
||||
// access to the maintainer's E2E secrets.
|
||||
func mustEnv(t *testing.T, key string) string {
|
||||
t.Helper()
|
||||
v := os.Getenv(key)
|
||||
@@ -203,27 +197,23 @@ func waitDocReady(t *testing.T, bin string, env []string, kbID, docID string, ti
|
||||
deadline := time.Now().Add(timeout)
|
||||
tick := 2 * time.Second
|
||||
for time.Now().Before(deadline) {
|
||||
out := runJSON(t, bin, env, "doc", "list", "--kb", kbID, "--page-size", "100", "--json")
|
||||
data, _ := out["data"].(map[string]any)
|
||||
items, _ := data["items"].([]any)
|
||||
for _, it := range items {
|
||||
m, ok := it.(map[string]any)
|
||||
if !ok {
|
||||
var docs []struct {
|
||||
ID string `json:"id"`
|
||||
ParseStatus string `json:"parse_status"`
|
||||
}
|
||||
runJSONInto(t, bin, env, &docs, "doc", "list", "--kb", kbID, "--page-size", "100", "--json")
|
||||
for _, d := range docs {
|
||||
if d.ID != docID {
|
||||
continue
|
||||
}
|
||||
id, _ := m["id"].(string)
|
||||
if id != docID {
|
||||
continue
|
||||
}
|
||||
status, _ := m["status"].(string)
|
||||
low := strings.ToLower(status)
|
||||
low := strings.ToLower(d.ParseStatus)
|
||||
switch {
|
||||
case low == "failed", low == "error":
|
||||
t.Fatalf("doc %s indexing failed: status=%q", docID, status)
|
||||
t.Fatalf("doc %s indexing failed: status=%q", docID, d.ParseStatus)
|
||||
case low == "pending", low == "processing", low == "":
|
||||
// keep waiting
|
||||
default:
|
||||
t.Logf("doc %s ready (status=%q)", docID, status)
|
||||
t.Logf("doc %s ready (status=%q)", docID, d.ParseStatus)
|
||||
return
|
||||
}
|
||||
}
|
||||
@@ -246,20 +236,16 @@ func run(bin string, env []string, args ...string) ([]byte, error) {
|
||||
return stdout.Bytes(), nil
|
||||
}
|
||||
|
||||
// runJSON runs the CLI expecting --json output and parses the envelope.
|
||||
// Test fails immediately on non-zero exit or unparseable JSON.
|
||||
func runJSON(t *testing.T, bin string, env []string, args ...string) map[string]any {
|
||||
// runJSONInto runs the CLI expecting bare JSON output and decodes stdout
|
||||
// into out (a struct, slice, or map pointer). Test fails on non-zero exit
|
||||
// or unparseable JSON.
|
||||
func runJSONInto(t *testing.T, bin string, env []string, out any, args ...string) {
|
||||
t.Helper()
|
||||
out, err := run(bin, env, args...)
|
||||
stdout, err := run(bin, env, args...)
|
||||
if err != nil {
|
||||
t.Fatalf("%v", err)
|
||||
}
|
||||
var env_ map[string]any
|
||||
if err := json.Unmarshal(out, &env_); err != nil {
|
||||
t.Fatalf("parse envelope from %v: %v\nstdout:\n%s", args, err, string(out))
|
||||
if err := json.Unmarshal(stdout, out); err != nil {
|
||||
t.Fatalf("parse JSON from %v: %v\nstdout:\n%s", args, err, string(stdout))
|
||||
}
|
||||
if ok, _ := env_["ok"].(bool); !ok {
|
||||
t.Fatalf("envelope ok=false from %v: %s", args, string(out))
|
||||
}
|
||||
return env_
|
||||
}
|
||||
|
||||
@@ -1 +0,0 @@
|
||||
{"ok":false,"error":{"code":"auth.unauthenticated","message":"fetch current user: HTTP error 401: {\"error\":\"unauthenticated\"}","hint":"run `weknora auth login`"}}
|
||||
@@ -1 +0,0 @@
|
||||
{"ok":false,"error":{"code":"auth.forbidden","message":"list knowledge bases: HTTP error 403: {\"error\":\"forbidden\"}","hint":"active context lacks permission for this resource"}}
|
||||
@@ -1 +0,0 @@
|
||||
{"ok":false,"error":{"code":"resource.not_found","message":"get knowledge base \"missing\": HTTP error 404: {\"error\":\"not found\"}","hint":"verify the resource ID; list available with `weknora kb list`"}}
|
||||
@@ -1 +0,0 @@
|
||||
{"ok":false,"error":{"code":"input.invalid_argument","message":"--no-vector and --no-keyword cannot both be set","hint":"see `weknora <command> --help` for valid usage"}}
|
||||
@@ -1 +0,0 @@
|
||||
{"ok":false,"error":{"code":"resource.not_found","message":"hybrid search: HTTP error 404: {\"error\":\"not found\"}","hint":"verify the resource ID; list available with `weknora kb list`"}}
|
||||
@@ -1 +0,0 @@
|
||||
{"ok":false,"error":{"code":"auth.unauthenticated","message":"fetch current user: HTTP error 401: {\"error\":\"unauthenticated\"}","hint":"run `weknora auth login`"}}
|
||||
@@ -1 +0,0 @@
|
||||
{"ok":true,"data":{"user_id":"usr_abc","tenant_id":42}}
|
||||
Reference in New Issue
Block a user