Files
Enterprise-Knowledge-Base/cli/acceptance/contract/helpers_test.go
T
nullkey e236be1ced fix(cli): correct KB id detection, SSE terminal-frame, and CI test isolation
Three defects surfaced during end-to-end RAG verification — the first two
block real chat usage, the third makes Linux CI flaky:

1. KB id detection — `IsKBID` was checking
   `strings.HasPrefix(s, "kb_")`, but WeKnora generates KB ids as bare
   UUIDs (internal/types/knowledge_base.go: `uuid.New().String()` stored
   in a `varchar(36)` column). Real ids therefore fell through to the
   name-resolution path:

     $ weknora chat ... --kb a32a63ff-fb36-4874-bcaa-30f48570a694
     Error: knowledge base not found: a32a63ff-...

   Switched the discriminator to a UUID regex
   (`^[0-9a-fA-F]{8}-…-[0-9a-fA-F]{12}$`). KB names are arbitrary
   user-supplied strings, so the canonical 8-4-4-4-12 form is an
   unambiguous signal. Mirrors gcloud `--project`'s id-vs-name detection.

2. SSE terminal-frame — the accumulator's `Append` was gating
   finalization on `r.Done`, but the server's KnowledgeQAStream protocol
   emits a leading `agent_query` frame with `done=true` to deliver
   session + message metadata *before* the answer fragments arrive:

     event: message
     data: {"response_type":"agent_query","content":"","done":true,…}

     event: message
     data: {"response_type":"answer","content":"你好","done":false}
     …
     event: message
     data: {"response_type":"complete","content":"","done":true}

   The accumulator therefore flipped to `finished=true` on frame #1 and
   discarded every subsequent answer fragment — `weknora chat … --json`
   returned `answer: ""` even though the LLM reported completion_tokens
   > 0. Fixed: terminate only on `response_type == complete`.
   References still captured opportunistically (they may arrive on a
   dedicated `references` event before the terminator).

3. doctor credential_storage CI isolation — the check probes the real
   OS keyring via `secrets.NewBestEffortStore()`: present on macOS dev
   machines → StatusOK; absent on Linux CI runners without libsecret /
   Gnome-Keyring → StatusWarn ("falling back to file store"). That
   host-dependence was leaking into two test classes that assumed
   StatusOK:

     * cmd/doctor/doctor_test.go: TestDoctor_AllOK and
       TestDoctor_NoConfig_StillRunsCredentialStorage already had a
       withCredStoreFactory seam but didn't use it. Added the pin.

     * acceptance/contract/envelope_test.go: doctor.success_offline
       and doctor.error_network golden cases. The contract test runs
       through the cobra tree in-process and shares cmd/doctor's
       package-level credStoreFactory var — but couldn't reach it
       because the existing seam was unexported.

   Fix: export `doctor.SetCredStoreFactoryForTest(fn) (restore func())`
   for out-of-package tests; acceptance/contract/helpers_test.go adds
   a TestMain that pins the factory to a MemStore-returning closure
   for the whole suite (MemStore is neither *FileStore nor a real
   keyring, so doctor's type-switch hits StatusOK). Production stays
   at secrets.NewBestEffortStore — only the test hook is now reachable
   from across packages.

Test fixtures and goldens that used the old `kb_xxx` literals or
`Done: true` terminators were rewritten to use real UUIDs and
`ResponseType: ResponseTypeComplete` respectively. Per-command --help
text and Long descriptions / Examples now show a UUID rather than
`kb_…` so users see the correct shape from the start. New
TestAccumulator_IgnoresAgentQueryDone pins the SSE terminator bug so
it can't regress.

Tests: 24 cli packages green on macOS dev + Linux/macOS/Windows CI
matrix. Verified end-to-end against a live WeKnora server: `weknora
chat "..." --kb <UUID> --no-stream --json` returns the full LLM answer
in the envelope, live token streaming in TTY mode works, and the
credential_storage check renders deterministic envelopes across hosts.
2026-05-12 13:20:42 +08:00

137 lines
5.0 KiB
Go

// cli/acceptance/contract/helpers_test.go
package contract_test
import (
"bytes"
"context"
"flag"
"net/http/httptest"
"os"
"path/filepath"
"testing"
"github.com/Tencent/WeKnora/cli/cmd"
"github.com/Tencent/WeKnora/cli/cmd/doctor"
"github.com/Tencent/WeKnora/cli/internal/cmdutil"
"github.com/Tencent/WeKnora/cli/internal/iostreams"
"github.com/Tencent/WeKnora/cli/internal/secrets"
sdk "github.com/Tencent/WeKnora/client"
)
// TestMain pins the doctor credential-storage outcome for the whole suite.
// Otherwise the check probes the real OS keyring, which differs between
// macOS dev machines (Keychain present → ok) and Linux CI runners without
// libsecret (file fallback → warn), making golden envelopes host-dependent.
// MemStore is neither *FileStore nor a real keyring, so the doctor's
// type-switch hits the StatusOK branch.
func TestMain(m *testing.M) {
restore := doctor.SetCredStoreFactoryForTest(func() (secrets.Store, error) {
return secrets.NewMemStore(), nil
})
defer restore()
os.Exit(m.Run())
}
// update is the standard Go test golden-update flag.
// go test -update ./acceptance/contract/...
// Mirrors gh / kubectl / golang-migrate convention.
var update = flag.Bool("update", false, "update golden files")
// newTestFactory builds a Factory whose Client returns mockClient.
// Caller must NOT use t.Parallel() — see iostreams.SetForTest contract.
//
// WEKNORA_BASE_URL is set when mockServer is non-nil. v0.0 buildClient does
// not currently honor this env var (it reads from config.Host); commands that
// need the mock URL must rely on the mockClient injection above. The env
// is set anyway as a forward-affordance for any direct net/http callers
// added in PR-7+ (e.g. doctor's PingBaseURL HEAD /health).
func newTestFactory(t *testing.T, mockServer *httptest.Server, mockClient *sdk.Client) *cmdutil.Factory {
t.Helper()
t.Setenv("XDG_CONFIG_HOME", t.TempDir())
t.Setenv("XDG_CACHE_HOME", t.TempDir())
if mockServer != nil {
t.Setenv("WEKNORA_BASE_URL", mockServer.URL)
}
f := cmdutil.New()
if mockClient != nil {
f.Client = func() (*sdk.Client, error) { return mockClient, nil }
}
return f
}
// runCmd executes the root command in-process and returns captured stdout/stderr.
// Replaces iostreams.IO singleton via SetForTest (auto-restored in t.Cleanup).
//
// Mirrors cmd.Execute() carefully: callers expect the same envelope-printing
// behavior the real entrypoint provides. The helper (a) wires the cobra Out /
// Err sinks to the same buffers it returns (the `version` leaf and any future
// command using c.OutOrStdout would otherwise leak to os.Stdout), and (b)
// re-runs the error-envelope path so failure cases produce the JSON envelope
// the contract test compares against. Without (b), every error scenario's
// golden would be empty.
func runCmd(t *testing.T, f *cmdutil.Factory, args ...string) (stdout, stderr string, exitCode int) {
t.Helper()
out, errBuf := iostreams.SetForTest(t)
root := cmd.NewRootCmd(f) // exported in cli/cmd/root.go (Task 16)
root.SetArgs(args)
root.SetContext(context.Background())
root.SetOut(out)
root.SetErr(errBuf)
leaf, err := root.ExecuteC()
if err != nil {
err = cmd.MapCobraError(err)
if cmd.WantsJSONOutput(leaf) {
cmdutil.PrintErrorEnvelope(iostreams.IO.Out, err)
} else {
cmdutil.PrintError(iostreams.IO.Err, err)
}
}
return out.String(), errBuf.String(), cmdutil.ExitCode(err)
}
// assertGolden compares got against the JSON golden file at path.
// With -update, writes got to path. Normalizes _meta.request_id to "<id>"
// before compare (only field known unstable in v0.0).
//
// CRLF normalization: Windows checkouts with the default core.autocrlf=true
// turn LF in tracked text files into CRLF on disk. The command output is
// always LF, so byte-equal would fail despite identical content. .gitattributes
// is the primary defense (forcing LF on testdata/**/*.json), but we also
// strip CR here so a misconfigured contributor checkout doesn't break the
// suite locally before they push.
func assertGolden(t *testing.T, got []byte, path string) {
t.Helper()
got = normalizeEnvelope(got)
if *update {
if err := os.MkdirAll(filepath.Dir(path), 0755); err != nil {
t.Fatalf("mkdir testdata: %v", err)
}
if err := os.WriteFile(path, got, 0644); err != nil {
t.Fatalf("write golden: %v", err)
}
return
}
want, err := os.ReadFile(path)
if err != nil {
t.Fatalf("read golden %s: %v (run with -update to create)", path, err)
}
want = stripCR(want)
got = stripCR(got)
if !bytes.Equal(want, got) {
t.Errorf("envelope mismatch for %s\nwant:\n%s\ngot:\n%s", path, want, got)
}
}
// stripCR removes CR bytes so CRLF golden files (from Windows autocrlf
// checkout) compare equal to LF runtime output.
func stripCR(b []byte) []byte {
return bytes.ReplaceAll(b, []byte{'\r'}, nil)
}
// normalizeEnvelope replaces unstable fields with placeholders for stable diff.
// Currently no-op (v0.0 commands don't set _meta.request_id, so output is stable).
// Hook for future fields.
func normalizeEnvelope(b []byte) []byte {
return b
}