Weknora/Enterprise-Knowledge-Base
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
2,124 Commits 8 Branches 2 Tags
74c845a1f48e32d4568b1b5d7c1925e4b0e6f8d1
Commit Graph

7 Commits

Author SHA1 Message Date
nullkey 2ce348d020 feat(cli): --format json default + NDJSON event stream + context→profile cascade + help calibration + docs (BREAKING) 
D1 — --format default flipped to json regardless of TTY:
- v0.6: smart default (text on TTY, json on pipe).
- v0.7: always json; TTY only affects indent (compact in pipe). Enum
  values unchanged (text | json | ndjson).
- Typed FormatMode enum replaces untyped string consts.
- --format / --jq promoted to persistent root flags so unknown-
  subcommand paths still reach the typed-envelope guard (per-command
  registration in v0.6 would have rejected --format on unknown
  commands as cobra-prose exit 2).
- WEKNORA_FORMAT env var added; precedence --format > env > default.
  Invalid env values silently ignored.

D2 — chat / session ask default to NDJSON event-stream:
- New cli/internal/output/ndjson_stream.go: InitEvent struct +
  EmitInit / EmitSDKEvent / WriteNDJSONLine helpers. EmitInit doc
  encodes the must-be-first-line invariant agents key on.
- chat / session ask: --format json AND --format ndjson both emit one
  JSON event per line (no envelope wrapping). CLI injects exactly one
  `init` event at stream head carrying session_id + optional kb_id /
  agent_id / profile. Subsequent events pass through verbatim from the
  SDK (passthrough discipline per spec §5.1).
- --format text keeps the SSE-style live renderer.

context → profile full cascade:
- Command group: cli/cmd/context/ → cli/cmd/profile/ (git mv;
  package contextcmd → profilecmd).
- Global flag --context → --profile. Factory.ContextOverride →
  ProfileOverride. WEKNORA_PROFILE env var honored
  (--profile flag > env > config.CurrentContext). When --profile or
  WEKNORA_PROFILE references a missing profile, the error is
  input.invalid_argument with hint "weknora profile list" — not the
  destructive local.config_corrupt path (which would have told users
  to delete their config file).
- Binding file .weknora/project.yaml field context: → profile:
  (no backwards-compat alias; re-run weknora link).
- profile use JSON fields current_context / previous_context →
  current_profile / previous_profile.
- weknora link JSON field context → profile.
- CodeLocalContextNotFound → CodeLocalProfileNotFound (typed code
  rename).
- Envelope top-level profile field populated via globalProfile (set
  by root PersistentPreRunE from Factory.ActiveProfile). chat /
  session ask NDJSON init event carries the same profile.
- Rationale: "context" collided with LLM context window / RAG context
  / Go context.Context; mainstream multi-credential CLIs (AWS /
  Stripe / OpenAI / Anthropic / lark) all use "profile".

H2/C1' help calibration:
- AgentHelp gains Warnings []string; single SetAgentHelp helper
  routes on WEKNORA_AGENT_HELP=1 (emits JSON blob including
  warnings) vs human help (appends "AI agents:" block from same
  source). Warnings surface as both a structured JSON field and
  visible help-text addendum without drift.
- 9 destructive commands carry warnings: kb / doc / agent / session /
  chunk delete; profile remove; kb / agent edit; auth logout.
- weknora doc wait dedups ids at entry; SIGINT mid-wait returns
  silently (root signal handler maps to exit 130) instead of being
  miscategorised as operation.timeout / operation.failed.

A4 — docs:
- cli/AGENTS.md gains four agent-facing sections: Wire contract for
  AI agents (stdout / stderr / NDJSON / _notice evolution / SDK
  contract boundary); Deliberate deviations + mainstream alignments;
  Pre-1.0 breaking policy; Exit-10 anti-patterns. ERROR_REFERENCE
  table extended.
- cli/README.md adds Agent quick start under Wire contract.
- cli/CHANGELOG.md v0.7 section: BREAKING entries with migration
  notes, Added (WEKNORA_FORMAT / WEKNORA_PROFILE / retry_command /
  retry_after_seconds / risk / _notice reserved infra / meta.count /
  meta.has_more / doc fetch / doc create / session ask / doc delete
  --all / NDJSON init), Changed (docs additions), Deprecated (none —
  pre-release one-shot breaking).

Spec: docs/superpowers/specs/2026-05-20-weknora-cli-v0.7-design.md §3 / §4 / §5 / §6 / §11
 2026-05-27 10:56:34 +08:00
nullkey 567d7ac74e feat(cli): --format / NDJSON / chat & agent invoke / MCP / SetAgentHelp / signal-aware ctx / kb create --storage-provider 
Adds the structured-output and agent-help surface plus root-level signal
handling so AI agents (and humans working through pipes) get a stable
wire contract.

* --format text|json|ndjson flag, registered per-command on outputs that
  need it; default text on TTY, json on pipe. --jq <expr> pairs with json
  / ndjson to filter or project. FormatOptions absorbs JQ; WantsJSON()
  helper for the JSON dispatch.
* WriteNDJSON helper in internal/format/ (per ndjson.org: one JSON value
  per line, arrays split element-per-line, empty slice → zero bytes).
* chat / agent invoke wire --format ndjson via SDK StreamResponse /
  AgentStreamResponse 1:1 passthrough. Both commands detect ctx.Cancelled
  in every stream + session-create path and emit a stable
  "operation.cancelled" code on Ctrl-C / SIGTERM.
* main.go wires signal.NotifyContext(SIGINT, SIGTERM) into the root
  context so long-running commands run their cancellation cleanup
  (re-emit auto-created session id, etc); the process exits 130 when
  the context was signal-cancelled, matching Unix convention.
* MCP chat / agent_invoke output schemas extended with thinking /
  tool_calls / assistant_message_id (server-side accumulated; MCP
  tools/call has no standard partial-response). doc_view and doc_download
  now use doc_id (not knowledge_id) so agents see a single id naming
  convention across all tools — matches the chunk_list / search_chunks
  schemas and the CLI's <doc-id> positional.
* SetAgentHelp(cmd, AgentHelp{...}) — opt-in machine-friendly --help
  payload activated by WEKNORA_AGENT_HELP=1. Applied to chat / kb list.
* kb create --storage-provider <local|minio|cos|tos|s3|oss|ks3> — sets
  the new KB's storage_provider_config.provider at creation time (server
  does not expose it on update). Required on self-hosted deployments
  where the server-side default doesn't pre-populate a provider —
  without it, subsequent doc upload returns a misleading "kb not found".
 2026-05-18 11:10:19 +08:00
nullkey e623e8208f refactor(cli): delete envelope infrastructure, errors to stderr 
Removes the entire envelope machinery now that every success path
emits bare JSON:

- cli/internal/format/envelope.go (Envelope, Success, Failure,
  SuccessWithRisk, WriteEnvelope, Meta, Notice, UpdateNotice,
  VersionSkewNotice, Risk, RiskLevel, ErrorBody) + tests.
- cli/internal/format/filter.go envelope-specific helpers
  (WriteEnvelopeFiltered, marshalEnvelope, applyFieldFilter,
  filterDataPayload, filterObjectData); the reusable
  filterArrayItems / filterObjectKeys / writeJQ stay for bare.go.
- cli/internal/cmdutil/exporter.go + tests (envelope-only).
- cli/internal/cmdutil/PrintErrorEnvelope + ToErrorBody +
  operationRiskOf + Error.OperationRisk field + OperationRisk struct.

Error path: all errors now go to stderr via cmdutil.PrintError in
`code: message\nhint: ...` form, regardless of --json. Stdout stays
empty (or holds the partial-success the command already wrote) so
downstream `--json | jq` pipelines never have to filter error shapes
out of the success stream. Typed exit codes (3 auth.* / 4
resource.not_found / 5 input.* / 6 server.rate_limited / 7 server.*
+ network.* / 10 input.confirmation_required) carry the failure
class for agents that branch on it.

Acceptance contract:
- envelope_test.go → wire_test.go (TestEnvelopeGolden → TestWireGolden).
- testdata/envelopes/ → testdata/wire/.
- Error-path cases assert the typed code substring on stderr.
- Orphan whoami.*.json goldens deleted.

AGENTS.md + README.md rewritten for the bare-data contract:
- Drop envelope schema section + dry-run rule.
- Document bare JSON on stdout + `code: msg\nhint: …` on stderr.
- ADR-3 reframed around bare data and why error separation matters
  for `--json | jq` pipelines.

WriteJSONFiltered short-circuits to WriteJSON when both filters are
empty (skip the marshal-buffer round-trip for the common case).

Final review pass:
- Fix wire-contract bug: `--json id,name` (space form) is broken by
  pflag's NoOptDefVal; AGENTS.md / README.md / SetAgentHelp + the
  field-discovery help text all switched to `--json=id,name`.
- Fix `weknora api --jq` silently ignored: api.go now routes through
  WriteJSONFiltered with jopts.JQ.
- AGENTS.md: drop the false claim that `auth logout` honors `-y`
  (logout is local-only with no ConfirmDestructive guard); list the
  actual destructive commands instead.
- Rewrite cli/acceptance/e2e/e2e_test.go for the bare-data wire shape
  (was still parsing `out["data"]` / `env["ok"]`).
- Add `JSONOptions.Emit(w, v)` helper; collapse ~33 repeated
  `format.WriteJSONFiltered(iostreams.IO.Out, X, jopts.Fields,
  jopts.JQ)` sites to `jopts.Emit(iostreams.IO.Out, X)` — drops the
  format import from 22 cmd/* files.
- Delete single-caller `cmdutil.MustRequireFlag`; inline as
  `_ = cmd.MarkFlagRequired(...)` everywhere.
- Add `_ = cmd.MarkFlagRequired("name")` to `kb create`; it was the
  only write command relying on runtime --name validation while
  `context add` already used the cobra-level mark.
- `context use`: register `--json` / `--jq` (was always emitting JSON
  unconditionally with no human path and no flag — diverged from
  every other write command); human mode now prints
  `✓ Switched context to X (was Y)`.
- Replace per-package `confirmPrompter` / `scriptedConfirm` /
  `errPrompter` test doubles with `testutil.ConfirmPrompter`.
- Rename `chatService` → `ChatService` (export to match siblings
  `ListService` / `ViewService`); rename `printUploadSuccess` →
  `renderUploadSuccess` (siblings use `render*`).
- `defaultHint(CodeResourceNotFound)`: drop the hardcoded
  "list available with `weknora kb list`" — misleading on agent /
  doc / session 404. Replaced with "verify the resource ID and try
  again".
- Strip stale `v0.2/v0.3` / "envelope" / "v0.0/v0.1 supports only"
  historical tags from production comments and a few test
  descriptions.
 2026-05-15 12:03:56 +08:00
nullkey cc8254f862 refactor(cli): drop --dry-run + introduce bare-JSON output path 
Two intertwined mainstream-alignment moves bundled because they share
the migration target (every command's --json path):

1. Drop --dry-run entirely. Survey of comparable API-wrapper CLIs
   (gh, aws, stripe, lark): none expose --dry-run. The mainstream that
   does (kubectl/git/helm/ansible) operates on declarative manifests
   or local state where the preview is materially different from the
   executed action. WeKnora's CLI just echoed the same parameters
   that would have gone on the wire — the preview added no real
   signal over `--help` + reading the call site. Removes:
   - root --dry-run persistent flag + cmdutil/dryrun.go
   - DryRun fields + EmitDryRun calls in 12 write commands
   - format.Envelope.DryRun field
   - 8 corresponding *_test.go cases
   - --dry-run mention from README.md and CHANGELOG.md
   - "dry_run":false from 16 golden envelopes

2. Migrate every --json output to bare data:
   - New format.WriteJSON / WriteJSONFiltered helpers
     (cli/internal/format/bare.go) share filterArrayItems /
     filterObjectKeys / writeJQ with the (still-live for now) envelope
     filter helpers.
   - Read commands (kb/doc/session list+view, search chunks/docs/
     sessions/kb, auth list/status, agent list/view, context list,
     doctor) emit bare arrays / objects on stdout.
   - Write commands (kb create/edit/delete/pin/empty, doc upload/
     upload_recursive/delete, session delete, auth login/logout/
     refresh/token, link/unlink, context add/use/remove, agent
     invoke, chat, api, version) emit bare result objects. Risk
     classification dropped — the resource + exit code already
     convey the action.

Per-command shape changes:
   list / search       → []T   (was {ok, data:{items:[…]}})
   view                → T     (was {ok, data:T, _meta:…})
   create / edit       → T
   delete / pin / etc. → {id, …action result…}
   doctor              → {summary, checks}
   api                 → {status, headers, body}

_meta dropped on the read path:
   pagination (page/page_size/total/has_more) — agents iterate with
   --all-pages or accept --limit (gh CLI parity);
   kb_id / context echo — caller already knows what it asked for.

Acceptance contract goldens regenerated for the new bare shape.
Error envelope on stdout (PrintErrorEnvelope) stays live for now —
the envelope-infra deletion lands in the next commit.
 2026-05-15 12:03:56 +08:00
nullkey 1b20b06f5e feat(cli): --json field-select, --jq, auth token, doc --from-url 
Output ergonomics:
- `--json` accepts a comma-separated field list (gh-parity); selects
  named keys from the per-command payload. Bare `--json` keeps the
  full shape.
- `--jq <expr>` evaluates a gojq expression over the JSON; pairs with
  `--json field-list` so projection runs before jq.
- `--version` is a global cobra flag in addition to the `version`
  subcommand; both render the same line.
- Per-command `--help` now renders the available JSON field list under
  "JSON fields available via `--json id,name,...`" (field-discovery
  parity with gh / kubectl `-o jsonpath`).

New commands:
- `auth token` — print the active context's credential to stdout for
  shell command substitution (`WEKNORA_TOKEN=$(weknora auth token)`).
  Default: raw secret, no trailing newline. `--json` emits
  `{token, mode, context}`.
- `doc upload --from-url <URL>` — ingest a remote URL via the SDK
  `CreateKnowledgeFromURL`. `--name` forwarded as `FileName` so the
  server's known-extension heuristic upgrades crawl-mode to
  file-download-mode where appropriate.

Includes the simplify post-review polish pass (field-filter unit
tests, --json/--jq compose check, agent_help copy fixes).
 2026-05-15 12:03:56 +08:00
nullkey e236be1ced fix(cli): correct KB id detection, SSE terminal-frame, and CI test isolation 
Three defects surfaced during end-to-end RAG verification — the first two
block real chat usage, the third makes Linux CI flaky:

1. KB id detection — `IsKBID` was checking
   `strings.HasPrefix(s, "kb_")`, but WeKnora generates KB ids as bare
   UUIDs (internal/types/knowledge_base.go: `uuid.New().String()` stored
   in a `varchar(36)` column). Real ids therefore fell through to the
   name-resolution path:

     $ weknora chat ... --kb a32a63ff-fb36-4874-bcaa-30f48570a694
     Error: knowledge base not found: a32a63ff-...

   Switched the discriminator to a UUID regex
   (`^[0-9a-fA-F]{8}-…-[0-9a-fA-F]{12}$`). KB names are arbitrary
   user-supplied strings, so the canonical 8-4-4-4-12 form is an
   unambiguous signal. Mirrors gcloud `--project`'s id-vs-name detection.

2. SSE terminal-frame — the accumulator's `Append` was gating
   finalization on `r.Done`, but the server's KnowledgeQAStream protocol
   emits a leading `agent_query` frame with `done=true` to deliver
   session + message metadata *before* the answer fragments arrive:

     event: message
     data: {"response_type":"agent_query","content":"","done":true,…}

     event: message
     data: {"response_type":"answer","content":"你好","done":false}
     …
     event: message
     data: {"response_type":"complete","content":"","done":true}

   The accumulator therefore flipped to `finished=true` on frame #1 and
   discarded every subsequent answer fragment — `weknora chat … --json`
   returned `answer: ""` even though the LLM reported completion_tokens
   > 0. Fixed: terminate only on `response_type == complete`.
   References still captured opportunistically (they may arrive on a
   dedicated `references` event before the terminator).

3. doctor credential_storage CI isolation — the check probes the real
   OS keyring via `secrets.NewBestEffortStore()`: present on macOS dev
   machines → StatusOK; absent on Linux CI runners without libsecret /
   Gnome-Keyring → StatusWarn ("falling back to file store"). That
   host-dependence was leaking into two test classes that assumed
   StatusOK:

     * cmd/doctor/doctor_test.go: TestDoctor_AllOK and
       TestDoctor_NoConfig_StillRunsCredentialStorage already had a
       withCredStoreFactory seam but didn't use it. Added the pin.

     * acceptance/contract/envelope_test.go: doctor.success_offline
       and doctor.error_network golden cases. The contract test runs
       through the cobra tree in-process and shares cmd/doctor's
       package-level credStoreFactory var — but couldn't reach it
       because the existing seam was unexported.

   Fix: export `doctor.SetCredStoreFactoryForTest(fn) (restore func())`
   for out-of-package tests; acceptance/contract/helpers_test.go adds
   a TestMain that pins the factory to a MemStore-returning closure
   for the whole suite (MemStore is neither *FileStore nor a real
   keyring, so doctor's type-switch hits StatusOK). Production stays
   at secrets.NewBestEffortStore — only the test hook is now reachable
   from across packages.

Test fixtures and goldens that used the old `kb_xxx` literals or
`Done: true` terminators were rewritten to use real UUIDs and
`ResponseType: ResponseTypeComplete` respectively. Per-command --help
text and Long descriptions / Examples now show a UUID rather than
`kb_…` so users see the correct shape from the start. New
TestAccumulator_IgnoresAgentQueryDone pins the SSE terminator bug so
it can't regress.

Tests: 24 cli packages green on macOS dev + Linux/macOS/Windows CI
matrix. Verified end-to-end against a live WeKnora server: `weknora
chat "..." --kb <UUID> --no-stream --json` returns the full LLM answer
in the envelope, live token streaming in TTY mode works, and the
credential_storage check renders deterministic envelopes across hosts.
 2026-05-12 13:20:42 +08:00
nullkey 3fb3583a92 feat(cli): add api passthrough, chat streaming, doctor warn status 
Close the v0.2 RAG demo loop and ship the validation infrastructure:

  - weknora api <method> <path> [--data X | --data-file F]
      Raw passthrough wrapping client.Raw, gh-style. JSON envelope mode
      surfaces status / headers / parsed body. Non-2xx routes through
      cmdutil.ClassifyHTTPStatus (factored out of ClassifyHTTPError so
      both SDK-error and direct-status paths stay aligned — reuse review).

  - weknora chat <text> [--session-id S] [--no-stream]
      KnowledgeQAStream consumer with two output modes:
        - TTY default: token streaming + references footer
        - --json / --no-stream / non-TTY: buffered single envelope
      Auto-creates a session when --session-id is omitted; the id prints
      to stderr at start AND on stream failure (^C scrolls past the
      first announcement, so the recovery hint is re-surfaced when the
      user is most likely to need it).

  - cli/internal/sse/Accumulator
      buffers Content / References / SessionID across SDK callbacks.
      Idempotent post-Done so misbehaving servers don't corrupt state.

  - doctor: ok → ok / warn / fail / skip
      warn marks soft issues that don't block: server within compat range
      but >=1 minor behind CLI; credential storage falling back to file
      because keyring is unavailable. Envelope.ok stays true on warn,
      flips false on fail (exit 1). doctor.error_network golden updated.

  - cli/acceptance/e2e/    real-server RAG full loop
      Build-tagged //go:build acceptance_e2e — kept out of the default
      `go test ./...`. Exercises kb create → doc upload → poll ready →
      search → chat → cleanup against a server pointed at by
      WEKNORA_E2E_HOST / _TOKEN.

  - .github/workflows/cli-e2e.yml
      manual workflow_dispatch + label-gated PR trigger
      ("acceptance-e2e"). No-ops gracefully when the secrets aren't set
      so cross-fork PRs can't accidentally fail the suite.
 2026-05-12 13:20:42 +08:00