Files
Enterprise-Knowledge-Base/internal/im/think.go
T
qilifan 9afb8b7b68 feat(im,datasource): support Lark, Feishu's international edition
Lark and Feishu are the same product deployed on two isolated clouds
(open.larksuite.com and open.feishu.cn). WeKnora touches both from two
places — the IM bot and the Wiki data-source connector — and this brings
Lark to parity across both. Neither API surface differs, so rather than
duplicate either implementation, each is parametrized by a Region and
registered twice.

IM (internal/im/feishu):
  Every API call routes through a.api(), which prefixes the region's
  host, and the WebSocket client passes larkws.WithDomain — the SDK
  defaults to open.feishu.cn, so Lark long connections would otherwise
  fail to authenticate.

Data source (internal/datasource/connector/feishu):
  Lark was half-supported: the client honoured a base_url override and a
  LarkBaseURL constant existed, but the connector registered only as
  "feishu" and the UI never exposed base_url, so Lark wikis could not be
  configured at all. Lark is now its own connector type. base_url is kept
  as an explicit override so data sources that pointed a "feishu"
  connector at open.larksuite.com keep working.

Bugs found and fixed along the way:

- Wiki resource links were hardcoded to https://feishu.cn/wiki/{token} in
  four places, so a Lark wiki linked into the wrong cloud entirely.
- Bot identity is now platform-prefixed (feishu:cli_x vs lark:cli_x).
  BotIdentity carries a unique index, so without this a Lark channel
  would be rejected as a duplicate of an unrelated Feishu bot sharing an
  app_id.
- The card image cache is now keyed by app_id. image_keys are issued per
  app on one cloud; a Feishu key handed to a Lark app fails the card
  update with code=200570.
- The datasource doc claimed Lark "adapts automatically" — it could not,
  since the UI had no way to reach base_url.

Permissions are where the clouds genuinely diverge. Identical APIs do not
mean identical permission catalogues: aily:file:read, aily:file:write and
corehr:file:download exist on Feishu but not on Lark, so pasting Feishu's
scope JSON into Lark fails to import. That JSON is also over-broad (it
covers products WeKnora never calls) and simultaneously incomplete (it
omits the drive:*/docx:* scopes the connector needs). Lark therefore gets
its own list, derived from the 17 endpoints the code actually calls and
checked identifier-by-identifier against Lark's own API reference, plus a
full Feishu→Lark mapping table marking each scope verified, unverified,
or absent. Both features use tenant_access_token only — no user_access_
token exists anywhere in the repo — so the user dimension stays empty.

Known limitation: Region.ThinkingText makes Lark's initial card
placeholder English, but once streaming starts the think-block headers
come from the shared MarkdownThinkStyle and are Chinese on every platform
(Lark, Slack, Telegram, Mattermost alike). Fixing that means threading a
locale through FormatIMDisplayContent, which has no platform context — a
cross-platform change, not a Lark one. Documented as such.

The channel-creation 400 message is now derived from validIMPlatforms so
the list cannot drift as platforms are added.

Tests: region invariants and per-region URL construction on both sides,
the Feishu/Lark bot-identity split, platform-to-channel mapping, base_url
override precedence, an end-to-end SendReply against a fake Open Platform
(token fetch, reply, thread-unsupported fallback, hard-error surfacing),
and a ListResources drive asserting wiki links never leak the other
cloud's host.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-17 11:05:54 +08:00

276 lines
8.7 KiB
Go

package im
import (
"regexp"
"strings"
)
// StreamDisplayPhase controls how raw IM stream content is rendered.
type StreamDisplayPhase int
const (
// StreamDisplayIntermediate shows in-progress thinking/tools (Web: expanded progress).
StreamDisplayIntermediate StreamDisplayPhase = iota
// StreamDisplayFinal shows answer-only text (Web: collapsed/hidden intermediate steps).
StreamDisplayFinal
)
var (
thinkBlockRe = regexp.MustCompile(`(?s)<think>.*?</think>`)
thinkOpenTailRe = regexp.MustCompile(`(?s)<think>.*$`)
)
// ragPipelineToolNames mirrors frontend RAG_PIPELINE_TOOL_NAMES.
var ragPipelineToolNames = map[string]bool{
"query_understand": true,
"knowledge_search": true,
}
// IsRAGPipelineToolName reports whether a tool is part of the quick-QA progress UI.
func IsRAGPipelineToolName(name string) bool {
return ragPipelineToolNames[name]
}
// StripThinkBlocks removes <think> blocks for final IM display.
func StripThinkBlocks(content string) string {
if content == "" {
return ""
}
cleaned := thinkBlockRe.ReplaceAllString(content, "")
cleaned = thinkOpenTailRe.ReplaceAllString(cleaned, "")
return trimIMOuterWhitespace(cleaned)
}
// IMStreamMode distinguishes agent reasoning from quick-QA RAG pipeline display.
type IMStreamMode int
const (
IMStreamModeAgent IMStreamMode = iota
IMStreamModeQuickQA
)
// IMStreamParts separates stream content for IM display.
type IMStreamParts struct {
Mode IMStreamMode
// Quick-QA (RagPipelineProgress on Web):
PipelineToolSteps []IMToolStep // query_understand / knowledge_search rows
ReasoningInner string // model reasoning_content — separate "思考" section
// Agent (AgentStreamDisplay on Web):
AgentInner string // retracted preambles + thoughts ("思考过程")
AgentToolSteps []IMToolStep // tool progress lines (one row per tool_call_id)
LiveAnswer string // optimistic answer stream before tools retract it
Answer string // final answer (complete / knowledge QA)
}
func agentThinkContent(parts IMStreamParts) string {
toolLines := renderIMToolSteps(parts.AgentToolSteps, FormatIMToolLine)
return mergeIMNarrativeAndTools(parts.AgentInner, toolLines)
}
func quickQAPipelineContent(parts IMStreamParts) string {
return renderIMToolSteps(parts.PipelineToolSteps, FormatIMRagPipelineLine)
}
// RAGThinkingStyle matches Web RagPipelineProgress thinking row (agent.think).
var RAGThinkingStyle = ThinkBlockStyle{
ThinkingHeader: "> 💭 **思考中...**\n",
ThoughtHeader: "> 💭 **思考**\n",
LinePrefix: "> ",
LineSuffix: "",
Separator: "\n---\n\n",
}
// WrapThinkBlock wraps inner content with think tags for formatting.
func WrapThinkBlock(inner string) string {
inner = strings.TrimSpace(inner)
if inner == "" {
return ""
}
return "<think>\n" + inner + "\n</think>"
}
// BuildIMAgentStreamRaw assembles agent-mode raw content.
func BuildIMAgentStreamRaw(parts IMStreamParts, agentInProgress bool) string {
thinkTagged := WrapThinkBlock(agentThinkContent(parts))
if agentInProgress || strings.TrimSpace(parts.Answer) == "" {
return thinkTagged
}
if thinkTagged == "" {
return parts.Answer
}
return thinkTagged + "\n\n" + parts.Answer
}
// formatIMAgentIntermediate mirrors Web AgentStreamDisplay:
// stream answer text first; when tools run, retract into "思考过程";
// once tools exist, keep the think block visible above the streaming answer.
func formatIMAgentIntermediate(parts IMStreamParts) string {
think := strings.TrimSpace(agentThinkContent(parts))
live := strings.TrimSpace(parts.LiveAnswer)
var sections []string
if think != "" {
sections = append(sections, FormatIMDisplayContent(WrapThinkBlock(think), StreamDisplayIntermediate))
}
if live != "" {
sections = append(sections, live)
}
return strings.Join(sections, MarkdownThinkStyle.Separator)
}
// formatIMQuickQAIntermediate mirrors Web RagPipelineProgress:
// pipeline steps as plain lines, model reasoning in a separate "思考" block,
// and once answer starts streaming the progress collapses to answer-only preview.
func formatIMQuickQAIntermediate(parts IMStreamParts) string {
if answer := strings.TrimSpace(parts.Answer); answer != "" {
return answer
}
var sections []string
if p := strings.TrimSpace(quickQAPipelineContent(parts)); p != "" {
sections = append(sections, p)
}
if r := strings.TrimSpace(parts.ReasoningInner); r != "" {
sections = append(sections, TransformThinkBlocks(WrapThinkBlock(r), RAGThinkingStyle))
}
return strings.Join(sections, "\n\n")
}
// FormatIMIntermediateFromParts formats in-progress IM stream display.
func FormatIMIntermediateFromParts(parts IMStreamParts, agentInProgress bool) string {
if parts.Mode == IMStreamModeQuickQA {
return formatIMQuickQAIntermediate(parts)
}
_ = agentInProgress
return formatIMAgentIntermediate(parts)
}
// FormatIMFinalFromParts returns the final replace frame (answer-only for all modes).
func FormatIMFinalFromParts(parts IMStreamParts) string {
answer := strings.TrimSpace(parts.Answer)
if answer != "" {
return answer
}
// Fallback for embedded think tags in non-agent answers.
return FormatIMDisplayContent(BuildIMAgentStreamRaw(parts, false), StreamDisplayFinal)
}
// FormatIMDisplayContent formats raw stream content for the given display phase.
func FormatIMDisplayContent(raw string, phase StreamDisplayPhase) string {
switch phase {
case StreamDisplayFinal:
return StripThinkBlocks(raw)
default:
return TransformThinkBlocks(raw, MarkdownThinkStyle)
}
}
func trimIMOuterWhitespace(s string) string {
return strings.TrimSpace(s)
}
// ThinkBlockStyle controls how <think>...</think> blocks are rendered.
type ThinkBlockStyle struct {
// ThinkingHeader is shown when the think block is still in-progress (no closing tag yet).
ThinkingHeader string // e.g. "> 💭 **思考中...**\n" or "💭 _思考中..._\n"
// ThoughtHeader is shown before the think block content.
ThoughtHeader string // e.g. "> 💭 **思考过程**\n" or "💭 *思考过程*\n"
// LinePrefix is prepended to each line of think content.
LinePrefix string // e.g. "> " or "> _"
// LineSuffix is appended to each line of think content (before newline).
LineSuffix string // e.g. "" or "_"
// Separator is inserted between the think block and the rest of the content.
Separator string // e.g. "\n---\n\n" or "\n"
}
// MarkdownThinkStyle renders think blocks as markdown blockquotes.
// Used by DingTalk and Feishu/Lark.
//
// Known limitation: the headers below are Chinese for every platform, including
// ones whose users are typically not Chinese speakers (Lark, Slack, Telegram,
// Mattermost). Localizing them means threading a locale into
// FormatIMDisplayContent, which is called from the shared streaming path with no
// platform context — a cross-platform change rather than a per-platform patch.
var MarkdownThinkStyle = ThinkBlockStyle{
ThinkingHeader: "> 💭 **思考中...**\n",
ThoughtHeader: "> 💭 **思考过程**\n",
LinePrefix: "> ",
LineSuffix: "",
Separator: "\n---\n\n",
}
// TelegramThinkStyle renders think blocks as blockquotes for Telegram.
// Uses the same blockquote format as other platforms for reliable rendering
// during streaming (where incomplete markdown can cause API failures).
var TelegramThinkStyle = ThinkBlockStyle{
ThinkingHeader: "> 💭 *思考中...*\n",
ThoughtHeader: "> 💭 *思考过程*\n",
LinePrefix: "> ",
LineSuffix: "",
Separator: "\n---\n\n",
}
// TransformThinkBlocks converts <think>...</think> blocks using the given style.
// Handles both complete blocks and in-progress blocks (where </think> has not
// yet arrived during streaming).
func TransformThinkBlocks(content string, style ThinkBlockStyle) string {
const (
openTag = "<think>"
closeTag = "</think>"
)
openIdx := strings.Index(content, openTag)
if openIdx < 0 {
return content
}
before := content[:openIdx]
after := content[openIdx+len(openTag):]
closeIdx := strings.Index(after, closeTag)
thinkClosed := closeIdx >= 0
var thinkContent, rest string
if thinkClosed {
thinkContent = after[:closeIdx]
rest = after[closeIdx+len(closeTag):]
} else {
thinkContent = after
}
thinkContent = strings.TrimSpace(thinkContent)
var result strings.Builder
result.WriteString(before)
if thinkContent == "" {
if !thinkClosed {
result.WriteString(style.ThinkingHeader)
return result.String()
}
result.WriteString(strings.TrimLeft(rest, "\n"))
return result.String()
}
result.WriteString(style.ThoughtHeader)
for _, line := range strings.Split(thinkContent, "\n") {
result.WriteString(style.LinePrefix)
result.WriteString(line)
result.WriteString(style.LineSuffix)
result.WriteString("\n")
}
if thinkClosed {
rest = strings.TrimLeft(rest, "\n")
if rest != "" {
result.WriteString(style.Separator)
result.WriteString(rest)
}
}
return result.String()
}