feat(observability): extend Langfuse tracing across asynq pipeline
Previously the Langfuse integration only traced in-process HTTP requests (chat / search / eval), so file uploads and every downstream asynq task (document parse, chunk embedding, OCR/VLM, summary / question gen, wiki ingest, datasource sync, etc.) produced either disconnected shallow traces or no observation at all. This change threads one trace end-to-end: - tracer: add SPAN observation type and StartSpan; add ResumeTrace so a worker can attach to an upstream trace without emitting a duplicate trace-create; StartGeneration now auto-picks parentObservationId from ctx so nested trace -> span -> generation trees render correctly. - types.TracingContext + LangfuseTracingCarrier: embed on all 17 asynq payloads so trace_id / parent_obs_id / user_id / session_id serialise into every job. - langfuse.InjectTracing: injected at 28 enqueue sites before json.Marshal so the HTTP-layer trace survives the Redis hop. - langfuse.AsynqMiddleware: mux.Use hook that peeks the payload, either resumes the upstream trace or opens a standalone asynq.<type> trace for scheduled jobs, and wraps the handler in a SPAN with task metadata (id / queue / retry / payload_bytes) plus ERROR level on failure. - GinMiddleware.shouldTrace: whitelist ingestion / knowledge-mutation / FAQ / wiki / datasource endpoints so the root trace actually starts. - Tests: tracer_test.go covers span nesting, error status, and ResumeTrace no-trace-create guarantee; asynq_test.go covers InjectTracing round-trip, middleware resume path, and standalone trace fallback. - Docs: docs/Langfuse\u96c6\u6210.md now lists the covered task types and documents the cross-process propagation model. No behavioural change when Langfuse is disabled (all new code paths are no-ops and carriers serialise to empty strings with omitempty).
This commit is contained in:
+32
-7
@@ -5,8 +5,9 @@ WeKnora 内置了对 [Langfuse](https://langfuse.com) 的轻量级集成,用
|
||||
## 1. 特性
|
||||
|
||||
- 自动上报 **chat / embedding / rerank / VLM(视觉语言模型)/ ASR(语音识别)** 全部 5 类模型调用的 prompt、响应和 token 使用量。
|
||||
- 为每个对话或检索请求创建一条 **trace**,对应 HTTP 请求级别,方便按 session / user 聚合。
|
||||
- 为每个对话、检索、**文件上传及后续异步处理**创建一条端到端 **trace**。HTTP 请求是根,asynq 任务以 SPAN 的形式挂在同一条 trace 下,文档解析 → chunk embedding → 多模态 OCR/Caption → 摘要 / 问题生成全部在同一棵树里可见。
|
||||
- 支持 **流式响应**:记录首 token 延迟(Time-To-First-Token),完整响应在流结束后一次性写入。
|
||||
- **跨进程 trace 透传**:HTTP 层把 `trace_id` / `parent_observation_id` 注入 asynq payload,worker 在 asynq middleware 层自动 resume;定时任务(例如数据源同步)则退化为独立 trace,依然按任务类型(`asynq.<type>`)聚合。
|
||||
- **完全可选**:不配置 `LANGFUSE_*` 环境变量时,Langfuse 相关代码路径是 no-op,不产生任何性能开销。
|
||||
- **异步批量上报**:不阻塞业务请求;队列满时静默丢弃,观测数据不会影响用户对话。
|
||||
- **开箱即用的部署方式**:Docker Compose(`docker-compose.yml` 已内置环境变量)、Helm Chart(通过 `extraEnv`)、Lite 版本(本地单机)均支持。
|
||||
@@ -226,15 +227,34 @@ Dev 相关容器都带 `-dev` 后缀、用独立网络 `WeKnora-network-dev`,
|
||||
|
||||
| Langfuse 概念 | WeKnora 对应 | 备注 |
|
||||
| --- | --- | --- |
|
||||
| Trace | 一次 HTTP 请求 | 自动在 `/api/v1/knowledge-chat`、`/api/v1/agent-chat`、`/api/v1/knowledge-search`、`generate_title`、模型连通性测试等端点创建。 |
|
||||
| Generation(type=GENERATION) | 每次 chat / embedding / rerank / VLM / ASR 调用 | 包含 model、prompt、响应、token usage、modelParameters(temperature / top_p / max_tokens 等)。 |
|
||||
| Trace | 一次 HTTP 请求(含其触发的所有 asynq 任务) | 对于 `knowledge-chat`、`agent-chat`、`knowledge-search`、`generate_title`、`evaluation`、模型连通性测试等在线请求;以及文件上传/URL 入库/manual/reparse/move/copy、FAQ 导入、知识修改、wiki auto-fix、数据源手工触发等入库请求,HTTP 层都会开启 trace,并把 `trace_id` / `parent_observation_id` 注入 asynq payload。 |
|
||||
| Span(type=SPAN) | 每个 asynq 任务的执行窗口 | 由 `internal/tracing/langfuse/AsynqMiddleware` 在 `mux.Use` 注册;对每个 handler 自动创建 `asynq.<task_type>` 的 SPAN,并记录 `task_id` / `queue` / `retry` / `payload_bytes`。定时任务(无上游 trace)会退化为 `asynq.<task_type>` 独立 trace。 |
|
||||
| Generation(type=GENERATION) | 每次 chat / embedding / rerank / VLM / ASR 调用 | 若位于 span 下会自动设置 `parentObservationId`,所以 Langfuse UI 呈现 trace → asynq-span → generation 的树状结构。 |
|
||||
| Input Tokens | `TokenUsage.PromptTokens` | 来自模型返回的 usage 字段。 |
|
||||
| Output Tokens | `TokenUsage.CompletionTokens` | 来自模型返回的 usage 字段。 |
|
||||
| Total Tokens | `TokenUsage.TotalTokens` | 大多数厂商返回;未返回时自动求和。 |
|
||||
| `userId` | `X-User-ID` / 租户 ID | 未登录时退化为 `tenant:<id>`,方便按租户汇总消耗。 |
|
||||
| `sessionId` | URL 中的 `:session_id` | 可以在 Langfuse 的 Sessions 视图聚合一整场对话。 |
|
||||
| `userId` | `X-User-ID` / 租户 ID | 未登录时退化为 `tenant:<id>`,方便按租户汇总消耗;enqueue 时会写入 payload,worker 在无上游 trace 的场景也能保留归属。 |
|
||||
| `sessionId` | URL 中的 `:session_id`(或 `RequestID` 兜底) | 可以在 Langfuse 的 Sessions 视图聚合一整场对话,或按单次异步批次聚合。 |
|
||||
| Time-To-First-Token | 流式调用首条有效 chunk 的时间 | 通过 `generation-update.completionStartTime` 上报。 |
|
||||
|
||||
### 覆盖到的 asynq 任务类型
|
||||
|
||||
下表列出当前会在 Langfuse 里自动出现对应 SPAN 的 asynq 任务;每种任务的 payload 均已嵌入 `types.TracingContext`,enqueue 时由 `langfuse.InjectTracing(ctx, &payload)` 从当前 HTTP trace 拷出 `trace_id` / `parent_observation_id`。
|
||||
|
||||
| 任务类型常量 | Handler | 典型触发来源 |
|
||||
| --- | --- | --- |
|
||||
| `document:process` | `knowledgeService.ProcessDocument` | 文件 / URL / 文本 / file_url 四种入库;reparse;知识库克隆内部重派发 |
|
||||
| `manual:process` | `knowledgeService.ProcessManualKnowledge` | 手工知识新建 / 更新 |
|
||||
| `image:multimodal` | `ImageMultimodalService.Handle` | 文档解析时发现图片 |
|
||||
| `knowledge:post_process` | `KnowledgePostProcessService.Handle` | 文档解析完成后统一调度 summary/question |
|
||||
| `summary:generation` / `question:generation` | `KnowledgePostProcessService` 子任务 | 由 `knowledge:post_process` 派发 |
|
||||
| `chunk:extract` | `ChunkExtractor.Handle` | 图谱提取(NEO4J 启用时) |
|
||||
| `datatable:summary` | `DataTableSummaryService.Handle` | 表格文件解析 |
|
||||
| `faq:import` | FAQ 批量导入 handler | FAQ 导入 / 批量创建 |
|
||||
| `knowledge:move` / `knowledge:list_delete` / `index:delete` / `kb:clone` / `kb:delete` | 知识移动 / 批量删除 / 索引清理 / 知识库复制 / 知识库删除 | 对应 HTTP 路由 |
|
||||
| `wiki:ingest` | `wikiIngestService.ProcessWikiIngest` | Wiki auto-fix / 重建链接 |
|
||||
| `datasource:sync` | `dataSourceSyncService.Handle` | 数据源手动触发 + 定时调度(定时场景下 trace 为 standalone) |
|
||||
|
||||
### 各模型的 usage 处理策略
|
||||
|
||||
| 模型类型 | 上报名称 | Token 计量方式 | 备注 |
|
||||
@@ -270,12 +290,17 @@ Dev 相关容器都带 `-dev` 后缀、用独立网络 `WeKnora-network-dev`,
|
||||
|
||||
## 8. 代码位置
|
||||
|
||||
- `internal/tracing/langfuse/` — Langfuse 客户端、异步批量上报、Gin 中间件。
|
||||
- `internal/tracing/langfuse/` — Langfuse 客户端、异步批量上报、Gin 中间件、**asynq middleware**、Span / Trace resume 实现。
|
||||
- `tracer.go` — 暴露 `Trace` / `Span` / `Generation` + `StartTrace` / `StartSpan` / `StartGeneration` / `ResumeTrace`。
|
||||
- `asynq.go` — `AsynqMiddleware()` 统一在 mux 上包 handler;`InjectTracing(ctx, payload)` 在 enqueue 侧把 trace/span ID 注入 payload。
|
||||
- `middleware.go` — Gin 中间件 + `shouldTrace` 白名单(覆盖 chat / 入库 / FAQ / wiki / 数据源等路径)。
|
||||
- `internal/types/tracing.go` — `TracingContext` POCO,所有 asynq payload 通过嵌入此结构携带 `lf_trace_id` / `lf_parent_obs_id` / `lf_user_id` / `lf_session_id`。
|
||||
- `internal/models/chat/langfuse_wrapper.go` — Chat 调用装饰器(含流式)。
|
||||
- `internal/models/embedding/langfuse_wrapper.go` — Embedding 调用装饰器。
|
||||
- `internal/models/rerank/langfuse_wrapper.go` — Rerank 调用装饰器。
|
||||
- `internal/models/vlm/langfuse_wrapper.go` — VLM(视觉语言模型)调用装饰器。
|
||||
- `internal/models/asr/langfuse_wrapper.go` — ASR(语音识别)调用装饰器。
|
||||
- `internal/router/router.go` — 注册 `langfuse.GinMiddleware()`。
|
||||
- `internal/router/task.go` — 在 asynq mux 上 `mux.Use(langfuse.AsynqMiddleware())`,使所有 handler 自动被 trace。
|
||||
- `internal/container/container.go` — 初始化 + 资源清理。
|
||||
- `internal/router/router.go` — 自动为 LLM 相关路由创建 trace。
|
||||
- `docker-compose.yml` / `.env.example` / `.env.lite.example` — 预置 `LANGFUSE_*` 环境变量直通。
|
||||
|
||||
@@ -12,6 +12,7 @@ import (
|
||||
|
||||
"github.com/Tencent/WeKnora/internal/datasource"
|
||||
"github.com/Tencent/WeKnora/internal/logger"
|
||||
"github.com/Tencent/WeKnora/internal/tracing/langfuse"
|
||||
"github.com/Tencent/WeKnora/internal/types"
|
||||
"github.com/Tencent/WeKnora/internal/types/interfaces"
|
||||
"github.com/hibiken/asynq"
|
||||
@@ -298,6 +299,7 @@ func (s *DataSourceService) ManualSync(ctx context.Context, dsID string) (*types
|
||||
SyncLogID: syncLog.ID,
|
||||
ForceFull: false,
|
||||
}
|
||||
langfuse.InjectTracing(ctx, payload)
|
||||
|
||||
payloadJSON, _ := json.Marshal(payload)
|
||||
task := asynq.NewTask(types.TypeDataSourceSync, payloadJSON)
|
||||
|
||||
@@ -16,6 +16,7 @@ import (
|
||||
"github.com/Tencent/WeKnora/internal/logger"
|
||||
"github.com/Tencent/WeKnora/internal/models/chat"
|
||||
"github.com/Tencent/WeKnora/internal/models/embedding"
|
||||
"github.com/Tencent/WeKnora/internal/tracing/langfuse"
|
||||
"github.com/Tencent/WeKnora/internal/types"
|
||||
"github.com/Tencent/WeKnora/internal/types/interfaces"
|
||||
"github.com/google/uuid"
|
||||
@@ -91,11 +92,13 @@ func NewChunkExtractTask(
|
||||
logger.Warn(ctx, "NEO4J is not enabled, skip chunk extract task")
|
||||
return nil
|
||||
}
|
||||
payload, err := json.Marshal(types.ExtractChunkPayload{
|
||||
taskPayload := types.ExtractChunkPayload{
|
||||
TenantID: tenantID,
|
||||
ChunkID: chunkID,
|
||||
ModelID: modelID,
|
||||
})
|
||||
}
|
||||
langfuse.InjectTracing(ctx, &taskPayload)
|
||||
payload, err := json.Marshal(taskPayload)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
@@ -118,12 +121,14 @@ func NewDataTableSummaryTask(
|
||||
summaryModel string,
|
||||
embeddingModel string,
|
||||
) error {
|
||||
payload, err := json.Marshal(DataTableSummaryPayload{
|
||||
taskPayload := DataTableSummaryPayload{
|
||||
TenantID: tenantID,
|
||||
KnowledgeID: knowledgeID,
|
||||
SummaryModel: summaryModel,
|
||||
EmbeddingModel: embeddingModel,
|
||||
})
|
||||
}
|
||||
langfuse.InjectTracing(ctx, &taskPayload)
|
||||
payload, err := json.Marshal(taskPayload)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
@@ -238,6 +243,7 @@ func (s *ChunkExtractService) Handle(ctx context.Context, t *asynq.Task) error {
|
||||
|
||||
// DataTableExtractPayload represents the table extract task payload
|
||||
type DataTableSummaryPayload struct {
|
||||
types.TracingContext
|
||||
TenantID uint64 `json:"tenant_id"`
|
||||
KnowledgeID string `json:"knowledge_id"`
|
||||
SummaryModel string `json:"summary_model"`
|
||||
|
||||
@@ -14,6 +14,7 @@ import (
|
||||
"github.com/Tencent/WeKnora/internal/logger"
|
||||
"github.com/Tencent/WeKnora/internal/models/utils/ollama"
|
||||
"github.com/Tencent/WeKnora/internal/models/vlm"
|
||||
"github.com/Tencent/WeKnora/internal/tracing/langfuse"
|
||||
"github.com/Tencent/WeKnora/internal/types"
|
||||
"github.com/Tencent/WeKnora/internal/types/interfaces"
|
||||
secutils "github.com/Tencent/WeKnora/internal/utils"
|
||||
@@ -419,6 +420,7 @@ func (s *ImageMultimodalService) enqueueKnowledgePostProcessTask(ctx context.Con
|
||||
KnowledgeBaseID: payload.KnowledgeBaseID,
|
||||
Language: payload.Language,
|
||||
}
|
||||
langfuse.InjectTracing(ctx, &taskPayload)
|
||||
payloadBytes, err := json.Marshal(taskPayload)
|
||||
if err != nil {
|
||||
logger.Warnf(ctx, "[ImageMultimodal] Failed to marshal post process payload: %v", err)
|
||||
|
||||
@@ -31,6 +31,7 @@ import (
|
||||
"github.com/Tencent/WeKnora/internal/models/embedding"
|
||||
"github.com/Tencent/WeKnora/internal/searchutil"
|
||||
"github.com/Tencent/WeKnora/internal/tracing"
|
||||
"github.com/Tencent/WeKnora/internal/tracing/langfuse"
|
||||
"github.com/Tencent/WeKnora/internal/types"
|
||||
"github.com/Tencent/WeKnora/internal/types/interfaces"
|
||||
secutils "github.com/Tencent/WeKnora/internal/utils"
|
||||
@@ -429,6 +430,7 @@ func (s *knowledgeService) CreateKnowledgeFromFile(ctx context.Context,
|
||||
Language: lang,
|
||||
}
|
||||
|
||||
langfuse.InjectTracing(ctx, &taskPayload)
|
||||
payloadBytes, err := json.Marshal(taskPayload)
|
||||
if err != nil {
|
||||
logger.Errorf(ctx, "Failed to marshal document process task payload: %v", err)
|
||||
@@ -603,6 +605,7 @@ func (s *knowledgeService) CreateKnowledgeFromURL(ctx context.Context,
|
||||
Language: lang,
|
||||
}
|
||||
|
||||
langfuse.InjectTracing(ctx, &taskPayload)
|
||||
payloadBytes, err := json.Marshal(taskPayload)
|
||||
if err != nil {
|
||||
logger.Errorf(ctx, "Failed to marshal URL process task payload: %v", err)
|
||||
@@ -825,6 +828,7 @@ func (s *knowledgeService) createKnowledgeFromFileURL(
|
||||
Language: lang,
|
||||
}
|
||||
|
||||
langfuse.InjectTracing(ctx, &taskPayload)
|
||||
payloadBytes, err := json.Marshal(taskPayload)
|
||||
if err != nil {
|
||||
logger.Errorf(ctx, "Failed to marshal file URL process task payload: %v", err)
|
||||
@@ -1043,6 +1047,7 @@ func (s *knowledgeService) createKnowledgeFromPassageInternal(ctx context.Contex
|
||||
Language: lang,
|
||||
}
|
||||
|
||||
langfuse.InjectTracing(ctx, &taskPayload)
|
||||
payloadBytes, err := json.Marshal(taskPayload)
|
||||
if err != nil {
|
||||
logger.Errorf(ctx, "Failed to marshal passage process task payload: %v", err)
|
||||
@@ -2206,12 +2211,14 @@ func (s *knowledgeService) processChunks(ctx context.Context,
|
||||
} else {
|
||||
// If there are no multimodal tasks, enqueue the post process task immediately
|
||||
lang, _ := types.LanguageFromContext(ctx)
|
||||
payloadBytes, err := json.Marshal(types.KnowledgePostProcessPayload{
|
||||
postProcessPayload := types.KnowledgePostProcessPayload{
|
||||
TenantID: knowledge.TenantID,
|
||||
KnowledgeID: knowledge.ID,
|
||||
KnowledgeBaseID: knowledge.KnowledgeBaseID,
|
||||
Language: lang,
|
||||
})
|
||||
}
|
||||
langfuse.InjectTracing(ctx, &postProcessPayload)
|
||||
payloadBytes, err := json.Marshal(postProcessPayload)
|
||||
if err == nil {
|
||||
task := asynq.NewTask(types.TypeKnowledgePostProcess, payloadBytes, asynq.Queue("default"), asynq.MaxRetry(3))
|
||||
if _, err := s.task.Enqueue(task); err != nil {
|
||||
@@ -3076,6 +3083,7 @@ func (s *knowledgeService) enqueueManualProcessing(ctx context.Context,
|
||||
Content: content,
|
||||
NeedCleanup: needCleanup,
|
||||
}
|
||||
langfuse.InjectTracing(ctx, &payload)
|
||||
payloadBytes, err := json.Marshal(payload)
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed to marshal manual process payload: %w", err)
|
||||
@@ -3202,6 +3210,7 @@ func (s *knowledgeService) ReparseKnowledge(ctx context.Context, knowledgeID str
|
||||
Language: lang,
|
||||
}
|
||||
|
||||
langfuse.InjectTracing(ctx, &taskPayload)
|
||||
payloadBytes, err := json.Marshal(taskPayload)
|
||||
if err != nil {
|
||||
logger.Errorf(ctx, "Failed to marshal reparse task payload: %v", err)
|
||||
@@ -3254,6 +3263,7 @@ func (s *knowledgeService) ReparseKnowledge(ctx context.Context, knowledgeID str
|
||||
Language: lang,
|
||||
}
|
||||
|
||||
langfuse.InjectTracing(ctx, &taskPayload)
|
||||
payloadBytes, err := json.Marshal(taskPayload)
|
||||
if err != nil {
|
||||
logger.Errorf(ctx, "Failed to marshal file URL reparse task payload: %v", err)
|
||||
@@ -3299,6 +3309,7 @@ func (s *knowledgeService) ReparseKnowledge(ctx context.Context, knowledgeID str
|
||||
Language: lang,
|
||||
}
|
||||
|
||||
langfuse.InjectTracing(ctx, &taskPayload)
|
||||
payloadBytes, err := json.Marshal(taskPayload)
|
||||
if err != nil {
|
||||
logger.Errorf(ctx, "Failed to marshal URL reparse task payload: %v", err)
|
||||
@@ -4077,6 +4088,7 @@ func (s *knowledgeService) UpsertFAQEntries(ctx context.Context,
|
||||
taskPayload.Entries = payload.Entries
|
||||
}
|
||||
|
||||
langfuse.InjectTracing(ctx, &taskPayload)
|
||||
payloadBytes, err := json.Marshal(taskPayload)
|
||||
if err != nil {
|
||||
logger.Errorf(ctx, "Failed to marshal FAQ import task payload: %v", err)
|
||||
@@ -8461,6 +8473,7 @@ func (s *knowledgeService) enqueueImageMultimodalTasks(
|
||||
ImageSourceType: metadata["image_source_type"],
|
||||
}
|
||||
|
||||
langfuse.InjectTracing(ctx, &payload)
|
||||
payloadBytes, err := json.Marshal(payload)
|
||||
if err != nil {
|
||||
logger.Warnf(ctx, "Failed to marshal image multimodal payload: %v", err)
|
||||
@@ -9676,6 +9689,7 @@ func (s *knowledgeService) moveKnowledgeReparse(
|
||||
Language: lang,
|
||||
}
|
||||
|
||||
langfuse.InjectTracing(ctx, &taskPayload)
|
||||
payloadBytes, err := json.Marshal(taskPayload)
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed to marshal document process payload: %w", err)
|
||||
|
||||
@@ -7,6 +7,7 @@ import (
|
||||
"time"
|
||||
|
||||
"github.com/Tencent/WeKnora/internal/logger"
|
||||
"github.com/Tencent/WeKnora/internal/tracing/langfuse"
|
||||
"github.com/Tencent/WeKnora/internal/types"
|
||||
"github.com/Tencent/WeKnora/internal/types/interfaces"
|
||||
"github.com/hibiken/asynq"
|
||||
@@ -142,6 +143,7 @@ func (s *KnowledgePostProcessService) enqueueSummaryGenerationTask(ctx context.C
|
||||
KnowledgeID: payload.KnowledgeID,
|
||||
Language: payload.Language,
|
||||
}
|
||||
langfuse.InjectTracing(ctx, &taskPayload)
|
||||
payloadBytes, err := json.Marshal(taskPayload)
|
||||
if err != nil {
|
||||
logger.Warnf(ctx, "[KnowledgePostProcess] Failed to marshal summary generation payload: %v", err)
|
||||
@@ -180,6 +182,7 @@ func (s *KnowledgePostProcessService) enqueueQuestionGenerationIfEnabled(ctx con
|
||||
QuestionCount: questionCount,
|
||||
Language: payload.Language,
|
||||
}
|
||||
langfuse.InjectTracing(ctx, &taskPayload)
|
||||
payloadBytes, err := json.Marshal(taskPayload)
|
||||
if err != nil {
|
||||
logger.Warnf(ctx, "[KnowledgePostProcess] Failed to marshal question generation payload: %v", err)
|
||||
|
||||
@@ -8,6 +8,7 @@ import (
|
||||
|
||||
"github.com/Tencent/WeKnora/internal/application/service/retriever"
|
||||
"github.com/Tencent/WeKnora/internal/logger"
|
||||
"github.com/Tencent/WeKnora/internal/tracing/langfuse"
|
||||
"github.com/Tencent/WeKnora/internal/types"
|
||||
"github.com/Tencent/WeKnora/internal/types/interfaces"
|
||||
"github.com/google/uuid"
|
||||
@@ -384,6 +385,7 @@ func (s *knowledgeBaseService) DeleteKnowledgeBase(ctx context.Context, id strin
|
||||
KnowledgeBaseID: id,
|
||||
EffectiveEngines: tenantInfo.GetEffectiveEngines(),
|
||||
}
|
||||
langfuse.InjectTracing(ctx, &payload)
|
||||
|
||||
payloadBytes, err := json.Marshal(payload)
|
||||
if err != nil {
|
||||
|
||||
@@ -10,6 +10,7 @@ import (
|
||||
"github.com/Tencent/WeKnora/internal/application/service/retriever"
|
||||
werrors "github.com/Tencent/WeKnora/internal/errors"
|
||||
"github.com/Tencent/WeKnora/internal/logger"
|
||||
"github.com/Tencent/WeKnora/internal/tracing/langfuse"
|
||||
"github.com/Tencent/WeKnora/internal/types"
|
||||
"github.com/Tencent/WeKnora/internal/types/interfaces"
|
||||
"github.com/google/uuid"
|
||||
@@ -282,6 +283,7 @@ func (s *knowledgeTagService) DeleteTag(ctx context.Context, id string, force bo
|
||||
TenantID: tenantID,
|
||||
KnowledgeIDs: knowledgeIDs,
|
||||
}
|
||||
langfuse.InjectTracing(ctx, &payload)
|
||||
payloadBytes, err := json.Marshal(payload)
|
||||
if err != nil {
|
||||
logger.Errorf(ctx, "Failed to marshal knowledge list delete payload: %v", err)
|
||||
@@ -351,6 +353,7 @@ func (s *knowledgeTagService) enqueueIndexDeleteTask(ctx context.Context,
|
||||
ChunkIDs: chunkIDs,
|
||||
EffectiveEngines: effectiveEngines,
|
||||
}
|
||||
langfuse.InjectTracing(ctx, &payload)
|
||||
payloadBytes, err := json.Marshal(payload)
|
||||
if err != nil {
|
||||
logger.Errorf(ctx, "Failed to marshal index delete payload: %v", err)
|
||||
|
||||
@@ -14,6 +14,7 @@ import (
|
||||
"github.com/Tencent/WeKnora/internal/logger"
|
||||
"github.com/Tencent/WeKnora/internal/models/chat"
|
||||
"github.com/Tencent/WeKnora/internal/searchutil"
|
||||
"github.com/Tencent/WeKnora/internal/tracing/langfuse"
|
||||
"github.com/Tencent/WeKnora/internal/types"
|
||||
"github.com/Tencent/WeKnora/internal/types/interfaces"
|
||||
"github.com/hibiken/asynq"
|
||||
@@ -92,6 +93,7 @@ func WikiDeletedTombstoneKey(kbID, knowledgeID string) string {
|
||||
// The actual document IDs are stored in a Redis list (wiki:pending:{kbID}).
|
||||
// KnowledgeID is only used as fallback in Lite mode (no Redis).
|
||||
type WikiIngestPayload struct {
|
||||
types.TracingContext
|
||||
TenantID uint64 `json:"tenant_id"`
|
||||
KnowledgeBaseID string `json:"knowledge_base_id"`
|
||||
Language string `json:"language,omitempty"`
|
||||
@@ -101,6 +103,7 @@ type WikiIngestPayload struct {
|
||||
|
||||
// WikiRetractPayload is the asynq task payload for wiki content retraction
|
||||
type WikiRetractPayload struct {
|
||||
types.TracingContext
|
||||
TenantID uint64 `json:"tenant_id"`
|
||||
KnowledgeBaseID string `json:"knowledge_base_id"`
|
||||
KnowledgeID string `json:"knowledge_id"`
|
||||
@@ -208,6 +211,7 @@ func EnqueueWikiIngest(ctx context.Context, task interfaces.TaskEnqueuer, redisC
|
||||
}}
|
||||
}
|
||||
|
||||
langfuse.InjectTracing(ctx, &payload)
|
||||
payloadBytes, _ := json.Marshal(payload)
|
||||
|
||||
t := asynq.NewTask(types.TypeWikiIngest, payloadBytes,
|
||||
@@ -248,6 +252,7 @@ func EnqueueWikiRetract(ctx context.Context, task interfaces.TaskEnqueuer, redis
|
||||
ingestPayload.LiteOps = []WikiPendingOp{op}
|
||||
}
|
||||
|
||||
langfuse.InjectTracing(ctx, &ingestPayload)
|
||||
payloadBytes, _ := json.Marshal(ingestPayload)
|
||||
t := asynq.NewTask(types.TypeWikiIngest, payloadBytes,
|
||||
asynq.Queue("low"),
|
||||
|
||||
@@ -11,6 +11,7 @@ import (
|
||||
"github.com/Tencent/WeKnora/internal/agent"
|
||||
"github.com/Tencent/WeKnora/internal/logger"
|
||||
"github.com/Tencent/WeKnora/internal/models/chat"
|
||||
"github.com/Tencent/WeKnora/internal/tracing/langfuse"
|
||||
"github.com/Tencent/WeKnora/internal/types"
|
||||
"github.com/google/uuid"
|
||||
"github.com/hibiken/asynq"
|
||||
@@ -29,6 +30,7 @@ func (s *wikiIngestService) scheduleFollowUp(ctx context.Context, payload WikiIn
|
||||
|
||||
logger.Infof(ctx, "wiki ingest: %d more documents pending for KB %s, scheduling follow-up", count, payload.KnowledgeBaseID)
|
||||
|
||||
langfuse.InjectTracing(ctx, &payload)
|
||||
payloadBytes, _ := json.Marshal(payload)
|
||||
t := asynq.NewTask(types.TypeWikiIngest, payloadBytes,
|
||||
asynq.Queue("low"),
|
||||
|
||||
@@ -8,6 +8,7 @@ import (
|
||||
"time"
|
||||
|
||||
"github.com/Tencent/WeKnora/internal/logger"
|
||||
"github.com/Tencent/WeKnora/internal/tracing/langfuse"
|
||||
"github.com/Tencent/WeKnora/internal/types"
|
||||
"github.com/Tencent/WeKnora/internal/types/interfaces"
|
||||
"github.com/hibiken/asynq"
|
||||
@@ -168,6 +169,7 @@ func (s *Scheduler) triggerSync(dataSourceID string, tenantID uint64) {
|
||||
SyncLogID: syncLog.ID,
|
||||
ForceFull: false,
|
||||
}
|
||||
langfuse.InjectTracing(ctx, payload)
|
||||
payloadJSON, _ := json.Marshal(payload)
|
||||
task := asynq.NewTask(types.TypeDataSourceSync, payloadJSON)
|
||||
|
||||
|
||||
@@ -18,6 +18,7 @@ import (
|
||||
"github.com/Tencent/WeKnora/internal/application/service"
|
||||
"github.com/Tencent/WeKnora/internal/errors"
|
||||
"github.com/Tencent/WeKnora/internal/logger"
|
||||
"github.com/Tencent/WeKnora/internal/tracing/langfuse"
|
||||
"github.com/Tencent/WeKnora/internal/types"
|
||||
"github.com/Tencent/WeKnora/internal/types/interfaces"
|
||||
"github.com/Tencent/WeKnora/internal/utils"
|
||||
@@ -686,6 +687,7 @@ func (h *KnowledgeHandler) ClearKnowledgeBaseContents(c *gin.Context) {
|
||||
TenantID: effectiveTenantID,
|
||||
KnowledgeIDs: knowledgeIDs,
|
||||
}
|
||||
langfuse.InjectTracing(ctx, &payload)
|
||||
payloadBytes, err := json.Marshal(payload)
|
||||
if err != nil {
|
||||
logger.Errorf(ctx, "Failed to marshal knowledge list delete payload: %v", err)
|
||||
@@ -1559,6 +1561,7 @@ func (h *KnowledgeHandler) MoveKnowledge(c *gin.Context) {
|
||||
TargetKBID: req.TargetKBID,
|
||||
Mode: req.Mode,
|
||||
}
|
||||
langfuse.InjectTracing(ctx, &payload)
|
||||
|
||||
payloadBytes, err := json.Marshal(payload)
|
||||
if err != nil {
|
||||
|
||||
@@ -13,6 +13,7 @@ import (
|
||||
"github.com/Tencent/WeKnora/internal/errors"
|
||||
apperrors "github.com/Tencent/WeKnora/internal/errors"
|
||||
"github.com/Tencent/WeKnora/internal/logger"
|
||||
"github.com/Tencent/WeKnora/internal/tracing/langfuse"
|
||||
"github.com/Tencent/WeKnora/internal/types"
|
||||
"github.com/Tencent/WeKnora/internal/types/interfaces"
|
||||
"github.com/Tencent/WeKnora/internal/utils"
|
||||
@@ -655,6 +656,7 @@ func (h *KnowledgeBaseHandler) CopyKnowledgeBase(c *gin.Context) {
|
||||
SourceID: req.SourceID,
|
||||
TargetID: req.TargetID,
|
||||
}
|
||||
langfuse.InjectTracing(ctx, &payload)
|
||||
|
||||
payloadBytes, err := json.Marshal(payload)
|
||||
if err != nil {
|
||||
|
||||
@@ -8,6 +8,7 @@ import (
|
||||
"time"
|
||||
|
||||
"github.com/Tencent/WeKnora/internal/application/service"
|
||||
"github.com/Tencent/WeKnora/internal/tracing/langfuse"
|
||||
"github.com/Tencent/WeKnora/internal/types"
|
||||
"github.com/Tencent/WeKnora/internal/types/interfaces"
|
||||
"github.com/hibiken/asynq"
|
||||
@@ -100,6 +101,14 @@ func RunAsynqServer(params AsynqTaskParams) *asynq.ServeMux {
|
||||
// Create a new mux and register all handlers
|
||||
mux := asynq.NewServeMux()
|
||||
|
||||
// Install Langfuse middleware BEFORE handler registration so every task
|
||||
// type is automatically wrapped. When Langfuse is disabled the middleware
|
||||
// is a pass-through; when enabled it resumes the upstream HTTP trace (if
|
||||
// the payload carries one) or opens a standalone trace, then wraps the
|
||||
// handler execution in a SPAN so all child generations (embedding / VLM /
|
||||
// chat / rerank / ASR) nest correctly in the Langfuse UI.
|
||||
mux.Use(langfuse.AsynqMiddleware())
|
||||
|
||||
// Register extract handlers - router will dispatch to appropriate handler
|
||||
mux.HandleFunc(types.TypeChunkExtract, params.ChunkExtractor.Handle)
|
||||
mux.HandleFunc(types.TypeDataTableSummary, params.DataTableSummary.Handle)
|
||||
|
||||
@@ -0,0 +1,209 @@
|
||||
package langfuse
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"strconv"
|
||||
|
||||
"github.com/Tencent/WeKnora/internal/types"
|
||||
"github.com/hibiken/asynq"
|
||||
)
|
||||
|
||||
// InjectTracing stamps the current trace/span ids (and a best-effort
|
||||
// user/session label) from ctx onto the given payload, provided the payload
|
||||
// embeds types.TracingContext (and thus implements LangfuseTracingCarrier).
|
||||
//
|
||||
// Safe to call unconditionally: when Langfuse is disabled or no trace is
|
||||
// present on ctx, it writes a zero-valued TracingContext — which round-trips
|
||||
// through JSON as absent fields and therefore costs nothing.
|
||||
//
|
||||
// Call sites live at every asynq.NewTask creation point. We deliberately
|
||||
// keep the API synchronous and mutating (rather than returning a new payload
|
||||
// copy) so that existing enqueue code needs only a single added line just
|
||||
// before json.Marshal, minimizing review surface and the risk of regressions.
|
||||
func InjectTracing(ctx context.Context, carrier types.LangfuseTracingCarrier) {
|
||||
if carrier == nil {
|
||||
return
|
||||
}
|
||||
mgr := GetManager()
|
||||
if !mgr.Enabled() {
|
||||
return
|
||||
}
|
||||
tc := types.TracingContext{}
|
||||
if trace, ok := TraceFromContext(ctx); ok && trace != nil {
|
||||
tc.LangfuseTraceID = trace.ID
|
||||
}
|
||||
if obs, ok := parentObservationFromCtx(ctx); ok {
|
||||
tc.LangfuseParentObservationID = obs
|
||||
}
|
||||
tc.LangfuseUserID = userIDFromCtx(ctx)
|
||||
tc.LangfuseSessionID = sessionIDFromCtx(ctx)
|
||||
carrier.SetLangfuseTracing(tc)
|
||||
}
|
||||
|
||||
// peekTracingContext pulls just the Langfuse tracing fields out of a raw
|
||||
// asynq payload. It's deliberately lax: every payload type in the project
|
||||
// is a JSON object at the top level, and absent/mismatched fields decode to
|
||||
// the zero value. If unmarshalling fails entirely (e.g. the payload isn't
|
||||
// JSON at all) we return a zero TracingContext and let the main handler
|
||||
// deal with its own error — we never want an observability bug to mask a
|
||||
// real task failure.
|
||||
func peekTracingContext(payload []byte) types.TracingContext {
|
||||
if len(payload) == 0 {
|
||||
return types.TracingContext{}
|
||||
}
|
||||
var tc types.TracingContext
|
||||
_ = json.Unmarshal(payload, &tc)
|
||||
return tc
|
||||
}
|
||||
|
||||
// AsynqMiddleware is the worker-side counterpart of GinMiddleware. It:
|
||||
//
|
||||
// 1. Parses any tracing ids embedded in the task payload and either
|
||||
// resumes the originating trace (so the Langfuse UI stitches the HTTP
|
||||
// request and the async processing into one tree), or — for tasks that
|
||||
// came from a scheduled job with no originating HTTP request — creates
|
||||
// a standalone trace tagged with the task type.
|
||||
//
|
||||
// 2. Opens a SPAN around the handler execution so every child generation
|
||||
// (embedding / VLM / chat / rerank / ASR) auto-attaches to it via
|
||||
// parentObservationId.
|
||||
//
|
||||
// 3. Enriches the span with asynq's own metadata: task id, queue, retry
|
||||
// count, payload size.
|
||||
//
|
||||
// When the manager is disabled it degrades to a pass-through; failure of
|
||||
// the Langfuse path never blocks task execution.
|
||||
//
|
||||
// Register it once in router/task.go via mux.Use.
|
||||
func AsynqMiddleware() asynq.MiddlewareFunc {
|
||||
return func(next asynq.Handler) asynq.Handler {
|
||||
return asynq.HandlerFunc(func(ctx context.Context, task *asynq.Task) error {
|
||||
mgr := GetManager()
|
||||
if !mgr.Enabled() {
|
||||
return next.ProcessTask(ctx, task)
|
||||
}
|
||||
|
||||
tc := peekTracingContext(task.Payload())
|
||||
taskID, _ := asynq.GetTaskID(ctx)
|
||||
retryCount, _ := asynq.GetRetryCount(ctx)
|
||||
maxRetry, _ := asynq.GetMaxRetry(ctx)
|
||||
queueName, _ := asynq.GetQueueName(ctx)
|
||||
|
||||
meta := map[string]interface{}{
|
||||
"task_type": task.Type(),
|
||||
"task_id": taskID,
|
||||
"queue": queueName,
|
||||
"retry": retryCount,
|
||||
"max_retry": maxRetry,
|
||||
"payload_bytes": len(task.Payload()),
|
||||
}
|
||||
|
||||
// If the upstream enqueuer stamped a trace id onto the payload,
|
||||
// we graft under that trace. Otherwise (scheduled jobs, legacy
|
||||
// in-flight tasks that predate this code, tests, etc.) we start
|
||||
// a standalone trace named after the task type so at least the
|
||||
// worker-side work is observable.
|
||||
var trace *Trace
|
||||
shouldFinishTrace := false
|
||||
if tc.LangfuseTraceID != "" {
|
||||
ctx, trace = mgr.ResumeTrace(ctx, tc.LangfuseTraceID, tc.LangfuseParentObservationID)
|
||||
} else {
|
||||
ctx, trace = mgr.StartTrace(ctx, TraceOptions{
|
||||
Name: "asynq." + task.Type(),
|
||||
UserID: firstNonEmptyString(tc.LangfuseUserID, userIDFromCtx(ctx)),
|
||||
SessionID: firstNonEmptyString(tc.LangfuseSessionID, sessionIDFromCtx(ctx)),
|
||||
Metadata: meta,
|
||||
Tags: []string{"asynq", task.Type()},
|
||||
})
|
||||
shouldFinishTrace = true
|
||||
}
|
||||
|
||||
ctx, span := mgr.StartSpan(ctx, SpanOptions{
|
||||
Name: "asynq." + task.Type(),
|
||||
Input: spanInputFromPayload(task.Payload()),
|
||||
Metadata: meta,
|
||||
})
|
||||
|
||||
err := next.ProcessTask(ctx, task)
|
||||
|
||||
outcome := "success"
|
||||
if err != nil {
|
||||
outcome = "error"
|
||||
}
|
||||
span.Finish(map[string]interface{}{
|
||||
"outcome": outcome,
|
||||
}, map[string]interface{}{
|
||||
"outcome": outcome,
|
||||
}, err)
|
||||
|
||||
if shouldFinishTrace {
|
||||
trace.Finish(map[string]interface{}{
|
||||
"outcome": outcome,
|
||||
}, map[string]interface{}{
|
||||
"task_type": task.Type(),
|
||||
"outcome": outcome,
|
||||
})
|
||||
}
|
||||
|
||||
return err
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// spanInputFromPayload surfaces a compact, human-readable summary of the
|
||||
// task payload for the Langfuse "Input" pane. We deliberately do NOT send
|
||||
// the full JSON blob because:
|
||||
//
|
||||
// - Manual/text-ingest payloads can be many kilobytes of prose;
|
||||
// - FAQ import payloads embed the full entry list;
|
||||
// - Document-process payloads contain file URLs that, while small, may
|
||||
// include presigned query strings that rotate (adding diff noise).
|
||||
//
|
||||
// Instead we preview the first ~1KB verbatim, which matches what Langfuse
|
||||
// itself would display and keeps ingestion bandwidth predictable.
|
||||
func spanInputFromPayload(payload []byte) interface{} {
|
||||
const preview = 1024
|
||||
if len(payload) == 0 {
|
||||
return nil
|
||||
}
|
||||
if len(payload) <= preview {
|
||||
return string(payload)
|
||||
}
|
||||
return map[string]interface{}{
|
||||
"preview": string(payload[:preview]) + "...",
|
||||
"bytes": len(payload),
|
||||
}
|
||||
}
|
||||
|
||||
// userIDFromCtx mirrors middleware.extractUserID but accepts a raw context
|
||||
// (no gin.Context) so both HTTP and asynq paths can share the same fallback
|
||||
// logic: explicit UserID → tenant:<id> → empty.
|
||||
func userIDFromCtx(ctx context.Context) string {
|
||||
if v, ok := ctx.Value(types.UserIDContextKey).(string); ok && v != "" {
|
||||
return v
|
||||
}
|
||||
if v, ok := ctx.Value(types.TenantIDContextKey).(uint64); ok && v != 0 {
|
||||
return "tenant:" + strconv.FormatUint(v, 10)
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
// sessionIDFromCtx pulls a best-effort "session" label. For HTTP chat this
|
||||
// is already set by GinMiddleware; for async work we fall back to the
|
||||
// request id so retries of the same logical task group together.
|
||||
func sessionIDFromCtx(ctx context.Context) string {
|
||||
if v, ok := types.RequestIDFromContext(ctx); ok && v != "" {
|
||||
return v
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
func firstNonEmptyString(vals ...string) string {
|
||||
for _, v := range vals {
|
||||
if v != "" {
|
||||
return v
|
||||
}
|
||||
}
|
||||
return ""
|
||||
}
|
||||
@@ -0,0 +1,272 @@
|
||||
package langfuse
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"io"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"sync"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/Tencent/WeKnora/internal/types"
|
||||
"github.com/hibiken/asynq"
|
||||
)
|
||||
|
||||
// dummyPayload is a minimal payload that embeds TracingContext, mirroring
|
||||
// how real asynq payloads opt into trace propagation.
|
||||
type dummyPayload struct {
|
||||
types.TracingContext
|
||||
KnowledgeID string `json:"knowledge_id"`
|
||||
}
|
||||
|
||||
// TestInjectTracing_DisabledIsZero verifies InjectTracing is a no-op when
|
||||
// Langfuse is disabled: no panics, no trace fields written.
|
||||
func TestInjectTracing_DisabledIsZero(t *testing.T) {
|
||||
// Explicitly install a disabled manager to shadow any globally-enabled
|
||||
// one from a previous test in the same package.
|
||||
_, _ = Init(Config{Enabled: false})
|
||||
|
||||
p := &dummyPayload{KnowledgeID: "k1"}
|
||||
InjectTracing(context.Background(), p)
|
||||
if p.LangfuseTraceID != "" || p.LangfuseParentObservationID != "" {
|
||||
t.Fatalf("expected no tracing fields on disabled manager, got %+v", p.TracingContext)
|
||||
}
|
||||
}
|
||||
|
||||
// TestInjectTracing_PopulatesFromContext checks that when a trace is active
|
||||
// on the context, its id is copied onto the payload and a subsequent
|
||||
// peekTracingContext round-trips it correctly.
|
||||
func TestInjectTracing_PopulatesFromContext(t *testing.T) {
|
||||
// Stand up a fake ingestion endpoint — Init starts a real flush worker
|
||||
// and we don't want it to panic on connection refused during the test.
|
||||
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
_, _ = io.Copy(io.Discard, r.Body)
|
||||
w.WriteHeader(200)
|
||||
_, _ = w.Write([]byte(`{}`))
|
||||
}))
|
||||
defer srv.Close()
|
||||
|
||||
m, err := Init(Config{
|
||||
Enabled: true,
|
||||
Host: srv.URL,
|
||||
PublicKey: "pk",
|
||||
SecretKey: "sk",
|
||||
FlushAt: 16,
|
||||
FlushInterval: 50 * time.Millisecond,
|
||||
QueueSize: 16,
|
||||
RequestTimeout: 2 * time.Second,
|
||||
SampleRate: 1.0,
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("init: %v", err)
|
||||
}
|
||||
defer m.Shutdown(context.Background())
|
||||
|
||||
ctx, trace := m.StartTrace(context.Background(), TraceOptions{Name: "parent"})
|
||||
ctx, span := m.StartSpan(ctx, SpanOptions{Name: "wrap"})
|
||||
if span == nil || span.ID == "" {
|
||||
t.Fatalf("expected a span with id, got %+v", span)
|
||||
}
|
||||
|
||||
p := &dummyPayload{KnowledgeID: "k1"}
|
||||
InjectTracing(ctx, p)
|
||||
|
||||
if p.LangfuseTraceID != trace.ID {
|
||||
t.Errorf("trace id mismatch: got %q want %q", p.LangfuseTraceID, trace.ID)
|
||||
}
|
||||
if p.LangfuseParentObservationID != span.ID {
|
||||
t.Errorf("parent observation id mismatch: got %q want %q", p.LangfuseParentObservationID, span.ID)
|
||||
}
|
||||
|
||||
// Round-trip the payload through JSON, which is what asynq does on the
|
||||
// wire, and make sure peekTracingContext recovers the ids.
|
||||
b, err := json.Marshal(p)
|
||||
if err != nil {
|
||||
t.Fatalf("marshal: %v", err)
|
||||
}
|
||||
got := peekTracingContext(b)
|
||||
if got.LangfuseTraceID != trace.ID || got.LangfuseParentObservationID != span.ID {
|
||||
t.Errorf("peek round-trip lost ids: %+v", got)
|
||||
}
|
||||
}
|
||||
|
||||
// TestAsynqMiddleware_ResumeTrace asserts the middleware grafts a resumed
|
||||
// trace onto the child handler's context, and that the ingestion endpoint
|
||||
// receives span-create / span-update events for the wrapper — without
|
||||
// emitting a duplicate trace-create (which would split the Langfuse UI
|
||||
// tree into two disconnected roots).
|
||||
func TestAsynqMiddleware_ResumeTrace(t *testing.T) {
|
||||
var mu sync.Mutex
|
||||
var batches []ingestionRequest
|
||||
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
body, _ := io.ReadAll(r.Body)
|
||||
var req ingestionRequest
|
||||
_ = json.Unmarshal(body, &req)
|
||||
mu.Lock()
|
||||
batches = append(batches, req)
|
||||
mu.Unlock()
|
||||
w.WriteHeader(200)
|
||||
_, _ = w.Write([]byte(`{}`))
|
||||
}))
|
||||
defer srv.Close()
|
||||
|
||||
m, err := Init(Config{
|
||||
Enabled: true,
|
||||
Host: srv.URL,
|
||||
PublicKey: "pk",
|
||||
SecretKey: "sk",
|
||||
FlushAt: 1,
|
||||
FlushInterval: 5 * time.Millisecond,
|
||||
QueueSize: 32,
|
||||
RequestTimeout: 2 * time.Second,
|
||||
SampleRate: 1.0,
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("init: %v", err)
|
||||
}
|
||||
|
||||
// Build a payload that already carries a trace id (as if the HTTP layer
|
||||
// injected one at enqueue time).
|
||||
payload := &dummyPayload{KnowledgeID: "k1"}
|
||||
payload.LangfuseTraceID = "trace-xyz"
|
||||
payload.LangfuseParentObservationID = "parent-span-abc"
|
||||
raw, _ := json.Marshal(payload)
|
||||
|
||||
var receivedTraceID, receivedParentID string
|
||||
var sawTrace bool
|
||||
handler := asynq.HandlerFunc(func(ctx context.Context, task *asynq.Task) error {
|
||||
if tr, ok := TraceFromContext(ctx); ok && tr != nil {
|
||||
sawTrace = true
|
||||
receivedTraceID = tr.ID
|
||||
}
|
||||
if pid, ok := parentObservationFromCtx(ctx); ok {
|
||||
receivedParentID = pid
|
||||
}
|
||||
return nil
|
||||
})
|
||||
|
||||
mw := AsynqMiddleware()
|
||||
wrapped := mw(handler)
|
||||
|
||||
task := asynq.NewTask("test:type", raw)
|
||||
if err := wrapped.ProcessTask(context.Background(), task); err != nil {
|
||||
t.Fatalf("handler err: %v", err)
|
||||
}
|
||||
|
||||
if !sawTrace {
|
||||
t.Fatal("expected handler ctx to carry a resumed trace")
|
||||
}
|
||||
if receivedTraceID != "trace-xyz" {
|
||||
t.Errorf("trace id mismatch: got %q want trace-xyz", receivedTraceID)
|
||||
}
|
||||
if receivedParentID == "" {
|
||||
t.Error("expected parent observation to be set to the wrapper span id")
|
||||
}
|
||||
|
||||
if err := m.Shutdown(context.Background()); err != nil {
|
||||
t.Fatalf("shutdown: %v", err)
|
||||
}
|
||||
|
||||
mu.Lock()
|
||||
defer mu.Unlock()
|
||||
var sawSpanCreate, sawSpanUpdate, sawTraceCreate bool
|
||||
for _, b := range batches {
|
||||
for _, ev := range b.Batch {
|
||||
switch ev.Type {
|
||||
case "span-create":
|
||||
sawSpanCreate = true
|
||||
// Resumed span should attach to the supplied trace id, not
|
||||
// a fresh one we made up in the worker.
|
||||
body, _ := json.Marshal(ev.Body)
|
||||
var ob observationBody
|
||||
_ = json.Unmarshal(body, &ob)
|
||||
if ob.TraceID != "trace-xyz" {
|
||||
t.Errorf("span-create trace id = %q, want trace-xyz", ob.TraceID)
|
||||
}
|
||||
if ob.ParentObservationID != "parent-span-abc" {
|
||||
t.Errorf("span-create parentObservationId = %q, want parent-span-abc", ob.ParentObservationID)
|
||||
}
|
||||
case "span-update":
|
||||
sawSpanUpdate = true
|
||||
case "trace-create":
|
||||
sawTraceCreate = true
|
||||
}
|
||||
}
|
||||
}
|
||||
if !sawSpanCreate {
|
||||
t.Error("missing span-create event")
|
||||
}
|
||||
if !sawSpanUpdate {
|
||||
t.Error("missing span-update event")
|
||||
}
|
||||
if sawTraceCreate {
|
||||
t.Error("resumed trace should not emit trace-create events (parent owns them)")
|
||||
}
|
||||
}
|
||||
|
||||
// TestAsynqMiddleware_StandaloneTrace asserts that when the payload carries
|
||||
// NO upstream trace id (e.g. a scheduled job), the middleware opens a
|
||||
// standalone trace tagged with the task type, so the worker-side work
|
||||
// still shows up in Langfuse.
|
||||
func TestAsynqMiddleware_StandaloneTrace(t *testing.T) {
|
||||
var mu sync.Mutex
|
||||
var batches []ingestionRequest
|
||||
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
body, _ := io.ReadAll(r.Body)
|
||||
var req ingestionRequest
|
||||
_ = json.Unmarshal(body, &req)
|
||||
mu.Lock()
|
||||
batches = append(batches, req)
|
||||
mu.Unlock()
|
||||
w.WriteHeader(200)
|
||||
_, _ = w.Write([]byte(`{}`))
|
||||
}))
|
||||
defer srv.Close()
|
||||
|
||||
m, err := Init(Config{
|
||||
Enabled: true, Host: srv.URL, PublicKey: "pk", SecretKey: "sk",
|
||||
FlushAt: 1, FlushInterval: 5 * time.Millisecond, QueueSize: 32,
|
||||
RequestTimeout: 2 * time.Second, SampleRate: 1.0,
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("init: %v", err)
|
||||
}
|
||||
|
||||
payload := &dummyPayload{KnowledgeID: "kX"}
|
||||
raw, _ := json.Marshal(payload)
|
||||
|
||||
handler := asynq.HandlerFunc(func(ctx context.Context, task *asynq.Task) error { return nil })
|
||||
wrapped := AsynqMiddleware()(handler)
|
||||
if err := wrapped.ProcessTask(context.Background(), asynq.NewTask("scheduled:ping", raw)); err != nil {
|
||||
t.Fatalf("handler err: %v", err)
|
||||
}
|
||||
|
||||
if err := m.Shutdown(context.Background()); err != nil {
|
||||
t.Fatalf("shutdown: %v", err)
|
||||
}
|
||||
|
||||
mu.Lock()
|
||||
defer mu.Unlock()
|
||||
// Langfuse uses trace-create events both on open and on update (the
|
||||
// server merges by id), so we filter to the initial event that carries
|
||||
// the Name — the finalizing update has no name.
|
||||
var sawNamedTraceCreate bool
|
||||
for _, b := range batches {
|
||||
for _, ev := range b.Batch {
|
||||
if ev.Type != "trace-create" {
|
||||
continue
|
||||
}
|
||||
body, _ := json.Marshal(ev.Body)
|
||||
var tb traceBody
|
||||
_ = json.Unmarshal(body, &tb)
|
||||
if tb.Name == "asynq.scheduled:ping" {
|
||||
sawNamedTraceCreate = true
|
||||
}
|
||||
}
|
||||
}
|
||||
if !sawNamedTraceCreate {
|
||||
t.Error("standalone run should emit a trace-create with name=asynq.scheduled:ping")
|
||||
}
|
||||
}
|
||||
@@ -15,6 +15,15 @@ import (
|
||||
// into many unrelated traces in the Langfuse UI.
|
||||
var traceCtxKey = types.LangfuseTraceContextKey
|
||||
|
||||
// parentObsCtxKey tracks the current "parent observation" id — i.e. the span
|
||||
// that encloses any generation/span started under it. Unlike traceCtxKey, this
|
||||
// key is private: spans are a pure tracer concern and logger.CloneContext
|
||||
// should reset the parent across request boundaries (we don't want a leftover
|
||||
// span id from one request leaking into another's LLM calls).
|
||||
type parentObsCtxKeyType struct{}
|
||||
|
||||
var parentObsCtxKey = parentObsCtxKeyType{}
|
||||
|
||||
// withTrace stores a *Trace on the context so downstream LLM wrappers can
|
||||
// attach their generations to it.
|
||||
func withTrace(ctx context.Context, t *Trace) context.Context {
|
||||
@@ -38,3 +47,21 @@ func traceFromCtx(ctx context.Context) (*Trace, bool) {
|
||||
func TraceFromContext(ctx context.Context) (*Trace, bool) {
|
||||
return traceFromCtx(ctx)
|
||||
}
|
||||
|
||||
// withParentObservation stores the id of the enclosing span so children
|
||||
// (sub-spans or generations) can attach via parentObservationId.
|
||||
func withParentObservation(ctx context.Context, id string) context.Context {
|
||||
if ctx == nil || id == "" {
|
||||
return ctx
|
||||
}
|
||||
return context.WithValue(ctx, parentObsCtxKey, id)
|
||||
}
|
||||
|
||||
// parentObservationFromCtx returns the enclosing span id if any.
|
||||
func parentObservationFromCtx(ctx context.Context) (string, bool) {
|
||||
if ctx == nil {
|
||||
return "", false
|
||||
}
|
||||
v, ok := ctx.Value(parentObsCtxKey).(string)
|
||||
return v, ok && v != ""
|
||||
}
|
||||
|
||||
@@ -57,14 +57,32 @@ func GinMiddleware() gin.HandlerFunc {
|
||||
}
|
||||
}
|
||||
|
||||
// shouldTrace restricts tracing to chat / search / agent / evaluation
|
||||
// endpoints where LLMs run. Everything else (auth, list, config, static…)
|
||||
// is skipped to keep the Langfuse dashboard signal-to-noise high.
|
||||
// shouldTrace restricts tracing to endpoints where LLM work (or the asynq
|
||||
// jobs that will run LLM work) originates. Everything else — auth, list,
|
||||
// config fetches, static assets, health checks — is skipped to keep the
|
||||
// Langfuse dashboard's signal-to-noise ratio high.
|
||||
//
|
||||
// The list below is grouped by purpose:
|
||||
// - online inference: knowledge-chat / agent-chat / knowledge-search /
|
||||
// generate_title are the existing chat/retrieval surface.
|
||||
// - ingestion: POST on knowledge-bases/:id/knowledge (file / url / manual)
|
||||
// and reparse / move / copy all kick off asynq jobs that later run
|
||||
// embedding/VLM/chat calls. Tracing the HTTP side means the Langfuse UI
|
||||
// shows a parent trace whose children are the worker spans.
|
||||
// - batch ops: FAQ import + knowledge batch delete also enqueue jobs.
|
||||
// - model/setup diagnostics: initialization endpoints exercise live
|
||||
// models; evaluation runs arbitrary chat pipelines.
|
||||
// - wiki: auto-fix kicks off wiki ingest, which calls embedding.
|
||||
//
|
||||
// Read-only listing / GET endpoints are deliberately excluded; they never
|
||||
// trigger LLM work and would only add noise.
|
||||
func shouldTrace(c *gin.Context) bool {
|
||||
path := c.FullPath()
|
||||
if path == "" {
|
||||
return false
|
||||
}
|
||||
method := c.Request.Method
|
||||
// Online inference
|
||||
switch {
|
||||
case strings.HasPrefix(path, "/api/v1/knowledge-chat"),
|
||||
strings.HasPrefix(path, "/api/v1/agent-chat"),
|
||||
@@ -72,9 +90,45 @@ func shouldTrace(c *gin.Context) bool {
|
||||
strings.HasPrefix(path, "/api/v1/sessions") && strings.Contains(path, "generate_title"),
|
||||
strings.HasPrefix(path, "/api/v1/initialization/remote/check"),
|
||||
strings.HasPrefix(path, "/api/v1/initialization/embedding/test"),
|
||||
strings.HasPrefix(path, "/api/v1/initialization/rerank/check"),
|
||||
strings.HasPrefix(path, "/api/v1/initialization/asr/check"),
|
||||
strings.HasPrefix(path, "/api/v1/initialization/multimodal/test"),
|
||||
strings.HasPrefix(path, "/api/v1/initialization/extract/"),
|
||||
strings.HasPrefix(path, "/api/v1/evaluation"):
|
||||
return true
|
||||
}
|
||||
// Ingestion (all POST/PUT that enqueue LLM-backed async work)
|
||||
if method == "POST" || method == "PUT" {
|
||||
switch {
|
||||
// Per-knowledge-base ingestion surface.
|
||||
case strings.Contains(path, "/knowledge-bases/") && strings.Contains(path, "/knowledge/"):
|
||||
return true
|
||||
// Knowledge-level mutations that trigger re-processing.
|
||||
case strings.HasPrefix(path, "/api/v1/knowledge/") &&
|
||||
(strings.HasSuffix(path, "/reparse") ||
|
||||
strings.HasSuffix(path, "/move") ||
|
||||
strings.Contains(path, "/manual/")):
|
||||
return true
|
||||
// Knowledge base copy (clones an entire KB, fanning out documents).
|
||||
case path == "/api/v1/knowledge-bases/copy":
|
||||
return true
|
||||
// FAQ bulk import.
|
||||
case strings.Contains(path, "/faq/entries") ||
|
||||
strings.Contains(path, "/faq/entry") ||
|
||||
strings.Contains(path, "/faq/import"):
|
||||
return true
|
||||
// Wiki auto-fix enqueues wiki ingest.
|
||||
case strings.Contains(path, "/wiki/auto-fix") ||
|
||||
strings.Contains(path, "/wiki/rebuild-links"):
|
||||
return true
|
||||
// Chunk-level mutations that rerun embeddings on update.
|
||||
case strings.HasPrefix(path, "/api/v1/chunks/") && method == "PUT":
|
||||
return true
|
||||
// Manual data source sync triggers asynq sync.
|
||||
case strings.Contains(path, "/datasource/") && strings.HasSuffix(path, "/sync"):
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
|
||||
@@ -16,13 +16,27 @@ type Trace struct {
|
||||
|
||||
// Generation represents a single model invocation (LLM / embedding / VLM).
|
||||
type Generation struct {
|
||||
ID string
|
||||
TraceID string
|
||||
manager *Manager
|
||||
sampled bool
|
||||
startTime time.Time
|
||||
model string
|
||||
name string
|
||||
ID string
|
||||
TraceID string
|
||||
ParentObservationID string
|
||||
manager *Manager
|
||||
sampled bool
|
||||
startTime time.Time
|
||||
model string
|
||||
name string
|
||||
}
|
||||
|
||||
// Span represents a logical unit of work that isn't itself an LLM call — for
|
||||
// example an asynq task execution, a pipeline stage, or a document-processing
|
||||
// step. Generations and nested spans attach to it via parentObservationId.
|
||||
type Span struct {
|
||||
ID string
|
||||
TraceID string
|
||||
ParentObservationID string
|
||||
manager *Manager
|
||||
sampled bool
|
||||
startTime time.Time
|
||||
name string
|
||||
}
|
||||
|
||||
// TraceOptions configures a new trace.
|
||||
@@ -46,6 +60,13 @@ type GenerationOptions struct {
|
||||
ModelParameters map[string]interface{}
|
||||
}
|
||||
|
||||
// SpanOptions configures a new SPAN observation.
|
||||
type SpanOptions struct {
|
||||
Name string
|
||||
Input interface{}
|
||||
Metadata map[string]interface{}
|
||||
}
|
||||
|
||||
// StartTrace opens a new trace, stores its ID in the returned ctx, and returns
|
||||
// a handle callers can finish with FinishTrace. When the manager is disabled
|
||||
// or sampling excludes the trace, the returned *Trace is non-nil but all
|
||||
@@ -110,8 +131,109 @@ func (t *Trace) Finish(output interface{}, metadata map[string]interface{}) {
|
||||
})
|
||||
}
|
||||
|
||||
// ResumeTrace reconstructs a *Trace handle from an upstream-provided trace id
|
||||
// (and optional parent observation id), without emitting a new trace-create
|
||||
// event. Used by the asynq middleware to graft async work onto the HTTP-level
|
||||
// trace that originated it: the HTTP layer already issued trace-create, and
|
||||
// the worker only needs to add child observations.
|
||||
//
|
||||
// When traceID is empty the returned *Trace is nil, signalling the caller
|
||||
// should fall back to StartTrace if it wants a standalone root.
|
||||
func (m *Manager) ResumeTrace(ctx context.Context, traceID, parentObservationID string) (context.Context, *Trace) {
|
||||
if m == nil || !m.cfg.Enabled || traceID == "" {
|
||||
return ctx, nil
|
||||
}
|
||||
t := &Trace{ID: traceID, manager: m, sampled: true}
|
||||
ctx = withTrace(ctx, t)
|
||||
if parentObservationID != "" {
|
||||
ctx = withParentObservation(ctx, parentObservationID)
|
||||
}
|
||||
return ctx, t
|
||||
}
|
||||
|
||||
// StartSpan opens a SPAN observation under the trace (and optional parent
|
||||
// observation) carried by ctx. When no trace is present, a shallow trace is
|
||||
// auto-created, mirroring StartGeneration's behaviour. Returns a ctx whose
|
||||
// parentObservationId is set to this span's id, so any nested span or
|
||||
// generation will properly attach as a child.
|
||||
func (m *Manager) StartSpan(ctx context.Context, opts SpanOptions) (context.Context, *Span) {
|
||||
if m == nil || !m.cfg.Enabled {
|
||||
return ctx, &Span{}
|
||||
}
|
||||
trace, ok := traceFromCtx(ctx)
|
||||
if !ok || trace == nil {
|
||||
newCtx, t := m.StartTrace(ctx, TraceOptions{Name: opts.Name})
|
||||
ctx = newCtx
|
||||
trace = t
|
||||
}
|
||||
if !trace.sampled {
|
||||
return ctx, &Span{}
|
||||
}
|
||||
parent, _ := parentObservationFromCtx(ctx)
|
||||
now := time.Now()
|
||||
s := &Span{
|
||||
ID: newID(),
|
||||
TraceID: trace.ID,
|
||||
ParentObservationID: parent,
|
||||
manager: m,
|
||||
sampled: true,
|
||||
startTime: now,
|
||||
name: opts.Name,
|
||||
}
|
||||
body := observationBody{
|
||||
ID: s.ID,
|
||||
TraceID: s.TraceID,
|
||||
ParentObservationID: parent,
|
||||
Type: "SPAN",
|
||||
Name: opts.Name,
|
||||
StartTime: isoTime(now),
|
||||
Input: opts.Input,
|
||||
Metadata: opts.Metadata,
|
||||
}
|
||||
m.enqueue(ingestionEvent{
|
||||
ID: newID(),
|
||||
Timestamp: isoTime(now),
|
||||
Type: "span-create",
|
||||
Body: body,
|
||||
})
|
||||
ctx = withParentObservation(ctx, s.ID)
|
||||
return ctx, s
|
||||
}
|
||||
|
||||
// Finish updates a span with its final output, extra metadata and any error.
|
||||
// A non-nil err marks the span as ERROR level in Langfuse.
|
||||
func (s *Span) Finish(output interface{}, metadata map[string]interface{}, err error) {
|
||||
if s == nil || s.manager == nil || !s.sampled {
|
||||
return
|
||||
}
|
||||
level := "DEFAULT"
|
||||
var statusMsg string
|
||||
if err != nil {
|
||||
level = "ERROR"
|
||||
statusMsg = err.Error()
|
||||
}
|
||||
body := observationBody{
|
||||
ID: s.ID,
|
||||
TraceID: s.TraceID,
|
||||
Type: "SPAN",
|
||||
EndTime: isoTime(time.Now()),
|
||||
Output: output,
|
||||
Metadata: metadata,
|
||||
Level: level,
|
||||
StatusMessage: statusMsg,
|
||||
}
|
||||
s.manager.enqueue(ingestionEvent{
|
||||
ID: newID(),
|
||||
Timestamp: isoTime(time.Now()),
|
||||
Type: "span-update",
|
||||
Body: body,
|
||||
})
|
||||
}
|
||||
|
||||
// StartGeneration opens a generation observation under the trace carried by
|
||||
// ctx (or a newly auto-created trace if none is present).
|
||||
// ctx (or a newly auto-created trace if none is present). If a parent span
|
||||
// is present on ctx, the generation attaches under it via parentObservationId
|
||||
// so the Langfuse tree shows: trace → span → generation.
|
||||
func (m *Manager) StartGeneration(ctx context.Context, opts GenerationOptions) (context.Context, *Generation) {
|
||||
if m == nil || !m.cfg.Enabled {
|
||||
return ctx, &Generation{}
|
||||
@@ -128,26 +250,29 @@ func (m *Manager) StartGeneration(ctx context.Context, opts GenerationOptions) (
|
||||
if !trace.sampled {
|
||||
return ctx, &Generation{}
|
||||
}
|
||||
parent, _ := parentObservationFromCtx(ctx)
|
||||
now := time.Now()
|
||||
g := &Generation{
|
||||
ID: newID(),
|
||||
TraceID: trace.ID,
|
||||
manager: m,
|
||||
sampled: true,
|
||||
startTime: now,
|
||||
model: opts.Model,
|
||||
name: opts.Name,
|
||||
ID: newID(),
|
||||
TraceID: trace.ID,
|
||||
ParentObservationID: parent,
|
||||
manager: m,
|
||||
sampled: true,
|
||||
startTime: now,
|
||||
model: opts.Model,
|
||||
name: opts.Name,
|
||||
}
|
||||
body := observationBody{
|
||||
ID: g.ID,
|
||||
TraceID: g.TraceID,
|
||||
Type: "GENERATION",
|
||||
Name: opts.Name,
|
||||
StartTime: isoTime(now),
|
||||
Input: opts.Input,
|
||||
Metadata: opts.Metadata,
|
||||
Model: opts.Model,
|
||||
ModelParameters: opts.ModelParameters,
|
||||
ID: g.ID,
|
||||
TraceID: g.TraceID,
|
||||
ParentObservationID: parent,
|
||||
Type: "GENERATION",
|
||||
Name: opts.Name,
|
||||
StartTime: isoTime(now),
|
||||
Input: opts.Input,
|
||||
Metadata: opts.Metadata,
|
||||
Model: opts.Model,
|
||||
ModelParameters: opts.ModelParameters,
|
||||
}
|
||||
m.enqueue(ingestionEvent{
|
||||
ID: newID(),
|
||||
|
||||
@@ -0,0 +1,195 @@
|
||||
package langfuse
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"io"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"sync"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
// newTestServer spins up a fake Langfuse ingestion endpoint and returns the
|
||||
// collected events + a cleanup func.
|
||||
func newTestServer(t *testing.T) (*httptest.Server, func() []ingestionEvent) {
|
||||
t.Helper()
|
||||
var mu sync.Mutex
|
||||
var batches []ingestionRequest
|
||||
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
body, _ := io.ReadAll(r.Body)
|
||||
var req ingestionRequest
|
||||
_ = json.Unmarshal(body, &req)
|
||||
mu.Lock()
|
||||
batches = append(batches, req)
|
||||
mu.Unlock()
|
||||
w.WriteHeader(200)
|
||||
_, _ = w.Write([]byte(`{}`))
|
||||
}))
|
||||
drain := func() []ingestionEvent {
|
||||
mu.Lock()
|
||||
defer mu.Unlock()
|
||||
var out []ingestionEvent
|
||||
for _, b := range batches {
|
||||
out = append(out, b.Batch...)
|
||||
}
|
||||
return out
|
||||
}
|
||||
return srv, drain
|
||||
}
|
||||
|
||||
func newTestManager(t *testing.T, host string) *Manager {
|
||||
t.Helper()
|
||||
m, err := Init(Config{
|
||||
Enabled: true,
|
||||
Host: host,
|
||||
PublicKey: "pk",
|
||||
SecretKey: "sk",
|
||||
FlushAt: 1,
|
||||
FlushInterval: 5 * time.Millisecond,
|
||||
QueueSize: 32,
|
||||
RequestTimeout: 2 * time.Second,
|
||||
SampleRate: 1.0,
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("init: %v", err)
|
||||
}
|
||||
return m
|
||||
}
|
||||
|
||||
// TestSpan_NestedHierarchy verifies that nested StartSpan calls produce a
|
||||
// trace → span₁ → span₂ → generation hierarchy, with parentObservationId
|
||||
// pointing to the direct ancestor at each level.
|
||||
func TestSpan_NestedHierarchy(t *testing.T) {
|
||||
srv, drain := newTestServer(t)
|
||||
defer srv.Close()
|
||||
m := newTestManager(t, srv.URL)
|
||||
|
||||
ctx, trace := m.StartTrace(context.Background(), TraceOptions{Name: "root"})
|
||||
ctx, outer := m.StartSpan(ctx, SpanOptions{Name: "outer"})
|
||||
ctx, inner := m.StartSpan(ctx, SpanOptions{Name: "inner"})
|
||||
_, gen := m.StartGeneration(ctx, GenerationOptions{Name: "llm", Model: "m"})
|
||||
|
||||
gen.Finish("out", &TokenUsage{Input: 1, Output: 2, Total: 3}, nil)
|
||||
inner.Finish("inner-out", nil, nil)
|
||||
outer.Finish("outer-out", nil, nil)
|
||||
trace.Finish("root-out", nil)
|
||||
|
||||
if err := m.Shutdown(context.Background()); err != nil {
|
||||
t.Fatalf("shutdown: %v", err)
|
||||
}
|
||||
|
||||
events := drain()
|
||||
parentOf := map[string]string{}
|
||||
traceOf := map[string]string{}
|
||||
kindOf := map[string]string{}
|
||||
for _, ev := range events {
|
||||
if ev.Type != "span-create" && ev.Type != "generation-create" {
|
||||
continue
|
||||
}
|
||||
b, _ := json.Marshal(ev.Body)
|
||||
var ob observationBody
|
||||
_ = json.Unmarshal(b, &ob)
|
||||
parentOf[ob.ID] = ob.ParentObservationID
|
||||
traceOf[ob.ID] = ob.TraceID
|
||||
kindOf[ob.ID] = ob.Type
|
||||
}
|
||||
|
||||
if traceOf[outer.ID] != trace.ID || parentOf[outer.ID] != "" {
|
||||
t.Errorf("outer span should sit directly under the trace, got parent=%q trace=%q", parentOf[outer.ID], traceOf[outer.ID])
|
||||
}
|
||||
if parentOf[inner.ID] != outer.ID {
|
||||
t.Errorf("inner span parent mismatch: got %q want %q", parentOf[inner.ID], outer.ID)
|
||||
}
|
||||
if parentOf[gen.ID] != inner.ID {
|
||||
t.Errorf("generation should nest under inner span, got parent=%q want %q", parentOf[gen.ID], inner.ID)
|
||||
}
|
||||
if kindOf[outer.ID] != "SPAN" || kindOf[inner.ID] != "SPAN" {
|
||||
t.Errorf("expected both wrappers to be SPAN observations, got outer=%q inner=%q", kindOf[outer.ID], kindOf[inner.ID])
|
||||
}
|
||||
if kindOf[gen.ID] != "GENERATION" {
|
||||
t.Errorf("expected generation kind, got %q", kindOf[gen.ID])
|
||||
}
|
||||
}
|
||||
|
||||
// TestSpan_FinishWithError records an error status on the span so failures in
|
||||
// asynq handlers surface as red observations in Langfuse.
|
||||
func TestSpan_FinishWithError(t *testing.T) {
|
||||
srv, drain := newTestServer(t)
|
||||
defer srv.Close()
|
||||
m := newTestManager(t, srv.URL)
|
||||
|
||||
ctx, _ := m.StartTrace(context.Background(), TraceOptions{Name: "root"})
|
||||
_, span := m.StartSpan(ctx, SpanOptions{Name: "boom"})
|
||||
span.Finish(nil, nil, errors.New("kaboom"))
|
||||
|
||||
if err := m.Shutdown(context.Background()); err != nil {
|
||||
t.Fatalf("shutdown: %v", err)
|
||||
}
|
||||
|
||||
var sawError bool
|
||||
for _, ev := range drain() {
|
||||
if ev.Type != "span-update" {
|
||||
continue
|
||||
}
|
||||
b, _ := json.Marshal(ev.Body)
|
||||
var ob observationBody
|
||||
_ = json.Unmarshal(b, &ob)
|
||||
if ob.Level == "ERROR" && ob.StatusMessage == "kaboom" {
|
||||
sawError = true
|
||||
}
|
||||
}
|
||||
if !sawError {
|
||||
t.Fatal("expected span-update with ERROR level and status message")
|
||||
}
|
||||
}
|
||||
|
||||
// TestResumeTrace_NoTraceCreateEvent verifies that ResumeTrace does NOT
|
||||
// emit a trace-create event — the originating HTTP handler already did,
|
||||
// and a duplicate would register as an orphan root in the Langfuse UI.
|
||||
func TestResumeTrace_NoTraceCreateEvent(t *testing.T) {
|
||||
srv, drain := newTestServer(t)
|
||||
defer srv.Close()
|
||||
m := newTestManager(t, srv.URL)
|
||||
|
||||
ctx, trace := m.ResumeTrace(context.Background(), "upstream-trace", "upstream-span")
|
||||
if trace == nil || trace.ID != "upstream-trace" {
|
||||
t.Fatalf("expected resumed trace with id upstream-trace, got %+v", trace)
|
||||
}
|
||||
if pid, ok := parentObservationFromCtx(ctx); !ok || pid != "upstream-span" {
|
||||
t.Errorf("expected parent observation upstream-span on ctx, got %q (ok=%v)", pid, ok)
|
||||
}
|
||||
|
||||
// Emit one child so there's *something* to flush, proving only child
|
||||
// observations reach the wire, not a trace-create for the resume.
|
||||
_, span := m.StartSpan(ctx, SpanOptions{Name: "child"})
|
||||
span.Finish(nil, nil, nil)
|
||||
|
||||
if err := m.Shutdown(context.Background()); err != nil {
|
||||
t.Fatalf("shutdown: %v", err)
|
||||
}
|
||||
|
||||
for _, ev := range drain() {
|
||||
if ev.Type == "trace-create" {
|
||||
t.Fatalf("ResumeTrace must not emit trace-create events, got %+v", ev)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// TestResumeTrace_DisabledIsSafe guards against nil deref when Langfuse is
|
||||
// off: ResumeTrace should return a nil *Trace and the original ctx unchanged.
|
||||
func TestResumeTrace_DisabledIsSafe(t *testing.T) {
|
||||
m, err := Init(Config{Enabled: false})
|
||||
if err != nil {
|
||||
t.Fatalf("init: %v", err)
|
||||
}
|
||||
ctx, trace := m.ResumeTrace(context.Background(), "x", "y")
|
||||
if trace != nil {
|
||||
t.Errorf("expected nil trace when disabled, got %+v", trace)
|
||||
}
|
||||
if _, ok := parentObservationFromCtx(ctx); ok {
|
||||
t.Error("disabled ResumeTrace should not attach parent observation")
|
||||
}
|
||||
}
|
||||
@@ -314,6 +314,8 @@ type SyncResult struct {
|
||||
|
||||
// DataSourceSyncPayload represents the asynq task payload for data source sync
|
||||
type DataSourceSyncPayload struct {
|
||||
TracingContext
|
||||
|
||||
// Data source ID to sync
|
||||
DataSourceID string `json:"data_source_id"`
|
||||
|
||||
|
||||
@@ -21,6 +21,7 @@ const (
|
||||
|
||||
// ExtractChunkPayload represents the extract chunk task payload
|
||||
type ExtractChunkPayload struct {
|
||||
TracingContext
|
||||
TenantID uint64 `json:"tenant_id"`
|
||||
ChunkID string `json:"chunk_id"`
|
||||
ModelID string `json:"model_id"`
|
||||
@@ -28,6 +29,7 @@ type ExtractChunkPayload struct {
|
||||
|
||||
// DocumentProcessPayload represents the document process task payload
|
||||
type DocumentProcessPayload struct {
|
||||
TracingContext
|
||||
RequestId string `json:"request_id"`
|
||||
TenantID uint64 `json:"tenant_id"`
|
||||
KnowledgeID string `json:"knowledge_id"`
|
||||
@@ -46,6 +48,7 @@ type DocumentProcessPayload struct {
|
||||
|
||||
// FAQImportPayload represents the FAQ import task payload (including dry run mode)
|
||||
type FAQImportPayload struct {
|
||||
TracingContext
|
||||
TenantID uint64 `json:"tenant_id"`
|
||||
TaskID string `json:"task_id"`
|
||||
KBID string `json:"kb_id"`
|
||||
@@ -60,6 +63,7 @@ type FAQImportPayload struct {
|
||||
|
||||
// QuestionGenerationPayload represents the question generation task payload
|
||||
type QuestionGenerationPayload struct {
|
||||
TracingContext
|
||||
TenantID uint64 `json:"tenant_id"`
|
||||
KnowledgeBaseID string `json:"knowledge_base_id"`
|
||||
KnowledgeID string `json:"knowledge_id"`
|
||||
@@ -70,6 +74,7 @@ type QuestionGenerationPayload struct {
|
||||
|
||||
// SummaryGenerationPayload represents the summary generation task payload
|
||||
type SummaryGenerationPayload struct {
|
||||
TracingContext
|
||||
TenantID uint64 `json:"tenant_id"`
|
||||
KnowledgeBaseID string `json:"knowledge_base_id"`
|
||||
KnowledgeID string `json:"knowledge_id"`
|
||||
@@ -78,6 +83,7 @@ type SummaryGenerationPayload struct {
|
||||
|
||||
// KBClonePayload represents the knowledge base clone task payload
|
||||
type KBClonePayload struct {
|
||||
TracingContext
|
||||
TenantID uint64 `json:"tenant_id"`
|
||||
TaskID string `json:"task_id"`
|
||||
SourceID string `json:"source_id"`
|
||||
@@ -86,6 +92,7 @@ type KBClonePayload struct {
|
||||
|
||||
// IndexDeletePayload represents the index delete task payload
|
||||
type IndexDeletePayload struct {
|
||||
TracingContext
|
||||
TenantID uint64 `json:"tenant_id"`
|
||||
KnowledgeBaseID string `json:"knowledge_base_id"`
|
||||
EmbeddingModelID string `json:"embedding_model_id"`
|
||||
@@ -96,6 +103,7 @@ type IndexDeletePayload struct {
|
||||
|
||||
// KBDeletePayload represents the knowledge base delete task payload
|
||||
type KBDeletePayload struct {
|
||||
TracingContext
|
||||
TenantID uint64 `json:"tenant_id"`
|
||||
KnowledgeBaseID string `json:"knowledge_base_id"`
|
||||
EffectiveEngines []RetrieverEngineParams `json:"effective_engines"`
|
||||
@@ -103,12 +111,14 @@ type KBDeletePayload struct {
|
||||
|
||||
// KnowledgeListDeletePayload represents the batch knowledge delete task payload
|
||||
type KnowledgeListDeletePayload struct {
|
||||
TracingContext
|
||||
TenantID uint64 `json:"tenant_id"`
|
||||
KnowledgeIDs []string `json:"knowledge_ids"`
|
||||
}
|
||||
|
||||
// KnowledgeMovePayload represents the knowledge move task payload
|
||||
type KnowledgeMovePayload struct {
|
||||
TracingContext
|
||||
TenantID uint64 `json:"tenant_id"`
|
||||
TaskID string `json:"task_id"`
|
||||
KnowledgeIDs []string `json:"knowledge_ids"`
|
||||
@@ -136,6 +146,7 @@ type KnowledgeMoveProgress struct {
|
||||
// ManualProcessPayload represents the manual knowledge processing task payload.
|
||||
// Used for both create (publish) and update operations.
|
||||
type ManualProcessPayload struct {
|
||||
TracingContext
|
||||
RequestId string `json:"request_id"`
|
||||
TenantID uint64 `json:"tenant_id"`
|
||||
KnowledgeID string `json:"knowledge_id"`
|
||||
@@ -146,6 +157,7 @@ type ManualProcessPayload struct {
|
||||
|
||||
// ImageMultimodalPayload represents the image multimodal processing task payload.
|
||||
type ImageMultimodalPayload struct {
|
||||
TracingContext
|
||||
TenantID uint64 `json:"tenant_id"`
|
||||
KnowledgeID string `json:"knowledge_id"`
|
||||
KnowledgeBaseID string `json:"knowledge_base_id"`
|
||||
@@ -160,6 +172,7 @@ type ImageMultimodalPayload struct {
|
||||
|
||||
// KnowledgePostProcessPayload represents the knowledge post process task payload.
|
||||
type KnowledgePostProcessPayload struct {
|
||||
TracingContext
|
||||
TenantID uint64 `json:"tenant_id"`
|
||||
KnowledgeID string `json:"knowledge_id"`
|
||||
KnowledgeBaseID string `json:"knowledge_base_id"`
|
||||
|
||||
@@ -0,0 +1,60 @@
|
||||
package types
|
||||
|
||||
// TracingContext is an embeddable struct that carries observability context
|
||||
// (currently Langfuse trace/span ids plus user/session hints) across process
|
||||
// boundaries — specifically, from an HTTP request into an asynq task payload
|
||||
// and back out inside the worker.
|
||||
//
|
||||
// It lives in the types package, not the langfuse package, so that:
|
||||
//
|
||||
// - asynq payload structs can embed it without pulling the langfuse
|
||||
// package into every service/handler import graph;
|
||||
// - the langfuse package can remain a leaf dependency that only types
|
||||
// (and its own tests) reference directly.
|
||||
//
|
||||
// The JSON tags all use the "lf_" prefix and omitempty so that payloads
|
||||
// constructed before the Langfuse feature landed remain byte-compatible
|
||||
// (empty fields collapse to nothing in the serialized output) and so that
|
||||
// Langfuse-specific columns don't collide with business fields that may
|
||||
// happen to be named similarly.
|
||||
type TracingContext struct {
|
||||
// LangfuseTraceID is the id of the root trace that originated this task.
|
||||
// When set, workers attach their observations to the same trace instead
|
||||
// of creating a standalone one, which is what makes the Langfuse UI
|
||||
// show the asynq work as a child of the originating HTTP request.
|
||||
LangfuseTraceID string `json:"lf_trace_id,omitempty"`
|
||||
// LangfuseParentObservationID, when set, points to the span that
|
||||
// enclosed the enqueue call. The worker's own span/generation will list
|
||||
// it as parentObservationId so Langfuse renders the tree as:
|
||||
// http-trace → http-span → asynq-span → generation.
|
||||
LangfuseParentObservationID string `json:"lf_parent_obs_id,omitempty"`
|
||||
// LangfuseUserID preserves the userId / tenant label across the async
|
||||
// boundary so that orphan async traces (when no upstream trace id is
|
||||
// available) still show up in the Langfuse "Users" view under the
|
||||
// right tenant.
|
||||
LangfuseUserID string `json:"lf_user_id,omitempty"`
|
||||
// LangfuseSessionID preserves the sessionId for the same reason.
|
||||
LangfuseSessionID string `json:"lf_session_id,omitempty"`
|
||||
}
|
||||
|
||||
// SetLangfuseTracing overwrites the embedded TracingContext. Method is
|
||||
// exported so helpers in internal/tracing/langfuse can populate it via the
|
||||
// LangfuseTracingCarrier interface without reflection.
|
||||
func (tc *TracingContext) SetLangfuseTracing(other TracingContext) {
|
||||
*tc = other
|
||||
}
|
||||
|
||||
// GetLangfuseTracing returns a copy of the embedded TracingContext.
|
||||
func (tc TracingContext) GetLangfuseTracing() TracingContext {
|
||||
return tc
|
||||
}
|
||||
|
||||
// LangfuseTracingCarrier is implemented automatically by any struct that
|
||||
// embeds TracingContext (thanks to Go's method promotion rules). The asynq
|
||||
// enqueue helper in internal/tracing/langfuse uses this interface to inject
|
||||
// the current trace/span ids into the payload without caring about the
|
||||
// concrete payload type.
|
||||
type LangfuseTracingCarrier interface {
|
||||
SetLangfuseTracing(TracingContext)
|
||||
GetLangfuseTracing() TracingContext
|
||||
}
|
||||
Reference in New Issue
Block a user