feat(tracing): migrate langfuse client to OTLP/OTel wire protocol

Migrate WeKnora's Langfuse tracing from the legacy /api/public/ingestion
batch API to the OpenTelemetry Go SDK emitting OTLP/HTTP to
/api/public/otel/v1/traces — the Langfuse v3+ standard ingestion protocol.

Why: Langfuse v3 deprecated the legacy ingestion API in favor of OTLP, and
OTel-only backends (e.g. the LiteFuse Doris fork) dropped it entirely, so the
legacy client was incompatible. Migrating to the OTel SDK also adds W3C
traceparent cross-service propagation: an upstream caller's trace id is
inherited so its run and the WeKnora call land under one trace in the UI.

What:
- Manager owns an OTel TracerProvider backed by an otlptracehttp exporter
  (Basic auth + the x-langfuse-ingestion-version gate header). Spans are
  buffered by BatchSpanProcessor and exported complete on End, so a long
  trace no longer fragments across flush batches.
- *Trace/*Span/*Generation wrap OTel spans; the public API (StartTrace/
  Span/Generation, handles, InjectTracing, asynq/gin middleware, Config,
  TokenUsage) is unchanged, so callers (agent/chat/handler/scheduler/asynqdl)
  need no edits.
- W3C traceparent: GinMiddleware extracts traceparent from incoming requests;
  AsynqMiddleware + InjectTracing propagate it across the asynq payload.
  types.TracingContext gains a LangfuseTraceparent field.
- logger.CloneContext preserves the active OTel span across context rebuilds
  so child spans (agent engine, summary generation) stitch to the HTTP trace
  instead of orphaning into separate traces.
- StartSpan/StartGeneration re-establish the parent from the *Trace handle
  when the ctx lost the active OTel span, covering background/derived contexts.

Compatibility: works with Langfuse v3+ (cloud / self-hosted / the bundled
docker-compose profile) and LiteFuse. Drops Langfuse v2 (legacy-only) support.
Uses the OTel Go SDK already (indirectly) in go.mod, promoting
otlptracehttp to direct; no third-party langfuse-go library.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
shenk-b
2026-07-17 11:41:45 +08:00
committed by lyingbug
parent 1a2021b3f8
commit 5ee96019e0
16 changed files with 654 additions and 1013 deletions
+4 -3
View File
@@ -68,6 +68,10 @@ require (
github.com/weaviate/weaviate-go-client/v5 v5.7.3
github.com/xuri/excelize/v2 v2.11.0
github.com/yanyiwu/gojieba v1.4.7
go.opentelemetry.io/otel v1.43.0
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp v1.43.0
go.opentelemetry.io/otel/sdk v1.43.0
go.opentelemetry.io/otel/trace v1.43.0
go.uber.org/dig v1.19.0
golang.org/x/crypto v0.53.0
golang.org/x/mod v0.36.0
@@ -312,12 +316,9 @@ require (
go.opentelemetry.io/auto/sdk v1.2.1 // indirect
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.67.0 // indirect
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.68.0 // indirect
go.opentelemetry.io/otel v1.43.0 // indirect
go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.43.0 // indirect
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.43.0 // indirect
go.opentelemetry.io/otel/metric v1.43.0 // indirect
go.opentelemetry.io/otel/sdk v1.43.0 // indirect
go.opentelemetry.io/otel/trace v1.43.0 // indirect
go.opentelemetry.io/proto/otlp v1.10.0 // indirect
go.uber.org/atomic v1.11.0 // indirect
go.uber.org/automaxprocs v1.5.3 // indirect
+2
View File
@@ -2685,6 +2685,8 @@ go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.43.0/go.mod h1:Vl1/iaggsuRl
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.0.1/go.mod h1:xOvWoTOrQjxjW61xtOmD/WKGRYb/P4NzRo3bs65U6Rk=
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.43.0 h1:RAE+JPfvEmvy+0LzyUA25/SGawPwIUbZ6u0Wug54sLc=
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.43.0/go.mod h1:AGmbycVGEsRx9mXMZ75CsOyhSP6MFIcj/6dnG+vhVjk=
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp v1.43.0 h1:3iZJKlCZufyRzPzlQhUIWVmfltrXuGyfjREgGP3UUjc=
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp v1.43.0/go.mod h1:/G+nUPfhq2e+qiXMGxMwumDrP5jtzU+mWN7/sjT2rak=
go.opentelemetry.io/otel/metric v1.19.0/go.mod h1:L5rUsV9kM1IxCj1MmSdS+JQAcVm319EUrDVLrt7jqt8=
go.opentelemetry.io/otel/metric v1.21.0/go.mod h1:o1p3CA8nNHW8j5yuQLdc1eeqEaPfzug24uvsyIEJRWM=
go.opentelemetry.io/otel/metric v1.22.0/go.mod h1:evJGjVpZv0mQ5QBRJoBF64yMuOf4xCWdXjK8pzFvliY=
+11
View File
@@ -15,6 +15,7 @@ import (
"github.com/Tencent/WeKnora/internal/types"
"github.com/sirupsen/logrus"
"go.opentelemetry.io/otel/trace"
"gopkg.in/natefinch/lumberjack.v2"
)
@@ -538,5 +539,15 @@ func CloneContext(ctx context.Context) context.Context {
}
}
// Preserve the active OpenTelemetry span across the rebuild. The Langfuse
// *Trace handle above carries the trace id, but span PARENTING flows through
// the OTel span context (trace.SpanFromContext), which CloneContext would
// otherwise drop — orphaning child spans opened after a CloneContext (e.g.
// the agent engine's agent.execute becoming a separate trace from the HTTP
// root). Re-inject the recording span so children stitch to the same trace.
if sp := trace.SpanFromContext(ctx); sp.IsRecording() {
newCtx = trace.ContextWithSpan(newCtx, sp)
}
return newCtx
}
+37 -39
View File
@@ -7,20 +7,24 @@ import (
"github.com/Tencent/WeKnora/internal/types"
"github.com/hibiken/asynq"
"go.opentelemetry.io/otel/propagation"
oteltrace "go.opentelemetry.io/otel/trace"
)
// InjectTracing stamps the current trace/span ids (and a best-effort
// InjectTracing stamps the current W3C traceparent (plus a best-effort
// user/session label) from ctx onto the given payload, provided the payload
// embeds types.TracingContext (and thus implements LangfuseTracingCarrier).
//
// Safe to call unconditionally: when Langfuse is disabled or no trace is
// present on ctx, it writes a zero-valued TracingContext — which round-trips
// through JSON as absent fields and therefore costs nothing.
// The traceparent is produced by propagator.Inject from the active OTel span
// context (the HTTP root span opened by GinMiddleware). The asynq worker
// re-extracts it so worker-side spans are children of the same trace —
// giving LiteFuse one stitched tree across the HTTP request and the async
// processing. This also makes a sop3 run's traceparent propagate through to
// any asynq jobs WeKnora enqueues while serving sop3's agent-chat call.
//
// Call sites live at every asynq.NewTask creation point. We deliberately
// keep the API synchronous and mutating (rather than returning a new payload
// copy) so that existing enqueue code needs only a single added line just
// before json.Marshal, minimizing review surface and the risk of regressions.
// Safe to call unconditionally: when Langfuse is disabled or no span is
// present on ctx, it writes a zero-valued TracingContext — which round-trips
// through JSON as absent fields and costs nothing.
func InjectTracing(ctx context.Context, carrier types.LangfuseTracingCarrier) {
if carrier == nil {
return
@@ -30,12 +34,14 @@ func InjectTracing(ctx context.Context, carrier types.LangfuseTracingCarrier) {
return
}
tc := types.TracingContext{}
c := propagation.MapCarrier{}
propagator.Inject(ctx, c)
tc.LangfuseTraceparent = c["traceparent"]
// Backward-compat: keep LangfuseTraceID = the W3C trace id for any legacy
// reader. LangfuseParentObservationID is no longer used by the OTLP path.
if trace, ok := TraceFromContext(ctx); ok && trace != nil {
tc.LangfuseTraceID = trace.ID
}
if obs, ok := parentObservationFromCtx(ctx); ok {
tc.LangfuseParentObservationID = obs
}
tc.LangfuseUserID = userIDFromCtx(ctx)
tc.LangfuseSessionID = sessionIDFromCtx(ctx)
carrier.SetLangfuseTracing(tc)
@@ -59,23 +65,20 @@ func peekTracingContext(payload []byte) types.TracingContext {
// AsynqMiddleware is the worker-side counterpart of GinMiddleware. It:
//
// 1. Parses any tracing ids embedded in the task payload and either
// resumes the originating trace (so the Langfuse UI stitches the HTTP
// request and the async processing into one tree), or — for tasks that
// came from a scheduled job with no originating HTTP request — creates
// a standalone trace tagged with the task type.
// 1. Extracts the W3C traceparent stamped onto the task payload by
// InjectTracing and resumes the originating trace (so the Langfuse UI
// stitches the HTTP request and the async processing into one tree). For
// scheduled jobs with no upstream trace it opens a standalone trace named
// after the task type.
//
// 2. Opens a SPAN around the handler execution so every child generation
// (embedding / VLM / chat / rerank / ASR) auto-attaches to it via
// parentObservationId.
// (embedding / VLM / chat / rerank / ASR) auto-attaches to it.
//
// 3. Enriches the span with asynq's own metadata: task id, queue, retry
// count, payload size.
//
// When the manager is disabled it degrades to a pass-through; failure of
// the Langfuse path never blocks task execution.
//
// Register it once in router/task.go via mux.Use.
// When the manager is disabled it degrades to a pass-through; failure of the
// Langfuse path never blocks task execution. Register once via mux.Use.
func AsynqMiddleware() asynq.MiddlewareFunc {
return func(next asynq.Handler) asynq.Handler {
return asynq.HandlerFunc(func(ctx context.Context, task *asynq.Task) error {
@@ -99,15 +102,16 @@ func AsynqMiddleware() asynq.MiddlewareFunc {
"payload_bytes": len(task.Payload()),
}
// If the upstream enqueuer stamped a trace id onto the payload,
// we graft under that trace. Otherwise (scheduled jobs, legacy
// in-flight tasks that predate this code, tests, etc.) we start
// a standalone trace named after the task type so at least the
// worker-side work is observable.
// If the upstream enqueuer stamped a traceparent, resume that trace
// (worker spans become children of the HTTP trace). Otherwise start
// a standalone trace named after the task type.
var trace *Trace
shouldFinishTrace := false
if tc.LangfuseTraceID != "" {
ctx, trace = mgr.ResumeTrace(ctx, tc.LangfuseTraceID, tc.LangfuseParentObservationID)
if tc.LangfuseTraceparent != "" {
ctx = propagator.Extract(ctx, propagation.MapCarrier{"traceparent": tc.LangfuseTraceparent})
if sc := oteltrace.SpanContextFromContext(ctx); sc.IsValid() {
ctx = withTrace(ctx, &Trace{ID: sc.TraceID().String(), manager: mgr})
}
} else {
ctx, trace = mgr.StartTrace(ctx, TraceOptions{
Name: "asynq." + task.Type(),
@@ -153,15 +157,9 @@ func AsynqMiddleware() asynq.MiddlewareFunc {
// spanInputFromPayload surfaces a compact, human-readable summary of the
// task payload for the Langfuse "Input" pane. We deliberately do NOT send
// the full JSON blob because:
//
// - Manual/text-ingest payloads can be many kilobytes of prose;
// - FAQ import payloads embed the full entry list;
// - Document-process payloads contain file URLs that, while small, may
// include presigned query strings that rotate (adding diff noise).
//
// Instead we preview the first ~1KB verbatim, which matches what Langfuse
// itself would display and keeps ingestion bandwidth predictable.
// the full JSON blob because manual/text-ingest payloads can be many
// kilobytes and FAQ import payloads embed the full entry list. Instead we
// preview the first ~1KB verbatim.
func spanInputFromPayload(payload []byte) interface{} {
const preview = 1024
if len(payload) == 0 {
@@ -177,7 +175,7 @@ func spanInputFromPayload(payload []byte) interface{} {
}
// userIDFromCtx mirrors middleware.extractUserID but accepts a raw context
// (no gin.Context) so both HTTP and asynq paths can share the same fallback
// (no gin.Context) so both HTTP and asynq paths share the same fallback
// logic: explicit UserID → tenant:<id> → empty.
func userIDFromCtx(ctx context.Context) string {
if v, ok := ctx.Value(types.UserIDContextKey).(string); ok && v != "" {
+61 -210
View File
@@ -3,12 +3,8 @@ package langfuse
import (
"context"
"encoding/json"
"io"
"net/http"
"net/http/httptest"
"sync"
"strings"
"testing"
"time"
"github.com/Tencent/WeKnora/internal/types"
"github.com/hibiken/asynq"
@@ -24,249 +20,104 @@ type dummyPayload struct {
// TestInjectTracing_DisabledIsZero verifies InjectTracing is a no-op when
// Langfuse is disabled: no panics, no trace fields written.
func TestInjectTracing_DisabledIsZero(t *testing.T) {
// Explicitly install a disabled manager to shadow any globally-enabled
// one from a previous test in the same package.
_, _ = Init(Config{Enabled: false})
p := &dummyPayload{KnowledgeID: "k1"}
InjectTracing(context.Background(), p)
if p.LangfuseTraceID != "" || p.LangfuseParentObservationID != "" {
if p.LangfuseTraceparent != "" || p.LangfuseTraceID != "" {
t.Fatalf("expected no tracing fields on disabled manager, got %+v", p.TracingContext)
}
}
// TestInjectTracing_PopulatesFromContext checks that when a trace is active
// on the context, its id is copied onto the payload and a subsequent
// peekTracingContext round-trips it correctly.
func TestInjectTracing_PopulatesFromContext(t *testing.T) {
// Stand up a fake ingestion endpoint — Init starts a real flush worker
// and we don't want it to panic on connection refused during the test.
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
_, _ = io.Copy(io.Discard, r.Body)
w.WriteHeader(200)
_, _ = w.Write([]byte(`{}`))
}))
defer srv.Close()
m, err := Init(Config{
Enabled: true,
Host: srv.URL,
PublicKey: "pk",
SecretKey: "sk",
FlushAt: 16,
FlushInterval: 50 * time.Millisecond,
QueueSize: 16,
RequestTimeout: 2 * time.Second,
SampleRate: 1.0,
})
if err != nil {
t.Fatalf("init: %v", err)
}
defer m.Shutdown(context.Background())
// TestInjectTracing_PopulatesTraceparent checks that when a trace is active
// on the context, a W3C traceparent is stamped onto the payload (so the
// asynq worker can resume the same trace).
func TestInjectTracing_PopulatesTraceparent(t *testing.T) {
m, _ := newTestManager(t)
ctx, trace := m.StartTrace(context.Background(), TraceOptions{Name: "parent"})
ctx, span := m.StartSpan(ctx, SpanOptions{Name: "wrap"})
if span == nil || span.ID == "" {
t.Fatalf("expected a span with id, got %+v", span)
}
p := &dummyPayload{KnowledgeID: "k1"}
InjectTracing(ctx, p)
if p.LangfuseTraceparent == "" {
t.Fatal("expected LangfuseTraceparent to be populated")
}
// The traceparent carries the trace id; trace.ID is the OTel trace id.
if !strings.HasPrefix(p.LangfuseTraceparent, "00-"+trace.ID) {
t.Errorf("traceparent %q does not carry trace id %s", p.LangfuseTraceparent, trace.ID)
}
if p.LangfuseTraceID != trace.ID {
t.Errorf("trace id mismatch: got %q want %q", p.LangfuseTraceID, trace.ID)
}
if p.LangfuseParentObservationID != span.ID {
t.Errorf("parent observation id mismatch: got %q want %q", p.LangfuseParentObservationID, span.ID)
}
// Round-trip the payload through JSON, which is what asynq does on the
// wire, and make sure peekTracingContext recovers the ids.
b, err := json.Marshal(p)
if err != nil {
t.Fatalf("marshal: %v", err)
}
got := peekTracingContext(b)
if got.LangfuseTraceID != trace.ID || got.LangfuseParentObservationID != span.ID {
t.Errorf("peek round-trip lost ids: %+v", got)
t.Errorf("LangfuseTraceID = %q, want %q", p.LangfuseTraceID, trace.ID)
}
}
// TestAsynqMiddleware_ResumeTrace asserts the middleware grafts a resumed
// trace onto the child handler's context, and that the ingestion endpoint
// receives span-create / span-update events for the wrapper — without
// emitting a duplicate trace-create (which would split the Langfuse UI
// tree into two disconnected roots).
func TestAsynqMiddleware_ResumeTrace(t *testing.T) {
var mu sync.Mutex
var batches []ingestionRequest
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
body, _ := io.ReadAll(r.Body)
var req ingestionRequest
_ = json.Unmarshal(body, &req)
mu.Lock()
batches = append(batches, req)
mu.Unlock()
w.WriteHeader(200)
_, _ = w.Write([]byte(`{}`))
}))
defer srv.Close()
// TestAsynqMiddleware_TraceparentPropagation is the cross-process correlation
// core test: InjectTracing stamps a traceparent onto the payload; the worker
// middleware re-extracts it, and the worker span inherits the upstream trace
// id — stitching the HTTP trace and the async job into one LiteFuse tree.
func TestAsynqMiddleware_TraceparentPropagation(t *testing.T) {
m, exp := newTestManager(t)
m, err := Init(Config{
Enabled: true,
Host: srv.URL,
PublicKey: "pk",
SecretKey: "sk",
FlushAt: 1,
FlushInterval: 5 * time.Millisecond,
QueueSize: 32,
RequestTimeout: 2 * time.Second,
SampleRate: 1.0,
})
if err != nil {
t.Fatalf("init: %v", err)
}
// Build a payload that already carries a trace id (as if the HTTP layer
// injected one at enqueue time).
// Upstream caller opens a span and injects a traceparent onto the payload.
upstreamCtx, upstreamSpan := m.Tracer().Start(context.Background(), "upstream-http")
remoteTraceID := upstreamSpan.SpanContext().TraceID()
payload := &dummyPayload{KnowledgeID: "k1"}
payload.LangfuseTraceID = "trace-xyz"
payload.LangfuseParentObservationID = "parent-span-abc"
InjectTracing(upstreamCtx, payload)
if payload.LangfuseTraceparent == "" {
t.Fatal("InjectTracing did not stamp a traceparent")
}
raw, _ := json.Marshal(payload)
var receivedTraceID, receivedParentID string
var sawTrace bool
handler := asynq.HandlerFunc(func(ctx context.Context, task *asynq.Task) error {
if tr, ok := TraceFromContext(ctx); ok && tr != nil {
sawTrace = true
receivedTraceID = tr.ID
}
if pid, ok := parentObservationFromCtx(ctx); ok {
receivedParentID = pid
}
return nil
})
mw := AsynqMiddleware()
wrapped := mw(handler)
task := asynq.NewTask("test:type", raw)
if err := wrapped.ProcessTask(context.Background(), task); err != nil {
mw := AsynqMiddleware()(asynq.HandlerFunc(func(context.Context, *asynq.Task) error { return nil }))
if err := mw.ProcessTask(context.Background(), asynq.NewTask("test:type", raw)); err != nil {
t.Fatalf("handler err: %v", err)
}
if !sawTrace {
t.Fatal("expected handler ctx to carry a resumed trace")
}
if receivedTraceID != "trace-xyz" {
t.Errorf("trace id mismatch: got %q want trace-xyz", receivedTraceID)
}
if receivedParentID == "" {
t.Error("expected parent observation to be set to the wrapper span id")
}
if err := m.Shutdown(context.Background()); err != nil {
t.Fatalf("shutdown: %v", err)
}
mu.Lock()
defer mu.Unlock()
var sawSpanCreate, sawSpanUpdate, sawTraceCreate bool
for _, b := range batches {
for _, ev := range b.Batch {
switch ev.Type {
case "span-create":
sawSpanCreate = true
// Resumed span should attach to the supplied trace id, not
// a fresh one we made up in the worker.
body, _ := json.Marshal(ev.Body)
var ob observationBody
_ = json.Unmarshal(body, &ob)
if ob.TraceID != "trace-xyz" {
t.Errorf("span-create trace id = %q, want trace-xyz", ob.TraceID)
}
if ob.ParentObservationID != "parent-span-abc" {
t.Errorf("span-create parentObservationId = %q, want parent-span-abc", ob.ParentObservationID)
}
case "span-update":
sawSpanUpdate = true
case "trace-create":
sawTraceCreate = true
}
for _, s := range exp.GetSpans() {
if s.Name != "asynq.test:type" {
continue
}
if s.SpanContext.TraceID() != remoteTraceID {
t.Errorf("worker span trace id = %s, want upstream %s (traceparent not propagated)",
s.SpanContext.TraceID(), remoteTraceID)
}
return
}
if !sawSpanCreate {
t.Error("missing span-create event")
}
if !sawSpanUpdate {
t.Error("missing span-update event")
}
if sawTraceCreate {
t.Error("resumed trace should not emit trace-create events (parent owns them)")
}
t.Fatal("asynq worker span not exported")
}
// TestAsynqMiddleware_StandaloneTrace asserts that when the payload carries
// NO upstream trace id (e.g. a scheduled job), the middleware opens a
// standalone trace tagged with the task type, so the worker-side work
// still shows up in Langfuse.
// NO upstream traceparent (e.g. a scheduled job), the middleware opens a
// standalone trace named after the task type.
func TestAsynqMiddleware_StandaloneTrace(t *testing.T) {
var mu sync.Mutex
var batches []ingestionRequest
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
body, _ := io.ReadAll(r.Body)
var req ingestionRequest
_ = json.Unmarshal(body, &req)
mu.Lock()
batches = append(batches, req)
mu.Unlock()
w.WriteHeader(200)
_, _ = w.Write([]byte(`{}`))
}))
defer srv.Close()
m, err := Init(Config{
Enabled: true, Host: srv.URL, PublicKey: "pk", SecretKey: "sk",
FlushAt: 1, FlushInterval: 5 * time.Millisecond, QueueSize: 32,
RequestTimeout: 2 * time.Second, SampleRate: 1.0,
})
if err != nil {
t.Fatalf("init: %v", err)
}
_, exp := newTestManager(t)
payload := &dummyPayload{KnowledgeID: "kX"}
raw, _ := json.Marshal(payload)
handler := asynq.HandlerFunc(func(ctx context.Context, task *asynq.Task) error { return nil })
wrapped := AsynqMiddleware()(handler)
if err := wrapped.ProcessTask(context.Background(), asynq.NewTask("scheduled:ping", raw)); err != nil {
mw := AsynqMiddleware()(asynq.HandlerFunc(func(context.Context, *asynq.Task) error { return nil }))
if err := mw.ProcessTask(context.Background(), asynq.NewTask("scheduled:ping", raw)); err != nil {
t.Fatalf("handler err: %v", err)
}
if err := m.Shutdown(context.Background()); err != nil {
t.Fatalf("shutdown: %v", err)
}
mu.Lock()
defer mu.Unlock()
// Langfuse uses trace-create events both on open and on update (the
// server merges by id), so we filter to the initial event that carries
// the Name — the finalizing update has no name.
var sawNamedTraceCreate bool
for _, b := range batches {
for _, ev := range b.Batch {
if ev.Type != "trace-create" {
continue
}
body, _ := json.Marshal(ev.Body)
var tb traceBody
_ = json.Unmarshal(body, &tb)
if tb.Name == "asynq.scheduled:ping" {
sawNamedTraceCreate = true
}
// The standalone run opens a root trace span ("asynq.scheduled:ping",
// type=trace) plus a worker span with the same name (type=span).
var sawRoot, sawSpan bool
for _, s := range exp.GetSpans() {
if s.Name != "asynq.scheduled:ping" {
continue
}
switch spanType(s) {
case obsTypeTrace:
sawRoot = true
case obsTypeSpan:
sawSpan = true
}
}
if !sawNamedTraceCreate {
t.Error("standalone run should emit a trace-create with name=asynq.scheduled:ping")
if !sawRoot {
t.Error("standalone run should open a root trace span named asynq.scheduled:ping")
}
if !sawSpan {
t.Error("standalone run should open a worker span")
}
}
-85
View File
@@ -1,85 +0,0 @@
package langfuse
import (
"bytes"
"context"
"encoding/base64"
"encoding/json"
"fmt"
"io"
"net/http"
"strings"
)
// client is a thin HTTP client for the Langfuse ingestion API.
type client struct {
host string
auth string // pre-computed "Basic <base64>"
httpClient *http.Client
debug bool
}
func newClient(cfg Config) *client {
credentials := cfg.PublicKey + ":" + cfg.SecretKey
return &client{
host: strings.TrimRight(cfg.Host, "/"),
auth: "Basic " + base64.StdEncoding.EncodeToString([]byte(credentials)),
httpClient: &http.Client{
Timeout: cfg.RequestTimeout,
},
debug: cfg.Debug,
}
}
type ingestionRequest struct {
Batch []ingestionEvent `json:"batch"`
}
// ingest posts a batch of events to Langfuse. The API accepts partial failure:
// individual event errors are returned in a 207 response, which we surface as
// a logged debug message rather than a hard error (the batch as a whole was
// accepted).
func (c *client) ingest(ctx context.Context, events []ingestionEvent) error {
if len(events) == 0 {
return nil
}
body, err := json.Marshal(ingestionRequest{Batch: events})
if err != nil {
return fmt.Errorf("langfuse: marshal batch: %w", err)
}
endpoint := c.host + "/api/public/ingestion"
req, err := http.NewRequestWithContext(ctx, http.MethodPost, endpoint, bytes.NewReader(body))
if err != nil {
return fmt.Errorf("langfuse: build request: %w", err)
}
req.Header.Set("Content-Type", "application/json")
req.Header.Set("Authorization", c.auth)
req.Header.Set("User-Agent", "weknora-langfuse/1.0")
resp, err := c.httpClient.Do(req)
if err != nil {
return fmt.Errorf("langfuse: ingest request failed: %w", err)
}
defer resp.Body.Close()
respBody, _ := io.ReadAll(resp.Body)
// 207 = "multi-status": batch accepted, some events may have failed.
// 2xx = success. Anything else is a transport/auth error worth surfacing.
if resp.StatusCode >= 200 && resp.StatusCode < 300 {
if c.debug && resp.StatusCode == http.StatusMultiStatus {
return fmt.Errorf("langfuse: partial ingest success (207): %s", truncate(string(respBody), 512))
}
return nil
}
return fmt.Errorf("langfuse: ingest failed with status %d: %s", resp.StatusCode, truncate(string(respBody), 512))
}
func truncate(s string, max int) string {
if len(s) <= max {
return s
}
return s[:max] + "..."
}
+18 -5
View File
@@ -1,9 +1,13 @@
// Package langfuse implements a lightweight client for the Langfuse ingestion
// API (https://langfuse.com/docs/api). It lets WeKnora record LLM traces,
// generations and token usage in Langfuse without pulling in a heavy SDK.
// Package langfuse implements WeKnora's Langfuse/LiteFuse LLM tracing client on
// top of the OpenTelemetry Go SDK. It records chat/embedding/rerank/VLM/ASR
// generations plus the surrounding agent/HTTP/asynq spans as OTLP/HTTP spans
// to a Langfuse v3+ or LiteFuse backend (POST /api/public/otel/v1/traces).
//
// The integration is fully opt-in: when disabled (the default), all public
// entry points are cheap no-ops, so callers can wire them unconditionally.
// The integration is fully opt-in: when disabled (the default), every public
// entry point is a cheap no-op, so callers can wire them unconditionally. A
// W3C traceparent propagated from upstream callers (e.g. sop3) is inherited so
// WeKnora's trace shares the upstream trace id — correlating runs across
// services in the LiteFuse UI.
package langfuse
import (
@@ -12,6 +16,8 @@ import (
"strconv"
"strings"
"time"
sdktrace "go.opentelemetry.io/otel/sdk/trace"
)
// Config holds the runtime configuration for the Langfuse client.
@@ -45,6 +51,13 @@ type Config struct {
SampleRate float64
// Debug enables verbose logging of batch send errors.
Debug bool
// testExporter, when non-nil, replaces the real OTLP exporter. Tests use
// it to inject an in-memory span exporter (tracetest.InMemoryExporter) so
// they can assert on exported spans deterministically without an HTTP
// server. Init registers it with a SimpleSpanProcessor (synchronous
// export on span End) when set. Unexported — production never sets it.
testExporter sdktrace.SpanExporter
}
// LoadConfigFromEnv builds a Config by reading the LANGFUSE_* environment
+5 -27
View File
@@ -13,17 +13,13 @@ import (
// CloneContext call would drop the trace and downstream LLM wrappers would
// each auto-create their own shallow trace, fragmenting a single HTTP request
// into many unrelated traces in the Langfuse UI.
//
// Span parenting is NOT carried by this key: it flows through the standard
// OpenTelemetry context (trace.SpanFromContext), which tracer.Start wires up
// automatically. The *Trace on this key is only for handlers/middleware that
// need to set trace-level input/output.
var traceCtxKey = types.LangfuseTraceContextKey
// parentObsCtxKey tracks the current "parent observation" id — i.e. the span
// that encloses any generation/span started under it. Unlike traceCtxKey, this
// key is private: spans are a pure tracer concern and logger.CloneContext
// should reset the parent across request boundaries (we don't want a leftover
// span id from one request leaking into another's LLM calls).
type parentObsCtxKeyType struct{}
var parentObsCtxKey = parentObsCtxKeyType{}
// withTrace stores a *Trace on the context so downstream LLM wrappers can
// attach their generations to it.
func withTrace(ctx context.Context, t *Trace) context.Context {
@@ -47,21 +43,3 @@ func traceFromCtx(ctx context.Context) (*Trace, bool) {
func TraceFromContext(ctx context.Context) (*Trace, bool) {
return traceFromCtx(ctx)
}
// withParentObservation stores the id of the enclosing span so children
// (sub-spans or generations) can attach via parentObservationId.
func withParentObservation(ctx context.Context, id string) context.Context {
if ctx == nil || id == "" {
return ctx
}
return context.WithValue(ctx, parentObsCtxKey, id)
}
// parentObservationFromCtx returns the enclosing span id if any.
func parentObservationFromCtx(ctx context.Context) (string, bool) {
if ctx == nil {
return "", false
}
v, ok := ctx.Value(parentObsCtxKey).(string)
return v, ok && v != ""
}
+42 -47
View File
@@ -4,17 +4,9 @@ import (
"time"
)
// Langfuse ingestion API event envelope.
// https://api.reference.langfuse.com/#tag/ingestion
type ingestionEvent struct {
ID string `json:"id"`
Timestamp string `json:"timestamp"`
Type string `json:"type"`
Body interface{} `json:"body"`
}
// TokenUsage captures the input/output/total token counts reported by the
// underlying model, in Langfuse's canonical schema.
// underlying model, in Langfuse's canonical schema. Set as the value of the
// langfuse.observation.usage_details span attribute (JSON-serialized).
type TokenUsage struct {
Input int `json:"input,omitempty"`
Output int `json:"output,omitempty"`
@@ -22,45 +14,48 @@ type TokenUsage struct {
Unit string `json:"unit,omitempty"`
}
// traceBody mirrors the /api/public/ingestion trace-create body.
type traceBody struct {
ID string `json:"id"`
Timestamp string `json:"timestamp,omitempty"`
Name string `json:"name,omitempty"`
UserID string `json:"userId,omitempty"`
SessionID string `json:"sessionId,omitempty"`
Release string `json:"release,omitempty"`
Environment string `json:"environment,omitempty"`
Input interface{} `json:"input,omitempty"`
Output interface{} `json:"output,omitempty"`
Metadata map[string]interface{} `json:"metadata,omitempty"`
Tags []string `json:"tags,omitempty"`
Public bool `json:"public,omitempty"`
}
// Langfuse / OpenTelemetry semantic-convention attribute keys mirrored from
// the official langfuse-python v4 SDK (_client/attributes.py). These are set
// as span attributes on the OTLP wire; LiteFuse (and Langfuse v3+) index
// traces/generations off them.
const (
attrObsType = "langfuse.observation.type"
attrObsInput = "langfuse.observation.input"
attrObsOutput = "langfuse.observation.output"
attrObsMetadata = "langfuse.observation.metadata"
attrObsLevel = "langfuse.observation.level"
attrObsStatusMessage = "langfuse.observation.status_message"
attrObsModel = "langfuse.observation.model.name"
attrObsModelParams = "langfuse.observation.model.parameters"
attrObsUsageDetails = "langfuse.observation.usage_details"
attrObsCompletionStart = "langfuse.observation.completion_start_time"
attrTraceName = "langfuse.trace.name"
attrTraceInput = "langfuse.trace.input"
attrTraceOutput = "langfuse.trace.output"
attrTraceMetadata = "langfuse.trace.metadata"
attrTraceTags = "langfuse.trace.tags"
attrUserID = "user.id"
attrSessionID = "session.id"
attrEnvironment = "langfuse.environment"
attrRelease = "langfuse.release"
attrLangfusePubKey = "langfuse.public.key"
// observationBody is shared between span-create / generation-create /
// span-update / generation-update events (different fields are populated
// depending on the event type).
type observationBody struct {
ID string `json:"id,omitempty"`
TraceID string `json:"traceId,omitempty"`
ParentObservationID string `json:"parentObservationId,omitempty"`
Type string `json:"type,omitempty"` // SPAN, GENERATION, EVENT
Name string `json:"name,omitempty"`
StartTime string `json:"startTime,omitempty"`
EndTime string `json:"endTime,omitempty"`
Input interface{} `json:"input,omitempty"`
Output interface{} `json:"output,omitempty"`
Metadata map[string]interface{} `json:"metadata,omitempty"`
Level string `json:"level,omitempty"` // DEFAULT | ERROR | WARNING
StatusMessage string `json:"statusMessage,omitempty"` // free-form
// The LiteFuse/Langfuse v3 OTel gate keys the "events_full"
// direct-write path on the instrumentation scope name. langfuse-python
// v4 uses "langfuse-sdk"; LiteFuse's getSdkInfoFromResourceSpans only
// requires the scope name to contain "langfuse" for server-side SDK
// classification. The actual gate is the x-langfuse-ingestion-version:4
// HTTP header (see exporter.go).
langfuseScopeName = "langfuse-sdk"
langfuseScopeVersion = "4.0.0"
)
// Generation-specific fields
Model string `json:"model,omitempty"`
ModelParameters map[string]interface{} `json:"modelParameters,omitempty"`
Usage *TokenUsage `json:"usage,omitempty"`
CompletionStart string `json:"completionStartTime,omitempty"`
}
// Observation types carried by the langfuse.observation.type attribute.
const (
obsTypeTrace = "trace"
obsTypeSpan = "span"
obsTypeGeneration = "generation"
)
func isoTime(t time.Time) string {
return t.UTC().Format("2006-01-02T15:04:05.000Z")
+41
View File
@@ -0,0 +1,41 @@
package langfuse
import (
"context"
"encoding/base64"
"fmt"
"strings"
"go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp"
sdktrace "go.opentelemetry.io/otel/sdk/trace"
)
// newExporter builds an OTLP/HTTP trace exporter pointed at the
// Langfuse v3+ / LiteFuse OTel endpoint (POST /api/public/otel/v1/traces).
//
// Auth is HTTP Basic (public_key:secret_key). The x-langfuse-ingestion-version
// header is the gate opt-in that LiteFuse/Langfuse v3 require for the OTel
// direct-write path (verified against directWriteHelpers.ts — without it the
// server returns 400 "requires Python SDK >= 4.0.0"). The x-langfuse-sdk-*
// markers are sent for parity; WeKnora is a Go client, not the Python SDK.
func newExporter(ctx context.Context, cfg Config) (sdktrace.SpanExporter, error) {
endpoint := strings.TrimRight(cfg.Host, "/") + "/api/public/otel/v1/traces"
creds := cfg.PublicKey + ":" + cfg.SecretKey
opts := []otlptracehttp.Option{
otlptracehttp.WithEndpointURL(endpoint),
otlptracehttp.WithHeaders(map[string]string{
"Authorization": "Basic " + base64.StdEncoding.EncodeToString([]byte(creds)),
"x-langfuse-ingestion-version": "4",
"x-langfuse-sdk-name": "python",
"x-langfuse-sdk-version": langfuseScopeVersion,
}),
}
if cfg.RequestTimeout > 0 {
opts = append(opts, otlptracehttp.WithTimeout(cfg.RequestTimeout))
}
exp, err := otlptracehttp.New(ctx, opts...)
if err != nil {
return nil, fmt.Errorf("langfuse: build otlp exporter: %w", err)
}
return exp, nil
}
+83 -121
View File
@@ -2,31 +2,42 @@ package langfuse
import (
"context"
"math/rand"
"sync"
"sync/atomic"
"time"
"github.com/Tencent/WeKnora/internal/logger"
"github.com/google/uuid"
"go.opentelemetry.io/otel"
"go.opentelemetry.io/otel/attribute"
"go.opentelemetry.io/otel/propagation"
"go.opentelemetry.io/otel/sdk/resource"
sdktrace "go.opentelemetry.io/otel/sdk/trace"
"go.opentelemetry.io/otel/trace"
"go.opentelemetry.io/otel/trace/noop"
)
// propagator is the W3C TraceContext propagator used to extract/inject
// traceparent across process boundaries (HTTP requests from sop3, asynq
// payloads). It is a package-level value rather than the global OTel
// propagator so tests remain isolated and Init never mutates global state.
var propagator = propagation.TraceContext{}
// Manager is the public façade of the langfuse package. A singleton is
// installed via Init(); callers should treat a nil *Manager as "disabled"
// and still invoke methods — every public method tolerates a nil receiver.
//
// Internally the manager owns an OpenTelemetry TracerProvider backed by an
// OTLP/HTTP exporter pointing at the Langfuse v3+ / LiteFuse OTel endpoint.
// The handles (*Trace / *Span / *Generation) wrap OTel spans; spans are
// buffered by the BatchSpanProcessor and exported complete on End, so there
// is no per-flush-batch duplication of root spans (the bug the legacy
// hand-rolled translator had on long traces spanning multiple flushes).
type Manager struct {
cfg Config
client *client
cfg Config
queue chan ingestionEvent
done chan struct{}
workerWG sync.WaitGroup
closed atomic.Bool
tp *sdktrace.TracerProvider
tracer trace.Tracer
// rng is used for sampling decisions. Guarded by rngMu because
// math/rand.Source isn't goroutine-safe.
rngMu sync.Mutex
rng *rand.Rand
closed atomic.Bool
}
var (
@@ -43,12 +54,48 @@ func Init(cfg Config) (*Manager, error) {
}
m := &Manager{cfg: cfg}
if cfg.Enabled {
m.client = newClient(cfg)
m.queue = make(chan ingestionEvent, cfg.QueueSize)
m.done = make(chan struct{})
m.rng = rand.New(rand.NewSource(time.Now().UnixNano()))
m.workerWG.Add(1)
go m.runWorker()
resAttrs := []attribute.KeyValue{
attribute.String("service.name", "weknora"),
attribute.String(attrLangfusePubKey, cfg.PublicKey),
}
if cfg.Environment != "" {
resAttrs = append(resAttrs, attribute.String(attrEnvironment, cfg.Environment))
}
if cfg.Release != "" {
resAttrs = append(resAttrs, attribute.String(attrRelease, cfg.Release))
}
res, err := resource.New(context.Background(), resource.WithAttributes(resAttrs...))
if err != nil {
return nil, err
}
var sp sdktrace.SpanProcessor
if cfg.testExporter != nil {
// Test mode: synchronous export on span End (deterministic).
sp = sdktrace.NewSimpleSpanProcessor(cfg.testExporter)
} else {
exp, err := newExporter(context.Background(), cfg)
if err != nil {
return nil, err
}
sp = sdktrace.NewBatchSpanProcessor(exp,
sdktrace.WithBatchTimeout(cfg.FlushInterval),
sdktrace.WithMaxExportBatchSize(cfg.FlushAt),
sdktrace.WithMaxQueueSize(cfg.QueueSize),
)
}
m.tp = sdktrace.NewTracerProvider(
sdktrace.WithResource(res),
sdktrace.WithSpanProcessor(sp),
sdktrace.WithSampler(sdktrace.ParentBased(sdktrace.TraceIDRatioBased(cfg.SampleRate))),
)
m.tracer = m.tp.Tracer(langfuseScopeName,
trace.WithInstrumentationVersion(langfuseScopeVersion),
trace.WithInstrumentationAttributes(attribute.String("public_key", cfg.PublicKey)),
)
// Make the W3C propagator available to any OTel-aware HTTP client
// in the process. Harmless to set globally; extraction/injection in
// this package use the package-level `propagator` directly.
otel.SetTextMapPropagator(propagator)
}
globalMu.Lock()
@@ -57,7 +104,7 @@ func Init(cfg Config) (*Manager, error) {
if cfg.Enabled {
logger.Infof(context.Background(),
"[Langfuse] enabled host=%s flush_at=%d flush_interval=%s sample_rate=%.2f",
"[Langfuse] enabled host=%s flush_at=%d flush_interval=%s sample_rate=%.2f (OTLP/OTel SDK)",
cfg.Host, cfg.FlushAt, cfg.FlushInterval, cfg.SampleRate,
)
}
@@ -72,114 +119,29 @@ func GetManager() *Manager {
return global
}
// Enabled reports whether the manager would actually emit events.
// Enabled reports whether the manager would actually emit spans.
func (m *Manager) Enabled() bool {
return m != nil && m.cfg.Enabled && !m.closed.Load()
return m != nil && m.cfg.Enabled && !m.closed.Load() && m.tp != nil
}
// Shutdown drains pending events, signals the worker to stop and waits for it.
// Safe to call multiple times.
// Tracer exposes the OTel tracer so middleware can create spans directly
// when needed (e.g. extracting a remote traceparent). Returns a no-op tracer
// when disabled.
func (m *Manager) Tracer() trace.Tracer {
if !m.Enabled() {
return noop.NewTracerProvider().Tracer(langfuseScopeName)
}
return m.tracer
}
// Shutdown flushes pending spans and releases the exporter. Safe to call
// multiple times.
func (m *Manager) Shutdown(ctx context.Context) error {
if m == nil || !m.cfg.Enabled {
if m == nil || !m.cfg.Enabled || m.tp == nil {
return nil
}
if !m.closed.CompareAndSwap(false, true) {
return nil
}
close(m.done)
doneCh := make(chan struct{})
go func() {
m.workerWG.Wait()
close(doneCh)
}()
select {
case <-doneCh:
return nil
case <-ctx.Done():
return ctx.Err()
}
}
// enqueue drops silently when either disabled, full or closed. Langfuse is
// observability, not business logic — back-pressure or failures must never
// block the request path.
func (m *Manager) enqueue(ev ingestionEvent) {
if !m.Enabled() {
return
}
select {
case m.queue <- ev:
default:
if m.cfg.Debug {
logger.Warnf(context.Background(), "[Langfuse] queue full, dropping event type=%s", ev.Type)
}
}
}
// sample decides whether to emit based on SampleRate. Sampling is applied once
// per trace; observations attached to an already-sampled trace are always kept
// (Langfuse itself would drop orphaned observations anyway).
func (m *Manager) sample() bool {
if !m.Enabled() {
return false
}
if m.cfg.SampleRate >= 1.0 {
return true
}
m.rngMu.Lock()
defer m.rngMu.Unlock()
return m.rng.Float64() < m.cfg.SampleRate
}
// runWorker batches queued events and flushes them either when the batch
// reaches FlushAt, when FlushInterval elapses, or when the manager shuts down.
func (m *Manager) runWorker() {
defer m.workerWG.Done()
ticker := time.NewTicker(m.cfg.FlushInterval)
defer ticker.Stop()
buf := make([]ingestionEvent, 0, m.cfg.FlushAt)
flush := func(reason string) {
if len(buf) == 0 {
return
}
ctx, cancel := context.WithTimeout(context.Background(), m.cfg.RequestTimeout)
defer cancel()
if err := m.client.ingest(ctx, buf); err != nil && m.cfg.Debug {
logger.Warnf(ctx, "[Langfuse] flush (%s) failed: %v", reason, err)
}
buf = buf[:0]
}
for {
select {
case ev := <-m.queue:
buf = append(buf, ev)
if len(buf) >= m.cfg.FlushAt {
flush("batch-full")
}
case <-ticker.C:
flush("interval")
case <-m.done:
// Drain whatever remains in the queue before exiting.
for {
select {
case ev := <-m.queue:
buf = append(buf, ev)
if len(buf) >= m.cfg.FlushAt {
flush("batch-full-on-shutdown")
}
default:
flush("shutdown")
return
}
}
}
}
}
// newID returns a Langfuse-compatible UUIDv4.
func newID() string {
return uuid.New().String()
return m.tp.Shutdown(ctx)
}
+2 -104
View File
@@ -2,17 +2,12 @@ package langfuse
import (
"context"
"encoding/json"
"io"
"net/http"
"net/http/httptest"
"sync"
"testing"
"time"
)
// TestManager_DisabledIsNoop verifies that when the manager is disabled the
// public API is safe to call and produces no side effects.
// public API is safe to call and produces no side effects (no spans exported,
// no panics).
func TestManager_DisabledIsNoop(t *testing.T) {
m, err := Init(Config{Enabled: false})
if err != nil {
@@ -31,100 +26,3 @@ func TestManager_DisabledIsNoop(t *testing.T) {
t.Fatalf("shutdown: %v", err)
}
}
// TestManager_FullRoundTrip boots a fake Langfuse server, runs a trace +
// generation through the manager, and asserts the ingested payload contains
// the expected ids, model name and usage.
func TestManager_FullRoundTrip(t *testing.T) {
var mu sync.Mutex
var batches []ingestionRequest
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path != "/api/public/ingestion" {
t.Errorf("unexpected path: %s", r.URL.Path)
w.WriteHeader(404)
return
}
if auth := r.Header.Get("Authorization"); auth == "" {
t.Errorf("missing Authorization header")
}
body, _ := io.ReadAll(r.Body)
var req ingestionRequest
if err := json.Unmarshal(body, &req); err != nil {
t.Errorf("decode body: %v", err)
w.WriteHeader(400)
return
}
mu.Lock()
batches = append(batches, req)
mu.Unlock()
w.WriteHeader(200)
w.Write([]byte(`{}`))
}))
defer srv.Close()
m, err := Init(Config{
Enabled: true,
Host: srv.URL,
PublicKey: "pk",
SecretKey: "sk",
FlushAt: 1,
FlushInterval: 10 * time.Millisecond,
QueueSize: 16,
RequestTimeout: 2 * time.Second,
SampleRate: 1.0,
})
if err != nil {
t.Fatalf("init: %v", err)
}
ctx, trace := m.StartTrace(context.Background(), TraceOptions{
Name: "test.trace",
UserID: "user-42",
})
_, gen := m.StartGeneration(ctx, GenerationOptions{
Name: "chat.completion",
Model: "gpt-test",
Input: []map[string]string{{"role": "user", "content": "hi"}},
})
gen.Finish("hello", &TokenUsage{Input: 10, Output: 20, Total: 30, Unit: "TOKENS"}, nil)
trace.Finish("hello", nil)
if err := m.Shutdown(context.Background()); err != nil {
t.Fatalf("shutdown: %v", err)
}
mu.Lock()
defer mu.Unlock()
// We should have received at least:
// trace-create, generation-create, generation-update, trace-create(update)
// possibly split across multiple HTTP calls depending on batching.
var events []ingestionEvent
for _, b := range batches {
events = append(events, b.Batch...)
}
if len(events) < 4 {
t.Fatalf("expected >=4 events, got %d: %+v", len(events), events)
}
var sawGenerationUpdate bool
for _, ev := range events {
if ev.Type != "generation-update" {
continue
}
b, _ := json.Marshal(ev.Body)
var body observationBody
_ = json.Unmarshal(b, &body)
if body.Usage == nil || body.Usage.Total != 30 {
t.Errorf("expected usage total=30, got %+v", body.Usage)
}
if body.TraceID != trace.ID {
t.Errorf("generation trace id mismatch: got %s want %s", body.TraceID, trace.ID)
}
sawGenerationUpdate = true
}
if !sawGenerationUpdate {
t.Fatalf("no generation-update event found")
}
}
+11 -5
View File
@@ -7,6 +7,7 @@ import (
"github.com/Tencent/WeKnora/internal/types"
"github.com/gin-gonic/gin"
"go.opentelemetry.io/otel/propagation"
)
// GinMiddleware returns a Gin handler that opens a Langfuse trace for each
@@ -25,6 +26,13 @@ func GinMiddleware() gin.HandlerFunc {
}
ctx := c.Request.Context()
// Extract a W3C traceparent from the incoming request so the WeKnora
// trace inherits the upstream caller's trace id. This is what lets a
// sop3 pipeline run (identified by its W3C trace_id) and the WeKnora
// agent-chat call it triggers land under the same trace in LiteFuse.
// When no traceparent is present (human UI calls, other clients) the
// root span starts a fresh trace as before.
ctx = propagator.Extract(ctx, propagation.HeaderCarrier(c.Request.Header))
userID := extractUserID(ctx)
sessionID := extractSessionID(c)
@@ -49,11 +57,9 @@ func GinMiddleware() gin.HandlerFunc {
c.Next()
trace.Finish(map[string]interface{}{
"status": c.Writer.Status(),
}, map[string]interface{}{
"http.status_code": c.Writer.Status(),
"response.size": c.Writer.Size(),
})
"status": c.Writer.Status(),
"response.size": c.Writer.Size(),
}, nil)
}
}
+185 -227
View File
@@ -2,41 +2,44 @@ package langfuse
import (
"context"
"encoding/json"
"time"
"github.com/Tencent/WeKnora/internal/logger"
"go.opentelemetry.io/otel/attribute"
"go.opentelemetry.io/otel/codes"
"go.opentelemetry.io/otel/trace"
)
// Trace represents an active root observation. A Trace is conceptually one
// "request" (e.g. a chat turn). Generations and spans attached to it roll up
// as children in the Langfuse UI.
// as children in the Langfuse UI. It wraps an OpenTelemetry root span; its
// ID is the OTel trace id (W3C 32-hex), which — when the request carried a
// traceparent header — is the upstream caller's trace id (sop3 correlation).
type Trace struct {
ID string
span trace.Span
manager *Manager
sampled bool
}
// Generation represents a single model invocation (LLM / embedding / VLM).
// Generation represents a single model invocation (LLM / embedding / VLM / ASR).
type Generation struct {
ID string
TraceID string
ParentObservationID string
manager *Manager
sampled bool
startTime time.Time
model string
name string
ID string
span trace.Span
manager *Manager
model string
name string
}
// Span represents a logical unit of work that isn't itself an LLM call — for
// example an asynq task execution, a pipeline stage, or a document-processing
// step. Generations and nested spans attach to it via parentObservationId.
// step. Generations and nested spans attach as children via the OTel span
// context (parenting is automatic through trace.SpanFromContext).
type Span struct {
ID string
TraceID string
ParentObservationID string
manager *Manager
sampled bool
startTime time.Time
name string
ID string
span trace.Span
manager *Manager
name string
}
// TraceOptions configures a new trace.
@@ -67,268 +70,223 @@ type SpanOptions struct {
Metadata map[string]interface{}
}
// StartTrace opens a new trace, stores its ID in the returned ctx, and returns
// a handle callers can finish with FinishTrace. When the manager is disabled
// or sampling excludes the trace, the returned *Trace is non-nil but all
// methods are no-ops so callers don't need nil checks.
// StartTrace opens a root span. When ctx carries a remote SpanContext (from a
// W3C traceparent extracted by GinMiddleware), the root span inherits the
// upstream trace id — this is what makes a sop3 run and its WeKnora call land
// under the same trace in LiteFuse. The returned *Trace is non-nil even when
// disabled (methods are no-ops), so callers don't need nil checks.
func (m *Manager) StartTrace(ctx context.Context, opts TraceOptions) (context.Context, *Trace) {
if m == nil || !m.cfg.Enabled {
return ctx, &Trace{}
if !m.Enabled() {
return ctx, &Trace{manager: m}
}
sampled := m.sample()
id := newID()
t := &Trace{ID: id, manager: m, sampled: sampled}
if sampled {
env := opts.Environment
if env == "" {
env = m.cfg.Environment
}
release := opts.Release
if release == "" {
release = m.cfg.Release
}
body := traceBody{
ID: id,
Timestamp: isoTime(time.Now()),
Name: opts.Name,
UserID: opts.UserID,
SessionID: opts.SessionID,
Input: opts.Input,
Metadata: opts.Metadata,
Tags: opts.Tags,
Environment: env,
Release: release,
}
m.enqueue(ingestionEvent{
ID: newID(),
Timestamp: isoTime(time.Now()),
Type: "trace-create",
Body: body,
})
name := opts.Name
attrs := []attribute.KeyValue{attribute.String(attrObsType, obsTypeTrace)}
if opts.Name != "" {
attrs = append(attrs, attribute.String(attrTraceName, opts.Name))
}
if opts.UserID != "" {
attrs = append(attrs, attribute.String(attrUserID, opts.UserID))
}
if opts.SessionID != "" {
attrs = append(attrs, attribute.String(attrSessionID, opts.SessionID))
}
env := opts.Environment
if env == "" {
env = m.cfg.Environment
}
if env != "" {
attrs = append(attrs, attribute.String(attrEnvironment, env))
}
rel := opts.Release
if rel == "" {
rel = m.cfg.Release
}
if rel != "" {
attrs = append(attrs, attribute.String(attrRelease, rel))
}
attrs = append(attrs, jsonAttr(attrTraceInput, opts.Input))
attrs = append(attrs, jsonAttr(attrTraceMetadata, opts.Metadata))
if len(opts.Tags) > 0 {
attrs = append(attrs, jsonAttr(attrTraceTags, opts.Tags))
}
ctx, span := m.tracer.Start(ctx, name, trace.WithTimestamp(time.Now()), trace.WithAttributes(attrs...))
t := &Trace{ID: span.SpanContext().TraceID().String(), span: span, manager: m}
return withTrace(ctx, t), t
}
// Finish updates the trace with its final output. Safe to call on a disabled
// trace (no-op).
// trace (no-op). The trace's metadata attribute (request_id, http.method,
// etc.) is set once at StartTrace and left untouched here — overwriting it
// with the finish metadata would lose the open's correlation fields.
func (t *Trace) Finish(output interface{}, metadata map[string]interface{}) {
if t == nil || t.manager == nil || !t.sampled {
_ = metadata
if t == nil || t.manager == nil || !t.manager.Enabled() || t.span == nil {
return
}
// "trace-create" events are also used to update traces in Langfuse —
// the server merges repeated events by ID. See the ingestion API docs.
body := traceBody{
ID: t.ID,
Output: output,
Metadata: metadata,
}
t.manager.enqueue(ingestionEvent{
ID: newID(),
Timestamp: isoTime(time.Now()),
Type: "trace-create",
Body: body,
})
t.span.SetAttributes(jsonAttr(attrTraceOutput, output))
t.span.End()
}
// ResumeTrace reconstructs a *Trace handle from an upstream-provided trace id
// (and optional parent observation id), without emitting a new trace-create
// event. Used by the asynq middleware to graft async work onto the HTTP-level
// trace that originated it: the HTTP layer already issued trace-create, and
// the worker only needs to add child observations.
//
// When traceID is empty the returned *Trace is nil, signalling the caller
// should fall back to StartTrace if it wants a standalone root.
func (m *Manager) ResumeTrace(ctx context.Context, traceID, parentObservationID string) (context.Context, *Trace) {
if m == nil || !m.cfg.Enabled || traceID == "" {
// ResumeTrace reconstructs a *Trace handle from an externally-provided W3C
// trace id (and optional parent span id), without creating a new root span —
// the originating process (e.g. an HTTP request that already opened a trace)
// owns the root. Used to graft async work onto an existing trace: it sets a
// remote SpanContext on ctx so any child span/generation started under it
// inherits the upstream trace id. When traceID is empty the returned *Trace
// is nil, signalling the caller should fall back to StartTrace.
func (m *Manager) ResumeTrace(ctx context.Context, traceID, parentSpanID string) (context.Context, *Trace) {
if m == nil || !m.Enabled() || traceID == "" {
return ctx, nil
}
t := &Trace{ID: traceID, manager: m, sampled: true}
ctx = withTrace(ctx, t)
if parentObservationID != "" {
ctx = withParentObservation(ctx, parentObservationID)
tid, err := trace.TraceIDFromHex(traceID)
if err != nil {
// Not a W3C 32-hex trace id (legacy UUID, etc.); cannot resume.
return ctx, nil
}
return ctx, t
var sid trace.SpanID
if parentSpanID != "" {
if s, err := trace.SpanIDFromHex(parentSpanID); err == nil {
sid = s
}
}
sc := trace.NewSpanContext(trace.SpanContextConfig{
TraceID: tid,
SpanID: sid,
TraceFlags: trace.FlagsSampled,
Remote: true,
})
ctx = trace.ContextWithRemoteSpanContext(ctx, sc)
t := &Trace{ID: traceID, manager: m}
return withTrace(ctx, t), t
}
// StartSpan opens a SPAN observation under the trace (and optional parent
// observation) carried by ctx. When no trace is present, a shallow trace is
// auto-created, mirroring StartGeneration's behaviour. Returns a ctx whose
// parentObservationId is set to this span's id, so any nested span or
// generation will properly attach as a child.
// reestablishParentSpan re-injects the active trace's root span as the OTel
// parent when ctx carries a *Trace but no active OTel span. This happens when
// a context rebuild drops the OTel span while the *Trace handle survives on
// the exported key (e.g. a background goroutine derived from a non-request
// context, or a CloneContext that predates/missed the span fix). Without
// this, child spans (e.g. a summary generation) start a fresh root and orphan
// off the HTTP trace.
func (m *Manager) reestablishParentSpan(ctx context.Context) context.Context {
if !m.Enabled() {
return ctx
}
if sp := trace.SpanFromContext(ctx); sp.IsRecording() {
return ctx // already has an active span
}
if t, ok := traceFromCtx(ctx); ok && t != nil && t.span != nil {
return trace.ContextWithSpan(ctx, t.span)
}
return ctx
}
// StartSpan opens a child span under the trace/span carried by ctx. When no
// trace is present, OTel creates a fresh root (mirroring StartGeneration's
// auto-trace behaviour). Returns a ctx whose active span is this span.
func (m *Manager) StartSpan(ctx context.Context, opts SpanOptions) (context.Context, *Span) {
if m == nil || !m.cfg.Enabled {
return ctx, &Span{}
if !m.Enabled() {
return ctx, &Span{manager: m}
}
trace, ok := traceFromCtx(ctx)
if !ok || trace == nil {
newCtx, t := m.StartTrace(ctx, TraceOptions{Name: opts.Name})
ctx = m.reestablishParentSpan(ctx)
if _, ok := traceFromCtx(ctx); !ok {
// No active trace: open a shallow root so the span isn't orphaned.
newCtx, _ := m.StartTrace(ctx, TraceOptions{Name: opts.Name})
ctx = newCtx
trace = t
}
if !trace.sampled {
return ctx, &Span{}
attrs := []attribute.KeyValue{
attribute.String(attrObsType, obsTypeSpan),
jsonAttr(attrObsInput, opts.Input),
jsonAttr(attrObsMetadata, opts.Metadata),
}
parent, _ := parentObservationFromCtx(ctx)
now := time.Now()
s := &Span{
ID: newID(),
TraceID: trace.ID,
ParentObservationID: parent,
manager: m,
sampled: true,
startTime: now,
name: opts.Name,
}
body := observationBody{
ID: s.ID,
TraceID: s.TraceID,
ParentObservationID: parent,
Type: "SPAN",
Name: opts.Name,
StartTime: isoTime(now),
Input: opts.Input,
Metadata: opts.Metadata,
}
m.enqueue(ingestionEvent{
ID: newID(),
Timestamp: isoTime(now),
Type: "span-create",
Body: body,
})
ctx = withParentObservation(ctx, s.ID)
return ctx, s
ctx, span := m.tracer.Start(ctx, opts.Name, trace.WithTimestamp(time.Now()), trace.WithAttributes(attrs...))
return ctx, &Span{ID: span.SpanContext().SpanID().String(), span: span, manager: m, name: opts.Name}
}
// Finish updates a span with its final output, extra metadata and any error.
// A non-nil err marks the span as ERROR level in Langfuse.
// A non-nil err marks the span as ERROR. The span's metadata attribute is set
// once at StartSpan and left untouched here.
func (s *Span) Finish(output interface{}, metadata map[string]interface{}, err error) {
if s == nil || s.manager == nil || !s.sampled {
_ = metadata
if s == nil || s.manager == nil || !s.manager.Enabled() || s.span == nil {
return
}
level := "DEFAULT"
var statusMsg string
s.span.SetAttributes(jsonAttr(attrObsOutput, output))
if err != nil {
level = "ERROR"
statusMsg = err.Error()
s.span.RecordError(err)
s.span.SetStatus(codes.Error, err.Error())
}
body := observationBody{
ID: s.ID,
TraceID: s.TraceID,
Type: "SPAN",
EndTime: isoTime(time.Now()),
Output: output,
Metadata: metadata,
Level: level,
StatusMessage: statusMsg,
}
s.manager.enqueue(ingestionEvent{
ID: newID(),
Timestamp: isoTime(time.Now()),
Type: "span-update",
Body: body,
})
s.span.End()
}
// StartGeneration opens a generation observation under the trace carried by
// ctx (or a newly auto-created trace if none is present). If a parent span
// is present on ctx, the generation attaches under it via parentObservationId
// so the Langfuse tree shows: trace → span → generation.
// ctx (or a newly auto-created trace). If a parent span is present on ctx,
// the generation attaches under it via the OTel span context.
func (m *Manager) StartGeneration(ctx context.Context, opts GenerationOptions) (context.Context, *Generation) {
if m == nil || !m.cfg.Enabled {
return ctx, &Generation{}
if !m.Enabled() {
return ctx, &Generation{manager: m, model: opts.Model, name: opts.Name}
}
// If the caller hasn't opened a trace yet, create a shallow auto-trace so
// the generation has a parent. This keeps single-shot internal callers
// (e.g. test connections) observable.
trace, ok := traceFromCtx(ctx)
if !ok || trace == nil {
newCtx, t := m.StartTrace(ctx, TraceOptions{Name: opts.Name})
ctx = m.reestablishParentSpan(ctx)
if _, ok := traceFromCtx(ctx); !ok {
newCtx, _ := m.StartTrace(ctx, TraceOptions{Name: opts.Name})
ctx = newCtx
trace = t
}
if !trace.sampled {
return ctx, &Generation{}
attrs := []attribute.KeyValue{
attribute.String(attrObsType, obsTypeGeneration),
attribute.String(attrObsModel, opts.Model),
jsonAttr(attrObsInput, opts.Input),
jsonAttr(attrObsMetadata, opts.Metadata),
jsonAttr(attrObsModelParams, opts.ModelParameters),
}
parent, _ := parentObservationFromCtx(ctx)
now := time.Now()
ctx, span := m.tracer.Start(ctx, opts.Name, trace.WithTimestamp(time.Now()), trace.WithAttributes(attrs...))
g := &Generation{
ID: newID(),
TraceID: trace.ID,
ParentObservationID: parent,
manager: m,
sampled: true,
startTime: now,
model: opts.Model,
name: opts.Name,
ID: span.SpanContext().SpanID().String(),
span: span,
manager: m,
model: opts.Model,
name: opts.Name,
}
body := observationBody{
ID: g.ID,
TraceID: g.TraceID,
ParentObservationID: parent,
Type: "GENERATION",
Name: opts.Name,
StartTime: isoTime(now),
Input: opts.Input,
Metadata: opts.Metadata,
Model: opts.Model,
ModelParameters: opts.ModelParameters,
}
m.enqueue(ingestionEvent{
ID: newID(),
Timestamp: isoTime(now),
Type: "generation-create",
Body: body,
})
return ctx, g
}
// Finish updates a generation with its final output, token usage and any
// error. A non-nil err marks the observation as ERROR level in Langfuse.
// error. A non-nil err marks the observation as ERROR.
func (g *Generation) Finish(output interface{}, usage *TokenUsage, err error) {
if g == nil || g.manager == nil || !g.sampled {
if g == nil || g.manager == nil || !g.manager.Enabled() || g.span == nil {
return
}
level := "DEFAULT"
var statusMsg string
attrs := []attribute.KeyValue{jsonAttr(attrObsOutput, output)}
if usage != nil {
attrs = append(attrs, jsonAttr(attrObsUsageDetails, usage))
}
g.span.SetAttributes(attrs...)
if err != nil {
level = "ERROR"
statusMsg = err.Error()
g.span.RecordError(err)
g.span.SetStatus(codes.Error, err.Error())
}
body := observationBody{
ID: g.ID,
TraceID: g.TraceID,
Type: "GENERATION",
EndTime: isoTime(time.Now()),
Output: output,
Usage: usage,
Level: level,
StatusMessage: statusMsg,
}
g.manager.enqueue(ingestionEvent{
ID: newID(),
Timestamp: isoTime(time.Now()),
Type: "generation-update",
Body: body,
})
g.span.End()
}
// MarkCompletionStart records the time at which the first token was received
// in a streaming generation. Langfuse surfaces this as time-to-first-token.
func (g *Generation) MarkCompletionStart(t time.Time) {
if g == nil || g.manager == nil || !g.sampled {
if g == nil || g.manager == nil || !g.manager.Enabled() || g.span == nil {
return
}
body := observationBody{
ID: g.ID,
TraceID: g.TraceID,
Type: "GENERATION",
CompletionStart: isoTime(t),
}
g.manager.enqueue(ingestionEvent{
ID: newID(),
Timestamp: isoTime(time.Now()),
Type: "generation-update",
Body: body,
})
g.span.SetAttributes(attribute.String(attrObsCompletionStart, isoTime(t)))
}
// jsonAttr serializes v to a compact JSON string and wraps it as a string
// OTel attribute — matching how langfuse-python stores structured fields
// (input/output/metadata/usage) on spans. nil/zero values return an empty
// KeyValue (harmless on SetAttributes).
func jsonAttr(key string, v interface{}) attribute.KeyValue {
if v == nil {
return attribute.KeyValue{Key: attribute.Key(key)}
}
b, err := json.Marshal(v)
if err != nil || len(b) == 0 || string(b) == "null" {
logger.Warnf(context.Background(), "[Langfuse] marshal attr %s failed: %v", key, err)
return attribute.KeyValue{Key: attribute.Key(key)}
}
return attribute.String(key, string(b))
}
+140 -133
View File
@@ -2,70 +2,62 @@ package langfuse
import (
"context"
"encoding/json"
"errors"
"io"
"net/http"
"net/http/httptest"
"sync"
"strings"
"testing"
"time"
"go.opentelemetry.io/otel/attribute"
"go.opentelemetry.io/otel/codes"
"go.opentelemetry.io/otel/propagation"
"go.opentelemetry.io/otel/sdk/trace/tracetest"
)
// newTestServer spins up a fake Langfuse ingestion endpoint and returns the
// collected events + a cleanup func.
func newTestServer(t *testing.T) (*httptest.Server, func() []ingestionEvent) {
t.Helper()
var mu sync.Mutex
var batches []ingestionRequest
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
body, _ := io.ReadAll(r.Body)
var req ingestionRequest
_ = json.Unmarshal(body, &req)
mu.Lock()
batches = append(batches, req)
mu.Unlock()
w.WriteHeader(200)
_, _ = w.Write([]byte(`{}`))
}))
drain := func() []ingestionEvent {
mu.Lock()
defer mu.Unlock()
var out []ingestionEvent
for _, b := range batches {
out = append(out, b.Batch...)
}
return out
}
return srv, drain
}
func newTestManager(t *testing.T, host string) *Manager {
// newTestManager builds a Manager wired to an in-memory span exporter via the
// SimpleSpanProcessor (synchronous export on span End), so tests can assert on
// exported spans deterministically without an HTTP server.
func newTestManager(t *testing.T) (*Manager, *tracetest.InMemoryExporter) {
t.Helper()
exp := tracetest.NewInMemoryExporter()
m, err := Init(Config{
Enabled: true,
Host: host,
Host: "http://test",
PublicKey: "pk",
SecretKey: "sk",
FlushAt: 1,
FlushInterval: 5 * time.Millisecond,
FlushInterval: 1 * time.Second,
QueueSize: 32,
RequestTimeout: 2 * time.Second,
SampleRate: 1.0,
testExporter: exp,
})
if err != nil {
t.Fatalf("init: %v", err)
}
return m
t.Cleanup(func() { _ = m.Shutdown(context.Background()) })
return m, exp
}
// TestSpan_NestedHierarchy verifies that nested StartSpan calls produce a
// trace → span₁ → span₂ → generation hierarchy, with parentObservationId
// pointing to the direct ancestor at each level.
// spanAttr returns the string value of a span attribute, or "" if absent.
func spanAttr(attrs []attribute.KeyValue, key string) string {
for _, kv := range attrs {
if string(kv.Key) == key {
if v, ok := kv.Value.AsInterface().(string); ok {
return v
}
}
}
return ""
}
// spanType returns the langfuse.observation.type of a span.
func spanType(s tracetest.SpanStub) string { return spanAttr(s.Attributes, attrObsType) }
// TestSpan_NestedHierarchy verifies nested StartSpan calls produce a
// trace → span → span → generation tree with correct OTel parent linking
// (parenting is automatic through trace.SpanFromContext, no manual ids).
func TestSpan_NestedHierarchy(t *testing.T) {
srv, drain := newTestServer(t)
defer srv.Close()
m := newTestManager(t, srv.URL)
m, exp := newTestManager(t)
ctx, trace := m.StartTrace(context.Background(), TraceOptions{Name: "root"})
ctx, outer := m.StartSpan(ctx, SpanOptions{Name: "outer"})
@@ -77,119 +69,134 @@ func TestSpan_NestedHierarchy(t *testing.T) {
outer.Finish("outer-out", nil, nil)
trace.Finish("root-out", nil)
if err := m.Shutdown(context.Background()); err != nil {
t.Fatalf("shutdown: %v", err)
spans := exp.GetSpans()
if len(spans) != 4 {
t.Fatalf("expected 4 spans, got %d", len(spans))
}
events := drain()
parentOf := map[string]string{}
traceOf := map[string]string{}
kindOf := map[string]string{}
for _, ev := range events {
if ev.Type != "span-create" && ev.Type != "generation-create" {
continue
}
b, _ := json.Marshal(ev.Body)
var ob observationBody
_ = json.Unmarshal(b, &ob)
parentOf[ob.ID] = ob.ParentObservationID
traceOf[ob.ID] = ob.TraceID
kindOf[ob.ID] = ob.Type
byName := map[string]tracetest.SpanStub{}
for _, s := range spans {
byName[s.Name] = s
}
if traceOf[outer.ID] != trace.ID || parentOf[outer.ID] != "" {
t.Errorf("outer span should sit directly under the trace, got parent=%q trace=%q", parentOf[outer.ID], traceOf[outer.ID])
root, outerS, innerS, genS := byName["root"], byName["outer"], byName["inner"], byName["llm"]
if root.Name == "" || outerS.Name == "" || innerS.Name == "" || genS.Name == "" {
t.Fatalf("missing expected spans: %+v", byName)
}
if parentOf[inner.ID] != outer.ID {
t.Errorf("inner span parent mismatch: got %q want %q", parentOf[inner.ID], outer.ID)
// All spans share the trace id.
if outerS.SpanContext.TraceID() != root.SpanContext.TraceID() ||
innerS.SpanContext.TraceID() != root.SpanContext.TraceID() ||
genS.SpanContext.TraceID() != root.SpanContext.TraceID() {
t.Errorf("all spans must share the root trace id")
}
if parentOf[gen.ID] != inner.ID {
t.Errorf("generation should nest under inner span, got parent=%q want %q", parentOf[gen.ID], inner.ID)
// Parent chain: outer → root, inner → outer, gen → inner.
if outerS.Parent.SpanID() != root.SpanContext.SpanID() {
t.Errorf("outer parent = %s, want root %s", outerS.Parent.SpanID(), root.SpanContext.SpanID())
}
if kindOf[outer.ID] != "SPAN" || kindOf[inner.ID] != "SPAN" {
t.Errorf("expected both wrappers to be SPAN observations, got outer=%q inner=%q", kindOf[outer.ID], kindOf[inner.ID])
if innerS.Parent.SpanID() != outerS.SpanContext.SpanID() {
t.Errorf("inner parent = %s, want outer %s", innerS.Parent.SpanID(), outerS.SpanContext.SpanID())
}
if kindOf[gen.ID] != "GENERATION" {
t.Errorf("expected generation kind, got %q", kindOf[gen.ID])
if genS.Parent.SpanID() != innerS.SpanContext.SpanID() {
t.Errorf("gen parent = %s, want inner %s", genS.Parent.SpanID(), innerS.SpanContext.SpanID())
}
// Observation types.
if spanType(root) != obsTypeTrace {
t.Errorf("root type = %q, want %q", spanType(root), obsTypeTrace)
}
if spanType(outerS) != obsTypeSpan || spanType(innerS) != obsTypeSpan {
t.Errorf("outer/inner type not span: %q %q", spanType(outerS), spanType(innerS))
}
if spanType(genS) != obsTypeGeneration {
t.Errorf("gen type = %q, want %q", spanType(genS), obsTypeGeneration)
}
}
// TestSpan_FinishWithError records an error status on the span so failures in
// asynq handlers surface as red observations in Langfuse.
func TestSpan_FinishWithError(t *testing.T) {
srv, drain := newTestServer(t)
defer srv.Close()
m := newTestManager(t, srv.URL)
m, exp := newTestManager(t)
ctx, _ := m.StartTrace(context.Background(), TraceOptions{Name: "root"})
_, span := m.StartSpan(ctx, SpanOptions{Name: "boom"})
span.Finish(nil, nil, errors.New("kaboom"))
if err := m.Shutdown(context.Background()); err != nil {
t.Fatalf("shutdown: %v", err)
}
var sawError bool
for _, ev := range drain() {
if ev.Type != "span-update" {
for _, s := range exp.GetSpans() {
if s.Name != "boom" {
continue
}
b, _ := json.Marshal(ev.Body)
var ob observationBody
_ = json.Unmarshal(b, &ob)
if ob.Level == "ERROR" && ob.StatusMessage == "kaboom" {
sawError = true
if s.Status.Code != codes.Error {
t.Errorf("span status = %v, want Error", s.Status.Code)
}
}
if !sawError {
t.Fatal("expected span-update with ERROR level and status message")
}
}
// TestResumeTrace_NoTraceCreateEvent verifies that ResumeTrace does NOT
// emit a trace-create event — the originating HTTP handler already did,
// and a duplicate would register as an orphan root in the Langfuse UI.
func TestResumeTrace_NoTraceCreateEvent(t *testing.T) {
srv, drain := newTestServer(t)
defer srv.Close()
m := newTestManager(t, srv.URL)
ctx, trace := m.ResumeTrace(context.Background(), "upstream-trace", "upstream-span")
if trace == nil || trace.ID != "upstream-trace" {
t.Fatalf("expected resumed trace with id upstream-trace, got %+v", trace)
}
if pid, ok := parentObservationFromCtx(ctx); !ok || pid != "upstream-span" {
t.Errorf("expected parent observation upstream-span on ctx, got %q (ok=%v)", pid, ok)
}
// Emit one child so there's *something* to flush, proving only child
// observations reach the wire, not a trace-create for the resume.
_, span := m.StartSpan(ctx, SpanOptions{Name: "child"})
span.Finish(nil, nil, nil)
if err := m.Shutdown(context.Background()); err != nil {
t.Fatalf("shutdown: %v", err)
}
for _, ev := range drain() {
if ev.Type == "trace-create" {
t.Fatalf("ResumeTrace must not emit trace-create events, got %+v", ev)
if s.Status.Description != "kaboom" {
t.Errorf("status description = %q, want kaboom", s.Status.Description)
}
return
}
t.Fatal("boom span not exported")
}
// TestResumeTrace_DisabledIsSafe guards against nil deref when Langfuse is
// off: ResumeTrace should return a nil *Trace and the original ctx unchanged.
func TestResumeTrace_DisabledIsSafe(t *testing.T) {
m, err := Init(Config{Enabled: false})
if err != nil {
t.Fatalf("init: %v", err)
}
ctx, trace := m.ResumeTrace(context.Background(), "x", "y")
if trace != nil {
t.Errorf("expected nil trace when disabled, got %+v", trace)
}
if _, ok := parentObservationFromCtx(ctx); ok {
t.Error("disabled ResumeTrace should not attach parent observation")
// TestManager_FullRoundTrip asserts a generation carries the model name and
// usage_details attribute (JSON with token counts), and shares the trace id
// of the root span.
func TestManager_FullRoundTrip(t *testing.T) {
m, exp := newTestManager(t)
ctx, trace := m.StartTrace(context.Background(), TraceOptions{Name: "test.trace", UserID: "user-42"})
_, gen := m.StartGeneration(ctx, GenerationOptions{
Name: "chat.completion",
Model: "gpt-test",
Input: []map[string]string{{"role": "user", "content": "hi"}},
})
gen.Finish("hello", &TokenUsage{Input: 10, Output: 20, Total: 30, Unit: "TOKENS"}, nil)
trace.Finish("hello", nil)
for _, s := range exp.GetSpans() {
if s.Name != "chat.completion" {
continue
}
if spanAttr(s.Attributes, attrObsModel) != "gpt-test" {
t.Errorf("model = %q, want gpt-test", spanAttr(s.Attributes, attrObsModel))
}
usage := spanAttr(s.Attributes, attrObsUsageDetails)
if !strings.Contains(usage, `"total":30`) {
t.Errorf("usage_details = %q, want total:30", usage)
}
if s.SpanContext.TraceID().String() != trace.ID {
t.Errorf("gen trace id = %s, want %s", s.SpanContext.TraceID(), trace.ID)
}
return
}
t.Fatal("generation span not exported")
}
// TestTraceparentPropagation is the sop3 correlation core test: an incoming
// W3C traceparent (as injected by an upstream caller like sop3) is extracted,
// and the WeKnora root span inherits the upstream trace id — so in LiteFuse
// the WeKnora trace and the upstream caller's trace are the same trace.
func TestTraceparentPropagation(t *testing.T) {
m, exp := newTestManager(t)
// Simulate an upstream caller carrying a traceparent.
upstreamCtx, upstreamSpan := m.Tracer().Start(context.Background(), "upstream-caller")
remoteTraceID := upstreamSpan.SpanContext().TraceID()
carrier := propagation.MapCarrier{}
propagator.Inject(upstreamCtx, carrier)
if carrier["traceparent"] == "" {
t.Fatal("no traceparent injected")
}
// The HTTP middleware extracts the traceparent into the request context.
httpCtx := propagator.Extract(context.Background(), carrier)
_, trace := m.StartTrace(httpCtx, TraceOptions{Name: "weknora-root"})
trace.Finish(nil, nil)
for _, s := range exp.GetSpans() {
if s.Name != "weknora-root" {
continue
}
if s.SpanContext.TraceID() != remoteTraceID {
t.Errorf("weknora root trace id = %s, want upstream %s (traceparent not inherited)",
s.SpanContext.TraceID(), remoteTraceID)
}
return
}
t.Fatal("weknora-root span not exported")
}
+12 -7
View File
@@ -19,15 +19,20 @@ package types
// happen to be named similarly.
type TracingContext struct {
// LangfuseTraceID is the id of the root trace that originated this task.
// When set, workers attach their observations to the same trace instead
// of creating a standalone one, which is what makes the Langfuse UI
// show the asynq work as a child of the originating HTTP request.
// Kept for backward compatibility with legacy payloads; the OTLP path now
// propagates correlation via LangfuseTraceparent (W3C) below.
LangfuseTraceID string `json:"lf_trace_id,omitempty"`
// LangfuseParentObservationID, when set, points to the span that
// enclosed the enqueue call. The worker's own span/generation will list
// it as parentObservationId so Langfuse renders the tree as:
// http-trace → http-span → asynq-span → generation.
// LangfuseParentObservationID is retained for backward compatibility only;
// the OTLP path no longer uses it (parent linking flows through the W3C
// traceparent's parent span id).
LangfuseParentObservationID string `json:"lf_parent_obs_id,omitempty"`
// LangfuseTraceparent carries the W3C Trace Context (`traceparent` header
// value: `00-<trace_id>-<span_id>-<flags>`) from the enqueuing request.
// The worker re-extracts it so its spans are children of the same trace —
// stitching the HTTP request and the async job into one LiteFuse tree. This
// is also what propagates a sop3 run's W3C trace_id into any asynq jobs
// WeKnora enqueues while serving sop3's agent-chat call.
LangfuseTraceparent string `json:"lf_traceparent,omitempty"`
// LangfuseUserID preserves the userId / tenant label across the async
// boundary so that orphan async traces (when no upstream trace id is
// available) still show up in the Langfuse "Users" view under the