feat(langfuse): upgrade SDK and OTel ingestion to v4 by devin-ai-integration[bot] · Pull Request #33391 · BerriAI/litellm

devin-ai-integration · 2026-07-15T14:43:56Z

Relevant issues

Fixes #33383

Linear ticket

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

I have added meaningful tests
My PR passes all CI/CD checks (e.g., lint, format, unit tests)
My PR's scope is as isolated as possible; it only solves 1 specific problem
I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review)

Screenshots / Proof of Fix

Live Langfuse ingestion proof was not captured because this environment has no Langfuse project credentials

Type

New Feature

Changes

Upgrades the runtime and development Langfuse dependency to v4.7 and raises the compatible OpenTelemetry and Traceloop pins. OpenTelemetry remains optional when LiteLLM is imported without proxy runtime dependencies

Langfuse.trace(...).generation(...)
# becomes
with propagate_attributes(...):
    generation = Langfuse.start_as_current_observation(
        as_type="generation",
        trace_context=trace_context,
        ...,
    )

Legacy callback correlation now normalizes valid trace IDs, seeds deterministic trace and parent observation IDs for arbitrary external identifiers, and applies the parent directly to the generation observation without an intermediate span. Requested generation IDs remain available in metadata

User, session, trace name, metadata, version, and release attributes are attached to exported generation, provider, and guardrail observations. Observation version takes precedence over propagated trace version on the shared Langfuse v4 langfuse.version attribute. Requests continuing an existing_trace_id do not propagate tags that could replace the trace's existing tag set

Langfuse v4 does not accept historical observation start timestamps. The callback closes observations using SDK timestamps instead of applying historical end and first-token timestamps that would predate the SDK-created span start

Langfuse OTLP configuration sends Authorization and x-langfuse-ingestion-version: 4 for static, dynamic, and OTel v2 preset exporters. Trace metadata is emitted with the v4 langfuse.trace.metadata.<key> shape, and both user_id and trace_user_id map to user.id

Legacy tests now query v4 observations and traces through client.api. The obsolete v2 /api/public/ingestion payload snapshots were removed in favor of v4 exported-span coverage

Final Attestation

The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

Link to Devin session: https://app.devin.ai/sessions/55ba7054762d48cc9fd95922bd7f6506

devin-ai-integration · 2026-07-15T14:43:58Z

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

Disable automatic comment, CI, and merge conflict monitoring

CLAassistant · 2026-07-15T14:44:03Z

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

codecov · 2026-07-15T14:50:28Z

Codecov Report

❌ Patch coverage is 83.33333% with 14 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
litellm/integrations/langfuse/langfuse.py	78.57%	12 Missing ⚠️
litellm/integrations/SlackAlerting/utils.py	0.00%	1 Missing ⚠️
litellm/integrations/langfuse/langfuse_otel.py	93.75%	1 Missing ⚠️

📢 Thoughts on this report? Let us know!

greptile-apps · 2026-07-15T14:51:23Z

Greptile Summary

This PR migrates the Langfuse integration from the v2 trace/generation callback API to the v4 start_as_current_observation / propagate_attributes OTel-based API, and bumps the runtime dependency to langfuse>=4.7,<5.0 with a matching OpenTelemetry 1.33.1 stack.

Core callback rewrite: _log_langfuse_v2 now uses propagate_attributes to attach trace-level attributes (user, session, version, tags, trace name) and start_as_current_observation as a context manager to create the generation span; historical start_time/end_time are dropped because v4 does not accept timestamps that predate the SDK span start.
OTel attribute alignment: langfuse.trace.metadata is now emitted as individual langfuse.trace.metadata.<key> attributes; GENERATION_VERSION and TRACE_VERSION both map to the shared langfuse.version key; both user_id and trace_user_id metadata fields now map to user.id; static and dynamic OTLP exporters include the required x-langfuse-ingestion-version: 4 header.
ID normalization: _create_langfuse_trace_context converts UUIDs and arbitrary external identifiers to valid 32-hex trace IDs (via Langfuse.create_trace_id(seed=…)) and 16-hex span IDs (via SHA-256 truncation), so parent observation IDs that were UUID-format or free-form strings now round-trip correctly instead of being silently dropped.

Confidence Score: 5/5

The migration is mechanically correct; all known regressions (tag isolation for existing traces, UUID-format parent observation IDs) were already fixed in follow-up commits before this review.

The logic changes are faithful translations of the v2 API to v4 context-manager semantics. _create_langfuse_trace_context ID normalization is tested against both UUID and arbitrary-string cases. Tag isolation for existing_trace_id is guarded and covered by a regression test. Remaining findings are style/cleanup items that do not affect correctness.

tests/logging_callback_tests/test_langfuse_unit_tests.py — two new tests instantiate a real Langfuse SDK client against a live host URL; worth verifying they pass in an offline CI environment.

Important Files Changed

Filename	Overview
litellm/integrations/langfuse/langfuse.py	Core migration from Langfuse v2 trace/generation API to v4 `start_as_current_observation` / `propagate_attributes`; logic is sound but `_supports_completion_start_time` is now dead code
litellm/integrations/langfuse/langfuse_otel.py	Adds `build_langfuse_otel_headers` helper; flattens `trace_metadata` dict into per-key OTEL attributes matching v4 `langfuse.trace.metadata.<key>` shape; maps both `user_id` and `trace_user_id` to `TRACE_USER_ID`
litellm/integrations/otel/presets/langfuse.py	Dynamic header builder now includes the `x-langfuse-ingestion-version: 4` header via the shared `build_langfuse_otel_headers` helper
litellm/types/integrations/langfuse_otel.py	Enum values updated to v4 attribute names (`langfuse.version`, `langfuse.release`); `TRACE_VERSION` and `GENERATION_VERSION` now share the same attribute key as intended by v4 spec
pyproject.toml	Bumps langfuse to `>=4.7,<5.0`, OTel stack to 1.33.1, and traceloop-sdk to 0.34.0; breaking for users pinned to langfuse 2.x but this is the intended v4 migration
tests/logging_callback_tests/test_langfuse_unit_tests.py	Replaces v2 mock-HTTP test with real Langfuse SDK + InMemorySpanExporter tests; folder rule restricts to mock-only tests — potential network call during SDK init is unverified
tests/test_litellm/integrations/test_langfuse.py	Mock chain updated from `trace.generation()` to `start_as_current_observation` context manager; adds tests for trace-context normalization and tag-isolation for existing traces; dead `_is_langfuse_v2` mock setup remains
litellm/integrations/SlackAlerting/utils.py	Fixes Langfuse base URL lookup from removed `Langfuse.base_url` attribute to the `LangFuseLogger.langfuse_host` instance attribute which is still set in `__init__`

_{Reviews (6): Last reviewed commit: "fix(langfuse): keep telemetry dependency..." | Re-trigger Greptile}

greptile-apps · 2026-07-15T14:51:29Z

-    "opentelemetry-sdk==1.28.0",
-    "opentelemetry-exporter-otlp==1.28.0",
-    "opentelemetry-instrumentation-fastapi==0.49b0",
+    "langfuse>=4.7,<5.0",


Hard breaking change for users on langfuse < 4.7

The minimum is raised from >=2.59.7,<3.0 to >=4.7,<5.0, skipping v3 entirely (which was never supported) and jumping two major versions. Any deployment that has langfuse pinned to v2.x or v3.x will encounter an import-time AttributeError or resolution conflict when they next update LiteLLM. Per the team's policy on backwards-incompatible changes, a feature flag or explicit opt-in mechanism should gate the v4 behaviour, giving users a migration window.

Rule Used: What: avoid backwards-incompatible changes without... (source)

codspeed-hq · 2026-07-15T15:05:49Z

Merging this PR will not alter performance

✅ 31 untouched benchmarks

_{Comparing litellm_langfuse_v4 (464c10d) with litellm_internal_staging (c1b6c40)}

devin-ai-integration · 2026-07-15T15:39:03Z

@greptileai

devin-ai-integration · 2026-07-15T23:41:14Z

@greptileai

devin-ai-integration · 2026-07-16T00:08:13Z

@greptileai

yucheng-berri · 2026-07-21T05:02:19Z

bugbot run

cursor

Cursor Bugbot has reviewed your changes using high effort and found 3 potential issues.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit aaeb263. Configure here.}

cursor · 2026-07-21T05:06:19Z

+                )
+                generation_client = trace.start_observation(**generation_params)
+                generation_client.end()
+                trace.end()


Parent ID nests wrong observation

High Severity

The parent_observation_id is now applied to the root trace observation via trace_context, rather than directly to the generation. This introduces an unintended intermediate span, breaking the expected direct parent-child relationship for generations and distorting the trace hierarchy, particularly for agent and tool nesting.

^{Reviewed by Cursor Bugbot for commit aaeb263. Configure here.}

Fixed in 2c1dfdb4f0e. The generation is now the root v4 observation created with trace_context, so parent_observation_id remains its direct parent without an intermediate span. Provider and guardrail observations remain children of the generation

cursor · 2026-07-21T05:06:19Z

    TRACE_METADATA = "langfuse.trace.metadata"
-    TRACE_VERSION = "langfuse.trace.version"
-    TRACE_RELEASE = "langfuse.trace.release"
+    TRACE_VERSION = "langfuse.version"


Version attributes collide on OTel

Medium Severity

The GENERATION_VERSION and TRACE_VERSION attributes now map to the same langfuse.version key. This causes _set_metadata_attributes to overwrite one value when both are present, preventing distinct generation and trace versions from being recorded for Langfuse OTel spans. The SDK callback path still treats these fields as distinct.

^{Reviewed by Cursor Bugbot for commit aaeb263. Configure here.}

Fixed in 2c1dfdb4f0e. Langfuse v4 uses the single langfuse.version semantic attribute; observation version now takes precedence over propagated trace_version, matching SDK behavior, with trace version used as the fallback

cursor · 2026-07-21T05:06:19Z

+                    version=propagated_version,
+                    level=level,
+                    status_message=trace_params.get("status_message"),
+                )


Trace release attribute dropped

Medium Severity

trace_release is still copied into trace_params as release, but neither propagate_attributes nor start_observation receives it. Per-request release metadata is therefore discarded on the SDK callback path, while the OTel path still emits langfuse.release.

Additional Locations (1)

litellm/integrations/langfuse/langfuse.py#L542-L555

^{Reviewed by Cursor Bugbot for commit aaeb263. Configure here.}

Fixed in 2c1dfdb4f0e. Per-request trace_release is applied as langfuse.release to the generation and its provider or guardrail child observations, with exported-span regression coverage

devin-ai-integration · 2026-07-21T06:11:31Z

@greptileai

devin-ai-integration · 2026-07-21T06:38:04Z

@greptileai

devin-ai-integration · 2026-07-21T07:47:40Z

@greptileai

ShellWen · 2026-07-22T11:16:52Z

A downstream release that cherry-picked this PR exposed a timing regression in the v4 callback. _log_langfuse_v2 creates and ends both observations inside the success callback, so Langfuse records callback latency instead of the supplied model start_time / end_time. The current Python SDK supports explicit end_time but not start_time, so a public-API-only fix is blocked on langfuse/langfuse-python#1774. Once that is released, this branch can pass the historical nanosecond start time to the trace and generation observations and restore the existing end times without accessing private SDK fields

feat(langfuse): upgrade SDK and OTel ingestion to v4

8d05102

devin-ai-integration Bot requested a review from a team July 15, 2026 14:43

greptile-apps Bot reviewed Jul 15, 2026

View reviewed changes

devin-ai-integration Bot added 3 commits July 15, 2026 15:09

fix(langfuse): align trace IDs and v4 tests

db87acd

fix(langfuse): preserve parent observation links

a380346

test(langfuse): remove remaining v2 SDK usage

5ac7059

chore: merge litellm_internal_staging

d86cb54

github-actions Bot mentioned this pull request Jul 15, 2026

🦞 OpenClaw 生态日报 2026-07-16 junlinzhao327-oss/big_model_radar#70

Open

fix(langfuse): avoid invalid v4 observation timing

fe4b39c

chore(langfuse): address review nits

6718676

chore: merge litellm_internal_staging

aaeb263

yucheng-berri mentioned this pull request Jul 21, 2026

Upgrade Langfuse integration to Python SDK v4 and v4 OTel ingestion #33383

Open

cursor Bot reviewed Jul 21, 2026

View reviewed changes

devin-ai-integration Bot added 3 commits July 21, 2026 05:12

chore: merge litellm_internal_staging

960d350

chore(langfuse): satisfy strict typing gate

620c1b4

fix(langfuse): preserve v4 observation hierarchy

2c1dfdb

greptile-apps Bot reviewed Jul 21, 2026

View reviewed changes

Comment thread litellm/integrations/langfuse/langfuse.py Outdated

fix(langfuse): preserve existing trace tags

5be1e40

fix(langfuse): keep telemetry dependency optional

464c10d

ShellWen mentioned this pull request Jul 22, 2026

fix(langfuse): detach explicit traces from external parents #34234

Closed

5 tasks

Uh oh!

Uh oh!

Conversation

devin-ai-integration Bot commented Jul 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Relevant issues

Linear ticket

Pre-Submission checklist

Delays in PR merge?

Screenshots / Proof of Fix

Type

Changes

Final Attestation

Uh oh!

devin-ai-integration Bot commented Jul 15, 2026

🤖 Devin AI Engineer

Uh oh!

CLAassistant commented Jul 15, 2026

Uh oh!

codecov Bot commented Jul 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

greptile-apps Bot commented Jul 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Greptile Summary

Confidence Score: 5/5

Important Files Changed

Uh oh!

Uh oh!

greptile-apps Bot Jul 15, 2026

Choose a reason for hiding this comment

Uh oh!

codspeed-hq Bot commented Jul 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Merging this PR will not alter performance

Uh oh!

devin-ai-integration Bot commented Jul 15, 2026

Uh oh!

devin-ai-integration Bot commented Jul 15, 2026

Uh oh!

devin-ai-integration Bot commented Jul 16, 2026

Uh oh!

yucheng-berri commented Jul 21, 2026

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

cursor Bot Jul 21, 2026

Choose a reason for hiding this comment

Parent ID nests wrong observation

Uh oh!

devin-ai-integration Bot Jul 21, 2026

Choose a reason for hiding this comment

Uh oh!

cursor Bot Jul 21, 2026

Choose a reason for hiding this comment

Version attributes collide on OTel

Uh oh!

devin-ai-integration Bot Jul 21, 2026

Choose a reason for hiding this comment

Uh oh!

cursor Bot Jul 21, 2026

Choose a reason for hiding this comment

Trace release attribute dropped

Uh oh!

devin-ai-integration Bot Jul 21, 2026

Choose a reason for hiding this comment

Uh oh!

devin-ai-integration Bot commented Jul 21, 2026

Uh oh!

Uh oh!

devin-ai-integration Bot commented Jul 21, 2026

Uh oh!

devin-ai-integration Bot commented Jul 21, 2026

Uh oh!

ShellWen commented Jul 22, 2026

Uh oh!

Reviewers

Assignees

devin-ai-integration Bot commented Jul 15, 2026 •

edited

Loading

codecov Bot commented Jul 15, 2026 •

edited

Loading

greptile-apps Bot commented Jul 15, 2026 •

edited

Loading

codspeed-hq Bot commented Jul 15, 2026 •

edited

Loading