chore: sync fork with upstream - #41
Closed
minhthao56 wants to merge 148 commits into
Closed
Conversation
* feat: apply the 2026 brand to the Phoenix docs Ports the design system already applied to the Arize AX docs, keeping Phoenix's own cyan identity. The layout does not change; this is colour, type, surfaces and icons. Phoenix keeps its own palette. The brand token set carries a dedicated [data-mode="phoenix"] block — cyan accent, neutral near-black surfaces — with the same Geist type and 2/8px radii as AX, so the structure is shared and the hue is not. docs.json - Accents: primary #00769E (light-mode emphasis), light #39C7FF (dark-mode emphasis), dark #009DD2 (button fills). Note the docs.json JSON-schema describes this mapping incorrectly; these are what actually render. - Backgrounds #FFFFFF / #08090A, gradient decoration dropped. - Geist for heading and body; Geist Mono for code via CSS, as there is no docs.json slot for it. - icons.library switched to lucide; brand marks pinned to fontawesome. The token set defines Phoenix as dark-only, so light-mode values are derived. The brand cyan #009DD2 is only 3.1:1 on white, so light-mode emphasis uses a darkened #00769E — 5.1:1 on the page and 4.6:1 on the accent-soft pill. docs/style.css - Replaces the neutral ramp, which Mintlify otherwise derives from the primary colour and which therefore tinted every surface and hairline. - Mode-scoped --px-* tokens, then component styling for the navbar, sidebar, TOC, cards, code blocks, callouts, tables, the copy-page control and the changelog. Callout families are keyed off data-callout-type. - Changelog entries stack so the date heads a full-width section. - Phoenix's existing rules are carried over unchanged. Icons - 184 fontawesome-only names remapped to lucide across 72 mdx files, plus 19 in docs.json navigation. Non-brand fontawesome names have no fallback under lucide and would render blank. - Brand marks (github, python, java, docker, aws, npm, …) are untouched: Mintlify resolves those against fontawesome automatically. - Eleven mappings are judgement calls where lucide has no close equivalent; they are listed in the PR description. Also drops three hardcoded generic-blue callout colours so they inherit the theme. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: drop the placeholder navigation groups Five of the six tabs wrapped their landing page in a group named " " — Documentation, Integrations, Self-Hosting, Cookbooks and Release Notes. Mintlify renders a group's name as an eyebrow above the page H1, so that single space produced a blank line there plus an empty header in the sidebar. A tab's `pages` array accepts nested group objects, so each wrapper's contents are hoisted to tab level and the wrapper dropped. Tabs may carry either `pages` or `groups` but not both, so the remaining real groups move into `pages` alongside the hoisted items, preserving order. Verified no page was added, dropped or reordered: 708 pages, byte-identical flattened order per tab, no blank-named groups left, and no tab carrying both keys. Pages that were inside a wrapper (self-hosting/architecture, license, cookbook) now have no eyebrow rather than an empty one; grouped pages keep theirs. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: mirror the light-mode corrections from the AX docs Four of the six AX fixes apply here; the other two have no Phoenix counterpart (there is no hero panel, and no coding-agent snippet). - Prose links: these render as `<a class="link">`, and the rule here was `a:not(.card):not(.link)` — so it excluded exactly the links it was meant to style, leaving them on Mintlify's link colour. Now scoped correctly and set to the brand accent. - Step number circles: shipped as a plain grey disc; now the design's accent-soft disc with a hairline accent ring and accent numeral. - Code inside callouts: sits on the page background rather than the code surface, so a tinted callout does not carry a second tint. - Table-of-contents active entry: marked by colour alone. Mintlify also bolds it, which the design does not. Also fixes two callouts that asked for a warning and did not get one. `type` is not a `<Callout>` prop, so `<Callout type="warning">` rendered as a neutral generic callout. Replaced with `<Warning>` in the Claude Code and OpenCode integration pages — same defect class as the AX callout fix, with the author's intent explicit in the markup. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * chore: empty commit to re-trigger CI checks Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Mc2tUw43f5K7tNEk6nU5Fk * fix: two icons that are absent from Mintlify's lucide v1.16.0 Mintlify serves lucide v1.16.0 from its CDN. user-shield and face-slightly-frowning are canonical ids in current lucide but postdate v1.16.0, so both 403 and render blank: user-shield -> shield-user (docs.json Security nav, self-hosting.mdx) face-slightly-frowning -> frown (07-22 release note) Addresses two of the four review comments. The other two (toolbox, list-check) were false positives — both return 200 from the v1.16.0 CDN and render correctly, so they are left alone. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Mc2tUw43f5K7tNEk6nU5Fk * revert: return icons to FontAwesome, dropping the lucide migration Mirrors the AX revert. FontAwesome's default style in Mintlify is already 'regular' — the lighter outline set — so removing icons.library restores light icons without touching a single iconType prop. Reverted at line level from the diff rather than by a global name map, and every original name was checked against Mintlify's FontAwesome build first. Three nav and card icons the line-level pass missed (their lines had other edits) were restored by hand: chef-hat -> user-chef, settings -> gear, layers -> layer-group. Verified: all 131 icon names now resolve — 121 via regular, 10 via brands, zero blank. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Mc2tUw43f5K7tNEk6nU5Fk --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
…tub (Arize-ai#15482) pydantic-ai reads the resource attribute matching the model class while constructing the model — `client.responses` for OpenAIResponsesModel, `client.chat.completions` for OpenAIChatModel — so a stand-in client must expose both to cover either `openai_api_type`. The stub only carried `kwargs` and `base_url`, so constructing a responses model raised AttributeError.
openai 3 moved its transport from httpx to httpx2. Type the openai SDK boundary against httpx2 and leave the other providers on httpx. - floor openai at >=3.1.0,<4 in the dev group, container extra, litellm override, and requirements/ci.txt - _HttpxClient accepts httpx and httpx2 clients since it proxies the anthropic client as well as openai's - rebuild the openai test transports on httpx2
A /chat route talks directly to the models AI search already knows about — the browser's built-in on-device model or a provider configured on the server — with no agent, tools, or persistence in between. Per-conversation token usage sits under the composer. Stacked on the AI search branch, which contributes the model menu, the browser AI adapter, and the server-side OpenAI-compatible model this page sends through.
…trols Adds a resizable sidebar to the chat page for setting a system prompt, temperature, top P, and max output tokens, persisted to localStorage. Starter prompts are reworked around Phoenix workflows (system prompts, LLM-as-a-judge evals, synthetic test data). The side nav groups the generative destinations (Prompts, Playground, Chat) behind a separator.
The experimental assistant settings section has shipped; show it unconditionally and leave the FeatureFlag union empty for the next pre-release gate.
Replace the hand-maintained provider z.enum in the chat and agent model storage schemas with the playground's pattern: a z.custom guard over the ModelProviders constant, which is already typed Record<ModelProvider, string> and so tracks the provider roster in one place.
Also in ChatPage: flush the debounced parameter store on pagehide so a hard unload doesn't drop the latest edits, and restore the draft after PromptInput's same-batch clear when a submit slips through with no model.
Use generateUUID so message ids don't throw in non-secure (plain-HTTP) contexts, abort an in-flight Browser AI model download on Stop and clear any stale download fraction at the start of each run, and drop the legacy bare-model localStorage migration no released build ever wrote.
toChatModelId and toServerModelId were verbatim copies of the /v1/chat/completions wire format; both now delegate to a single encodeServerModelId next to createServerLanguageModel, and the duplicate chat-side test suite is collapsed into the AI-query one.
With PHOENIX_AGENTS_DISABLE_BASH set the mutations capability is never constructed server-side, so the now-unconditional Experimental section offered a switch that did nothing — gate it like the sibling subagents setting. Also give the ctrl+shift+f feature-flags dialog an empty-state line now that no flags are in flight.
The route had crumb and agentRoute handles but no navigation metadata, so global search returned nothing for 'Chat' despite it being a primary side-nav destination.
Phoenix Cloud serves multiple workspaces at distinct basenames on one origin, so origin-wide keys let one workspace's system prompt and model selection hydrate another. Extract the assistant store's basename scoping into a shared scopeStorageKeyToBasename and resolve the chat keys through it at read/write time.
…ource The shared menu counts browser-local credentials because the playground can send them with its requests. Surfaces that execute through the server's chat-completions proxy authenticate with server-side credentials only, so a credentialSource="server" option computes readiness/provisioned/needs-credentials as if no local keys exist.
- Carve out a slim header bar (like the PXI panel) so the New Chat button has reserved top-right space instead of floating over content - Match PXI's input-meta padding so token usage isn't shoved against the composer - Judge the chat's provider menu and default-model pick by server credentials, matching its execution path - Abort the in-flight completion when the page unmounts — the conversation is discarded, so keeping the stream alive only burns tokens - Stop accumulating token usage for Browser AI turns: the Prompt API reports no real counts and the adapter's estimates are wrong (e.g. completion tokens on Gemini Nano)
Export the provider-key zod schema once from generativeUtils (built on isModelProvider) instead of three private copies, and generalize the basename-scoped localStorage pattern into createScopedStorageItem — the chat model/parameters stores become declarations over it, the duplicated scoping tests collapse into one direct suite, and the agent model-config key picks up the same workspace scoping.
- Write chat parameters to localStorage synchronously, deleting the debounce, its pagehide flush, and the module-load listener the batch existed to patch - Flush streamed deltas at most once per animation frame instead of one render per token, with a final flush guaranteeing completeness - Reuse isProviderProvisioned for the default-model ranking instead of an inline re-derivation - Reuse the ChatSurface's model-catalog response for the child ModelMenu's identical query (store-or-network) - Pass credentialSource="server" from the AI-query picker, the other server-proxied ModelMenu surface - Drop the single-child scroll-frame wrapper and the unreachable no-model draft-restore hack; fix encodeServerModelId's stacked docblocks; rename the settings gate to isServerAgentRuntimeEnabled
* fix(auth): polish password reset and user management
Make Agent-Assisted Setup (px setup) the recommended entry path in the Get Started flow instead of a standalone side page: - Add reusable snippet docs/snippets/agent-assisted-onboarding.mdx - Lead the Get Started overview "Where to Start" with the snippet - Replace the old Copilot-era callout on both tracing quickstarts (Python + TypeScript) with the snippet, before "Before We Start" - Move agent-assisted-setup into the Quick Start nav group as the first entry (removed from Coding Agents; group retained)
Surface Agent-Assisted Setup on the evaluations, prompt-playground, and
datasets-and-experiments quickstarts (Python + TypeScript). These guides
assume traces already exist, so they use a lighter prerequisite-framed
callout ("no traces yet? set up in one command") linking to
Agent-Assisted Setup, rather than the full setup snippet used on the
tracing quickstarts.
Lead the Quick Starts section on docs/phoenix.mdx with the agent-assisted-onboarding callout so the coding-agent setup path is visible on the landing page, mirroring the Arize AX docs home.
Add the agent-assisted-onboarding callout to the tracing setup entry points (Setup Tracing overview, Using Tracing Helpers, Setup OTEL) so the coding-agent path is offered alongside manual instrumentation, matching the Arize AX instrumentation docs.
…rize-ai#15701) The unit-test tox env installs with -U and resolves pydantic-ai-slim fresh, so it picked up 2.34.0, whose source no longer matches the vendored data-stream protocol types in src/phoenix/db/types. test_data_stream_protocol_compatibility now fails on every PR (verified: 2.33.0 passes, 2.34.0 fails the request-type and schema parity tests). Cap the unit env at 2.33 parity until the vendored types are re-synced. Claude-Session: https://claude.ai/code/session_014gvDFFS2FTKCnjCnQpcdng
…rize-ai#15709) * chore: complete removal of the legacy phoenix.session.client module * chore: total removal of the legacy client shims and dead code Addresses review feedback on Arize-ai#15709: drop the removal-shim scaffolding entirely rather than restructuring it. - src/phoenix/__init__.py: remove _REMOVED_MODULES / _REMOVED_ATTRIBUTES, the sys.meta_path finder+loader, and the PEP 562 __getattr__. Removed modules and attributes now fail with the standard ModuleNotFoundError / AttributeError; MIGRATION.md remains the migration reference. - Delete tests/unit/session/test_legacy_client_removal.py (nothing left to cover). - Delete src/phoenix/utilities/re.py and its test — parse_env_headers lost its last caller with get_env_client_headers; arize-phoenix-client and arize-phoenix-otel ship their own copies. - MIGRATION.md: importing phoenix.session.client now raises ModuleNotFoundError, not a guided ImportError. Co-authored-by: Mikyo King <5640648+mikeldking@users.noreply.github.com> --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com> Co-authored-by: Mikyo King <5640648+mikeldking@users.noreply.github.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: MoraVigoMalusardi <93168878+MoraVigoMalusardi@users.noreply.github.com>
…ta-stream types (Arize-ai#15714) * chore(deps): upgrade pydantic-ai-slim to 2.34 and re-sync vendored data-stream types pydantic-ai 2.34.0 added an `id` field to `ReasoningUIPart` so the Vercel AI adapter no longer rejects reasoning parts carrying a client UI part ID. Add the same field to the vendored copy, point the vendored headers at v2.34.0, raise the floor to >=2.34.0, and drop the <2.34 unit-test cap. 2.35.x carries identical vercel_ai sources but is still inside the 3-day exclude-newer quarantine, so the lock resolves to 2.34.0 for now. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017TyTrmUmSxJirG1vjijmFJ * chore: regenerate OpenAPI schema and clients for ReasoningUIPart.id Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017TyTrmUmSxJirG1vjijmFJ --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
…ize-ai#15614) * feat(datagen): recording toolkit and hand-recorded OpenInference corpora Adds scripts/datagen (mock OpenAI-compatible provider plus two PEP 723 scenario scripts using real OpenInference instrumenters) and two recorded corpora (openai_chat_sessions, langchain_agent_rag) as OTLP protobuf-JSON lines with manifests under src/phoenix/datagen/corpora. Claude-Session: https://claude.ai/code/session_01YF3zGrMPmFKZhQUjowsCJi * feat(datagen): add OTLP corpus replayer and phoenix datagen CLI Adds src/phoenix/datagen (corpus loader for local paths and URLs, session-aware replayer with ID/timestamp rewriting, contamination-mixture anomaly injection with ground-truth manifests, OTLP/HTTP export) and wires a lazily imported 'phoenix datagen' subcommand with env-var/flag config. Claude-Session: https://claude.ai/code/session_01YF3zGrMPmFKZhQUjowsCJi * fix(datagen): group corpus spans across requests Spans are grouped by recorded trace_id across all corpus lines, so corpora batched one-request-per-span replay as intact traces; manifest validation now counts distinct trace ids and total spans. Claude-Session: https://claude.ai/code/session_01YF3zGrMPmFKZhQUjowsCJi * fix(datagen): preserve replay fidelity and package corpora Bundles the recorded corpora in the wheel, restores parent-child end containment after latency redraws, rebases span-event timestamps, honors PHOENIX_CLIENT_HEADERS, preserves dangling recorded parents, and pins the numeric, export, and corpus-fidelity contracts in tests. Claude-Session: https://claude.ai/code/session_01YF3zGrMPmFKZhQUjowsCJi * fix(datagen): address acceptance findings Adds --project/PHOENIX_PROJECT_NAME with a datagen-<corpus> default emitted as the openinference.project.name resource attribute, derives emitted IDs from the seed plus a per-run nonce so same-seed reruns stay comparable without colliding, and retries OTLP export with capped exponential backoff instead of exiting on transport errors. Claude-Session: https://claude.ai/code/session_01YF3zGrMPmFKZhQUjowsCJi * feat: add optional datagen deployment recipes Adds an opt-in docker-compose datagen profile, a disabled-by-default Helm datagen deployment, a kustomize datagen overlay, a commented-out Render worker, and a self-hosting docs page covering local, Compose, Helm, Kustomize, Render, Railway, and Cloud Run flows. Claude-Session: https://claude.ai/code/session_01YF3zGrMPmFKZhQUjowsCJi * refactor(datagen): rename corpora to datagen assets and scenarios Claude-Session: https://claude.ai/code/session_01YF3zGrMPmFKZhQUjowsCJi * feat(datagen): fragment banks, generation lanes, session composer, distribution Adds the v2 fragment-bank schema and loader, a resumable USD-capped generation control plane with OpenAI Batch support, deterministic fake tools, self-play and scripted recording lanes, six archetype recorders, MinHash dedup and quality gating with atomic bank packaging, an archetype-safe replay session composer with lognormal session/gap knobs, checksum-verified bank fetch/pull with lazy resolution, a datagen-assets release workflow (--latest=false), a wheel starter-assets size gate, and checksum-pinned container asset baking. Claude-Session: https://claude.ai/code/session_01YF3zGrMPmFKZhQUjowsCJi * fix(datagen): make generation-tooling tests importable without PYTHONPATH Claude-Session: https://claude.ai/code/session_01YF3zGrMPmFKZhQUjowsCJi * fix(datagen): resolve recorder environments and verify offline recording Every recorder script's PEP 723 block now resolves under the repository's three-day `exclude-newer` window, and each recorder was run end-to-end, keyless, against the in-repo mock provider to confirm it emits its archetype's span kinds with session.id attached. - tool_agent, graph_multi_agent: langchain-core 1.5.6, langchain-openai 1.5.1, openinference-instrumentation-langchain 0.1.70. The previous langchain-openai/openai pins were mutually unsatisfiable, and the 2024-era LangChain instrumenter predates get_attributes_from_context. - langchain_agent_rag: llama-index-core 0.14.23, the newest release outside the freshness window. - guardrailed_app: guardrails-ai 0.5.0. Every published OpenInference Guardrails instrumenter supports only `>=0.4.5,<0.5.1`, so the 0.6.7 pin left the instrumentor disabled and no GUARDRAIL span was recorded. - Each recorder that imports `openinference.instrumentation` directly now declares it, and openai is pinned at one version across the directory. The mock provider now synthesizes tool-call arguments from the caller's own declared tool schema instead of a fixed delivery-estimate shape, so the tool-agent registry validates them, and it serves a server-sent-event stream when a request asks for one, which the plain-chat recorder needs. * chore(datagen): re-record starter assets under current instrumenter pins Claude-Session: https://claude.ai/code/session_01MjZmsruXdy3Wg2KQYQUCVo * feat(datagen): move assets to GCS * feat(datagen): application profiles, profile-scoped matrix, structured model backends Versioned application-profile contract with canonical run snapshots; matrix v2 draws every conversational field inside one profile with reproducible per-field streams, 10% targeted-seed cells, and Beta(2,8) seed intensities; structured OpenAI and codex-exec backends with provider-aware attempt accounting (priced vs subscription); scripted and self-play lanes consume profile draws. Claude-Session: https://claude.ai/code/session_01Jrru1FDRB5uKGFGq6Rwxst * feat(datagen): add customer support profiles * feat(datagen): add coding agent application profiles * feat(datagen): add data analyst application profiles * feat(datagen): add deep research profiles * feat(datagen): deterministic seed mechanics and materialized environments Profile seeds gain authored subtle/moderate/strong effect variants with a closed vocabulary (corpus edits, tool-result overlays, simulator traits); a pure materializer maps (cell_id, seed_id, intensity) to one variant and projects a seed-metadata-free environment into both lanes; fake tools overlay successful results after native handling; targeted cells add only a natural route; transcripts reject internal seed language. Claude-Session: https://claude.ai/code/session_01Jrru1FDRB5uKGFGq6Rwxst * test(datagen): add seed mechanics to generation fixture * feat(datagen): add deep research seed mechanics * feat(datagen): add data analyst seed mechanics * feat(datagen): add coding agent seed mechanics * feat(datagen): add customer support seed mechanics * feat(datagen): judged outcomes with engagement-based routing Immutable judging-input and judgment sidecars keyed by cell and fragment; recorded seed-engagement signals (tool overlay applications, perturbed- document serving, trait seeds by construction) determine ambient proximity; deterministic routing judges every trap-proximate fragment plus a 5% stratified baseline; versioned three-label rubric (survived/degraded/failed) through the frontier ModelBackend binding; outcomes project into schema-v2 quality metadata and never gate acceptance. Claude-Session: https://claude.ai/code/session_01Jrru1FDRB5uKGFGq6Rwxst * fix(datagen): align profile and composition boundaries * fix(datagen): make asset publication owner-run * fix(datagen): reject invalid conversation structure * feat(datagen): replay rate schedule, backfill, and error injection Add --rate-schedule (flat|business-hours), --timezone, --backfill, and --error-rate to phoenix datagen. The replayer paces a virtual timeline that follows weekday/weekend hourly tiers, can start in the past and catch up at exporter throughput, and can mark recorded LLM/TOOL spans as failed with an exception event and ancestor status propagation. Anomaly manifest rows gain an additive kind and timing fields. Flat defaults leave emitted requests unchanged. Claude-Session: https://claude.ai/code/session_01EvQfhu4vtHASNDPReHB5t9 * feat(datagen): supplemental fault runs and bank merge Runs can allocate one provider or tool fault per matrix cell (--fault-fraction, --fault-modes) with base-bank lineage (--base-scenario-name, --base-archive-sha256). The mock provider injects a selected fault once so the real SDK and instrumenters record retry and exception topology; tool exceptions cross the agent loop as error tool messages. Every fault fragment is judged through a dedicated route. bank.py gains package and merge commands that union a validated supplement into its base archive, rebuilding aggregates and recording per-input instrumenter provenance. README documents the supplemental procedure and the prepare-only publication handoff. Claude-Session: https://claude.ai/code/session_01EvQfhu4vtHASNDPReHB5t9 * refactor(datagen): trim runtime verification to its floor Verify scenario bytes once, where they can actually change: size and sha256 at download, member safety and manifest metadata at extract. Cache freshness is now a checksum-file presence and size check instead of a full re-hash on every start, and the loader validates only what replay needs (schema v2, fragment parse, unknown trace ids). - publish the cache directory by rename-aside instead of rmtree, so a concurrent reader never sees a half-deleted scenario - drop the unsound PID-based lock reclaim; a stale lock now fails with the file path to remove - drop the unverified HTTP-directory scenario source and schema-v1 support; test fixtures upgraded to schema v2 - collapse composer defaults to the dataclass fields plus CLI flags, removing the manifest channel that duplicated them - rename asset vocabulary to scenario (ScenarioFetchError, load_scenario_index, scenario_base_url, PHOENIX_DATAGEN_SCENARIO_BASE_URL); published index format unchanged Claude-Session: https://claude.ai/code/session_01Jb1jAxuoy8BeYgAuchjH3L * refactor(datagen): remove the cost plane and the batch lane The generation lane no longer models money or batch submission. Deleted the price catalog, budget pools, reserve/reconcile accounting, the cost-invariant latch, and the OpenAI Batch adapter, along with the costs and jobs journals and the CLI flags that fed them. Attempts keep their lane cap, which never depended on pricing. Callers in the judging and self-play paths lose their prices arguments; the scripted lane keeps its direct half. Tests pinning budgets, reservations and batch submission are removed; the codex_exec sandbox argv is now pinned as an exact sequence. * refactor(datagen): one shared serialization module for the sidecar scripts The datagen scripts carried seven JSON canonicalizers across three incompatible serialization policies, three journal readers and two deep-copy helpers. serialization.py now holds one of each: canonical_bytes, plain_json, json_copy, write_immutable_bytes, write_immutable_json, append_json and read_jsonl. The single policy is ensure_ascii=False, which writes real UTF-8 into journals and archives instead of escapes. Seed digests move for non-ASCII application state; run directories are disposable, and no test fixture carries non-ASCII seed state. The journal helpers take the caller's exception class as an error argument, so each module keeps raising its own error type. The module imports only the standard library, since the recorders in that directory run without the phoenix package on the path. * refactor(datagen): share the transcript hygiene names across the guards The reserved-phrase tuple was duplicated between the scripted and self-play recorders, and the bare-role-name check was written inline at three sites plus a fourth set in the quality gate. transcript.py now holds one copy of each, along with the strip-and-casefold predicate the guards were repeating. The four guards stay where they are and keep their own exception types and messages, since each defends a different boundary. The shared name set is the four-name superset, so a message whose whole visible content is 'system' or 'tool' is now rejected at the scripted decoder and both self-play boundaries; the two new tests cover that widening. * refactor(datagen): rename bank to scenario and enforce judged outcomes once The archive module is scenario.py and speaks scenario vocabulary throughout. Its reader extracts the archive and calls the runtime loader for manifest, fragment and OTLP parsing, keeping only the checks publish time actually owns: per-file size and digest, fragment/trace/span counts, span_kinds equality and fragment trace ownership. validate_archive no longer parses every archive twice through two independent implementations. The judged-outcome coupling now has one enforcement site, GenerationRun.record_judgment, which is the sole writer of judgments.jsonl. The archive projection, the two record parsers and the route builder drop their copies of the check. New manifests no longer carry composer_defaults. The atomic archive write with read-back-before-replace is unchanged; it is what makes a published scenario immutable. * refactor(datagen): scenario vocabulary and one owner per publish check prepare_publication no longer writes the staged index and reads it back to compare it against the record it was just serialized from. validate_archive drops its schema-v1 branch and the flag narrows to 2, matching a runtime that no longer loads v1. The duplicate instrumenter-version parser is gone. The workflow stops re-implementing the archive-name regex and the schema version guard that publish validate already applies, and drops the v1 option its dropdown could no longer produce a passing run for. The README now states the actual reason publication is safe: the tool holds no credentials and makes no network write, so nothing reaches the bucket until someone runs the two printed commands, and --no-clobber on the digest-namespaced archive is what keeps a published scenario immutable. Documented commands and env var names are corrected to what exists. * style(datagen): format test_codex_exec.py Applies the repo formatter to a file this change already touches. * refactor(datagen): default the destination project to phoenix-datagen The default project name no longer derives from the scenario manifest; zero-config replay always lands in the phoenix-datagen project, and --project / PHOENIX_PROJECT_NAME still override it. Claude-Session: https://claude.ai/code/session_01Jb1jAxuoy8BeYgAuchjH3L * refactor(datagen): zero-config replay with bundled or sole published scenario phoenix datagen now runs with no configuration: it replays the scenario bundled into the installation (Docker images bake one in at build time) or, failing that, the sole scenario in the published index, into the phoenix-datagen project. When the index holds several scenarios the command asks for --scenario instead of silently picking one. - delete every PHOENIX_DATAGEN_* environment variable; rate, epsilon, seed, burstiness, scenario, and the anomaly manifest are flags only - drop the base-URL override and the "default" scenario name convention along with its alphabetical fallback - phoenix datagen pull with no argument primes the sole published scenario - add the bundled-assets placeholder index the Docker bake stage copies - deploy configs (compose, helm, kustomize, render, docs) pass flags instead of the removed environment variables Claude-Session: https://claude.ai/code/session_01Jb1jAxuoy8BeYgAuchjH3L * refactor(datagen): drop the seven session-shape tuning flags The session-shape distributions (fragments per session, fragment gaps, archetype mix) keep their built-in defaults; the CLI no longer exposes per-distribution tuning. Programmatic callers and tests can still pin distributions through Replayer's composer_config parameter. Claude-Session: https://claude.ai/code/session_01Jb1jAxuoy8BeYgAuchjH3L * refactor(datagen): remove backfill, rate schedules, and the anomaly manifest The replay loop is one shape again: emit, export, sleep. The virtual- cursor loop, the business-hours rate profile with its timezone handling, the backfill duration parsing, and the anomaly-manifest JSONL writer are gone. Anomaly and error ground truth remains available in memory on each EmittedTrace for tests and programmatic callers. Claude-Session: https://claude.ai/code/session_01Jb1jAxuoy8BeYgAuchjH3L * Relax datagen replay validation and cache checks * Trim datagen generation checks and tests * Flatten datagen's published banks into a single corpus * Trim the datagen replayer to its live paths * feat(datagen): simplify corpus archive pipeline * refactor(datagen): simplify trace replay * feat(datagen): replace generation runs with recorder fixtures * refactor(datagen): record archetypes from fixed fixtures * refactor(datagen): align deployment with corpus replay * fix(datagen): satisfy repository type checks * feat(datagen): add recorder condition materialization * feat(datagen): add conditioned live recording lane * fix(datagen): skip llama-index recorder test when instrumenter is absent Claude-Session: https://claude.ai/code/session_01BWBfzCgBofoRLpGua3kihU * fix(datagen): skip guardrail recorder test when framework is absent Claude-Session: https://claude.ai/code/session_01JXjpZ2FuYsz35M2ndhLqCC * test(datagen): trim suite to one happy path per surface Collapse the datagen suite to a single executable example per public surface: loader, fetcher, composer, replayer, exporter, CLI run and pull, corpus pipeline, conditions materialization, recording reset/append, and one per recorder entry point. Deleted test_mock_openai_provider.py and test_fake_tools.py — both helpers are exercised through the recorder and conditions tests, and the provider test pinned an internal response counter. Removed the signature-introspection tests on the graph and guardrail recorders, the offline pointer fallback, the live-client error path, and the invalid-tool-path and multi-intensity condition parametrizations. Merged coverage that was worth keeping into the survivors: the tool-agent test now records under the shipped repository condition and asserts both framework span kinds and the authored tool result; the recording test proves reset truncates before append preserves; the CLI test drives register -> parse -> run instead of pinning the private _resolve_config helper. Claude-Session: https://claude.ai/code/session_01EJA3HS5EzqKqcHuLzHdRSN * feat(datagen): add iterative coding tool traces * feat(datagen): enrich authored corpus inputs * feat(datagen): report corpus depth statistics * feat(datagen): simulate live chat follow-up users * fix(datagen): suppress simulated user spans * fix(datagen): resolve luna recorder model * fix(datagen): configure luna tool calls * feat(datagen): vary simulated user dispositions * feat(datagen): add manual agent phase spans * Weight replay session sampling by fragment count The composer picked its archetype uniformly (then domain uniformly within it), so a 3-fragment archetype fired as often as a 110-fragment one and coding sessions replayed at ~14% despite being 25% of the corpus. Sample the (archetype, domain) cell proportionally to its fragment count so replay traffic tracks the corpus's authored composition. Claude-Session: https://claude.ai/code/session_014gvDFFS2FTKCnjCnQpcdng * Add a fat-tail slow-span outlier to replay jitter Recorded latencies top out around 12s, so the ±10% duration jitter never produces the slow outliers real latency distributions carry. With small probability per trace, stretch its longest leaf span by a lognormal factor (median 4x); parent end-time extension propagates the stall upward, yielding occasional 20s+ spans for latency filters to find. Claude-Session: https://claude.ai/code/session_014gvDFFS2FTKCnjCnQpcdng * docs: replace internal vocabulary with plain terms The recorder playbook and phoenix-cli skill docs had accumulated dense insider vocabulary. Swap "grain" for "level" and "legislates" for "enforces" throughout the filter-DSL skill docs, and in the recorder README define fragment/archetype/domain at first use, name the fragment row fields, drop "recording contract"/"generation stream"/"operating agent" phrasing, and render the intensity thresholds as a table. Claude-Session: https://claude.ai/code/session_014gvDFFS2FTKCnjCnQpcdng * Fix CI: formatting, redundant cast, and datagen script type checking Format scripts/update_kustomize.py; drop a cast mypy flags as redundant in the composer; skip mypy import-following for the PEP 723 recorder scripts, whose dependencies are not installed in the development environment; and guard two recorder tests with importorskip so collection succeeds where langchain is absent. Claude-Session: https://claude.ai/code/session_014gvDFFS2FTKCnjCnQpcdng * Fix datagen container start commands for the distroless image The Phoenix image's ENTRYPOINT is the Python interpreter and its console scripts carry interpreter paths that are not valid in the final image, so overriding the command with "phoenix datagen" cannot start. Use module arguments instead: compose overrides CMD, Kubernetes manifests use args to keep the ENTRYPOINT, and Render/Railway/Cloud Run instructions invoke python3 -m phoenix.server.main. Also align the kustomize datagen image tag with the base (20.4.0); release automation now bumps both together. Claude-Session: https://claude.ai/code/session_014gvDFFS2FTKCnjCnQpcdng * Prefix replayed session ids with their domain Session ids read customer_support-<hex> instead of datagen-<hex>, so the sessions view identifies what kind of conversation each row holds at a glance. Claude-Session: https://claude.ai/code/session_014gvDFFS2FTKCnjCnQpcdng * Give each archetype its own session-length profile One global median of two fragments per session made every replayed session read like a one-or-two-question visit: 38 of 150 sampled sessions held a single trace. Real sessions differ by application shape, so the composer now draws fragments per session from a per-archetype lognormal profile: agent work sessions string together a median of six episodes (p90 ~15 traces), chat and retrieval conversations run ~10 turns at the median, extraction stays batch-like. Claude-Session: https://claude.ai/code/session_014gvDFFS2FTKCnjCnQpcdng * Steer conversation length organically and diversify coding seeds Live plain-chat conversations now run until the simulated user closes them: a per-fixture target turn count controls when the simulator is told to wrap up once its concern is addressed, so lengths cluster near the target while every conversation ends at a natural closing message. Conversational archetypes accordingly compose sessions from one recorded conversation instead of concatenating unrelated ones. Coding seed diversity: eight new tool-agent coding tasks against an expanded fake repository (metrics, config, broker, changelog files with matching issues and tests), plus authored prompt paraphrases picked per live run so repeated recordings do not open with identical text. The corpus packer now reports distinct opening inputs per domain so thin seed variety is visible at packaging time. Adds the terra live-model alias. Claude-Session: https://claude.ai/code/session_014gvDFFS2FTKCnjCnQpcdng * Let chat sessions chain a few whole conversations Multi-topic sessions chaining recorded conversations are acceptable realism; keep the chain short now that each conversation records at full length. Claude-Session: https://claude.ai/code/session_014gvDFFS2FTKCnjCnQpcdng * Apply ruff formatting to datagen recorder and test Claude-Session: https://claude.ai/code/session_014gvDFFS2FTKCnjCnQpcdng * Vary chat conversation openings per live run Chat fixtures carry authored opening phrasings; live recording picks one per run so the session list does not repeat identical first messages. Conditioned runs keep their materialized opening. Claude-Session: https://claude.ai/code/session_014gvDFFS2FTKCnjCnQpcdng * Restore scripts/ in the unit-test checkout and pin the chat test opening The datagen unit tests import the recorder modules under scripts/, but the unit-test job's sparse checkout dropped that directory when the DSL compile checks moved to their own job, so collection failed with ModuleNotFoundError. Re-add it. Also pin the live chat test to the authored opening so the new per-run phrasing choice cannot make its assertions flaky. Claude-Session: https://claude.ai/code/session_014gvDFFS2FTKCnjCnQpcdng * Scope the scripted coding-agent test to fixtures with scripted episodes The test passed every coding fixture to the scripted recorder, which raises for the new live-only tasks. Select only the fixtures that carry a deterministic scripted episode. Claude-Session: https://claude.ai/code/session_014gvDFFS2FTKCnjCnQpcdng * Format the coding-agent test selection Claude-Session: https://claude.ai/code/session_014gvDFFS2FTKCnjCnQpcdng * Move datagen tooling tests out of the unit suite The recorder and corpus-pipeline tests import scripts.datagen.*, which forced the Unit Tests CI job to check out all of scripts/ and forced a mypy override for the un-followable script modules. That put dev tooling on the wrong side of the product test boundary. - Move the ten scripts-importing test files to scripts/datagen/tests/ with their own conftest and fragment_bank fixture copy - Restore the Unit Tests sparse-checkout to main's list - Add a path-filtered Datagen Tooling Tests job that runs uv run pytest scripts/datagen/tests - Drop the scripts.datagen.* mypy override Claude-Session: https://claude.ai/code/session_014gvDFFS2FTKCnjCnQpcdng * Narrow this PR to the datagen replayer runtime Split per review feedback: the generation tooling (scripts/datagen) and the deployment integration + docs (compose, kustomize, helm, render, self-hosting page) move to follow-up PRs. This PR keeps the feature that stands alone: the phoenix.datagen runtime (fetcher, loader, composer, replayer, exporter), the datagen server subcommand, and their unit tests. Reverts the langchain test pins and mypy carve-outs the generation tests had pushed into the shared dev environment. Claude-Session: https://claude.ai/code/session_014gvDFFS2FTKCnjCnQpcdng * fix(ci): cap pydantic-ai-slim below 2.34 in unit test requirements The unit-test tox env installs with -U and resolves pydantic-ai-slim fresh, so it picked up 2.34.0, whose source no longer matches the vendored data-stream protocol types in src/phoenix/db/types. test_data_stream_protocol_compatibility now fails on every PR (verified: 2.33.0 passes, 2.34.0 fails the request-type and schema parity tests). Cap the unit env at 2.33 parity until the vendored types are re-synced. Claude-Session: https://claude.ai/code/session_014gvDFFS2FTKCnjCnQpcdng * Keep jittered token totals consistent when one component is missing A span carrying only prompt and total counts (the embedding-span shape) fell through to the independent total jitter, so the emitted total could drop below the jittered prompt count. Recompute the total from the jittered components whenever either one is present, leaving the independent jitter for spans that carry only a total. Claude-Session: https://claude.ai/code/session_014gvDFFS2FTKCnjCnQpcdng * Drop the pydantic-ai-slim unit-test cap after the vendored re-sync Main upgraded pydantic-ai-slim to 2.34 and re-synced the vendored data-stream types (Arize-ai#15714), so the stopgap cap this branch carried is no longer needed. Claude-Session: https://claude.ai/code/session_014gvDFFS2FTKCnjCnQpcdng * Inline PHOENIX_CLIENT_HEADERS parsing in the datagen command Main removed phoenix.utilities.re with the legacy client cleanup (Arize-ai#15709), which deleted parse_env_headers from the server package. Parse the W3C Baggage-style header string locally instead of reaching into the client package. Claude-Session: https://claude.ai/code/session_014gvDFFS2FTKCnjCnQpcdng * Mark datagen as internal tooling and move it under experimental Review feedback: hide the datagen subcommand from the top-level help (help=SUPPRESS, matching db) while documenting it for direct --help invocation, add the no-compatibility disclaimer to the package docstring, and move the package to phoenix.experimental.datagen so the wheel's import surface carries the same signal. Claude-Session: https://claude.ai/code/session_014gvDFFS2FTKCnjCnQpcdng
Co-authored-by: axiomofjoy <15664869+axiomofjoy@users.noreply.github.com>
…#15724) * chore(skills): track third-party skills in skills-lock.json Install gh-stack, mintlify, and vercel-react-best-practices through the skills CLI so they join agent-browser in skills-lock.json, and refresh all four from upstream. A lockfile entry is what hides a vendored skill from `npx skills add Arize-ai/phoenix`, so the hand-added `internal: true` flags are no longer needed and would be reverted by the weekly update. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore(skills): drop third-party section from skills README Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* chore(js): upgrade pnpm to 12.0.0 * chore(js): keep pnpm install hint generic
…ize-ai#15723) * docs: add Arize AX callout to top-level Phoenix overview page * docs: simplify AX callout and add Phoenix vs AX card to Other Resources * docs: reword AX callout to managed enterprise platform * docs: drop AX info callout, keep Other Resources card * docs: promote AX callout to a horizontal card * docs: present AX callout as paired cards with product and comparison links * docs: add intro sentence above AX cards
Merges upstream/main (145 commits, 8f31d5d..c1da84b) into the fork. Conflicts resolved: - src/phoenix/config.py: kept the fork's media env-var helpers; also dropped get_env_client_headers, which upstream removed (replaced get_env_phoenix_api_key with get_env_phoenix_secret) and which had no callers left in src/phoenix/ — the real implementation lives in packages/phoenix-otel and packages/phoenix-client. - src/phoenix/server/api/routers/v1/__init__.py: kept both the fork's media router import and upstream's new model_providers router, alphabetically ordered. - Generated artifacts (schema.graphql, openapi.json, generated TS/Relay clients): took upstream's copies and regenerated via make graphql && make openapi. Migration graph: the fork-local media migration already chains off upstream's current head; no re-pointing needed.
minhthao56
added
Ok to test
Claude
4 tasks
Upstream bumped pyproject.toml's [tool.uv] required-version to 0.12.5 in this sync (0430e0c), and updated its own workflow files to match. The fork's own CI wrappers (pre-merge.yml, post-merge.yml) pin UV_VERSION independently and upstream's commit couldn't touch them, so they were left at 0.12.1 — every job invoking make targets that shell out to uv failed with a required-version mismatch. Also regenerates Dockerfile.fork via `make dockerfile-fork`, which carries the same uv version pin.
minhthao56
added
Ok to test
Claude
pre-merge.yml's openapi-compat job excludes agent/PXI routes from the backward-compatibility gate (they're still fast-moving upstream), but matched only a bare "/agent" prefix. This sync's upstream range moved those routes from /agents/... to /v1/agent_sessions/..., so the filter stopped matching and the job failed on exactly the routes it was built to ignore. Broaden it to match an optional /v1/ prefix. A previous sync PR (#40, closed/superseded rather than merged) already carried this exact fix on its own branch; porting it here since it never reached main.
minhthao56
added
Ok to test
Claude
4 tasks
Author
|
Superseded by #42 — 9 more commits landed upstream since this PR was opened ( Generated by Claude Code |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
20 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Syncs this fork with
Arize-ai/phoenixupstream. The fork was 145 commits behindupstream/main(merge-base8f31d5d54→c1da84b5).This supersedes #40, which was opened yesterday against an older upstream tip
(
8adf3376, 124 commits) — 21 more commits landed upstream since then, including twothat changed this environment's own tooling requirements:
0430e0c9achore(deps): bump uv to 0.12.5 (chore(deps): bump uv to 0.12.5 to match Dependabot Arize-ai/phoenix#15712)fc12523a6chore(js): upgrade pnpm to 12.0.0 (chore(js): upgrade pnpm to 12.0.0 Arize-ai/phoenix#15719)Other notable commits in that additional range:
b86acaccafeat(datagen): add a sidecar that replays a recorded trace corpus (feat(datagen): add a sidecar that replays a recorded trace corpus Arize-ai/phoenix#15614)0e3b2c2aefeat(evals): add session-level pii_detection evaluator (feat(evals): add session-level pii_detection evaluator Arize-ai/phoenix#15096)22ee34342fix(atif): link system subagents to emitted parent spans (fix(atif): link system subagents to emitted parent spans Arize-ai/phoenix#15585)269986ce1fix(dsl): correlate the parent test to the span an annotation filter is reading (fix(dsl): correlate the parent test to the span an annotation filter is reading Arize-ai/phoenix#15710)dc8489197fix(cost): update built-in model token prices (fix(cost): update built-in model token prices Arize-ai/phoenix#15711)116a0ec17fix(ui): truncate long annotation names in experiment comparisons (fix(ui): truncate long annotation names in experiment comparisons Arize-ai/phoenix#15722)743d32032chore: complete removal of the legacy phoenix.session.client module (chore: complete removal of the legacy phoenix.session.client module Arize-ai/phoenix#15709)#40 is being closed with a comment pointing here.
Highlights from the full 145-commit range, grouped by area:
Features
PATCH /projects/{id}/retention,GET /model_providersapproval gating, unified PXI agent builders, MCP skill tools in PXI
annotation summary table columns; 404 not-found handling for redirects
Fixes
truncate in experiment comparisons
/v1/agent_sessions/*response schemas changed (upstream replaced the old/agents/{agent_id}/sessions/{session_id}/chatand.../summaryroutes) — this is theexpected OpenAPI break the
upstream-synclabel exists to skip; no fork code touchesthose routes
Releases / tooling
updates to run codegen for this sync)
arize-phoenix,arize-phoenix-client,arize-phoenix-evals,arize-phoenix-sqleanreleases and weekly dependency upgradesphoenix.session.clientmodule fully removedDocs
good-first-issue triage CI workflow, terminology cleanup in the phoenix-cli skill docs
Conflict resolution
Two hand-written conflicts, both in files the fork-ownership rules already document:
src/phoenix/config.py: kept the fork'sget_env_max_media_bytesandget_env_media_orphan_grace_period_hours. Also found and removedget_env_client_headers, which had been carried along in the same conflict hunk butturned out to be dead code — upstream removed
get_env_phoenix_api_key(renamed toget_env_phoenix_secret),ENV_PHOENIX_CLIENT_HEADERS, andparse_env_headersentirely, and nothing in
src/phoenix/called this function (the real client-headersimplementations live in
packages/phoenix-otelandpackages/phoenix-client, each withtheir own copies). Confirmed via
make typecheck-python, which failed on the orphanedreferences until this was removed.
src/phoenix/server/api/routers/v1/__init__.py: kept both the fork'smediarouter import and upstream's new
model_providersrouter, alphabetically ordered — nologic conflict.
Generated-artifact conflicts (
schemas/openapi.json,js/app/schema.graphql, and thegenerated TS/Relay/OpenAPI clients — 7 files) were resolved by taking upstream's copy and
regenerating via
make graphql && make openapi, per the fork-ownership rule of neverhand-editing generated files.
The fork-local media migration (
e0307b79758d_add_media_files_table.py) already chainsoff upstream's current migration head — no re-pointing needed.
Verification
uv run pytest tests/unit/db/test_migration_heads.py— single migration head, passesmake typecheck-python— no issues (1112 source files)pnpm typecheck(js/app, aftermake buildto buildphoenix-config/phoenix-otel/phoenix-client/phoenix-evals) — cleanmake format-python && make lint-python && git diff --exit-code— clean, no diff(this is what CI's "Format and Lint" job gates on)
uv run pytest tests/unit/server/api/helpers tests/unit/db -q— 1217 passed, 324skipped (Postgres-only)
uv run pytest tests/unit/server/api/helpers/test_playground_media.py tests/unit/server/api/helpers/test_message_media.py -q— 73 passed on sqlite,including
TestEveryProviderAcceptsImagesNow— no fork media-provider payloaddelegation was dropped by the merge
pnpm vitest run(js/app) — 249 test files, 2608 passed, 12 skippedcontentPartSelectionSets.test.tsandmediaContentPartFragment.test.ts— pass; no newGraphQL document silently drops media selections
Test plan
— see below)
test_migration_heads.pyconfirms a single alembic headTestEveryProviderAcceptsImagesNow) confirm no forkmedia logic was dropped by the merge
Note on OpenAPI Backward Compatibility
This sync includes upstream's replacement of
/agents/{agent_id}/sessions/{session_id}/chatand.../summarywith/v1/agent_sessions/*, a breaking schema change that is entirely upstream's — no forkcode called those routes. The
upstream-synclabel (applied to this PR) makes.github/workflows/openapi-schema.yamlskip that check for exactly this reason.Generated by Claude Code