What it does
Phoenix is an observability and evaluation tool from Arize for AI applications and agents. It records the steps an agent takes, including prompts, retrievals, tool calls and outputs, so you can see why a response came out the way it did. Beyond tracing, it supports annotations added through human review or an LLM acting as judge, datasets built from traces, experiments that rerun a change under the same conditions, and evaluations that score output. The page also describes a Prompt IDE and an in-product agent called PXI that lets you ask questions about your traces. The site says Phoenix is vendor agnostic across models, frameworks and languages and has native OpenTelemetry support. You can run it on your laptop with one command, in a Docker container, or on Kubernetes with Helm, which keeps traces in your own environment. A cloud platform is available for teams that prefer not to self-host. The page states the project is ELv2 licensed.
Capabilities
- Tracing of prompts, retrievals, tool calls and outputs for each agent step
- Annotations from human review or an LLM judge
- Datasets and experiments to compare changes under the same conditions
- Built-in evaluations and a Prompt IDE
- PXI agent for asking questions about your traces
- Native OpenTelemetry support and no model or framework lock-in
When to use it
- Debugging why an agent or LLM app gave a particular answer
- Scoring output quality before shipping a prompt or model change
- Keeping trace data on your own infrastructure by self-hosting
- Adding tracing to an app that mixes different models or frameworks
