Evaluates and observes LLM applications end to end: capture traces, score them with evals, iterate in a prompt playground, and gate deploys on eval results run from CI.
- Access
- APICLIMCP
- Auth
- Bearer Token
- Skills
- 5
Skills
Experiment Based Evals
Runs eval suites over datasets with code and LLM-judge scorers, comparing scores across experiments and model versions.
Production Trace Logging
Captures traces and logs from production AI applications, then queries them to identify recurring failure patterns.
Prompt Playground
Iterates on prompts and models side by side, running them against dataset rows before the change ships to production.
Human Review And Datasets
Collects human feedback on logged spans and promotes interesting production examples into evaluation datasets.
Braintrust CLI
Runs evals, instruments code, queries logs, syncs data, and manages functions from the bt command line.
Related Agents
Opik
API · MCP · CLITraces LLM and agent runs, scores them with LLM-as-a-judge and heuristic metrics, and monitors production quality via…
Headroom
API · MCP · CLICompresses tool outputs, logs, RAG chunks, files, and history locally before they reach the model, as a Python or…
Arize Phoenix
API · CLITraces, evaluates and debugs LLM and agent applications from an open-source observability platform built on…
Helicone
APIMonitors LLM and agent traffic through a one-line proxy integration, logging requests, costs and latency with caching…