Best tools for monitoring agents in production (2026)
Observability for agents in production: traces, costs, latency and errors in one place. We list 47 tools for monitoring agents in production below, each with a link-checked page, pricing as the maker states it and alternatives.
Updated
47 tools: 29 open source, 3 source-available, 11 free or freemium, 4 paid, usage-based or enterprise.
Evals and benchmarks (14)
Openlayer MCP
Openlayer's MCP server, letting coding agents pull evaluation results, failing tests and monitoring data from Openlayer so they can debug and improve AI systems from the IDE.
Added
Opik
Comet's open-source platform for tracing, evaluating and monitoring LLM apps, RAG systems and agent workflows, self-hosted or in the cloud.
Added
Future AGI
Open-source platform for evaluating, tracing, simulating and guarding LLM and AI agent applications, self-hostable or cloud.
Added
Arize AI
Agent observability and evaluation platform with tracing, evals and experiments for improving AI agents.
Added
Confident AI
Enterprise platform for AI evals and observability that gives teams one standard for measuring and monitoring AI quality.
Added
MLflow
Open-source platform for tracing, evaluating and managing agents, LLM apps and ML models.
Added
Arize Phoenix
Open-source tracing and evaluation for LLM apps.
Added
Agenta
Open-source LLMOps platform for prompt management, evaluation and observability.
Added
Braintrust
Evals and observability platform for AI products.
Added
Kore.ai Autoloop
Optimization engine in Kore.ai Agent Platform Artemis that evaluates agents against business goals, diagnoses failures from traces, applies targeted fixes, and re-verifies; GA Oct 2026.
Added
Langfuse
Open-source LLM observability, tracing and evals.
Added
LangSmith
Tracing, evals and monitoring for LLM apps and agents.
Added
Respan
Observability and evaluation platform for LLM apps and agents.
Added
W&B Weave
Weights & Biases toolkit for tracing and evaluating agents.
Added
Guardrails and policies (7)
OpenAPPA
OpenAPPA is a deterministic guardrail that sits between an AI agent and its tools and tracks how data flows across tool calls, instead of asking a second model to judge each call. Policies decide whether untrusted or sensitive data may reach a given action, which blocks data exfiltration caused by prompt injection or hallucination while letting agents finish their tasks. It plugs into Claude Code, Pi, OpenCode, kAgent and Archestra, comes with reusable policy "batteries", validation, observability and reporting, and is backed by a NeurIPS workshop paper with published benchmarks. It is in preview as an open RFC.
Added
LiteLLM
Open-source AI gateway and Python SDK that calls 100+ LLM providers in the OpenAI format, with cost tracking, guardrails, load balancing and logging.
Added
coucou
Mac notch and phone companion that monitors your Claude Code, Codex, Cursor and Gemini CLI sessions and lets you approve permissions on the go.
Added
Metorial
Open-source identity and access layer that connects AI agents to 1,200+ integrations via MCP, CLI or API with auth, permissions and observability.
Added
ToolHive
Stacklok platform that runs MCP servers in isolated containers with per-request identity, access policy and observability.
Added
Failproof AI
Observability and policy enforcement for AI agent harnesses.
Added
NVIDIA Open Agent Safety Platform
Reference stack pairing OpenShell with NVIDIA Sentry on BlueField-4 DPUs for hardware-level agent monitoring.
Added
More tools (26)
OneUptime MCP
OneUptime's official MCP server, giving agents access to monitors, incidents, on-call and status pages in this open-source observability platform. Free plan, paid plans from $22 per user, or self-host.
Added
Beacon
An open-source memory layer that captures traces from coding agents across harnesses and turns them into self-improving memory and visibility into what your agents did.
Added
OpenOSINT
An open-source OSINT engine with about 20 passive investigation tools usable from a CLI, a web UI or an MCP server for AI assistants, with hosted monitoring and due-diligence plans.
Added
Agent Barn
Agent Barn is an open-source (Apache-2.0) control plane for running a fleet of AI agents on your own Kubernetes: create, configure, run and monitor agents across Slack, Microsoft Teams and Telegram from one place.
Added
sem
Semantic layer on top of Git that diffs, blames and traces impact by function and class instead of by line, with an MCP server for coding agents.
Added
Xquik
Xquik is an X (Twitter) data service for searching and extracting tweets, replies, followers and profiles, monitoring accounts and posting, over REST, an OAuth MCP server and SDKs; pay-as-you-go from $10 or monthly plans from $20.
Added
Latitude
Observability platform for AI agents that captures traces, groups failures into signals and verifies fixes against real traces.
Added
Jarvis Registry
Open-source MCP and A2A agent gateway that connects copilots and agents to enterprise tools with identity, access control and observability.
Added
Laminar
Open-source observability platform for AI agents with OpenTelemetry tracing, plain-English behavior tracking and alerts.
Added
Zeron
Native control plane for running and monitoring Claude Code, Codex, Cursor, Devin and other coding agents locally.
Added
Pezzo
Cloud-native LLMOps platform for prompt design, versioning, delivery, observability and cost monitoring.
Added
ADR
Uber's Agentic AI Detection and Response system that secures employee and customer-facing AI agents with observability and threat detection.
Added
OpenInference
OpenTelemetry-based instrumentation that traces LLM and agent framework calls so any observability backend can show them.
Added
Judgeval
Open-source SDK for tracing agents and scoring them with agent judges, so failures in production can be found and fixes checked.
Added
Sentry MCP
Sentry's MCP server and CLI that give coding agents access to Sentry issues, errors and traces for debugging.
Added
UptimeRobot MCP
UptimeRobot's MCP server for creating monitors, triaging incidents and querying uptime stats for sites, APIs and servers.
Added
Axiom MCP
Axiom's MCP server for listing datasets and schemas and running APL queries for log exploration and monitoring.
Added
Bland AI
Enterprise voice AI platform for building, deploying and monitoring phone agents with self-hosted models.
Added
Picsellia MCP Server
Official MCP server for Picsellia's computer vision MLOps platform, giving AI assistants access to datasets, experiments, models, deployments and monitoring.
Added
Sim Studio
Workspace for visually building, deploying and monitoring AI agents and workflows.
Added
agentgateway
Proxy for AI agents and MCP servers with security and observability.
Added
Cloudflare MCP Servers
Cloudflare's collection of MCP servers for Workers, observability, DNS analytics and other Cloudflare services.
Added
abtop
htop-style monitor for Claude Code and Codex sessions and token use.
Added
New Relic MCP
New Relic's MCP server for querying metrics, logs, traces and alerts through an agent.
Added
AgentOps
Monitoring and session replay for AI agents.
Added
Helicone
Open-source LLM observability and gateway.
Added
GitHub repos (31)
| Repo | GitHub stars | Licence | Last push |
|---|---|---|---|
| LiteLLM ↗ | 61,465 | MIT | 2026-10-21 |
| Sim Studio ↗ | 29,835 | Apache License 2.0 | 2026-10-21 |
| MLflow ↗ | 28,418 | Apache License 2.0 | 2026-10-21 |
| Opik ↗ | 22,700 | Apache License 2.0 | 2026-10-21 |
| Arize Phoenix ↗ | 11,828 | Elastic License 2.0 (source-available) | 2026-10-21 |
| OneUptime MCP ↗ | 7,751 | — | 2026-10-21 |
| Failproof AI ↗ | 5,560 | MIT | 2026-10-20 |
| coucou ↗ | 5,468 | — | 2026-10-20 |
| agentgateway ↗ | 5,373 | Apache License 2.0 | 2026-10-20 |
| Agenta ↗ | 4,840 | MIT License, except ee/ directories (separate licence) | 2026-10-20 |
| Latitude ↗ | 4,723 | — | 2026-10-20 |
| Cloudflare MCP Servers ↗ | 4,387 | Apache License 2.0 | 2026-10-20 |
| abtop ↗ | 3,752 | MIT | 2026-09-14 |
| Jarvis Registry ↗ | 3,648 | — | 2026-10-19 |
| sem ↗ | 3,415 | — | 2026-10-19 |
Top 15 of 31 by stars. Stars and licence come from each repo, checked when the listing was last verified.
Related tasks
Popular with: AI engineers · DevOps and SRE engineers · Startup CTOs · Related searches: Open-source LLM observability and eval tools
Tools are matched to tasks by category, use cases and what each tool says it does. Missing one? Submit it free.