AgentOps

Agent observability and evaluation

About AgentOps

Monitoring for agent runs that records sessions, tool calls and costs, with SDK integrations for common frameworks.

Details

Nothing confirmed yet. Check the tool's own site for current plans and features.

Listing written by WithAutobot, checked 2026-11-04. Report a correction or request removal. Back to agent observability and evaluation.

More tools in agent observability and evaluation

Langfuse

Open-source platform for tracing, prompt management and evaluation of LLM apps, self-hosted or as a cloud service.

  • Open source
  • Can run on your own servers
  • Developer API

Arize Phoenix

by Arize AI

Open-source tool for tracing and evaluating LLM apps, built on OpenTelemetry and runnable on a laptop or server.

  • Open source
  • Can run on your own servers

W&B Weave

by Weights & Biases

Weights & Biases toolkit for tracing LLM and agent calls and running evaluations, tied to its experiment tracking.

  • Open source

Helicone

Open-source proxy and dashboard that logs model requests, cost and latency with a one-line change to your API base URL.

  • Open source
  • Can run on your own servers

Opik

by Comet

Open-source tool from Comet for logging traces, running evaluations and testing prompts for LLM apps and agents.

  • Open source
  • Can run on your own servers

promptfoo

by Promptfoo

Open-source command-line tool for testing prompts and agents against test cases and probing them for security issues.

  • Open source
  • Can run on your own servers