Five categories. Fifty-plus tools. Updated weekly with community signals. The directory is structured for scanability: category leads, then dense comparison tables.
| Tool | Tier | Pricing | HN Signal | Best For |
|---|---|---|---|---|
| Forge | Open Source | Free | 687 pts ⭐ | Full-stack agent observability |
| Langfuse | Starter+ | Free → $59 → $249/mo | 100+ pts | LLM tracing + prompt mgmt |
| AgentOps | Starter+ | Free → $79 → $199/mo | — | Multi-agent fleet monitoring |
| Helicone | Pro+ | Free → $99 → $299/mo | — | LLM proxy + cost optimization |
| LangSmith | Team+ | Free → $99/mo → Enterprise | — | LangChain-native tracing |
| Arize AI | Pro+ | Free → $199 → Enterprise | — | ML + LLM observability |
| SmithDB | Open Source | Free | 7 pts | Data layer for agent observability |
| Tool | Tier | Pricing | HN Signal | Best For |
|---|---|---|---|---|
| GLiGuard | Open Source | Free | 36 pts | Local LLM guardrail (16x faster) |
| Aperion Shield | Open Source | Free | 3 pts | Destructive coding agent blocking |
| Guardrails AI | Starter+ | Free → Custom | — | Configurable LLM safety rules |
| NeMo Guardrails | Open Source | Free | — | NVIDIA guardrails toolkit |
| Lakera Guard | Pro+ | Custom | — | Enterprise LLM firewalls |
| Tool | Tier | Pricing | Best For |
|---|---|---|---|
| Lightbox (Inkeep) | Open Source | Free | Agent workflow replay + A/B testing |
| AgentSight | Open Source | Free | eBPF-based system-wide agent tracing |
| AgentShield | Beta | Free | Agent spend control + monitoring |
| Hollow | Open Source | Free | Local multi-agent OS with debugging |
| Stoic AgentOS | Open Source | Free | Open-source agent fleet orchestration |
| Tool | Tier | Pricing | Best For |
|---|---|---|---|
| Braintrust | Team+ | Free → $249/mo | Automated eval + human feedback |
| Weights & Biases | Team+ | Free → Custom | Experiment tracking + prompt eval |
| Confident AI | Starter+ | Free → Custom | LLM testing + regression detection |
| Tool | Tier | Pricing | Best For |
|---|---|---|---|
| Voker | Starter+ | Free → Custom | Analytics for AI agents (YC S24) |
| Helicone | Pro+ | Free → $99 → $299/mo | Per-request cost breakdown |
| AgentShield | Beta | Free | Budget caps + spending alerts |