Langfuse

A managed eval tool adopted but not relied on, with a single item-018 score delta flagged as noise

The Eval Tool I Adopted but Won't Rely On

I migrated my eval harness to a managed tool, got it working, and then decided not to trust it as canonical. A single per-item delta on identical code is why.

June 13, 2026 · 5 min · Jack Monte
RAG Starter pipeline and evaluation architecture

RAG Starter

A production-patterned RAG pipeline with a full evaluation harness: faithfulness, relevance, and Precision@3 scoring, error rate as a first-class metric, and end-to-end Langfuse tracing.

June 11, 2026 · 2 min · Jack Monte
Production LLM observability: my eval dashboard showed 1 trace, I expected 120

My Eval Dashboard Showed 1 Trace. I Expected 120.

Adding Langfuse to a RAG pipeline looked done until the dashboard showed one trace and zero scores. The real problem was trace structure, not instrumentation. Here is the gotcha, the fix, and the numbers.

May 28, 2026 · 5 min · Jack Monte