Emma's Blog

Some recent learnings and general musings on evals

I've been building an LLM-backed chatbot. Retrieval-grounded, citing sources, the standard RAG pattern. I used Replit Agent to scaffold most of it and Braintrust to tell me whether it actually works.

Twelve months ago, it took three tools (Lovable, Supabase & n8n) and some orchestration. This year, it took Replit and an afternoon.

For years, organizations invested in centralizing knowledge. LLMs might make that entire approach obsolete, and the biggest barrier isn't technical.

If ‘time-saved’ is the only goal of your agentic automations, then you’re missing a trick. Automations shouldn't just save time; they should feed into your business goals.