Writing
Notes from client work and the lab.
Mostly about running language models in production, sometimes about the web and data systems around them.
12 posts
- AI engineering
Notes on running RAG in production
What we learned running RAG for thousands of queries a day: hybrid retrieval, chunking messy documents, evaluation in CI, and the metrics we watch.
12 min readRead - AI engineering
How we structure multi-agent systems in LangGraph
Why we moved from single agents to multi-agent graphs, and how we handle state, error recovery and tool routing after 18 months in production.
10 min readRead - Full stack
How we organize large Next.js codebases
How we organize routes and features in large Next.js apps, where the server and client boundary goes, data fetching, middleware, and what we got wrong at first.
9 min readRead - AI engineering
How we test and tune prompts
Structured chain-of-thought and outputs, few-shot calibration, prompt versioning, and the evaluation a prompt change goes through before production.
8 min readRead - AI engineering
pgvector, Pinecone and Qdrant on the same workload
We ran one production workload on three vector databases and compared latency, recall, cost and how much work each one was to operate.
11 min readRead - System design
Getting WebSockets to 100k concurrent connections
What we changed when a client's real-time collaboration platform grew from 2,000 to 40,000 daily users: Redis pub/sub, sticky sessions, backpressure and monitoring.
10 min readRead - Full stack
End-to-end types with tRPC and Next.js
Why we replaced REST controllers with tRPC, how we structure routers, handle errors and write tests, and when we would not use it.
7 min readRead - Dev tools
Putting evals in CI
How we test model output in CI: pipeline stages, dataset versioning, feature flags for models, cost checks, and rolling back a regression.
9 min readRead - AI engineering
When we fine-tune, and when we don't
How we choose between prompting, RAG and fine-tuning, and what a LoRA project involves: preparing data, training settings, evaluation and cost.
14 min readRead - Dev tools
Tools we kept using in 2024
The tools we picked up in 2024 and still use: Cursor, Turborepo, Biome, Linear and tRPC, plus the ones we tried and dropped.
6 min readRead - Career
The work between a prototype and production
What a working demo still needs before real users arrive: error handling, database migrations, monitoring, security, performance work and a readiness checklist.
8 min readRead - AI engineering
Our local LLM setup
Why and how we run models locally: Ollama, GPU memory, wiring local models into the dev stack and tests, and what it cost over six months.
7 min readRead