Our Prefix Cache Died at Token 1,067
A 47,000-token agent prompt with a carefully designed stable-first layout, getting 49.8% cache reuse. The culprit was upstream of the system prompt entirely.
./robotnurse.io — a working notebook, not a blog
robotnurse.io is a working notebook about running large language models on hardware you own and can touch.
Field notes from building, breaking, and fixing local LLM setups: quantization, inference tuning, hardware that actually matters, and the honest benchmarks behind each claim. Written slow, tested twice, updated when something stops working.
✎
Margins and diagrams welcome. Entries are field notes: dated, a little stained, and honest about what broke.
[ series ]
01The Model Zoo1 live post03Inference Engines1 live post04Running It Daily2 live posts05Hardware Notes1 live post
[ latest entries ]
Your SSH-Launched Benchmark Is Running at nice 5
On macOS, backgrounding a process from a non-interactive SSH shell silently deprioritizes it. Every number you compare against a system daemon is invalid.
A Safety Hook Truncated Every Code Generation
frequency_penalty=0.1 sounds harmless. On code it compounds until the model can't emit a newline, and it quits with finish_reason=stop so nothing looks wrong.
The Benchmark Said 18%. Production Said 0.07%.
A benchmark found a serious failure rate in our serving stack. We nearly built a gateway routing layer to fix it. Then we measured real traffic.
Our GPU Cluster Is Idle 97.4% of the Time
Thirty days of gateway logs, one SQL sweep, and the number that retired a 29% throughput upgrade before we built it.