robotnurseLarge language models on hardware you own and can touch.
Field notes from building, breaking, and fixing local LLM setups: quantization, inference tuning, hardware that actually matters, and the honest benchmarks behind each claim. Written slow, tested twice, updated when something stops working. If you would rather have it set up than read about it, book a free 30-minute call.
- lived
- 11 months
- hardware
- gpu ×8 · apple silicon ×2
- entries
- 74 field notes
- hype
- off
The notebook is what we learned. This is what we do with it.
Private, on-site AI on machines you own. Your hardware. Your weights. No invoices from a cloud provider.
On-site in NYC. We install, quantize, and tune models on your own machines, then leave you running.
A short, paid look at your current stack: what is slow, what is wasted, what to fix first. Numbers, not opinions.
Fixed-scope bundles for individuals, small teams, and clinics that need private inference with no cloud dependency.
Latest entries
All 74 posts →Two Crash Fixes That Cost Throughput and Fixed Nothing
A daily inference-server crash got two mitigations, both matched to the symptom by name, both 'validated' by a load test that passed 12 of 12. One mechanism was impossible, the other flag routed traffic into the failing path, and one of them cost 24% of decode.
The Win Was Four Slots, Not the Prefix Cache
Three agent conversations against two serving slots: time to first token grew from 16 to 61 seconds a turn. Against four slots it stayed flat at 1.6-1.9 seconds, and the prefix cache we had just switched on recorded zero hits.
The Chat Template Ignored enable_thinking
Three different ways of switching thinking off all left one model thinking at its maximum setting, because its chat template never reads any of them. It reads only reasoning_effort, and treats every value except two as 'Max'. A two-render check at startup finds this in any template.
fp16 Residual Streams and a Two-Million Activation
A model whose hyper-connection streams peak around 2.0e6 on an ordinary prompt cannot store them in fp16, whose ceiling is 65,504: every logit becomes NaN. Our fixture never left fp16's range, so no test could see it. The fix is preventive, and it is not the fix for our open copy bug.
The Tag Was 128 Tokens Back: An Open Bug
Our engine's copy of a model cannot finish its own tool-call tags; another engine running the same prompt can. Here is everything we ruled out, the one test that 'failed' by 10% without finding anything, and the suspect our reference cannot see. Still open.
We Sent a V4 Model a V3.1 Tool Prompt
Our renderer wrote the previous generation's tool-call format for a model trained on a different one. The model copied the placeholder words from the instructions, or ended its turn with one token. The fix was to port the vendor's encoder step for step and test it byte for byte.
Tools
Free calculators we use before buying anything.
Buyers’ notes
- 01Best Mac Studio config for local LLMs — 2026
- 02Mac mini vs Mac Pro for local AI — which is worth it
- 03How much RAM do you need for 7B, 13B, and 30B models?
- 04amdgpu vs Mac for local inference — a fair fight
- 05Mac Studio vs GPU PC for local AI — an honest comparison
- 06Best local AI setup under $3,000 — three builds that work
- 07RAM vs VRAM: how much memory do local LLMs actually need?
- 08Turn an old gaming PC into a local AI box
More in all buyers’ notes.