Local LLM Field Notes

Local LLM Field Noteshttps://robotnurse.ioReal findings from self-hosted LLM tuning. Your SSH-Launched Benchmark Is Running at nice 5https://robotnurse.io/posts/12-your-ssh-benchmark-runs-at-nice-5.html https://robotnurse.io/posts/12-your-ssh-benchmark-runs-at-nice-5.htmlWed, 02 Sep 2026 00:00:00 +0000On macOS, backgrounding a process from a non-interactive SSH shell silently deprioritizes it. Every number you compare against a system daemon is invalid. https://robotnurse.io/posts/12-your-ssh-benchmark-runs-at-nice-5.htmlrobotnurseOur Prefix Cache Died at Token 1,067https://robotnurse.io/posts/15-prefix-cache-died-at-token-1067.html https://robotnurse.io/posts/15-prefix-cache-died-at-token-1067.htmlWed, 02 Sep 2026 00:00:00 +0000A 47,000-token agent prompt with a carefully designed stable-first layout, getting 49.8% cache reuse. The culprit was upstream of the system prompt entirely. https://robotnurse.io/posts/15-prefix-cache-died-at-token-1067.htmlrobotnurseA Safety Hook Truncated Every Code Generationhttps://robotnurse.io/posts/18-a-safety-hook-truncated-our-code-generation.html https://robotnurse.io/posts/18-a-safety-hook-truncated-our-code-generation.htmlSat, 15 Aug 2026 00:00:00 +0000frequency_penalty=0.1 sounds harmless. On code it compounds until the model can't emit a newline, and it quits with finish_reason=stop so nothing looks wrong. https://robotnurse.io/posts/18-a-safety-hook-truncated-our-code-generation.htmlrobotnurseThe Benchmark Said 18%. Production Said 0.07%.https://robotnurse.io/posts/04-benchmark-said-18-percent-production-said-0.07.html https://robotnurse.io/posts/04-benchmark-said-18-percent-production-said-0.07.htmlMon, 10 Aug 2026 00:00:00 +0000A benchmark found a serious failure rate in our serving stack. We nearly built a gateway routing layer to fix it. Then we measured real traffic. https://robotnurse.io/posts/04-benchmark-said-18-percent-production-said-0.07.htmlrobotnurseOur GPU Cluster Is Idle 97.4% of the Timehttps://robotnurse.io/posts/06-our-cluster-is-idle-97-percent-of-the-time.html https://robotnurse.io/posts/06-our-cluster-is-idle-97-percent-of-the-time.htmlSun, 09 Aug 2026 00:00:00 +0000Thirty days of gateway logs, one SQL sweep, and the number that retired a 29% throughput upgrade before we built it. https://robotnurse.io/posts/06-our-cluster-is-idle-97-percent-of-the-time.htmlrobotnurse