serving

5 posts tagged “serving”.

1. The Chat Template Ignored enable_thinking (2026-09-25)
2. The Win Was Four Slots, Not the Prefix Cache (2026-09-25)
3. Your Headline tok/s Is a Short-Prompt Number (2026-09-18)
4. Sizing a Local-AI Box for a 5-to-20-Person Office (2026-09-12)
5. Our Chunked-Prefill Fix Was 72x Slower. Every Test Passed. (2026-09-07)