LLM CPU Benchmarks
Tool-calling accuracy, throughput, and memory for small (~4B) open-weight LLMs running on a commodity x86 CPU box. Qwen3.5-4B · Gemma-4-E4B-it · Phi-4-mini, each measured with standard and TurboQuant KV-cache compression.
What's measured
Apples-to-apples
Three modern ~4B tool-calling models on the same CPU box, with the same harness, the same scenarios, the same warm-up budget. The numbers are comparable.
Standard vs TurboQuant
Every model is measured with default KV cache and again with TurboQuant compression — so you can see what compression actually costs in tool-calling accuracy.
CPU-only target
No GPU. Commodity x86. The benchmark exists for teams who need on-prem, air-gapped, or budget-constrained deployments — the same teams Deemwar builds for.
Methodology
- BFCL test harness adapted for tool-calling accuracy on a fixed scenario set
- Per-cell driver scripts so every run is reproducible from a single command
- Standard vs TurboQuant KV-cache compression measured side by side
- Throughput, memory footprint, and tool-call success rate captured per run
- Aggregate summary published as static JSON — wire it into your own dashboards
Need a custom CPU-LLM benchmark?
Open the live results, or talk to us about running the same harness against your target hardware, quantisation, or tool surface.