AI Hashrate is a fit & speed estimator for local / self-hosted LLM inference. Most cells are estimated. A small set of public anchors are measured. This is not a lab leaderboard.
| Label | Meaning |
|---|---|
| measured | Published or curated number (engine noted when known). Overrides the estimate for that model × GPU × quant bucket. |
| estimated | Computed from hardware peak memory bandwidth and model size. Relative ranking is more reliable than absolute tok/s. |
Decode (generation) is typically memory-bandwidth bound for large models at batch size 1:
tok/s ≈ (memory_bandwidth_GB/s × utilization)
÷ (active_params_billions × bytes_per_param)
0.35 (35% of theoretical peak)0.08(vram / model_size)² (rough offload penalty)Real engines (llama.cpp, vLLM, MLX, …), batch size, and kernels can move numbers by tens of percent. Treat estimates as a shopping guide, not a purchase certificate.
Fit is not weights-only. Default context is 8K tokens:
total_GB ≈ weight_GB + KV_GB + 1.0 weight_GB = total_params_B × bytes_per_param KV_GB = active_params_B × (ctx/1024) × 0.025
Example: ~8B Q4 weights ≈ 4.4 GB; KV at 8K ≈ 1.6 GB; +1 GB overhead → ~7 GB. A 8 GB card may fit weights alone but fail at 8K/32K — we mark that as No.
UI lets you switch 4K / 8K / 32K. Longer context is more conservative (fewer false Yes).
| Field | Description |
|---|---|
| tok/s | Approximate decode tokens per second (batch 1) |
| Quant | Q4 (typical local) or FP16 (full precision) |
| Fits? | weights + KV(ctx) + overhead ≲ 95% of device memory |
| Cost-Effect | tok/s per $1 of MSRP (list price; used market ignored) |