Tool Suggestions
What LLM Fits Your GPU? The Index That Measured 122 Models on Real Hardware
Every local AI discussion starts with the same question: will this model run on my graphics card? The usual answer is a formula — parameters multiplied by bytes per weight — which ignores the two things that actually decide whether a model loads: the memory its context window consumes, and the overhead of the runtime. The result is a familiar ritual of downloading a 20 GB file and watching it fail. A site launched on Show HN on October 23 takes the opposite approach. Quantized.uk — titled, plainly, "What LLM fits my GPU?" — measures models on real hardware instead of estimating them. It opened with 117 models and was already at 122 within the day, with new releases added as they ship.
Measured, not guessed
The index's test bench is deliberately ordinary: an RTX 4090 and an RTX 3090, both 24 GB cards, plus Apple Silicon machines (M3 Max 48 GB and M2 Ultra), running the stacks people actually use — llama.cpp, ExLlamaV2, vLLM and Ollama. Every row of its benchmark table is a real run with the hardware, the runtime and the quantization recorded next to the result. Each model page lists the VRAM it needs at a 4K context window for each quant level, the speed it reached, and the class of card it is, in the site's word, "comfortable" on. Llama 3.1 8B, for example, measured 5.4 GB and 235 tokens per second on the 4090 under ExLlamaV2 — and 5.7 GB at 68 tokens per second on the M3 Max under Ollama. Qwen3 30B-A3B at Q4_K_M measured 19 GB and 95 tokens per second on the 4090. A calculator on each page adjusts the figures for context length and quantization, which is the essential point: a model's footprint is not one number.
The numbers that change buying decisions
The site's clearest lesson is that context length, not usage, is the hidden cost. A 7B model at Q4_K_M needs about 5.1 GB at a 4K context — 4.2 GB of weights, 0.5 GB of KV cache, 0.5 GB of activation buffer. Run the same file at a 32K context and the requirement rises to about 9.0 GB, because the KV cache grows to roughly 4.0 GB. Calculators that count only weights are precisely how an 8 GB card ends up out of memory running a model it was told would fit.
Quantization level matters as much as model size. Mistral Small 24B Instruct on a 16 GB card fits two ways with opposite risk profiles: at AWQ INT4 it needs about 13.2 GB, leaving 2.8 GB of margin, while at Q4_K_M it needs 15.9 GB — 99 percent of the card, loadable only on an otherwise empty GPU with no room for a longer window. The index calculates that a 16 GB card runs 66 of its 122 models comfortably. It also quantifies what 4-bit costs in quality: across the 79 models that publish a figure, the median perplexity loss at Q4_K_M is 2.9 percent against FP16, ranging from 1.4 to 5.2 percent — with the site's own warning that smaller models lose more, and that perplexity is a language-modelling measure, not a verdict on your task. Its format advice is equally direct: GGUF unless you are running a server — all 122 models ship it, against 53 for AWQ, whose real case is batched throughput under vLLM.
The honest limits
The index is maintained by hand and checked against public model cards, and it says so. Entries marked "estimated" are calculated from a model's architecture rather than measured on a card, and the site invites corrections when a figure does not match what users see on their own hardware. Its benchmarks come from a small set of machines and runtime versions, so your driver, settings and build will move the numbers somewhat. None of that undermines the basic trade it offers: measured figures with a stated method, over formulas with none. If you are choosing a model for an 8, 16 or 24 GB card — or choosing the card itself — this is now the fastest way to answer the question before the download starts.