How is the fit estimate calculated?

We estimate weight size from parameter count and quantization, add KV-cache for the requested context, and compare the total against your VRAM and system RAM.

Is the estimate accurate?

It is a heuristic. Real usage depends on runtime, GPU layers, GQA settings and other factors. Look at the measured benchmarks below for real numbers.

What if the model does not fit?

You can try a smaller quantization, reduce context length, or offload some layers to system RAM.

Can I run this model?

Pick a GPU, a model, quantization and target context length. We show measured numbers - and fall back to the closest matches if an exact setup isn't available.

No measured data yet

Pick at least one filter to look up measured results.

Next steps

Find hardware

Cheapest setup that reaches your target speed

Compare GPUs

Side-by-side measured performance