Skip to content
ToolQivo

GPU VRAM Calculator for LLMs

Work out how much GPU memory a local LLM needs to run or fine-tune.

AI Tools Reviews Free · no sign-up
Architecture — filled in from the model

Read these from the model's config.json: num_hidden_layers, num_key_value_heads and head_dim (or hidden_size ÷ num_attention_heads).

Quick Tool Overview

The GPU VRAM Calculator works out how much video memory a local language model needs — weights at your chosen quantization, KV cache from the real layer and attention-head counts, and runtime overhead — for inference or QLoRA, LoRA and full fine-tuning, and checks it against common GPUs and Macs.

What it does
Estimates GPU memory for running or training an LLM
You give it
A model, a precision, a context length and a batch size
You get back
A memory breakdown and which GPUs fit
Best for
Ollama, llama.cpp, LM Studio and local fine-tuning
AI Tools Free, unlimited use No sign-up Runs in your browser

About the GPU VRAM Calculator for LLMs

Whether a local model runs on your card comes down to three numbers: the size of the weights at the quantization you choose, the KV cache that grows with every token of context, and the runtime overhead. Guess wrong and the model either refuses to load or spills into system RAM and slows to a crawl.

Pick a model — Llama, Qwen, Mistral, Mixtral, Gemma, Phi or Command R — or enter any parameter count, then choose the weights precision from F16 down to 2-bit GGUF quants, the KV cache precision, the context length and the number of parallel sequences. The breakdown shows where every gigabyte goes, using the real architecture — layers, key-value heads and head size — so grouped-query attention is accounted for rather than guessed.

Switch to QLoRA, LoRA or full fine-tuning to add gradients, optimizer states and activations. A table then checks the result against common cards from the RTX 3060 to the H100 and Apple Silicon Macs, and for inference shows the longest context each card can hold beside the weights.

Key features of the GPU VRAM Calculator for LLMs

Architecture-accurate KV cache

Uses layers, key-value heads and head size, so grouped-query attention is counted correctly.

Every GGUF precision

From F16 down to IQ2_XS, with KV cache at F16, Q8_0 or Q4_0.

Fine-tuning modes

QLoRA, LoRA and full fine-tuning add adapters, gradients, optimizer states and activations.

GPU and Mac fit table

RTX 3060 to H100 and Apple Silicon, marked fits, tight or needs more cards.

Maximum context per card

Shows the longest context each GPU can hold beside the weights.

How to use the GPU VRAM Calculator for LLMs

  1. 1

    Choose a model

    Pick from the list, or enter the parameter count and architecture from config.json.

  2. 2

    Pick the precision

    Q4_K_M is the usual choice for running locally; F16 for full quality.

  3. 3

    Set the context

    Context length and parallel sequences decide the size of the KV cache.

  4. 4

    Check the GPU table

    See which cards fit, which are tight, and the longest context each can handle.

Frequently asked questions

About 6 GB at Q4_K_M with an 8K context, and about 17 GB at F16. A 12 GB card runs an 8B model comfortably at 4 or 5 bits.

Reviews of the GPU VRAM Calculator for LLMs

What people who use the GPU VRAM Calculator for LLMs think of it. Reviews come from visitors and are posted as written.

No ratings yet

Yours would be the first.

  • 5 0 reviews rated 5 stars
  • 4 0 reviews rated 4 stars
  • 3 0 reviews rated 3 stars
  • 2 0 reviews rated 2 stars
  • 1 0 reviews rated 1 star

Write a review

Used this tool? Your rating helps the next person decide.

Your rating

Plain text only — links, email addresses and HTML are not accepted. Your name and review are shown publicly; nothing else is collected. See the privacy policy.

0 reviews

Newest first

No reviews yet

Be the first to rate the GPU VRAM Calculator for LLMs. It takes half a minute and no account.

Write the first review

More AI tools