VRAM
Also: video memory, unified memory
The memory on a graphics card. A model must fit in it (alongside its context) to run on the GPU, so VRAM is the single biggest constraint on what you can run.
VRAM is the fast memory attached to a graphics card. To run on the GPU, a model’s weights and its working memory (the KV cache for the current context) must fit in VRAM. This is why VRAM, more than raw compute, decides which models you can run.
On Apple Silicon there is no separate VRAM. Instead, CPU and GPU share unified memory, so most of the machine’s RAM is available to models. That is why a Mac with plenty of unified memory can run larger models than a discrete card with less VRAM.
VRAM figures quoted for models are approximate. Real usage grows with context length and batch size, so leave headroom rather than filling memory exactly.