Glossary

The vocabulary of local AI, explained plainly. Technical but accessible, and cross-linked throughout the site.

C

Context window The maximum amount of text, measured in tokens, a model can consider at once, covering both your input and its output.

G

GGUF The file format, introduced by llama.cpp, that packages a quantised model into a single file for local inference across a wide range of hardware.

Q

Quantisation Storing a model's weights at lower numerical precision to shrink its memory footprint, trading a little quality for a much smaller size.

V

VRAM The memory on a graphics card. A model must fit in it (alongside its context) to run on the GPU, so VRAM is the single biggest constraint on what you can run.