Run AI on your own hardware.

the tools people arrive for

Answer the questions you arrive with

• 10 September 2026

  DeepSeek releases V4.1 Flash, an MIT-licensed multimodal model with an unusual architecture

  DeepSeek has published V4.1 Flash under the MIT licence: a 552-billion-parameter multimodal mixture-of-experts that reads images and text and activates only 8 to 16 billion parameters per token. It uses an encoder-decoder design rather than the decoder-only shape almost every other current model uses, and it carries a 1-million-token context. It is server-class, but genuinely open.

• 10 September 2026

  Several open video models exclude UK users by licence, outputs included

  MiniMax H3, a widely shared open-weight video model, is licensed for use everywhere except the EU, the UK, Korea and the US, and the restriction covers the model's outputs as well as its weights. It is not alone. For a UK reader, this is a case where the licence, not the hardware, is what stops you, and it is invisible to a licence badge.

• 3 September 2026

  NVIDIA upstreams llama.cpp optimisations it says make local inference up to 1.9x faster

  At IFA 2026 NVIDIA announced local-AI updates, and the quietly important part is software: kernel optimisations it has contributed to the open-source llama.cpp and vLLM projects, which it says lift throughput by up to 1.9x on a GeForce RTX 5090. Because they are upstream, existing NVIDIA GPU owners get the gain by updating, with no new hardware. A new RTX Spark desktop with 128GB of unified memory was also shown.

• 2 September 2026

  GLM-5.3 open weights land, under a bespoke licence rather than MIT

  Z.ai has published the open weights of GLM-5.3, roughly two weeks after its hosted launch and following a security review. The full model is a server-class mixture-of-experts of around 744 billion parameters under a bespoke GLM-5.3 licence, not the MIT terms its predecessor carried. A smaller, natively multimodal GLM-5.3-Flash also shipped, and that one is MIT.

• Gemma 3 27B

  Google · 27B

  Google's 27-billion-parameter open model, and the largest of the Gemma 3 family. It is multimodal, handling images as well as text, supports over 140 languages, and is designed to run on a single 24GB card at 4-bit.

  Permitted with conditions Text + Image→Text

• Llama 3.3 70B Instruct

  Meta · 70B

  Meta's 70-billion-parameter instruction-tuned model, delivering performance close to their much larger 405B model at a fraction of the hardware cost. A strong general-purpose choice if you have the VRAM for it.

  Permitted with conditions

• Muse Glimmer 30B

  Meta · 30B

  Meta's 30-billion-parameter open model, distilled from its larger closed Muse Spark and built for agentic work: tool use, multi-step reasoning, and recovering from failures. It accepts images as well as text, runs on a single 24GB card at 4-bit, and, unusually for Meta, ships under Apache 2.0 rather than the Llama licence.

  Commercial use permitted Text + Image→Text

• Qwen2.5-Coder 7B Instruct

  Alibaba · 7B

  Alibaba's code-specialised 7B model, strong at code completion and generation well beyond what its size would suggest. A practical choice for a local coding assistant on a mid-range GPU, and permissively licensed.

  Commercial use permitted

• Qwen3 30B-A3B

  Alibaba · MoE 30B-A3B

  A mixture-of-experts model with 30 billion total parameters but only 3 billion active per token, so it generates far faster than a dense 30B while keeping much of the quality. Apache 2.0, with an optional thinking mode.

  Commercial use permitted

• Qwen3 32B

  Alibaba · 32B

  The largest dense Qwen3 model, and a strong single-card flagship: at Q4 it fits a 24GB card with short context. Apache 2.0 licensed, with an optional thinking mode that makes it a credible reasoning model.

  Commercial use permitted

• Ollama The fastest route to a working local model. Ollama wraps llama.cpp with simple model management, a one-line pull-and-run workflow, and a local API that many other tools speak to. The usual first recommendation for newcomers.
• LM Studio A polished desktop app for discovering, downloading, and chatting with local models through a graphical interface. A good fit if you would rather not touch a command line, with a built-in local server for connecting other tools.
• llama.cpp The inference engine that much of the local AI ecosystem is built on. A C/C++ implementation that runs GGUF models efficiently across CPU, CUDA, Metal, Vulkan, and more, with fine control over how inference behaves.
• vLLM A high-throughput inference and serving engine built to maximise GPU utilisation and handle many concurrent requests. Its PagedAttention and continuous batching make it the common choice once you are serving a model to more than one user.
• ComfyUI A node-graph interface for local image generation, where you build a workflow by wiring together nodes. It is the most flexible and powerful local image tool, quick to support new models, and it saves workflows as reusable JSON.
• Continue An open-source IDE extension for VS Code and JetBrains that adds AI autocomplete, chat, and multi-file edits, with first-class support for local models through Ollama. A common starting point for a fully local coding setup.
• LangChain The most widely used framework for building LLM applications, with a huge set of integrations for models, vector stores, and tools. It is a common backbone for RAG systems, and works with local models through Ollama and llama.cpp.
• Qdrant A fast, production-grade vector database written in Rust, with strong metadata filtering and a clean operational story. A common choice once a RAG system outgrows a prototype store, and free to self-host under Apache 2.0.

• 🚀 Getting Started What local AI is, why it matters, and how to run your first model.
• 🖥️ Hardware GPUs, unified memory, VRAM, and what you actually need for what you want to do.
• ⚙️ Running Models Quantisation, inference engines, model formats, and getting good performance.
• 💻 Local AI Coding Code assistants, autocomplete, and agentic coding tools that run on your machine.
• 📚 RAG & Knowledge Systems Building retrieval systems over your own documents, entirely locally.
• 🤖 Agentic AI & Harnesses Tool use, agent frameworks, and orchestration with local models.
• 🎯 Fine-Tuning Adapting models to your domain: LoRA, QLoRA, full fine-tunes, and when each is worth it.
• 🎨 Beyond LLMs Image generation, speech recognition, text-to-speech, and other local models.
• 🏢 Server & Enterprise Self-hosting at organisational scale, from a small team on a couple of GPUs to enterprise clusters running open models beyond 100 billion parameters.
• 🛡️ AI Regulation & Sovereignty AI-specific regulation like the EU AI Act, alongside data residency, GDPR, and the practical case for keeping data in-house.

• Agent
• Chunking
• Context window
• ControlNet
• Copyleft (GPL / AGPL)
• Diffusion
• Embedding
• Fine-tuning
• GGUF
• Inference engine
• Instruction tuning
• KV cache