Local AI Starter Guide: Everything You Need to Know Before Buying Hardware
New to running AI models on your own PC? Learn why VRAM is king, how model sizes work, what hardware to look for, and how free tools like Ollama make local setup straightforward.
Running artificial intelligence models locally on your own machine used to require a computer science PhD and tens of thousands of pounds in enterprise servers. Today, modern open-weight models (like Llama 3, Mistral, and DeepSeek) run beautifully on consumer PCs, laptops, and compact workstations.
If you are new to local AI, this guide walks you through the core concepts in plain English so you can choose the right hardware without wasting money.
Why Run AI Locally?
There are three huge reasons why enthusiasts, developers, and privacy-conscious professionals run models at home or in their office:
- Complete Data Privacy: Your prompts, proprietary source code, personal documents, and financial data never leave your local machine. Nothing is sent to cloud servers or used to train third-party models.
- Zero Recurring Subscriptions: Instead of paying £20/month per user for ChatGPT Plus, Claude Pro, or GitHub Copilot, you own the hardware and can query models as much as you want without rate limits.
- Offline & Uncensored Operation: Local models run completely without an internet connection and are not subject to arbitrary API outages or censorship filters.
The Golden Rule: VRAM Is King
In normal PC computing, you care mostly about the CPU (central processing unit) and system RAM. In local AI, Video RAM (VRAM) on your graphics card is the single most critical specification.
When you load an AI model, the entire neural network must be loaded into memory. While a CPU and regular system RAM can run models, standard system DDR5 memory is roughly 5 to 10 times slower than GPU memory (GDDR6X, GDDR7, or Apple Unified Memory).
If your model fits entirely inside your GPU's VRAM, generation is instantaneous (25 to 60+ tokens per second). If the model overflows into system RAM, token generation drops to a sluggish crawl (1 to 3 tokens per second).
Understanding Model Sizes (Parameters)
Models are measured by their parameter count:
- 8B Parameters (e.g. Llama 3.1 8B, Mistral 7B): Lightweight and fast. Requires 12GB to 16GB of VRAM. Fantastic for everyday chat, summarization, email drafting, and fast coding help.
- 14B to 32B Parameters (e.g. Qwen 2.5 32B, DeepSeek Coder 33B): The sweet spot for software development and logical analysis. Requires 24GB to 32GB of VRAM. Rivals earlier commercial closed models.
- 70B Parameters (e.g. Llama 3.3 70B, DeepSeek R1 70B): Powerhouse models capable of complex multi-step reasoning, mathematical problem solving, and nuanced creative writing. Requires 48GB to 128GB of memory.
What Hardware Should You Look For?
There are three main paths to consider:
- NVIDIA Desktop Graphics (The Gold Standard):
- Dedicated NVIDIA GeForce RTX cards (RTX 4060 Ti 16GB, RTX 4080 16GB, RTX 3090/4090 24GB, or RTX 5090 32GB).
- Benefit: Maximum software compatibility (CUDA), lightning-fast token generation, and versatile for gaming or video editing.
- Apple Silicon Unified Memory (The Quiet Workstation Pick):
- Mac Mini, Mac Studio, or MacBook Pro with M2/M3/M4/M5 chips.
- Benefit: Apple shares high-speed memory between the CPU and GPU. You can get 64GB, 128GB, or 256GB of high-speed memory without multiple noisy graphics cards or massive power consumption.
- AMD High-Memory APUs / Mini PCs (The Budget Value Pick):
- Systems like the Ryzen AI Max+ 395 with 128GB of unified LPDDR5x memory.
- Benefit: Unbeatable cost-per-GB of memory for running large 70B models on a strict budget.
Free Software That Makes It One-Click Simple
You do not need to know Python or machine learning to run models locally:
- Ollama: A simple, terminal-based tool that lets you download and run models with a single command (e.g.
ollama run llama3.1). - LM Studio: A gorgeous desktop application for Windows, Mac, and Linux with a built-in search engine for downloading models and a ChatGPT-style interface.
- Jan.ai: An open-source, 100% offline desktop app that turns your computer into a private AI assistant.
Next Steps
Ready to explore specific systems and prices?
- Read our How Much VRAM Do You Actually Need? sizing breakdown.
- Browse the £/GB VRAM Index to compare the cost efficiency of every verified prebuilt rig.