Can I run AI on my existing PC or laptop?
Yes. If your computer is from the last 4–5 years, you can download free tools like LM Studio or Ollama and run lightweight models—such as Llama 3.2 3B or Qwen 2.5 7B—right now. Once downloaded, they run offline with no subscription required.
Why is GPU VRAM so much more important than regular RAM?
Running local AI is a memory-bandwidth bottleneck, not a raw compute problem. Every time a model generates a single word (or "token"), your system has to read the entire model—billions of parameters called "weights"—from memory.
What software do I need to actually use these models?
You don't need to write code or deal with complicated setup to run local AI. The most popular tools are free and run across Windows, Mac, and Linux. Which one makes sense comes down to how you want to use it:
What is \"Quantization\" (like Q4_K_M or Q8)?
Quantization is essentially JPEG compression for AI models. Out of the box, models store their weights as high-precision 16-bit numbers (FP16). That makes them huge—an 8-billion-parameter model needs roughly 16GB of VRAM at full precision. Quantization rounds those numbers down into smaller 8-bit or 4-bit integers, shrinking the file by 50% to 75% without wrecking the model's smarts. Because the file is smaller, your GPU reads it faster and spits out text quicker.
Open WebUI
Open WebUI is a self-hosted, browser-based interface that gives your local AI models the exact look, feel, and everyday features of ChatGPT—running completely offline on your own hardware or home server.
How much VRAM do I need for 8B, 32B, and 70B models?
To run a local model at full speed, the entire model and your active chat history must fit inside your GPU's VRAM. If any part spills over into system RAM, token generation slows to a crawl.
Apple Unified Memory vs. NVIDIA GPU: Which is better for local AI?
Neither architecture is universally better—the right choice comes down to a trade-off between raw token speed (NVIDIA) and massive memory capacity (Apple).
Should I buy a used RTX 3090 (24GB) or a newer RTX 4060 Ti / 5070 (16GB)?
For running local AI, buy the used RTX 3090 (24GB). Newer mid-range cards like the RTX 4060 Ti (16GB) and RTX 5070 Ti (16GB) are more power-efficient, yet NVIDIA keeps their memory capacity capped at 16GB to protect its workstation lineup.
Do I need an NVIDIA GPU, or will AMD / Intel cards work?
No, you do not strictly need an NVIDIA GPU anymore—both AMD Radeon and Intel Arc cards can run local AI. NVIDIA still gives you the smoothest "install and go" experience, while AMD and Intel trade a bit of setup convenience for better VRAM-per-dollar on brand-new hardware.
Can I add a second GPU to my desktop later to double my VRAM?
Yes—for running large language models (LLMs), adding a second GPU effectively pools your VRAM together. Unlike PC gaming, where multi-GPU setups (`SLI` and `CrossFire`) are dead, AI engines like Ollama, LM Studio, and `llama.cpp` automatically divide a large model's layers across multiple graphics cards.
How noisy and power-hungry are dedicated AI rigs at home?
Much less expensive to run than most people expect, though a high-end PC rig will temporarily sound like a gaming tower under full load.
What is a \"good\" tokens-per-second (tok/s) speed for chatting vs. coding?
For casual chatting, 10 to 15 tokens per second (`tok/s`) feels comfortable—it tracks right around normal human reading speed. For coding and AI agents, aim for at least 30 to 50+ `tok/s`. When writing software, you are usually waiting on entire blocks of code to finish rather than reading along word by word.
How does a longer conversation or pasting big code files affect VRAM (Context Window & KV Cache)?
Every word in your chat and every line of pasted code eats into a growing pool of VRAM called the KV Cache. Even when a model file fits comfortably on your graphics card, dropping in a massive codebase or maxing out the context slider can push VRAM past its limit and crash the app.
Why does a 70B model crash on a 32GB graphics card?
A standard 4-bit (`Q4_K_M`) 70B model takes up 40GB to 42GB on disk. That is already bigger than a 32GB graphics card like an RTX 5090 before you even type a single word.
How much regular system RAM and SSD storage do I need alongside VRAM?
Aim for at least 32GB of regular system RAM and a 1TB NVMe SSD (2TB if you plan to keep several large models on disk). Even when an AI model runs entirely on your graphics card, your PC still depends on system memory and storage speed to stage, load, and manage files behind the scenes.
Ollama vs. LM Studio vs. Jan.ai: Which one should a beginner pick?
If you are a complete beginner, start with LM Studio. All three apps are free, run across Windows, Mac, and Linux, and generate text at nearly identical speeds—the real difference comes down to how you interact with them and which workflow tools come built in.
Can local models match ChatGPT, Claude, or Gemini?
For roughly 80% of everyday tasks, yes. When you move into deep multi-step reasoning, sprawling multi-file codebases, or obscure trivia, flagship cloud models like ChatGPT, Claude, and Gemini still take the lead.
Can I run local image generation (Flux / SDXL) and LLMs on the same rig?
Yes. The exact same hardware built for local LLMs will also handle image generators like Flux and Stable Diffusion XL (SDXL) well. If your machine already has enough VRAM for a solid text model, it already meets the baseline for local image generation.
Is local AI truly 100% private and offline once downloaded?
For plain chatting and document Q&A, yes. Once a model finishes downloading onto your machine, you can unplug your Ethernet cable or switch off Wi-Fi completely—tools like LM Studio, Ollama, and Jan will keep generating text at full speed with zero internet access. Your prompts, pasted code, and uploaded PDFs stay on your drive and are never used to train third-party models.
Do I need to use Linux, or does everything work on Windows and macOS?
No, you do not need Linux—everything a beginner needs works natively on Windows and macOS. Popular apps like LM Studio, Ollama, Jan, and ComfyUI ship with standard one-click installers for both Windows and Mac, and they generate text at nearly identical speeds across all three operating systems.
How do I connect my local AI to coding tools like VS Code or Cursor?
Both Ollama and LM Studio run a built-in local server that mimics the OpenAI API, letting you wire them into VS Code, Cursor, or terminal coding agents using a single `localhost` URL. You do not need a paid API key or an active internet connection—your source code stays 100% on your machine.
When does buying a workstation make more sense than renting cloud GPUs (RunPod / Lambda)?
Buying a workstation makes more sense when you rely on AI every day for coding, private documents, or multi-purpose work like gaming and video editing. Renting cloud GPUs (such as RunPod or Lambda) makes more sense for occasional weekend experiments, heavy training runs, or testing massive models without dropping thousands of dollars upfront.
How much will running a local AI rig add to my UK/US electricity bill?
For typical daily chat and coding—roughly 2 hours of active generation per day plus normal desktop idle time—expect a single-GPU desktop to add £8 to £18 per month in the UK, or $5 to $12 per month in the US. On an Apple Silicon Mac, it stays under £3 / $2 per month.
How does FindBestRig calculate £/GB and verify retailer prices?
FindBestRig ranks hardware by dividing a rig's lowest verified retail price by its fast, AI-accessible memory (`VRAM` or `Unified Memory`)—never by total system RAM. Because memory capacity sets the hard ceiling on which models you can load at full speed, `£/GB` (or `$/GB`) gives you an apples-to-apples value metric across wildly different machines.
VRAM (Video Random Access Memory)
VRAM is the high-speed, dedicated memory built directly onto a graphics card (GPU). While regular system RAM holds your operating system, browser tabs, and everyday apps, VRAM acts as a private, ultra-fast workspace exclusively for graphics and AI calculations.
Unified Memory
In a traditional desktop PC, the processor (CPU) and the graphics card (GPU) live in separate worlds with separate memory pools—system RAM for the CPU and VRAM for the GPU—forcing data to be copied back and forth between them. Unified Memory eliminates that wall by placing a single, shared pool of high-speed memory directly into the main chip package so both the CPU and GPU can read from the exact same space simultaneously.
Discrete GPU (dGPU)
In plain English: A standalone graphics card with its own dedicated processor, cooling fans, and private pool of high-speed VRAM—such as an NVIDIA GeForce RTX 3090, RTX 5090, or AMD Radeon RX 9070 XT. Unlike integrated graphics built directly inside a standard laptop or desktop CPU that have to share slow motherboard RAM, a discrete GPU plugs into a motherboard's `PCIe` expansion slot (or sits as a separate chip inside a gaming laptop) and handles heavy parallel math completely on its own.
Memory Bandwidth (GB/s)
Memory bandwidth measures how much data your memory can feed to the processor each second, expressed in gigabytes per second (`GB/s`). Picture VRAM capacity as the size of a warehouse, and memory bandwidth as the width of the highway connecting that warehouse to the factory floor—no matter how fast the workers are, they can only assemble products as fast as trucks deliver the parts.
HBM3e (High Bandwidth Memory)
`HBM3e` is an ultra-fast type of memory built by stacking thin memory dies vertically on top of one another like floors in a skyscraper, rather than laying them out flat side-by-side on a circuit board. Thousands of microscopic vertical wires called Through-Silicon Vias (`TSVs`) link those stacked layers together and sit right beside the GPU die on a shared silicon interposer, creating an extraordinarily wide data highway while drawing far less power than traditional graphics memory.
TDP (Thermal Design Power)
TDP stands for Thermal Design Power, measured in Watts (W). It tells you the maximum amount of heat a processor or graphics card generates when running at full tilt—and how beefy your cooling system needs to be to keep it from overheating.
PCIe (Peripheral Component Interconnect Express)
PCIe is the high-speed slot on your motherboard where you plug in a discrete graphics card. Think of it as the data highway connecting your GPU to your CPU and the rest of your computer.
SXM5
SXM5 is a specialized server socket used in enterprise AI hardware—such as NVIDIA H100 and H200 rigs—that mounts graphics processors directly onto the server motherboard instead of plugging them into standard PCIe slots.
Parameter Count (7B, 8B, 32B, 70B)
When you see labels like 7B, 8B, 32B, or 70B next to an AI model, the B stands for Billion parameters. Parameters are the internal adjustable dials—called weights—that the model tuned during its training to learn language, logic, coding, and facts.
Quantization (Q4, Q4_K_M, Q8, Unquantized/FP16)
Quantization is the process of shrinking an AI model so it fits into consumer graphics cards without losing much of its intelligence. It works by rounding off the precision of the numbers (weights) inside the model—much like saving a massive raw photo as a high-quality JPEG.
Weights
Weights are the billions of learned numerical values stored inside a model file—such as a `.gguf` or `.safetensors` download—that represent everything the AI knows.
Context Window
The context window is an AI model's short-term working memory. Measured in tokens (roughly ¾ of a word), it defines the maximum amount of text the model can read, hold in mind, and reference at one time during a single conversation.
KV Cache (Key-Value Cache)
The KV Cache (Key-Value Cache) is the extra slice of VRAM your system reserves to remember your ongoing conversation so the AI does not have to re-read the entire chat from scratch after every single word it writes.
Inference vs. Fine-Tuning
Inference and Fine-Tuning are the two main ways you interact with an AI model, and they place completely different demands on your hardware.
Tokens & Tokens per Second (t/s or tok/s)
AI models do not read or write whole words the way humans do; they break text down into chunks called tokens. In English, one token equals roughly ¾ of a word (so 100 tokens is around 75 words). Common short words are single tokens, while long technical terms, punctuation, and code indentation get split into multiple pieces.
£/GB (Cost per Gigabyte of Memory)
£/GB (Cost per Gigabyte of Memory) is the core value metric used across our hardware index. You calculate it by taking the total purchase price of a graphics card or complete system and dividing it by the amount of usable, high-bandwidth memory (VRAM or Apple Unified Memory) you get.
CPU Offloading
CPU Offloading is a clever safety net built into local AI engines like Ollama and LM Studio that lets you run a model even when it is slightly too big to fit inside your graphics card's VRAM.
CUDA
CUDA is NVIDIA's proprietary software platform that lets developers tap directly into the thousands of parallel cores inside GeForce, RTX, and datacenter graphics cards for heavy math instead of just gaming graphics.
Metal & ROCm
Metal and ROCm are the two main alternatives to NVIDIA's CUDA that allow Apple and AMD hardware to accelerate local AI workloads.
Local Inference Engines (Ollama, LM Studio, Jan.ai)
Local inference engines are the free desktop apps and background services that turn raw, unplayable AI model files into a working private assistant on your own computer.
GGUF
GGUF (GPT-Generated Unified Format) is the universal single-file format used by local AI tools like Ollama, LM Studio, and `llama.cpp` to store and run quantized language models.
ComfyUI
ComfyUI is the go-to open-source interface for running local image and video generation models like Flux.1, Stable Diffusion XL (SDXL), and Wan on your own graphics card.
Not Ready to Buy a Full Rig?
Don't want to spend £3,500+ upfront? Rent the GPU power instead.
Buying a dedicated AI workstation is a serious investment—and if you aren't ready to spend that much right now, you don't have to. Renting cloud GPUs gives you instant access to high-VRAM hardware like RTX 4090s and H100s on demand. Spin up an instance for fine-tuning or running massive 70B+ models, pay strictly for the minutes you use, and avoid the upfront hardware cost, electricity bills, and maintenance.
Vast.ai Global GPU Marketplace
Lowest $/hrRunPod Community & Secure Cloud
Community & Secure CloudFindBestRig independently benchmarks cloud GPU providers to compare rental economics against physical hardware.