Home/FAQ & Glossary/CPU Offloading
GLOSSARY DEFINITIONMetrics
← Back to FAQ & Glossary

CPU Offloading

CPU Offloading is a clever safety net built into local AI engines like Ollama and LM Studio that lets you run a model even when it is slightly too big to fit inside your graphics card's VRAM.

Instead of crashing with an Out-Of-Memory error, the software loads as many layers of the model onto your fast GPU as possible and dumps the leftover layers into your motherboard's regular system RAM, where your CPU processes them.

The upside is flexibility—you can test out a massive 70B model on a modest 16GB or 24GB rig without buying new hardware. The downside is a steep drop in tokens per second. Standard DDR4 or DDR5 system RAM has a fraction of the memory bandwidth of dedicated VRAM, so offloading even 20% of a model can cut your generation speed in half or worse.

Looking for verified hardware rigs?

Check our real-time £/GB VRAM index to find systems optimized for local model inference.

Explore VRAM Index →
Rent Cloud GPU Power

Not Ready to Buy a Full Rig?

Don't want to spend £3,500+ upfront? Rent the GPU power instead.

Buying a dedicated AI workstation is a serious investment—and if you aren't ready to spend that much right now, you don't have to. Renting cloud GPUs gives you instant access to high-VRAM hardware like RTX 4090s and H100s on demand. Spin up an instance for fine-tuning or running massive 70B+ models, pay strictly for the minutes you use, and avoid the upfront hardware cost, electricity bills, and maintenance.

Vast.ai Global GPU Marketplace

Lowest $/hr
Unbeatable $/hr rates on community-hosted RTX 4090 & 3090 rigs
Deploy Pod →

RunPod Community & Secure Cloud

Community & Secure Cloud
RTX 4090 from $0.34/hr • H100 SXM from $1.99/hr (Billed in USD)
Deploy Pod →

FindBestRig independently benchmarks cloud GPU providers to compare rental economics against physical hardware.