How a 284B AI Model Runs on a Laptop With No GPU

9 Просмотры

Спасибо! Поделитесь с друзьями!

Вам не понравилось видео. Спасибо за то что поделились своим мнением!

Издатель Aug 3, 2026

This is a 284-billion-parameter AI model running on a laptop with no GPU — and it's not a trick. Here's the mechanism that makes it possible, and the honest three-tier build so you can do it yourself tonight.

Why it works (the mechanism):
• Mixture-of-Experts — a 284B model only fires ~13B parameters per token. A router picks a few of many expert sub-networks, so compute scales with the ACTIVE params, not the total.
• RAM, not VRAM — the weights live in your system RAM (memory-mapped from SSD), and the CPU reads the experts it needs. You don't need a graphics card's VRAM to hold them.
• Quantization — shrinking each weight from 16-bit down to 3-bit collapses ~568 GB to ~100 GB, and dynamic quant (Unsloth) protects the layers that matter.

The three-tier build (honest about what runs on what):
• TIER 1 — Daily driver: gpt-oss-20b (OpenAI, open-weight), 21B / 3.6B active, ~13 GB. Fits a normal 16 GB laptop, no GPU. Run: ollama run gpt-oss:20b
• TIER 2 — Prestige: DeepSeek-V4-Flash (284B / 13B active), ~110 GB RAM at 3-bit. Wants a big-memory workstation or a maxed-out Mac.
• TIER 3 — Flex: Kimi K3 (2.8T), ~590 GB even at 1-bit. That's a server, not a laptop.

The honest catches: no GPU doesn't mean fast (CPU inference is slow — a few tokens/sec on the big ones), doesn't mean any laptop (the frontier models are RAM-hungry), and quantization does cost some quality. On the giant models, "it runs" and "it's usable" are different sentences. Note: the ~20 tokens/sec figure you'll see quoted for Kimi K3 is on a data-center GPU, not a CPU.

The takeaway: you don't need a GPU to run AI locally — the memory that matters is RAM, and Mixture-of-Experts + quantization does the rest. The trophy is the trillion-parameter run; the one that actually changes your workflow is tier one, tonight.

Chapters:
0:00 A 284B model on a laptop, no GPU
0:21 Why this should be impossible
0:41 Mixture-of-Experts: total vs active
1:06 RAM, not VRAM
1:29 Quantization: shrinking the model
1:57 Tier 1 — the daily driver (gpt-oss-20b)
2:21 Tier 2 — the prestige run (DeepSeek V4)
2:42 Tier 3 — the flex (Kimi K3)
3:05 The honest catch
3:33 The takeaway

Sources on screen: Unsloth model docs, OpenAI gpt-oss, Ollama, llama.cpp.

Subscribe for honest, no-hype breakdowns of how AI actually works and breaks.

#localllm #ollama #nogpu #ai #llm

Support: https://www.patreon.com/cybercreed
UPI : cybercreed@ibl