🧠
How DeepSeek Slashed Costs: MLA & DeepSeekMoE
DeepSeek-V3 compresses Key-Value (KV) cache by up to 93% using Multi-Head Latent Attention (MLA). Combined with Multi-token Prediction (MTP) and DualPipe 1-forward-1-backward overlapping, training cost was only $5.5M for 671B parameters. This enables providers to offer $0.14/1M pricing while remaining profitable.
🍏
Mac Studio (192GB): Running 671B MoE Locally
Thanks to Apple Silicon unified memory architecture, the 192GB Mac Studio M2 Ultra can allocate ~150GB directly to the GPU. Running llama.cpp with Metal acceleration on the DeepSeek-V3-Q3_K_M or IQ3_XXS GGUF quantization delivers 12-16 tokens/sec while consuming less than 140 Watts total power.
⚡
Dual RTX 3090: The $1,450 48GB VRAM Champion
Two second-hand RTX 3090 24GB GPUs combined with PCIe 4.0 and vLLM Tensor Parallelism (--tensor-parallel-size 2) allows seamless execution of the DeepSeek-R1-Distill-Llama-70B model at Q4_K_M. Throughput reaches 22+ tok/s with zero external API dependencies.
💾
Prompt Caching Masterclass: 90% Bill Reduction
DeepSeek Official API automatically identifies identical prompt prefix blocks (multi-turn system prompts, documentation chunks, codebase trees). Cache hits cost merely $0.014 / 1M tokens—a 90% discount over standard prompt pricing. Always prepend static context at the beginning of messages to maximize cache hits.