Three top-ranked open-source models (MIT or Apache 2.0 licenses) from the Arena.ai leaderboard — with the memory math shown step by step.
Why 1 GB per billion? When stored at 16-bit (fp16/bf16) precision — the standard for inference — each parameter takes 2 bytes. One billion parameters = 2 GB. We use 1 GB as a conservative per-billion estimate for 8-bit quantized loading, which most open-source tools support. At full fp16, multiply by 2.
Why 1.2× overhead? The model weights aren't the only thing in GPU memory. The KV cache (storing attention state during generation), activations, and runtime buffers add roughly 15–25% on top of the weights themselves. We use 1.2× as a minimum — production workloads often budget 1.5–2×.
One of the largest open-source language models available, ranked #10 on the Arena Agent leaderboard. Mixture-of-Experts architecture with 40B active params per token. Excels at complex reasoning, code, and multilingual tasks.
A single 8×H200 server (1,128 GB) covers the 904 GB needed with ~224 GB headroom.
Tencent's open frontier model. Strong performance on Chinese and English benchmarks, designed for enterprise deployment.
A single 8×H200 server (1,128 GB) covers the 354 GB needed with 3× headroom — great throughput.
DeepSeek's efficient frontier model. Delivers frontier-level reasoning at lower latency than full-scale MoE models, making it practical for production inference.
A single 8×H200 server (1,128 GB) covers the 341 GB needed. Strong choice for a startup that wants cutting-edge open-source inference.