Spheron Network

@spheron

One platform, every GPU cloud. Deploy H100s to B300s across multiple providers. On-demand, no lock-in, from a single dashboard.

Remote

spheron.network

Replies Articles Media

Spheron Network
@spheron
Aug 28
A 70B model at FP16 needs about 140GB of VRAM before you've loaded a single token of context. Quantization cuts that 2 to 4x. H200 reaches 31,712 tokens/s on Llama 2 70B, about 42% above H100's 22,290. Both numbers matter less than what's driving them: LLM inference is
Spheron Network
@spheron
Aug 28
A standard LLM emits maybe 400 tokens of internal reasoning before answering. DeepSeek R2 emits up to 40,000. That's 100x the pressure on the KV cache, the memory that holds every token's context while the model generates. MLA compression is what makes it survivable: roughly
Spheron Network
@spheron
Aug 27
H100 SXM5 moves data 40% faster than H100 PCIe. Same chip name on the invoice, different performance. SXM5: 3,350 GB/s memory bandwidth, NVLink up to 900 GB/s PCIe: 2,000 GB/s memory bandwidth, no NVLink Both have 80GB VRAM. Pick by workload: - High batch size, serving many
Spheron Network
@spheron
Aug 27
Qwen3-32B needs 64GB of VRAM at FP16. Drop to FP8 and that's 32GB. Drop to AWQ INT4 and it's 16GB. That range is the difference between needing two GPUs and fitting on a single H100 or A100 at FP8 or INT4. The MoE variant, Qwen3-235B-A22B, only activates 22B parameters per
Spheron Network
@spheron
Aug 26
At batch 1 on a 70B FP16 model, token ceiling tracks memory bandwidth almost exactly, not compute. - H100 (3.35 TB/s): about 24 tokens/sec - H200 (4.8 TB/s): about 34 - B200 (8 TB/s): about 57 - B300 (10 TB/s): about 71 - R100 (projected 22 TB/s): about 157 That last row is a