5 LLM quantization techniques, clearly explained:
(bookmark this)
A 70B model in FP16 needs 140GB for weights alone. At 4-bit, that drops to 35GB, which fits on one card.
But naive rounding fails on large models. Roughly 0.1% of hidden dimensions carry values up to 20x larger
Anthropic did something you'll regret ignoring:
They split one coding task across four agents by role, as a planner, implementer, tester, and reviewer.
The goal was to test whether splitting agents by job title is a good way to divide the work.
And they found agents spent more
