Local LLMs
2 min read · Mastodon
GGUF VRAM Calculator Estimates GPU Footprint Including Context and KV Cache
A zero-dependency HTML calculator estimates VRAM requirements for GGUF models before downloading. It computes model weights, context length, and architecture-specific KV cache footprint to avoid CUDA allocation crashes, with the caveat that MoE and mixed-quant files land a few percent off the table.