The best local LLM for coding isn't one model, it's a hardware-tier decision: what fits, and fits usefully, in 16GB, 32GB, or 48GB and up of unified memory or VRAM. A 14B-class coding model quantized to GGUF Q4 or Q5 is the honest ceiling on a 16GB machine, a 32B model opens up on 32GB, and 48GB and beyond is where 70B-class models and datacenter GPUs enter the picture. Quantization format decides most of that math, not raw model size alone.
Every "best local LLM" list I've read assumes you already know your own hardware budget, then buries the actual constraint three paragraphs deep in a spec sheet nobody asked for. I've run the same coding model on a 16GB M2 MacBook Air, a 32GB workstation, and a rented 80GB A100, and the honest answer changes at every step. Not because one number in a benchmark table moves. Because what fits, and what fits with a context window long enough to actually hold a real codebase, is a different question at each tier.
Reddit gets closer to the truth than most vendor pages, but it's scattered across a hundred threads that contradict each other on the same model. One thread swears a 14B model chokes past 8K tokens on 16GB. Another says it's fine, ignoring that they're running Q4 while the first person tried Q8. Both are right. Neither says which quant they used, which is the actual variable.
So this is organized the way the decision actually gets made: by what memory you have, not by which model has the most GitHub stars. Three tiers, one quantization table you can apply to any model that ships next month, and the arithmetic to check it yourself instead of trusting mine.
The 16GB Tier: What a Laptop Can Actually Run
Sixteen gigabytes of unified memory or VRAM is the floor where a genuinely useful coding model still fits, and it's tighter than most buying guides admit. The operating system, your editor, and whatever else is open all take a cut before the model even loads, so budgeting the full 16GB to the model is a mistake that shows up as a swap-to-disk stall the first time you open a second browser tab.
Qwen2.5-Coder-14B-Instruct at GGUF Q4_K_M is the model that actually fits here with room to spare. At roughly 8GB of weights, it leaves enough headroom on a 16GB Apple Silicon Mac or a 16GB consumer GPU like the RTX 4060 Ti to run a real context window, somewhere in the 8K to 16K token range, rather than the bare minimum. Push to Q5_K_M and the same model runs a little sharper on code-completion-adjacent tasks, at roughly 10GB, still comfortable inside the budget.
What you give up at this tier isn't reasoning quality so much as headroom. Long files, multi-file context, and agentic tool-calling loops that stack several rounds of context all eat into the same 16GB pool the model weights already claimed most of. A 7B model buys more context room for the same memory if your actual workload is short completions rather than whole-repository reasoning, which is the tradeoff this tier forces you to make explicitly instead of ignoring.
The 32GB Tier: Where the Ceiling Moves
Doubling to 32GB doesn't just let you run a bigger model, it changes which constraint binds first. At 16GB the model's own weights were the limiting factor. At 32GB, with a 32B-class model landing in the 18 to 22GB range depending on quant, context length becomes the thing you're actually trading off.
Qwen2.5-Coder-32B-Instruct at Q4_K_M is the pick here, and it's a meaningfully different tool than the 14B model one tier down, not just a bigger version of it. On real coding tasks, the step from 14B to 32B shows up more in multi-step reasoning and catching subtler bugs than in raw completion speed. Running it on a 32GB Mac (M2 Pro or M3 Pro-class unified memory) or a single 24 to 32GB consumer GPU still leaves several gigabytes for a working context window once the roughly 18 to 22GB of weights are loaded.
This is also the tier where the runtime you load the model into starts to matter more than it did at 16GB, since context handling and batching behavior diverge between them; our LM Studio vs Ollama comparison covers that choice directly if you haven't settled on one.
48GB and Up: Where Consumer Hardware Meets the Datacenter
Forty-eight gigabytes is the point where two audiences that never talk to each other start running the same math. A developer with a 48GB workstation GPU (an RTX 6000 Ada, or a Mac Studio with 64GB or more of unified memory) and a team renting an 80GB A100 in the cloud are solving the same fitting problem, just at different points on the same curve.
Llama 3.3 70B-Instruct at Q4_K_M is the model that defines this tier. Its roughly 70 billion parameters come in at around 40GB of weights at Q4, which leaves a workable, if not generous, margin on a 48GB card once context and KV cache are accounted for. This is also the point where Q4 stops being merely convenient and starts being close to required. At Q8, the same model needs on the order of 75GB just for weights, past what a single 48GB card can hold without offloading part of the model to system RAM and taking a real speed penalty for it.
Past 48GB, the tier stops being about a single card at all. True datacenter scale means multiple 80GB-class GPUs (A100s, H100s) working together, which is what it actually takes to run something like a 405B-parameter model such as Llama 3.1 405B-Instruct even at 4-bit, where the weights alone land north of 220GB and have to be split across several cards. That's a different operational problem than anything a single-machine setup faces, and it's the boundary most "best local LLM" content never crosses, because almost nobody covering the consumer side has also run the datacenter side.
The same fitting problem at three scales: laptop, workstation, and datacenter.
Quantization Formats, Compared on One Model
Every format in the table below is applied to the same base model, Qwen2.5-Coder-32B-Instruct (32.5 billion parameters), so the comparison is apples to apples rather than six different models with six different footprints muddying the picture. The memory figures are computed, not benchmarked: parameter count times bytes-per-weight, which is the same arithmetic the next section walks through so you can redo it for whatever model ships next month.
| Format | Memory (32.5B model) | Quality vs FP16 | Runs well on |
|---|---|---|---|
| FP16 (baseline) | ~65GB | Reference, unquantized | Datacenter GPU only |
| GGUF Q8_0 | ~35GB | Effectively FP16 | CPU + GPU, llama.cpp |
| GGUF Q5_K_M | ~22GB | Small, hard to notice | CPU + GPU, llama.cpp |
| GGUF Q4_K_M | ~18GB | Small, the usual sweet spot | CPU + GPU, llama.cpp |
| AWQ (4-bit) | ~16-17GB | Small, calibration-based | NVIDIA GPU only |
| GPTQ (4-bit) | ~16-17GB | Small, calibration-based | NVIDIA GPU, older kernels |
What's well established: the drop from FP16 to Q8 is close to imperceptible on most coding work, and Q5 is close behind it. Q4 is the real inflection point, still the sweet spot for most constrained-memory setups, but it's also where you start reading claims that contradict each other, because "quality" at Q4 depends heavily on which specific k-quant variant and which base model you're testing. Below Q4, degradation becomes visible on exactly the tasks a coding model gets used for: multi-step reasoning, longer functions, and edge cases in logic rather than simple autocomplete, where a confidently wrong token is worse than a slow correct one.
That last failure mode is worth naming precisely, because it's the one this whole table can't protect you from. A quantized model's mistakes don't announce themselves as mistakes. They come back as code that compiles, reads plausibly, and is subtly wrong about an edge case, which is exactly the class of defect that a lower quant level makes more likely and that no amount of memory arithmetic will catch. The check has to happen on the running application, not on the weights, which is why Autonoma derives behavioral end-to-end tests from your codebase and runs them against the live app on every pull request, whichever quant level produced the diff.
The format families differ in more than their bit count. GGUF's Q-levels are the llama.cpp-family k-quants, built to run well split across CPU and GPU, which is why they dominate the 16 and 32GB tiers where offloading some layers to system RAM is common. AWQ and GPTQ are both GPU-oriented 4-bit schemes built around calibration data (activation-aware weight rounding for AWQ, layer-wise reconstruction for GPTQ), and both depend on CUDA-specific kernel libraries (AutoAWQ, AutoGPTQ, ExLlama) to get their speed advantage, which is also why they're the two formats least available outside NVIDIA hardware. FP16 is the unquantized baseline everything else is measured against, and it's realistically a datacenter-only format at 32B and up, since 65GB of weights alone rules out anything but a multi-GPU or very high-memory single-card setup.
The mapping that actually matters when you're choosing: GGUF for anything CPU-adjacent, Apple Silicon, or mixed CPU/GPU offload, AWQ or GPTQ if you're purely on NVIDIA and want the fastest inference at a given bit width, FP16 only once memory stops being the constraint at all.
Bar height is memory footprint; Q4 is where the memory savings stop costing much quality.
How to Check Whether Any Model Fits
The arithmetic behind every number above is reproducible, and it's the part of this article that won't go stale when a new model ships. Start with parameters times bytes-per-weight: a model's listed parameter count multiplied by the bytes each weight takes up at your chosen quantization level. FP16 is 2 bytes per weight, Q8 is roughly 1.06, Q5_K_M is roughly 0.69, Q4_K_M is roughly 0.56. Multiply, and you get the weight footprint, which is the number every marketing page stops at, and the number that quietly leaves out the part that actually breaks people's budgets.
That missing part is the KV cache, and it's not optional overhead. It's memory the model needs for every token of context you actually use. The formula: 2 (one for keys, one for values) times the number of layers, times the number of KV attention heads, times the head dimension, times the context length, times the bytes per element. As an illustration, take a model with 40 layers, 8 KV attention heads (a common grouped-query-attention configuration that trades a little quality for a much smaller cache), and a head dimension of 128, running at fp16 KV cache and a 32,768-token context: 2 x 40 x 8 x 128 x 32,768 x 2 bytes comes out to roughly 5.4GB, on top of whatever the weights themselves take. Push that same setup to a 128K context and the cache alone crosses 20GB, which is exactly the kind of number that turns a model that "fit" at a short context into one that doesn't at the context length you actually need for a real codebase.
That's the arithmetic to run before trusting any "fits in 16GB" claim, including the ones in this article: weight footprint from the parameter count and quant level, plus KV cache from the layer count, head count, head dimension, and context length you actually plan to use, both added to whatever the OS and your other running applications already claim.
What Actually Works on AMD and ROCm
Every deep benchmark in this category, without exception, is run on NVIDIA or Apple Silicon. AMD owners get a paragraph at the bottom of somebody else's article, if they get anything, which is strange given how much cheaper VRAM is on the AMD side of the market.
The concrete data point: the Radeon RX 7900 XTX (24GB) sits on ROCm's officially supported card list, and llama.cpp's HIP backend runs GGUF-quantized models on it directly, no translation layer, no CUDA emulation. That means the entire GGUF path, Q4 through Q8, works on AMD hardware today in a way that's genuinely comparable to the NVIDIA path for the same quant.
Where the parity ends is AWQ and GPTQ. Both formats get their speed from CUDA-specific kernel libraries (AutoAWQ, AutoGPTQ, ExLlama and its successors), and ROCm support for those kernels has historically ranged from partial to nonexistent depending on the library and the card. In practice, that means an AMD owner's path to a quantized coding model runs through GGUF and a llama.cpp-family runtime, not through AWQ or GPTQ, and treating those two formats as available on AMD the way they are on NVIDIA is the mistake to avoid.
The other real friction point is version coupling. ROCm's runtime, the kernel driver, and whatever inference library sits on top all need to line up with each other more tightly than the CUDA equivalent does, and a mismatch tends to fail with an unhelpful error rather than a graceful fallback. None of that makes AMD a bad choice. The price-per-gigabyte-of-VRAM case is real, but it does mean budgeting some setup friction that the NVIDIA path mostly doesn't charge you.
The Roster, Verified as of August 2026
Everything above (the tier structure, the quantization arithmetic, the AMD path) holds regardless of which specific model is newest. The model names below don't. This roster moves monthly, and treating it as permanent is the mistake that ages an article like this one badly within a quarter.
Verified as of August 4, 2026: 16GB tier, Qwen2.5-Coder-14B-Instruct at GGUF Q4_K_M or Q5_K_M. 32GB tier, Qwen2.5-Coder-32B-Instruct at GGUF Q4_K_M, the pick this article builds the quantization table around. 48GB tier, Llama 3.3 70B-Instruct at GGUF Q4_K_M, the practical ceiling for a single high-memory card. True datacenter scale, multi-GPU setups running 400B-plus parameter models like Llama 3.1 405B-Instruct, still 4-bit, still split across several 80GB cards.
Check current standings before committing hardware budget around any single name here. llm-stats.com and Artificial Analysis track current releases and benchmark standings better than any static article can, this one included.
The Verdict: Best For, Not For
None of this converges on one best local LLM for coding, and that's not a hedge, it's the actual shape of the answer. Every tier trades something specific for something else, and the right pick depends on which side of that trade you're actually on.
Best for a 16GB laptop: Qwen2.5-Coder-14B at Q4 or Q5, if your work is mostly single-file completion and shorter functions. Not for: multi-file agentic workflows or long-context codebase reasoning, where the model will run but the context budget will fight you constantly.
Best for a 32GB workstation: Qwen2.5-Coder-32B at Q4, genuinely useful multi-step reasoning without needing a dedicated GPU rig. Not for: anyone expecting FP16-level nuance on the hardest edge cases. That gap is real, even if it's smaller than the marketing around quantization sometimes admits.
Best for 48GB and up: Llama 3.3 70B at Q4 on a single high-memory card, or a hosted equivalent if the hardware cost doesn't pencil out. Not for: anyone who actually needs 405B-class reasoning, which only exists at true multi-GPU datacenter scale and isn't coming to a single card anytime soon.
Best for an AMD-only budget: the GGUF path through llama.cpp's HIP backend on a supported card like the 7900 XTX, real coverage across every quant level that matters. Not for: anyone specifically counting on AWQ or GPTQ, since that tooling is a CUDA-first world for the foreseeable future.
And if the actual constraint is that local isn't worth the hardware spend at all, that's a real answer too, one worth working through on its own terms before buying a GPU to solve a problem an API might solve for less; our guide to running an LLM locally walks through that decision directly. The same goes for the tool-choice question one layer up: if you're weighing a local model against a hosted assistant, our GitHub Copilot alternatives roundup covers that side of the decision, and the hardware math in this article is only half of it.
What the Hardware Tier Doesn't Decide
Every number in this article is about fitting a model into memory. None of it is about whether the code that comes out the other side actually works.
That gap matters more at the constrained end of the tiers than at the top, and it's the part a hardware-shopping guide structurally leaves out. A 14B model at Q4 on a 16GB laptop will happily produce a diff that compiles, passes a linter, and quietly mishandles the concurrency case you never described to it. So will a 70B model at Q4 on a 48GB card, just less often. Picking the right tier lowers the rate of that outcome. It doesn't change the fact that something has to run the application and confirm the behavior is right.
That check is what Autonoma is built for. It reads your codebase, derives behavioral end-to-end tests from it, and runs them against the running application on every pull request, so a wrong turn from a quantized local model surfaces as a failing user flow instead of a bug report weeks later. It isn't a model and it doesn't compete with anything in the roster above; it sits one layer up, and the answer it gives doesn't change based on how much VRAM you bought. If you switch tiers next quarter, or swap Qwen for whatever ships next month, the verification layer is the one piece of this stack you don't have to re-derive.
Frequently Asked Questions
Qwen2.5-Coder-14B-Instruct quantized to GGUF Q4_K_M or Q5_K_M is the realistic ceiling. At roughly 8 to 10GB of weights, it leaves enough of a 16GB budget free for the operating system, your editor, and a usable context window in the 8K to 16K token range. A 7B model buys more headroom for context if your work is short completions rather than whole-file reasoning.
For most coding tasks, yes. The drop from FP16 to Q4 is small and usually hard to notice on completions, refactors, and typical debugging. It becomes more visible on multi-step reasoning, longer functions, and subtle logic edge cases, which is exactly where a coding model gets tested hardest. Below Q4, that degradation gets easier to spot, which is why Q4 is treated as the practical floor rather than Q3 or lower.
For the GGUF path, yes, with real coverage. llama.cpp's HIP backend runs GGUF-quantized models directly on ROCm-supported cards like the Radeon RX 7900 XTX, no CUDA emulation required. AWQ and GPTQ are a different story: both depend on CUDA-specific kernel libraries, and ROCm support for them has historically been partial to nonexistent. An AMD setup for local coding models today means GGUF and a llama.cpp-family runtime, not AWQ or GPTQ.
GGUF is the llama.cpp-family format built for flexibility across CPU and GPU, including split offload when a model doesn't fully fit in VRAM, and its Q-levels (Q4, Q5, Q8, and others) are k-quants that mix precision across layers to hit a target average bits-per-weight. AWQ is a GPU-oriented 4-bit format that uses calibration data to identify which weights are most sensitive and protect them during rounding, and it depends on CUDA-specific kernels for its speed, which is why it's effectively NVIDIA-only in practice.
For some workloads, yes, and for others the tradeoff doesn't pencil out. A local model removes the per-token cost and the network dependency, but it caps out at whatever your hardware can hold, and a hosted assistant backed by a much larger model will usually out-reason a quantized 14B or 32B model on the hardest tasks. The honest framing is a hardware-and-privacy decision as much as a capability one, not a strict replacement in either direction.
At GGUF Q4_K_M, Llama 3.3 70B-Instruct's weights alone run to roughly 40GB, which fits on a single 48GB card with a workable but not generous margin once context and KV cache are added. At Q8, the same model needs on the order of 75GB for weights, past what one 48GB card can hold without offloading part of the model to system RAM and taking a real speed penalty for it.
It catches the defect quantization makes more likely. Dropping to Q4 to fit 16GB doesn't produce output that looks broken; it produces code that compiles, passes a linter, and mishandles an edge case you never described, which no memory arithmetic will surface. Autonoma derives behavioral end-to-end tests from your codebase and runs them against the running application on every pull request, so that mistake fails as a user flow instead of reaching production. Change quant level or model size later and the suite stays the same. Autonoma isn't a model and doesn't run on your GPU.
