Can an 8GB laptop run a useful LLM?

The question sounds like a benchmark question, but it's really a definitions question. 'Useful' on a workstation with 80GB of GPU memory means something close to 'as capable as possible.' Useful on an 8GB laptop with no dedicated GPU means something closer to 'reliable enough to trust for a narrow set of tasks, running fast enough that a person will actually wait for it.'

Once RAM is fixed at 8GB — shared with the operating system, the browser, and whatever else is open — the model budget realistically lands somewhere under 2GB after quantization. That rules out anything close to frontier scale and puts the practical ceiling around 1.5–3B parameters at 4-bit quantization. The interesting engineering problem isn't squeezing a bigger model in; it's deciding what to give up first.

In practice, the tradeoff that matters most isn't raw benchmark accuracy — it's reliability under structure. A 1.5B model asked to write a free-form paragraph will sometimes ramble or drift. The same model, constrained at decode time to only emit tokens that satisfy a formal grammar, becomes something closer to a dependable component: it can't produce a malformed tool call even if it 'wants to,' because the invalid tokens are never in its sampling distribution to begin with.

That reframes the original question. An 8GB laptop probably can't run a useful general-purpose chatbot in the way people mean when they compare against GPT-class models. But it can run a useful orchestrator — something that reliably parses intent, calls the right tool, and produces valid structured output, even on modest hardware. The ceiling isn't 'how smart can the model be,' it's 'how much can the surrounding system compensate for a model that isn't very smart, but is fast, local, and constrained.'

That's the framing this project is built around: not chasing capability at a fixed hardware budget, but redesigning the system so a small model's weaknesses matter less.