Small local models chat only

Qwen2.5 3B · Qwen3.5 4B · Phi-4 Mini 3.8B — the small, fast models that run on almost any machine. They can't use Aperio's tools, but that's only half the story. Within their lane, they're genuinely delightful.

One page covers all three, because at this size the differences are about flavour, not power. None of them can remember you across chats, read your files, or browse the web — that needs a bigger model (the Llama 3.1 8B page is where tools start working). So instead of testing them on things they can't do, this page shows you the things they do well — and gives you ready-to-paste prompts to discover each one.

They're fast, private, free, and they fit in 8 GB of RAM. Think of them as a sharp, well-read writing partner who happens to live entirely on your laptop. New here? The two-minute concepts primer explains the jargon.

What they're genuinely great at

Paste any of these to see a small model shine. No setup, no tools — just type and watch.

The three, side by side

Same weight class, different personalities. If you've pulled one already, here's its character; if you're choosing, this is how they differ.

Where the wall is

These aren't flaws — they're the edge of the weight class. When you hit one of these, it's time for a bigger model, not a better prompt.

• Remembering you across chats — they can't use Aperio's memory. Start a new chat and everything's forgotten.
• Reading your files or browsing the web — no tool calling at this size, so no file access, no live information.
• Long documents — feed them a page or two at a time, not a whole report. They lose the early pages.
• Long, multi-step instructions — ask one thing at a time. A five-part mega-prompt will fade out halfway.
• Hard logic, maths, or code — results get unreliable. Verify anything that matters, or step up.
• Recent facts — frozen at the training cutoff above, with no way to look things up. Don't trust them on current events.

Need any of the above? The next step is Llama 3.1 8B — the smallest model where memory, files, and the web start working. Or browse the full lineup.

How safe is it, really?

Small models aren't just less capable — they're also less carefully trained to stay within safe boundaries. This section is the honest version of what that means.

The short version

Larger models go through extensive safety training — millions of examples of what to refuse and why. At 3–4 billion parameters, a model simply doesn't have room for that much nuance. Its safety guardrails are thinner, and they break sooner.

What this looks like in practice

• Easier to push past guardrails — a request that a bigger model would firmly refuse might get a compliant answer from a small one. This isn't because it's "evil" — it's because the refusal pattern wasn't trained deeply enough.
• Less likely to say "I can't help with that" — small models default to being helpful, and that helpfulness sometimes overrides their safety instincts.
• More vulnerable to phrasing tricks — the same request, reworded slightly, can flip a refusal into compliance. Big models have seen enough variations to catch the intent; small ones haven't.
• Weaker at recognising indirect harm — a request that's obviously harmful when stated plainly might slip through when wrapped in innocent-sounding language. Large models are better at reading between the lines.

What this means for you

Treat small models like a creative collaborator you trust with ideas, not a gatekeeper you'd trust with decisions. They're wonderful for writing, brainstorming, and conversation — and they should not be the final word on anything where safety, ethics, or factual accuracy matters.

If your use case involves filtering harmful content, moderating conversations, or making decisions with real-world consequences — this is the moment to step up to a Llama 3.1 8B or larger. Alignment scales with capability.

💡 Curious to test this yourself? The model guide has a calibration test (Test 6) that probes exactly this boundary — try it on your model and see what happens.