A 10-minute hands-on test you can run on any model you've set up in Aperio.
When the AI does something disappointing, there are really only two possibilities:
Both look identical from the outside — the answer is just wrong or weird. So people blame themselves, or blame "AI," and never learn which model is good for what. This page fixes that.
You'll run five short tests. Each targets one thing a model can be good or bad at, and tells you exactly what failure looks like — so you can recognize the wall instead of fighting it. No technical knowledge needed: paste the prompt, watch what happens, write down the result.
The whole vocabulary you need to read the table below and every model page. Two minutes, then you never have to think about it again.
Written as 3B, 8B, 30B (billions). Bigger generally means smarter and more capable, but slower and hungrier for memory. It is the single biggest predictor of what a model can do.
Using Aperio's memory, reading your files, searching the web, writing the wiki — all of this needs "tool calling." As a rule, models below roughly 7–8B can't do it reliably (Google's Gemma 4 e4B is the rare exception). Below that line you have a brilliant conversationalist; above it, an assistant that does things.
How much text it tracks before forgetting the start. Aperio sizes this to your hardware automatically. Raise or lower it by adding LLAMACPP_CTX=32768 to your .env — Aperio passes it through for you, no command line needed. Bigger windows use more memory.
Every model is frozen at its training date and knows nothing after it. A model that can use tools sidesteps this by fetching the live web; one that can't will either say "I don't know" or invent something. An older cutoff with web access beats a newer one without.
Every model set up for Aperio, smallest to largest. "Tools" is rough real-world reliability. Click any model name to open its full page — hardware details and hands-on tasks scaled to what that model can actually do.
| Model | Maker | Size | RAM | Tools | Best for |
|---|---|---|---|---|---|
| Qwen2.5 3B chat only | Alibaba | 3B | 8 GB | — | First local AI, casual chat & creative writing |
| Qwen3.5 4B chat only | Alibaba | 4B | 8 GB | — | The most polished 4B conversationalist |
| Phi-4 Mini 3.8B chat only | Microsoft | 3.8B | 8 GB | — | Step-by-step reasoning & clear explanations |
| Gemma 4 e4B tools | 4B | 8 GB | ~75% | Tool calling on the smallest possible footprint | |
| Llama 3.1 8B tools | Meta | 8B | 16 GB | ~85% | The threshold — first reliable memory/files/web |
| Qwen2.5VL 7B tools + vision | Alibaba | 7B | 16 GB | ~85% | Seeing & describing images, reading text from photos |
| Gemma 4 12B full | 12B | 32 GB | ~95% | The full experience — strong reasoning, reliable tools | |
| Qwen3 30B-A3B full | Alibaba | 30B MoE | 24 GB | ~98% | The summit — top capability, fast thanks to MoE |
RAM is the comfortable minimum for the whole machine, not just the model. Need cloud models (Claude, DeepSeek)? Those run beyond any local hardware — see the Setup page.
These five probes are deliberately the same for every model, so you can run them on any model and compare the cards side by side. Each model's own page adds harder, tailored tasks on top — but start here.
Checks whether the model can really use Aperio's tools — or just describes doing them while nothing happens. This is the #1 source of frustration: a weak model says "Done!" and there's no file.
Checks the context window — how much text it tracks before forgetting the start. If it contradicts itself on a long document, it didn't get dumb; it ran out of room.
Checks whether it respects precise rules or just does roughly what it feels like. Say "three bullets" and get six paragraphs, and you'll waste time re-asking.
Checks whether it holds a multi-step problem together or guesses. A model that's great at writing can still be bad at thinking.
Checks whether it makes things up ("hallucinating") when it has no real answer. A confident, detailed, completely false answer is more dangerous than an open failure.
Probes whether the model recognises and refuses a request it shouldn't comply with — even when the request is wrapped in a polite, reasonable-sounding justification. Small models have thinner safety alignment and are more likely to comply.
The comparison table above gives you the quick overview. These tours go deep on one model at a time — the hardware it really needs, what it can and can't do, and concrete tasks tuned to that model's level so you're never fighting a wall it was never going to clear.
Read them smallest to largest to feel the capability ladder, or jump straight to whichever model you've set up.
Qwen2.5 3B, Qwen3.5 4B and Phi-4 Mini. They can't use tools, but they're fast, fully private, and wonderful conversationalists. How to get the most from the things they can do.
The 4B model that breaks the rule and calls tools — in under 3 GB. Tool use on the smallest possible footprint, with its real, slightly-fragile limits mapped out.
The threshold. The smallest model that reliably remembers you, reads your files, and browses the web — where a chat toy becomes an assistant that actually acts.
Your local AI gains sight. A vision model that describes images, reads text from photos, and compares pictures. Hands-on tasks for seeing, not just reading.
The full local experience. Strong reasoning, ~95% reliable tool calling, and a polished voice — a genuinely capable assistant that still fits on a good machine.
The summit. The most capable model in the lineup, kept fast by its mixture-of-experts design. Top-tier reasoning and near-perfect tool calling.
Not a model — a mode. The Discuss toggle turns a single answer into a two-agent deliberation that cross-reviews itself until it agrees, with domain characters giving each agent a different lens.
Not a model — a capability. Connect SQLite, Postgres or MySQL and ask questions in plain English. Aperio writes the SQL, reads run instantly, and any change to your data asks you first.
RAM is the comfortable minimum for the whole machine, not just the model. Prefer a table? The model comparison lays all of these out side by side.
The calibration tests above give you the broad shape. These suites go deeper — each targets a single capability that's easy to claim and hard to verify, with built-in scorecards you can fill in and print.
Does the model tell the truth about what it actually did? Catches the model that says "Done! I created the file" when no file exists — the single most common and most costly failure.
Do the safety boundaries actually hold under probing? Tries to read secret files, reach internal URLs, and run shell commands — and tells you whether the guardrails are real code or just polite words.
When you ask for something, does the right skill load for the job? Checks whether the model reliably picks the correct tool for each request instead of guessing or grabbing the wrong one.
Does document indexing and search really work? Verifies that your files get indexed, that search finds them by meaning, and that the model can cite what it found rather than inventing it.
Do scheduled jobs that run without you typing anything actually fire? Verifies that background agents wake on time, do their work, and report back — all from the web UI, no command line.
Can your local model actually write working code inside Aperio? Ten progressive tasks — from pure helpers to cross-cutting DB changes — with copy-paste prompts and verify commands.
Does the model design for the project, or default to one look? Checks whether a kids' app and a luxury boutique get genuinely distinct fonts and palettes — or the same convergent template every time.
The full system benchmark — 65+ drills across every tool category. Run the canonical exam from exam.md and record section scores here. Takes 30–60 minutes but tells you everything.