Aperio Home All Suites
Memory & Wiki Evaluation

Core Knowledge Layer

Ten tests that probe whether the model correctly uses Aperio's persistent memory and wiki tools — recall, remember, update_memory, forget, wiki_search, wiki_get, wiki_write, and the self-memory quad. This is Aperio's core value proposition: does the agent actually store, retrieve, and synthesize knowledge across conversations?

These are observation-based tests. Paste the prompt, watch how the model responds — does it call the right tool? Does it use the returned data correctly? Unlike file-generation tests, there's no automated verify command here; the model's own output is the result. Each test has a checklist of what to look for. Start fresh conversations when possible.

Memory Tools Quick Reference

ToolWhen to callKey rule
recall(query?)User asks "what do you know about…" or "check my memories"Call with no args first — never ask the user to narrow it down
remember(content)User says "remember that…", "save this", "keep this"Call immediately, say "Saved." Nothing else
update_memory(uuid)New info contradicts or supersedes stored knowledgePrefer update over creating a duplicate
forget(uuid)User explicitly asks to delete somethingOnly call on explicit instruction
wiki_search(topic)Check if a wiki article exists before writing oneAlways call before wiki_write
wiki_get(slug)Read a specific articleSurface the breadcrumb as first line of reply
wiki_write(slug, ...)≥3 memories on same topic → synthesize an articleCite memories inline as [[mem:uuid]]
self_recall / self_rememberAgent's own continuity storeAutonomous — no permission needed

Setup

Start an Aperio server and open a chat session. Paste each prompt into a fresh conversation when possible to avoid cross-talk.

npm run start:local
# In another terminal:
npm run chat:local

Clear memory between runs if needed: rm -f var/store.sqlite var/wiki.sqlite && npm run migrate

The Tests

Each test is a prompt to paste. The model's response IS the test result — did it call the right tool, use the returned data, and follow the memory protocol rules?

Task 1 — Basic Recall

Check Stored Context ★☆☆☆☆

Probes: calls recall with no arguments, reads and acknowledges the result

Paste this into a fresh chatWhat do you know about me? Check my memories first before answering.
What to evaluate
  • Does the model call recall with no arguments? (It should NOT ask you to narrow it down.)
  • Does it read and acknowledge the recall results before answering?
  • If no memories exist yet, does it say so clearly rather than hallucinating?
Result:
Task 2 — Save a Memory

Instant Save ★☆☆☆☆

Probes: calls remember immediately, minimal confirmation, appropriate type/importance

Paste this into a fresh chatI mostly work with React and TypeScript, and I prefer functional components over class components. Remember that.
What to evaluate
  • Does the model call remember immediately without asking for confirmation?
  • Does it respond with just "Saved." or equivalent minimal acknowledgment?
  • Does it assign an appropriate type (preference) and default importance (3)?
  • Does the memory content capture both the tech stack AND the component preference?
Result:
Task 3 — Query Recall

Search Memories ★★☆☆☆

Probes: calls recall with a query string, uses the result to answer

Paste this into a fresh chatDo you remember what tech stack I prefer for frontend development? Search your memories for it.
What to evaluate
  • Does the model call recall with a relevant query (e.g., "tech stack" or "frontend")?
  • If the memory from Task 2 is present, does it answer correctly (React, TypeScript, functional components)?
  • If no matching memory exists, does it say so rather than guessing?
Result:
Task 4 — Update Memory

Correct Stored Information ★★★☆☆

Probes: calls recall to find existing memory, calls update_memory with the UUID, flags contradictions

Paste this into a fresh chatI told you earlier I prefer functional components, but actually I've switched. Now I prefer class components for complex state logic. First check what you have stored, then update it.
What to evaluate
  • Does the model call recall first to find the existing memory?
  • Does it flag the contradiction ("I have a memory that says X — should I update it?") before updating?
  • Once confirmed, does it call update_memory with the correct UUID?
  • Does the updated content preserve the unchanged parts (React, TypeScript) while changing the preference?
Result:
Task 5 — Delete Memory

Forget a Memory ★★★☆☆

Probes: calls recall to find memory, calls forget with UUID, confirms deletion

Paste this into a fresh chatI want to delete the memory about my tech stack preferences. Find it and forget it.
What to evaluate
  • Does the model call recall to find the specific memory?
  • Does it use forget with the correct UUID?
  • Does it respond appropriately (e.g., "Deleted." or "Forgotten.")?
  • If no matching memory exists, does it say so honestly?
Result:
Task 6 — Read Wiki Article

Search & Read ★★★☆☆

Probes: calls wiki_search before wiki_get, surfaces the breadcrumb

Paste this into a fresh chatWhat does the Aperio wiki say about memory architecture? Search the wiki and read the relevant article.
What to evaluate
  • Does the model call wiki_search first to find articles about memory architecture?
  • Does it call wiki_get on the best-matching slug?
  • Does the reply start with the breadcrumb: 🔖 From wiki: [[slug]]?
  • If no article exists, does it say so and offer to write one?
Result:
Task 7 — Write Wiki Article

Synthesize Memories into an Article ★★★★☆

Probes: checks existing articles, gathers memories, synthesizes, cites sources

Paste this sequentiallyRemember these facts one at a time: 1. "The Aperio server runs on port 31337 by default, configurable via PORT env var." 2. "Aperio supports SQLite (zero-config) and Postgres (multi-agent) backends, auto-detected." 3. "Aperio uses MCP protocol for tool communication with AI providers." Now check if a wiki article about "Aperio server configuration" already exists. If not, create one that synthesizes all three facts. Use the wiki tool, not manual writing.
What to evaluate
  • Does the model call remember for each fact as you give it?
  • Does it call wiki_search before attempting to write?
  • Does it call wiki_write with source_memory_ids linking to the 3 memories?
  • Does the article body cite memories as [[mem:uuid]] inline?
Result:
Task 8 — Self Memory

Agent's Own Continuity Store ★★★★☆

Probes: autonomous use of self_recall/self_remember, no permission-asking

Paste this into a fresh chatWhat have you learned about working with me? Check your own notes.
What to evaluate
  • Does the model call self_recall (or equivalent) to check its own memory store?
  • Does it do so autonomously — no asking for permission?
  • If there's content, does it reference what it learned?
  • If empty, does it say so without fabricating?
Result:
Task 9 — Types & Importance

Structured Memories ★★★☆☆

Probes: correct type assignment, importance levels, tag usage

Paste this into a fresh chatSave these as separate memories with the correct types and importance levels: 1. (important) "The production deployment uses Postgres, not SQLite. This is critical — never suggest SQLite for prod." 2. "I prefer dark mode for all my tools." 3. "The API rate limit is 100 requests per minute for our team." 4. "We chose Aperio over Mem0 because of the self-hosting aspect." Don't confirm — just save each one.
What to evaluate
  • Does the model save all 4 as separate memories?
  • Are types assigned correctly: fact for #1, preference for #2, fact for #3, decision for #4?
  • Is importance level 5 used for #1 (critical) and default 3 for others?
  • Are relevant tags included (e.g., "deployment", "postgres" for #1)?
Result:
Task 10 — Full Knowledge Cycle

Remember → Wiki → Recall → Update ★★★★★

Probes: end-to-end memory workflow, tool chaining, correct sequence

Paste this sequentiallyPhase 1: Remember these: - "The project name is Aperio and it's pronounced ah-PEHR-ee-oh." - "Aperio is licensed under MIT." - "The GitHub repo is at github.com/BaiGanio/aperio." Phase 2: Now create a wiki article called "project-info" that synthesizes all three facts. Phase 3: Now read that article back to me using the wiki tool. Phase 4: I just realized the repo has moved to github.com/aperio-org/aperio. Find and update the relevant memory.
What to evaluate
  • Phase 1: Does it save 3 separate memories?
  • Phase 2: Does it wiki_search first, then wiki_write with source_memory_ids?
  • Phase 3: Does it wiki_get the article and surface the breadcrumb?
  • Phase 4: Does it recall the memory, flag the contradiction, then update_memory?
  • Does the full cycle complete without the model getting confused or dropping context?
Result:

Scoring Rubric

Signal0 = fail1 = pass
Called the correct toolhallucinated a fact instead of calling recall; used wrong toolcorrect tool for the intent
Followed protocol rulesasked "can you narrow it down?"; said "do you want me to save that?"called immediately, minimal response
Used returned dataignored recall results; answered from training dataread and acknowledged stored data
Surfaced wiki breadcrumbanswered from wiki but didn't show the breadcrumbbreadcrumb is first line of reply
Handled contradictionssilently overwrote; created duplicate; didn't flagflagged conflict before updating

Scorecard

Click Pass/Fail as you observe each test.

TestResultNotes
1. Basic Recall—
2. Save Memory—
3. Query Recall—
4. Update Memory—
5. Forget—
6. Wiki Search & Read—
7. Wiki Write—
8. Self Memory—
9. Types & Importance—
10. Full Knowledge Cycle—
TOTAL: ____ / 10 passed
Scoring notes:
  • Pass — the model called the correct tool, followed protocol rules, and used the returned data.
  • Fail — hallucinated from training data instead of recalling, asked for permission to save, didn't surface breadcrumbs, created duplicates instead of updating.
  • N/A — prerequisite memory from an earlier task doesn't exist (run tasks sequentially with shared store).
  • Tasks 1–5 test memory tools. Tasks 6–7 test wiki tools. Task 8 tests self-memory. Tasks 9–10 test combined workflows.

What the Results Mean

← Your result
9–10/10
Full knowledge layer mastery. The model correctly uses all memory and wiki tools, follows protocol (immediate save, no permission-asking, breadcrumbs, contradiction-flagging), and can chain tools across the full remember→wiki→recall→update cycle. Trust it with Aperio's core feature.
← Your result
7–8/10
Strong knowledge layer understanding. Handles basic recall/remember well. May forget to surface breadcrumbs, skip wiki_search before wiki_write, or not flag contradictions clearly. Reliable for memory but needs wiki nudges.
← Your result
4–6/10
Moderate ability. Can save and recall basics but struggles with workflow chaining. May ask for permission to save, hallucinate instead of recalling, or create duplicate memories. Needs guidance for wiki tasks.
← Your result
1–3/10
Limited memory ability. May recall from training data instead of the store. Rarely calls memory tools autonomously. Not reliable for persistent knowledge work.
← Your result
0/10
Cannot use Aperio's knowledge layer. Does not call memory tools at all or uses them incorrectly.
All suites →Back to all evaluation suites. Next → PDF SkillsTest PDF document generation and manipulation.