The big idea
Grounded in a life, not the web
This isn't search over the public internet — it's answering questions using years of someone's own journals, notes, and documents. That makes two things true at once: the corpus is deeply personal (privacy has to be central, not bolted on), and it's messy, heterogeneous, and constantly growing in a way a curated knowledge base never is.
We'll build retrieval and ingestion designed for that reality: incremental updates, disambiguation using personal context, citations back to the real source document, and privacy as an architectural constraint that shapes the whole design, not a feature added at the end.
How to read this: Each step opens with a real design decision — make the call before I show you what ships. Watch the diagram grow, hover the boxes, and at the end ask a genuinely ambiguous question and request a deletion to see the assistant's honesty and its forgetting actually hold. Hit Begin.
Step 1 · The baseline
Dump everything into the context window
Simplest version: for every question, stuff all the user's personal content directly into the LLM's context window and ask it to answer. What happens as years of notes and documents accumulate?
Design decision: Every query dumps the user's entire personal corpus into the context window. What happens as it grows over years?
The call: The corpus eventually exceeds any practical context window, and even before that point, sending the ENTIRE history for every query is wasteful, slow, and expensive when only a small, specific part of it is actually relevant to any given question. — Personal corpora genuinely grow without bound over years of use — dumping everything in doesn't scale, both because of hard context limits eventually and because it's a wasteful, expensive way to answer a question that usually only needs a small, specific slice of the whole history.
A personal corpus grows without bound over years of real use, eventually exceeding any context window — and even before that, sending everything for every query is wasteful when only a small, specific slice is actually relevant. This needs real retrieval, not brute-force context stuffing.
A personal corpus is an unbounded, growing dataset: Unlike a fixed document set, a personal knowledge base keeps growing for as long as the user keeps living and writing — any design that assumes it fits in a context window is assuming away the actual long-term shape of the problem.
Step 2 · Retrieval over messy personal content
Not a curated knowledge base
Build real retrieval (RAG) over the personal corpus. Personal documents — journal entries, scanned receipts, meeting notes, half-finished drafts — are far more heterogeneous and noisier than a curated knowledge base. What does that mean for retrieval?
Design decision: Personal documents are heterogeneous and noisy compared to a curated knowledge base. What does retrieval need to account for?
The call: Retrieval has to handle wide format heterogeneity (short fragments, scanned images, structured and unstructured notes) and be robust to noisier, less consistently-structured content than a curated public knowledge base would ever contain. — Real personal content spans everything from a two-word reminder to a scanned multi-page document to a rambling journal entry — retrieval built assuming consistent, well-structured documents will underperform on this genuine heterogeneity, so the retrieval approach has to be designed for messiness from the start, not adapted later.
Build the Personal Corpus Index and Retriever to handle genuine format heterogeneity and noise from the start — short fragments, scanned content, unstructured notes — rather than assuming the relatively clean, consistent structure a curated knowledge base would have.
Design for the data you actually have, not the data that's convenient: Personal content is genuinely messier than most retrieval systems are built to handle by default — acknowledging and designing for that reality up front avoids a retrieval system that quietly underperforms on exactly the kind of content real users actually produce.
Step 3 · Continuous, incremental ingestion
New content arrives constantly
New personal content — a note, an email, a document — arrives continuously, not in scheduled batches. Should the index require a full reprocessing job to absorb each new item?
Design decision: New personal content arrives constantly, in an ongoing stream. Should the index require a full batch reprocessing for each addition?
The call: No — the index updates incrementally as each new item arrives, so newly-added content becomes searchable essentially immediately, without waiting for a batch job. — Incremental indexing means a note written five minutes ago is already retrievable — matching the reality that personal knowledge accumulates continuously, not in scheduled batches, and that a user reasonably expects the assistant to know about something they just wrote down.
Make the Personal Corpus Index update incrementally — each new item becomes searchable as it arrives, without waiting for a scheduled batch reprocessing job. This matches the reality that personal knowledge genuinely accumulates continuously.
Match the update mechanism to how the data actually arrives: A personal knowledge base is fed by an ongoing stream of small additions, not periodic bulk uploads — the ingestion architecture should reflect that actual pattern rather than imposing a batch model that doesn't fit it.
Step 4 · Disambiguate using personal context — or ask
"The contract" could mean several things
A user asks "what did the lawyer say about the contract" — but they've discussed several contracts over the years. How should the assistant resolve which one is actually meant?
Design decision: A query could plausibly refer to any of several things in the user's history. How should the assistant handle this?
The call: Use recent conversation context to disambiguate where possible, and when the query is genuinely ambiguous even with that context, ask a clarifying question rather than guessing. — Recent context often resolves ambiguity naturally (if the user was just discussing a specific contract, "the contract" likely means that one) — but when it genuinely doesn't, asking rather than guessing avoids confidently answering about the wrong referent, which is a far worse outcome than a brief clarifying question.
Add a Disambiguator that uses recent conversation context to resolve which specific personal referent a query means, and — when genuinely ambiguous even with that context — asks a clarifying question rather than guessing and risking a confidently wrong answer about the user's own life.
Personal ambiguity deserves the same honesty as any other: The same "recognize uncertainty and ask rather than guess" principle that matters for customer support applies here with even higher stakes — a wrong guess about someone's own private information and history is a uniquely trust-damaging failure mode.
Step 5 · Cite the source, always
The user needs to verify, not just trust
An answer is generated from retrieved personal content. Should it just present the answer, or something more?
Every answer includes source attribution — exactly which personal document or note it was grounded in, so the user can verify the answer, find the original context, or catch a retrieval mistake. For an assistant reasoning about someone's own life and documents, being able to trace an answer back to its actual source is what makes the system trustworthy rather than a confident black box.
Citations are how trust is earned with personal data: Unlike general knowledge questions where a wrong answer is merely unhelpful, a wrong or unverifiable claim about the user's OWN documents and history is uniquely unsettling — source attribution gives the user a concrete way to check the assistant's work, not just hope it's right.
Step 6 · Privacy as the central architectural constraint
This is uniquely sensitive data
Personal notes, journals, and documents are about as sensitive as data gets. Should this system handle privacy the same way a typical product handles any other sensitive-but-routine data?
Design decision: How central should privacy be in this system's architecture, compared to a typical product handling sensitive data?
The call: Make privacy a PRIMARY architectural driver: strong encryption at rest, and deliberate minimization of what — if anything — leaves the system for third-party processing (redacting highly sensitive fields, or preferring more private/on-device inference for the most sensitive parts of the corpus). — Given how uniquely sensitive this specific corpus is, privacy has to shape core architectural decisions (not just be a compliance checkbox) — including seriously considering HOW MUCH of a user's private content should ever leave the system boundary for third-party model inference, and minimizing that exposure deliberately.
Build a Privacy Layer that makes privacy a primary architectural driver: encryption at rest for the entire personal corpus, and deliberate minimization of what content leaves the system boundary for any third-party processing — redacting the most sensitive fields where feasible, or preferring more private inference options for the most sensitive parts of the corpus.
Some domains make privacy the primary constraint, not one of several: Most systems in this series treat security/privacy as an important but parallel concern alongside performance and correctness. For a system built entirely around someone's own private life data, privacy isn't parallel to the core function — it IS close to the core function, and should shape architecture from the start.
Step 7 · "Forget this" should mean forgotten
Genuine deletion, not hiding
A user asks the assistant to forget a specific note. Should that just hide it from a default view, or something more thorough?
Build a Deletion Handler that removes a requested item genuinely — from the retrieval index AND from any summary, cached derivation, or memory that referenced its content — so a future query can't surface it, directly or indirectly. Marking something merely "hidden" while its content still lives in the index or a derived summary doesn't honor what the user actually asked for.
Deletion has to reach every derived copy, not just the original: The same verifiable-deletion principle from multi-tenant offboarding applies here for a single user's own explicit request — genuine forgetting means tracing and removing every place the content's influence persists, not just the original document.
Step 8 · The sharp edges
Facts change, and not everything is text
Two real complications: the user's own notes can genuinely contradict each other as facts change over time (an old address, a since-changed plan), and a meaningful share of personal content — scanned documents, photos with text — isn't plain text to begin with.
For facts that change over time, prefer recency when retrieved content genuinely conflicts — "my address is X" from three years ago shouldn't be presented as equally valid alongside "my address is Y" from last month; the retriever and answer generation should weight more recent, contradicting information appropriately rather than surfacing whichever matches the query best regardless of when it was written. For multi-modal content, run OCR/vision processing at ingestion time to extract text from scanned documents and images before they can be meaningfully indexed and retrieved alongside plain-text notes.
Design for the unhappy path: Genuine ambiguity → ask, don't guess. A deletion request → genuinely removed everywhere, not just hidden. Facts that changed → recency-aware retrieval, not blind similarity matching. Non-text content → processed into retrievable text at ingestion. A personal assistant that only works for clean, consistent, never-contradicting, all-text content is a demo; one that handles a real messy personal life is a product.
You did it
You just designed a personal knowledge assistant.
- A personal corpus grows unbounded over years — dumping everything into a context window per query doesn't scale, in cost or eventually in hard limits.
- Retrieval has to handle genuine format heterogeneity and noise, since personal content is far messier than a curated knowledge base.
- Incremental ingestion makes new content searchable immediately, matching how personal knowledge actually accumulates.
- The assistant disambiguates using recent conversation context, and asks rather than guesses when genuinely uncertain — a wrong guess about someone's own life is uniquely trust-damaging.
- Every answer cites its source document, letting the user verify rather than just trust a confident-sounding claim about their own history.
- Privacy is a primary architectural driver here, not a parallel concern — encryption at rest and deliberate minimization of third-party exposure.
- A deletion request genuinely removes content from the index and every derived summary, not just from a default view.
- Recency-aware retrieval handles contradicting facts that changed over time, and OCR/vision processing brings non-text personal content into the retrievable corpus.