Own the memory, rent the model

Five months of keeping a digital self

On April 2, 2026, Andrej Karpathy posted a thread about using LLMs to build personal knowledge bases, and two days later published llm-wiki.md, an “idea file” describing the pattern: raw sources the model may only read, a wiki of markdown the model owns and maintains, and a schema document that turns a generic chatbot into a disciplined maintainer. Not a library. A pattern, meant to be handed to an agent so the agent can instantiate it with you.

A week after that gist went up, I started mine. Four markdown files describing how I think, how I communicate, how I work, and what I value. Five months later it’s a few dozen domain pages, a daily log going back to April, a corrections file, and a weekly job that keeps proposing changes to it.

No tool owns it

The pitch for a system like this is usually “the AI remembers you.” That’s real, and it’s the smaller half.

The return has been that my tools changed and my memory didn’t.

Over five months the assistant I work in day to day changed more than once. One IDE was my daily driver, then my usage of it went to zero over a couple of weeks while a different assistant picked up every session. Harnesses got swapped. One of them retired and was replaced by a successor with a different name and a different config format. Each of those could have been a small amnesia event: a re-briefing, a rebuilt set of preferences, another round of explaining the same standards to something new.

None of them were, because the memory never lived inside a tool. It’s a directory of markdown on disk, synced between my two machines, and every new assistant gets pointed at the same path. The schema document at the root tells whatever shows up how the place is organised and what it’s allowed to touch. Switching harnesses became a config change instead of a migration.

Tools change every few months. Memory shouldn't have to.

That was the intent from the beginning, and it’s written into the schema. There’s a portability section listing every tool I expected to read the vault, on the explicit assumption that any of them could be replaced. It ends on the line I still think is the most useful sentence in the file:

Any future tool: if it reads files, it reads me.

Nothing in here is allowed to depend on a particular runtime, a database, an export format, or a vendor’s memory feature.

The shape, and the one place I diverge

Three layers of my own, cutting differently from the gist’s, and the boundaries between them are the only design decision that really mattered.

Identity. A curated description of how I work. Four files. Read at the start of every session.

Knowledge. Domain pages. Systems, processes, decisions, people. Agents may update these, and are asked to flag significant changes.

Memory. Daily logs, a corrections file, an inbox of pending observations. Agents write here freely.

Two scheduled jobs move material upward. A weekday digest scans my sessions, tickets, chat and mail and writes one dated log. A Sunday job reads the week, updates the knowledge pages, and proposes candidate additions to the identity layer. That second job is essentially Karpathy’s Lint operation with a promotion step bolted on.

Here’s where I break from the pattern. The gist is emphatic that the human doesn’t write the wiki: “You never (or rarely) write the wiki yourself — the LLM writes and maintains all of it.” For research notes that’s exactly right; bookkeeping is the burden and the model is tireless.

For a description of a person, I think it’s wrong. My identity layer is hand-written, and agents are forbidden from editing it unless I explicitly ask. Everything else follows the gist. The reason is an asymmetry of cost. A stale entity page in a research wiki gives you a slightly wrong answer. A self-description drifting on its own gives you confident, plausible statements about your judgement that you never made and wouldn’t endorse, and every session reads that file first. An agent proposing “here’s a trait I observed, do you agree?” is useful. An agent quietly editing who I am is not a feature.

So the model proposes and I ratify. Candidates land in an inbox with a confidence note and wait for me.

What it’s for, and what it isn’t

Systems knowledge lives elsewhere, and that’s the right place for it. How a system works was never mine to hold. It belongs to the team that owns it, the org that depends on it, and the company that paid for it. That knowledge has to survive me changing teams, be correctable by anyone who finds it wrong, and be trusted by people who have never met me. A personal vault offers none of those properties, so documentation, process docs and shared knowledge bases carry it, as they should.

What this holds is narrower and isn’t written down anywhere else: how I work. My standards, the corrections I’ve already had to give, the patterns in my decisions, the trail of what I shipped and when. Keyed to a person, deliberately. Five months of running it confirms exactly that shape: a rich picture of working method and almost none of my domain’s business logic, which is the right outcome for something aimed at one person’s continuity rather than a team’s knowledge.

Three other things it turned out to be good for:

Corrections stop repeating. Roughly one a week gets written down. Things like an accuracy number that includes inconclusive verdicts is not precision, or never assume how a system behaves, go and check. Not a dramatic rate. But each one is a conversation I don’t have to have twice, and the file is the cheapest thing in the whole setup.

An accomplishment trail beats my recollection. The digest harvests achievements with dates and links as they happen. Anyone who has tried to reconstruct six months of work the week before a review knows why that matters. It’s the least clever part of the system and possibly the highest return.

The interview itself was worth it. Half the value of the identity layer arrived while writing it. Being asked “what makes you lose trust in someone’s work” and having to answer in full sentences is clarifying in a way that vaguely believing it never is.

What maintenance actually costs

Karpathy’s argument for why this works is that LLMs don’t get bored, so the bookkeeping cost goes to near zero. Mostly true. Three places where it isn’t:

A human gate rate-limits truth as well as error. One of my identity files claimed a particular IDE was my daily driver for everything, and stayed wrong for three weeks after it stopped being true. The weekly job flagged it at five days, then ten, then fifteen, and couldn’t fix it, because the rule protecting that layer is the same rule that lets it go stale when I don’t act. Any human-in-the-loop design inherits the human’s backlog. Choose which files really need the gate, and keep that set small.

Collectors outlive their sources. One of my scanners kept running against a tool I had stopped using, returning nothing week after week while its state file grew anyway. Nothing was broken. It just wasn’t earning its place, and nothing was watching for a component that had quietly stopped mattering.

Abandonment is silent. I once documented a prototype mid-build, in confident present tense. The work later stopped. No announcement, it just ended. Months afterwards an assistant read that page, concluded the capability existed, and cited it as live infrastructure while arguing against a colleague’s proposal for something similar. Nothing generates a log entry when work simply stops, so a file’s modification date tells you when the words were last touched, not whether the thing is alive. Every page describing a system now carries an explicit status, because the absence of one defaults to “real” in the reader’s mind, and the reader might be a machine acting on my behalf.

Worth copying

Take the pattern from the gist. It’s better than what you’d invent from scratch, and it’s free.

Then: keep the human-owned layer small enough that reviewing it stays a five-minute job. Put a status on anything describing a system. Log the corrections from day one. And keep the whole thing as plain files on a path you control, because that independence turned out to be the best property of the whole setup, and in a year where the tools kept changing underneath me, it’s the part that mattered.