The graveyard of second brains

I had a second brain once. Obsidian vault, a CouchDB LiveSync backend, even a weekly agent that summarised my notes. It worked — for a while. Then the sync started fighting itself across my laptop, the homelab, and my phone, and the day syncing becomes a chore is the day you stop opening the thing. The notes were still there. I just never looked at them again.

That’s how most second brains die. Not from bad notes — from the plumbing. The sync breaks, or the upkeep outpaces the payoff, or the whole thing is trapped in one app’s database and moving it feels like surgery. The knowledge was never the problem. The container was.

So when I rebuilt it, I started from the failure modes, not the features.

What I actually wanted

Three things, none of them “more notes”:

  1. Memory I share with my AIs. Every time I open a fresh Claude session, it starts from zero — I re-explain my homelab, my projects, what we decided last week. I wanted a place both of us read and write, so the context survives the session.
  2. Something that outlives any tool. No lock-in. If the app of the month dies, my brain shouldn’t die with it.
  3. Sync that can’t rot. The thing that killed v1.

The one decision that matters

The store and the intelligence are different layers, and only the store is sacred.

The store is a folder of plain markdown in git. That’s it. Human-readable, diffable, greppable, yours. Everything clever sits above it and is fully rebuildable:

L5  Visualisation   3D graph, Obsidian, whatever reads markdown
L4  Automation      scheduled "gardener" runs
L3  Agent interface MCP servers — search, graph, note CRUD
L2  Index           SQLite: full-text + vectors + materialised edges
L1  Structure       typed frontmatter + [[wikilinks]]
L0  Substrate       markdown files in git   ← the only thing that's truth

Delete L1–L5 and nothing is lost — you rebuild them from L0 with one command. That property is the whole design. The index can corrupt, the embedding model can change, the viewer can break (mine did, spectacularly — that’s another post), and the knowledge doesn’t care. It’s text in git.

And sync is just git pull. No LiveSync daemon to wedge itself, no proprietary replication. The exact thing that killed v1 is now the most boring, battle-tested part of the stack. Three devices, one git pull, done.

Search that explains itself

The retrieval layer is deliberately not “throw it all at embeddings.” It fuses three signals — keyword (BM25), vector similarity, and graph expansion (pull in the neighbours of strong hits) — and every result reports which signals fired.

exo search "hybrid retrieval"
→ hybrid-retrieval   matched_on: [bm25, graph]

That matched_on matters more than it looks. An embeddings-only system gives you a ranked list and no reason — you can’t tell a real match from a vibe. For a brain I’m supposed to trust over years, “why did this surface?” is a feature, not a nicety.

The AI is a librarian, not a hoarder

Here’s the part I care about most. The AI doesn’t just read the brain — it writes to it. Through an MCP server it can search, walk the graph, and author notes. But under a hard rule: every write is a reviewable git diff.

It searches before it writes (extend a note, don’t spawn a duplicate). It links instead of piling. A scheduled “gardener” pass finds orphaned notes and stale summaries and proposes fixes — as commits I can read and git revert if it gets something wrong. No black-box mutation of my memory. Just a librarian that files things while I’m asleep and leaves a paper trail.

So now “what am I building?” is a question with an instant, honest answer: a single map note, kept current, that every project links into. I ask, the AI pulls it, and neither of us has to remember.

Why not just…

  • Obsidian alone? It’s a lovely viewer — and I still use it as one. But it can’t give an agent structured read/write or explainable retrieval, and its sync is what burned me. Here Obsidian reads the same markdown; it’s a window, not the house.
  • Embeddings RAG? Opaque and one-directional. It can rank, but it can’t tell you why, and it can’t write back. This is transparent and bidirectional.
  • Notion / a SaaS brain? Lock-in by design. git clone is my backup and any text editor is my fallback.
  • A graph database? Unnecessary infra. The graph lives in the wikilinks; SQLite just materialises it. I’ll add Neo4j the day my queries actually outgrow a single file, and not a day sooner.

What it changes

The vault is small still — that’s fine; it grows by use. But the loop already pays off: I work, the AI checkpoints decisions into markdown, and the next session — fresh model, no memory of its own — searches the brain and is caught up in seconds. The knowledge stopped living only in my head and in dead chat logs.

I’m a team of one. There’s no colleague who remembers why I made a call six months ago, no handover doc someone else maintains. Continuity isn’t a nice-to-have; it’s the whole job. A second brain that the AI helps keep alive — and that I can git clone onto any machine in thirty seconds — is the first version of this idea that I actually trust to still be here in five years.

The notes from v1? They’re sitting in a folder, waiting to be triaged into v2. This time I’ll still be opening it.