Give Your Writing a Brain

The shelf and the brain

Most bodies of writing are shelves. The essays are all there, indexed by date, searchable by keyword — and completely inert. A reader with a real question (“does this apply to a company like mine?”) has to become a librarian first: guess the right terms, skim five posts, assemble the answer themselves. The writing knows things. It just can’t tell you.

The shift is small to describe and large in effect: put a retrieval-grounded answer layer on top of the corpus. Not a chatbot that makes things up — an engine that answers only from what you actually wrote, cites the source every time, and says “the writing doesn’t cover that” instead of inventing. Then expose it where people already are: inside ChatGPT and Claude as a connector, and as a chat box on the site itself.

I built exactly this for this corpus — roughly seventy essays, a 380-term glossary, the playbooks and standards pages — over an afternoon, on infrastructure that costs roughly nothing at this traffic. This is the whole recipe, including the parts that went wrong. If you have a shelf, you can give it a brain.

The shape: one brain, two mouths

Resist the urge to make this complicated. The entire system is one small server that does four things — index, retrieve, generate, and expose — and everything else is a surface on top of it.

• Index: turn the corpus into searchable vectors + text, and re-run it automatically whenever you publish.
• Retrieve: given a question, pull the handful of passages most likely to contain the answer.
• Generate: hand those passages to an LLM under strict rules — answer only from these, cite them, refuse otherwise.
• Expose: one HTTP endpoint for programmatic use (the MCP connector), one for the site chat, one for a semantic search box.

I ran all of it on a single Cloudflare Worker — Vectorize for embeddings, D1 (SQLite) for the text and keyword index, KV for rate limits and caching, Workers AI for the model. Pick your own stack; the shape is what matters. Now, part by part.

Step 1 — The corpus is already structured. Chunk it that way.

Retrieval works on chunks, not whole documents — a 2,000-word essay is too coarse to match a specific question against. The instinct is to slice by a fixed character count. Don’t. Your writing already has structure: headings. Chunk on those, carry the metadata (title, URL, section heading, category, date), and each chunk arrives pre-labeled with where it came from — which is exactly what you need for citations later.

Two decisions here pay off later. First, keep a content hash per chunk so re-indexing only touches what changed — publishing one essay shouldn’t re-embed the whole corpus. Second, wire the indexer into your deploy pipeline, not a separate cron you’ll forget. When publishing is the training event, the brain is never stale and there’s nothing to maintain. That single property — the corpus updates itself — is most of why this is worth building rather than paying for a SaaS that silos your content.

Step 2 — Retrieval is a two-lane problem

Here’s the mistake almost everyone makes: they reach for vector search alone. Embeddings are magic for meaning — they’ll connect “are my retail-media numbers real?” to an essay titled iROAS Is Not a Number, It’s a Negotiation that shares not one keyword with the question. But they’re mediocre at exact terms — acronyms, product names, a specific phrase — where old-fashioned keyword search wins. Use both, then fuse the rankings.

Fusing is simpler than it sounds. Reciprocal rank fusion just says: a chunk’s score is the sum of 1/(k + rank) across every list it appears in. No weight-tuning, no calibration — a passage that both lanes rank highly floats to the top; one that only appears deep in a single lane sinks. Take the top handful and pass them on. This one technique closes most of the gap between a demo that impresses you and a system that actually finds the right passage on a real, oddly-worded question.

Step 3 — Grounding is the entire game

This is where a knowledge tool is won or lost. A model handed some passages and a question will happily blend what it retrieved with what it already “knows” from training — and the moment it does, you’re shipping confident, plausible, uncited fabrication under your own name. The whole discipline is forcing the model to answer only from the passages, cite them, and refuse when they don’t cover the question.

Three techniques make grounding real, not aspirational:

• Prompt rules are necessary but not sufficient. Tell the model to answer only from the passages, cite them with markers, refuse otherwise, and — for a personal corpus — speak about the author, never impersonate them. Good models mostly comply. “Mostly” is the problem, which is why the next two exist.
• Map citations server-side. The model emits markers like [S2]; your code translates those to real titles and URLs from the passages you actually retrieved. The model never writes a URL on its own authority, so it can’t invent one.
• Tier your sources, and filter the output. If some material is private (drafts, unlisted pieces), let it inform answers as “background” but tag it so it can never be cited, named, or quoted — then run a server-side filter that strips any sentence referencing it. Belt and suspenders: the prompt asks the model to keep secrets; the filter enforces it when the model forgets.

One more piece belongs here: detecting a refusal. Models decline in prose (“the passages don’t mention…”) while still emitting citation markers, so you can’t infer refusal from “no citations.” Anchor the check on the answer’s opening — a real answer starts with substance; a refusal starts by disclaiming — and when you detect one, return a clean, on-brand “not covered” message instead of the model’s clumsy internal framing. Then log the question. That log becomes the most valuable output of the whole system.

Step 4 — Publishing makes it smarter (the loop)

The reason to own this rather than rent it is compounding. Three loops run once it’s live, and none of them need you.

The corpus loop: publish, auto-index, done — the brain never drifts from the writing. The quality loop: a small golden set of test questions (definitional, cross-essay synthesis, “should decline” traps) you run before shipping any prompt or retrieval change, so quality is measured, not vibes-based — a tweak that quietly breaks grounding shows up as a failing case, not a customer complaint. And the gap loop: the log of unanswerable questions feeds a monthly pass that ranks genuine, repeat-asked, on-topic gaps into an essay backlog. Your readers tell you exactly what to write next — in their own words, weighted by how many asked.

Two mouths: the connector and the chat box

One brain, two ways in — and the second one is the interesting one.

The obvious surface is a chat box on your site: a floating button, a panel, questions to the same engine, cited answers rendered with links. Keep it lazy — the button is static HTML and the server is only called when someone actually asks, so it costs your page-load budget nothing.

The less obvious, higher-leverage surface is a remote MCP server — the Model Context Protocol, the emerging standard that lets any AI client call external tools. Expose your brain as an MCP endpoint and it drops into ChatGPT, Claude, or any MCP client as a connector: now people query your entire body of work from inside the assistant they already use, without visiting your site at all. Tools like ask_evgeny (grounded answer), search_writings, get_essay, get_glossary_term. This is distribution most content sites never get — your writing, available at the point of thought, in someone else’s tool. Gate it with a free key issued by email, and the distribution channel doubles as a lead list.

The four bugs that actually cost me an afternoon

Every clean architecture diagram hides the debugging. Here’s the honest part — the specific things that broke, because you’ll hit versions of them too:

1. The vector store rejected my IDs. Long essay slugs blew past the index’s 64-byte ID limit and the whole batch failed. Fix: hash the slug into a fixed-width ID. The lesson generalizes — don’t use human-readable strings as primary keys in systems with length caps; hash and keep the readable version in your own metadata.
2. Verbose pages buried the essays. My reference pages are keyword-dense, so raw retrieval kept surfacing standards docs over the thought-leadership essays that actually answered the question. Fix: a light doc-type boost — essays and their summaries up, reference pages down — so the writing you’re proud of wins on thematic questions while pages still win on direct lookups.
3. Graceful refusals counted as answers. The model would decline and emit citation markers, so my “did it answer?” check (based on citation count) called a polite “I don’t know” a successful answer. Fix: detect refusal from the answer’s opening sentence, not its citations, and route it to the gap log.
4. The vocabulary-mismatch miss. A question phrased with none of an essay’s words (“privacy as an engine, not a brake”) sat right at the retrieval cutoff. Fix: widen the candidate pool before fusion and give the model a few more passages of context, so a borderline-but-correct source still gets a seat at the table.

None of these show up in a weekend demo. All of them show up the first time a real person asks a real question. Budget the afternoon.

What it costs, and when it’s worth it

At this scale — a few thousand chunks, low-hundreds of questions a month — the honest number is near zero: embeddings and generation on a serverless AI tier, a vector index and a database both inside free-or-cheap allowances, one worker. Per-IP and per-key rate limits plus an answer cache keep a viral day from becoming a surprise bill. You can start on a free model tier and flip to a premium one with a config change if the answers deserve it.

It’s worth building when three things are true: you have a corpus (dozens of pieces, not five), the content is evergreen enough that answers stay useful, and you’d rather own the intelligence layer than rent one that silos your writing behind someone else’s login. If you’re there, the whole thing is an afternoon and a handful of files.

The attention economy taught us to publish and hope the archive gets found. This is the other move: make the archive answer. You already did the hard part — you wrote the things. Giving them a brain is mostly plumbing, and the plumbing is worth it, because a shelf that can talk back is a fundamentally more useful thing than a shelf.

Want to see it running? Ask this corpus anything — the chat box in the corner, or add it to your own ChatGPT or Claude as a connector.

This build is also a practice area: the same entity, citable-content, and machine-surface engineering, applied to AdTech, MarTech, and data companies — AI Visibility & GEO Advisory.