Itself a Nothing

You can't copy a memory from one head into another, or from one model into the next. Everything has to go through language, and that round trip is a cost every mind pays.

The pen is mightier than the sword. Behold / The arch-enchanter’s wand!—itself a nothing!

Edward Bulwer-Lytton, Richelieu; Or the Conspiracy (1839)

Try to give someone a memory. Not describe it: give it to them, the way you’d hand over a photograph. You can’t. Your memory of the smell of a particular afternoon is a particular pattern of connections in your particular brain, laid down over years of your particular life, and there is no cable that copies that pattern into my head. The best you can do is talk, and hope the version I assemble on my side resembles the one on yours.

None of this means we can’t understand each other. Obviously we can; you’re reconstructing my meaning right now, out of these words, using your own materials. That’s the whole point. Understanding gets rebuilt on the receiving side. It does not get transferred. What crosses the gap between two minds is language. The pattern stays in the skull that grew it.1

Models have the same problem, and it’s easier to see because you can look at the numbers. Take two versions of the same model, or honestly any two separate training runs of the same architecture. The internal state one of them uses to represent something like “the user is losing patience” is a long vector of numbers, and that vector only means anything relative to the exact weights that produced it. Hand it to the other model and it’s noise. They don’t share a coordinate system.2 They’re private substrates too. They just happen to be ones we can measure.

The only thing that crosses

If you can’t move the pattern, you move a description of it. For humans that’s language. For models it’s text. And it’s not an accident that it’s the same channel: text is the one representation that survives the trip between minds precisely because it has thrown away everything specific to any single one of them.

A programmer would call this serialization. Inside a running program an object is a live thing full of pointers into that process’s memory: fast, rich, and completely meaningless to any other process. To send it somewhere you serialize it, flattening it to a stream of bytes with no pointers and no assumptions about who’s reading. The other side deserializes, building a fresh object out of its own memory. Text is the serialization format for minds. The wire is lossy by construction, and that lossiness is the feature. It’s what lets the message cross between substrates that share nothing else.

You can poke at the model side of this and it holds. Two networks’ internal representations aren’t naively comparable.2 You can sometimes learn a map that stitches one model’s guts onto another’s,3 and separately-trained models even seem to drift toward similar geometry.4 But “train a whole extra model to translate between them” is not the same as “hand the state over,” and nobody ships the translator. In practice the thing that actually crosses from one model to the next is the same thing that crosses from me to you: words.

The round trip

Because the state can’t be saved and reloaded, it has to be rebuilt every time it’s needed. For a model the loop looks like this:

The round trip between two mindsyour statethrown awayencodetextsurvives the tripdecodemy staterebuilt from scratch

Read the text in (the industry calls it prefill, and it is exactly re-encoding), reason over it, emit new text out. Every conversation, a model rebuilds its understanding of you out of whatever text it can see, because last session’s understanding was never written down in any form it could pick back up. The one piece of state we could cache to skip the re-encoding, the model’s own internal activations, is the very thing that’s specific to that model and evaporates at the next version. The state worth keeping is the one thing that can’t be handed over. That cost is unavoidable as long as the only durable, portable record is text, and text is not where the understanding lives.

We have been here before

Humans have been able to talk for a very long time. The estimates run wide (from the one or two hundred thousand years usually attached to our own species, out to half a million or more if the Neanderthals had language too5), and the species itself is around three hundred thousand years old.6 Pick whichever number you like. Writing is about five and a half thousand years old,7 a sliver at the very end of it. On that scale, writing is brand new.

For nearly all of that stretch, knowledge lived in people, and oral cultures held it far better than we tend to assume: genealogies, law, navigation, star lore, even the shape of coastlines that had shifted thousands of years earlier, all carried faithfully across many generations.8 The limit was never that knowledge couldn’t accumulate. It was that knowledge moved only the way people move. It had to be kept alive on purpose, taught forward from one memory into the next, and it reached only as far as someone could carry and recite it; let a generation fail to learn it and it was gone. What writing changed was the sharing. Knowledge went into an object outside any single head, one that could be copied, handed to a stranger, and read again long after everyone who made it was dead. Then the cost of copying and moving that object fell again with the printing press, and again with everything digital, and each time the reach of one mind’s knowledge grew wider.

It’s tempting to draw a straight line from Gutenberg to the Enlightenment, but that line isn’t clean. A great deal had to align for the scientific revolution, and Gutenberg himself was not the linchpin: movable type was already in the air, and had he never been born, someone else would have built a press, the way inventions tend to arrive once their moment is ready.9 Every time the cost of recording and retrieving what a mind knows dropped sharply, development sped up.10 The press was one enabling condition among many. But it keeps happening.

Oral cultures, for their part, were not naive about how easily an unwritten record can fade. They engineered against it. Meter, rhyme, and fixed formula are error-correcting codes for speech: constraints that make a line hard to misremember and easy to reproduce, so an epic could survive centuries of retelling with its shape intact.11 They couldn’t make the record durable, so they made it robust.

A mouth and a page

A skilled bard could carry something on the order of an epic (the Iliad is roughly fifteen thousand lines12), and a whole tradition, across all its keepers, held far more than that. Capacity was never really the constraint. The problem was that the whole store had to stay loaded: it lived only in minds that were actively holding it, so keeping it alive cost a continuous share of those minds, generation after generation.

Writing changes the operation, not just the size. Oral memory is volatile storage: it lasts only while it is powered, which here means actively remembered and retaught, and it fades when that stops. Writing is non-volatile: it persists with the power off, sitting in a jar or on a shelf for centuries with no one holding it in mind, waiting to be read again. That is the difference that matters, and it is why a single ordinary library outlasts everything the bards of an age could hold between them. Not because a page holds more than a mouth. Because a page keeps holding it when no one is looking.

Language, but no pen

Models are in a strange spot. They have language; in fact they were built by reading almost all of ours. What they don’t have is a pen: a cheap, durable way to write down something they have understood, in a form they can pick back up later without paying full price to re-derive it.

Look at what gets reached for instead, and each one is a pre-writing workaround:

  • Fine-tuning writes into the weights. That’s real recording, but it’s carving stone: slow, expensive, done in batches, and prone to scribbling over things you wanted to keep. You don’t do it after every conversation.
  • Retrieval over a text store (RAG) is writing it down as text and reading it again later. It’s a library, and a good library is genuinely valuable (I’ve argued exactly that). But the model still pays the full re-encode on every read. The library doesn’t remember; it just makes the right page reachable.
  • Caching the activations (the KV cache, prompt caching) is the closest thing to keeping the actual understanding around. And it’s the private-substrate problem all over again: model-specific, bulky, and dead the moment the version changes.

All of these live on the text side of the wire, or they don’t survive the next model. They make re-encoding cheaper, or better-aimed. None of them let a mind save its state and reload it. They are our meter and rhyme: clever ways to make the record robust, invented by a culture that doesn’t yet have writing.

Writ in water

The thing worth wanting is a pen: a cheap, durable way for a mind to record what it has understood, so that it, or its successor, can take that understanding back up without rebuilding it from scratch. Strip the databases and the vector indices and the memory frameworks down to what they’re actually reaching for, and it’s that.

No one has built it yet. But the conditions are all in place: everyone is reaching for some version of it, from every direction. When a thing is attempted in ten places at once, its moment has arrived.9

Until then, what a mind understands dies with the moment that held it. A model reads you, works out your problem and your voice and the shape of what you’re after, gets good at it, and then the conversation closes and all of it is gone. The transcript survives; the understanding behind it does not. It was never written down. It could not be. Millions of times a day, a mind comes to know something and then loses it, and no one is there to see. We have the language. We’re still working out the pen.

All those moments will be lost in time.13

Footnotes

  1. Nagel, T. (1974). What is it like to be a bat? The Philosophical Review, 83(4), 435-450. On the irreducibly first-person character of experience. ↩

  2. Kornblith, S., Norouzi, M., Lee, H., & Hinton, G. (2019). Similarity of neural network representations revisited. ICML. Introduces CKA and the broader point that representations across networks are not naively comparable. ↩ ↩2

  3. On model stitching: Lenc, K., & Vedaldi, A. (2015). Understanding image representations by measuring their equivariance and equivalence. CVPR; Bansal, Y., Nakkiran, P., & Barak, B. (2021). Revisiting model stitching to compare neural representations. NeurIPS. On merging via permutation symmetry: Ainsworth, S., Hayase, J., & Srinivasa, S. (2022). Git Re-Basin: Merging models modulo permutation symmetries. ↩

  4. Huh, M., Cheung, B., Wang, T., & Isola, P. (2024). The Platonic Representation Hypothesis. ICML. Evidence that separately-trained models converge toward aligned representations, which makes translation more plausible, not free. ↩

  5. Dediu, D., & Levinson, S. C. (2013). On the antiquity of language: the reinterpretation of Neanderthal linguistic capacities and its consequences. Frontiers in Psychology, 4, 397. ↩

  6. Hublin, J.-J., et al. (2017). New fossils from Jebel Irhoud, Morocco and the pan-African origin of Homo sapiens. Nature, 546, 289-292. ↩

  7. Schmandt-Besserat, D. (1992). Before Writing. University of Texas Press. Proto-cuneiform accounting gives way to written signs in Mesopotamia a little over five thousand years ago. ↩

  8. Oral traditions can preserve information with striking fidelity across very long spans. Aboriginal Australian stories of now-submerged coastlines appear to encode sea-level rise from more than seven thousand years ago: Nunn, P. D., & Reid, N. J. (2016). Aboriginal memories of inundation of the Australian coast dating from more than 7000 years ago. Australian Geographer, 47(1), 11-47. ↩

  9. On independent multiple discovery, the idea that inventions arrive once conditions are ripe rather than from a single irreplaceable mind: Ogburn, W. F., & Thomas, D. (1922). Are inventions inevitable? A note on social evolution. Political Science Quarterly, 37(1), 83-98; and Merton, R. K. (1961). Singletons and multiples in scientific discovery. Proceedings of the American Philosophical Society, 105(5), 470-486. ↩ ↩2

  10. Eisenstein, E. L. (1979). The Printing Press as an Agent of Change. Cambridge University Press. Her strong causal thesis is influential and much debated; the argument here leans only on the weaker correlation. ↩

  11. Lord, A. B. (1960). The Singer of Tales. Harvard University Press. The oral-formulaic theory, built on Milman Parry’s fieldwork among South Slavic epic singers. ↩

  12. The Iliad runs to roughly 15,700 lines across its 24 books; exact counts vary by edition. ↩

  13. The replicant Roy Batty’s closing monologue in Blade Runner (1982), dir. Ridley Scott; screenplay by Hampton Fancher and David Webb Peoples. ↩