LLMs Don’t Understand — And That’s Fine.

The Mirror Has No Opinion

I've spent considerable time working alongside large language models, and the most clarifying realization I arrived at was also the simplest: these systems do not understand anything.

They compress, pattern-match, and reproduce. The output feels coherent because human writing is coherent, and LLMs are, at their core, a statistical portrait of that writing. There is no line of thought behind the response — only the weighted residue of lines of thought written by others.

This distinction matters not as a criticism, but as a precise description. A mirror is not inferior to a face. It serves a different function. The problem begins when we confuse the reflection for the thing itself.

Compiled Knowledge Is Not Curated Knowledge

There is a structural difference between compilation and curation.

Compilation aggregates. Curation selects, rejects, and arranges with intent — with a point of view. What LLMs do is closer to the former, executed at an unprecedented scale. The model ingests the written record of human reasoning and reassembles fragments of it in response to prompts. It does not know why one idea matters more than another. It cannot hold an opinion across time, because it has no experience of time.

The result is output that can be encyclopedic without being insightful, fluent without being precise, and useful without being original. These are not paradoxes. They are the natural properties of a system that has access to the map but has never traveled the territory.

What "New" Means in This Context

When an LLM produces a combination of ideas that appears novel, it is not generating novelty — it is surfacing latent combinations that already existed in the training data, dormant in the space between documents.

The recombination can be useful. It can even be surprising. But surprise is not the same as discovery. A kaleidoscope produces patterns no one has seen before. We do not credit it with creativity.

Genuine intellectual production involves:

• Selective rejection — knowing what to discard and why
• Positional commitment — holding a view that excludes others
• Accumulated experience — grounding abstractions in consequences encountered over time
• Authorial continuity — ideas that build on each other across a body of work

An LLM satisfies none of these criteria. It has no position it defends, no experience it draws from, and no body of work it is extending.

Each response is generated in isolation, without memory of what came before and without stake in what comes after.

The Useful Illusion

None of this means LLMs are without value. Compiled and well-indexed human knowledge, made accessible through natural language, is a significant practical achievement. The ability to retrieve, summarize, and recombine existing ideas with low friction has real utility — particularly in domains where the constraint is access to information rather than the production of it.

The danger is in mistaking the tool for a thinker.

When organizations begin substituting LLM output for reasoned analysis, when writers use it as a replacement for developed perspective rather than a research aid, the illusion becomes costly.

What gets lost is not productivity — it's the curatorial function that gives knowledge its shape and authority.

Understanding Requires Stakes

I find the concept of understanding inseparable from the concept of consequence.

To understand something is to have a stake in whether it is true, to have arrived at it through a process that could have gone otherwise, and to be changed by having arrived there.

None of these conditions apply to a language model. It has no stake in the accuracy of what it produces. Its process cannot go otherwise in any meaningful sense — given the same weights and the same input, it will produce the same distribution. And it is not changed by what it generates.

This is not a limitation to be solved with more parameters or better training data. It is a structural property of what the technology is. Accepting it clearly is the first step toward using it well.

The Human Function That Remains

What remains irreducibly human is the function of judgment under uncertainty — knowing which questions deserve sustained attention, which answers to reject despite their fluency, and which ideas to pursue despite their apparent incompleteness.

LLMs can assist the process of thinking. They cannot replace the thinker, because the thinker is not primarily an information-processing unit. The thinker is the one who decides what the information is for.

The written record that LLMs are trained on was produced by people who had stakes, made choices, and lived with the consequences of their ideas. The model inherits the vocabulary and structure of that record without inheriting its weight.

Recognizing this gap — clearly, without disappointment — is what allows the tool to be used precisely.