The Rothko Test for AI

Why language models generate the right answers without undergoing the question

Mark Rothko Orang and Yellow 1956

In a previous essay, I introduced the beholder’s share — the art historian Alois Riegl’s term for the active role the viewer plays in making sense of what they see. Ernst Gombrich called it “the act of completion by the beholder.” The core claim: a painting doesn’t deliver meaning, it invites it. The viewer’s prior knowledge, emotional history, and expectations fill in the rest. This isn’t a bug in human cognition. It’s the mechanism by which meaning gets made at all.

I ended that essay on a cautionary note — beholder’s share can introduce bias, distort communication, enable manipulation. All true. But I left a more basic question untouched: is completion the same thing as interpretation?

Large language models are forcing the question, because they complete brilliantly and constantly. The question is whether that’s the same act Gombrich was describing, or something that only looks like it from the outside.

The Pipe Problem, Revisited

Recall Magritte’s The Treachery of Images. Below a meticulously rendered pipe, the caption reads: Ceci n’est pas une pipe. This is not a pipe.

The painting works because it creates a contradiction you feel. Your visual system confidently classifies the object. Then the text pulls the rug out. There’s a moment of real disorientation as you navigate between representation and reality. Magritte is exploiting your beholder’s share against itself.

Now ask: does a large language model experience this?

An LLM can tell you that the painting is about the gap between representation and reality. It can explain the French, connect the work to semiotics, situate it in the history of surrealism. The output is accurate, often impressive.

But something is missing. The model was never fooled by the pipe in the first place. There was no expectation that got violated, no tension that needed resolving. The contradiction didn’t land anywhere — it was simply described.

It completed the prompt. It did not interpret the painting.

Having, Doing, Being — Applied to a Painting

In an earlier essay, I borrowed Aristotle’s three economies of human activity — poiesis (making), praxis (acting), theoria (contemplating) — and mapped them onto what John Vervaeke calls having, doing, and being. The argument there was that AI disrupts having almost completely, barely touches doing, and cannot touch being at all, because being is not a performance but a process: the sediment of sustained attention, accumulating over time, that makes you the kind of person who can see something clearly.

That framework turns out to fit the Magritte case exactly.

The having with respect to the painting is the information about it — the facts, the references, the correct account of what the work is doing. An LLM has all of this, instantly, more completely than most art historians.

The doing is the encounter itself — standing in front of the work, reading the caption, feeling your own categories wobble. This is praxis: not producing an output, but undergoing something.

The being is what the encounter does to you afterward — the way your sense of the relationship between image and word is slightly different than it was before. This is theoria in Aristotle’s strict sense: not information acquired, but a change in how you see.

A model can supply the having. It cannot undergo the doing. And without the doing, there’s no being to be changed. The model’s account of the painting is real, in the sense that it’s accurate. But it’s an account produced without ever having been in front of anything. There was no encounter to interpret — only a prompt to complete.

This is the distinction I want to sharpen: completion can fully substitute for having. It cannot substitute for doing, and it certainly cannot manufacture being. Gombrich’s “act of completion by the beholder” was never describing the having. It was describing the doing — the active, felt, time-bound act of meeting a stimulus with everything you bring to it.

The Rothko Test

Mark Rothko No 3/no 13 , 1949

Rothko’s color field paintings make the same point in a more extreme form, because Rothko removes almost everything that having could grab onto.

There’s no pipe to be fooled by, no narrative to follow, no iconography to decode. What remains is large fields of color, soft edges, a strange luminosity. And people still weep in front of them. Others feel oppressed. Others feel nothing at all, which is its own kind of data.

Almost everything here is coming from the viewer. Rothko’s paintings function less like representations and more like constraints — they narrow the space of possible responses without determining which one you’ll have. They are tuned to resonate with whatever the beholder brings, and if the beholder brings nothing, there’s nothing to resonate.

This is why Rothko is the harder test than Magritte. Magritte at least gives a model something to have — a joke, a philosophical point, a citation trail. Rothko gives it almost nothing to have. What’s left is pure doing, or nothing.

A model shown a high-resolution image of a Rothko can produce fluent commentary: composition, period, critical reception, the artist’s stated intentions. All true, all retrievable, all having. What it cannot produce is the part Rothko’s work is actually for — the encounter that either changes something in you or doesn’t.

To experience that requires a physical body to occupy the room and a lifetime of continuous memory to meet the canvas. There is no “in front of” for a system with no place to stand and no continuity for the painting to interrupt.

The beholder’s share requires a beholder who can be changed by the encounter, not just informed by it.

The Other Side of the Loop

So far I’ve treated the painting as a finished thing the beholder encounters — a fixed stimulus that either lands or doesn’t. But the painting wasn’t always finished. Before there was a Rothko to stand in front of, there was Rothko standing in front of his own canvas, and that encounter deserves the same scrutiny I’ve been giving the viewer’s.

His color choices look intuitive from the outside, but they weren’t generated in one pass. They were the residue of decades of looking — at his own work, at what specific combinations did to his own felt sense, refining a private vocabulary of what worked and what didn’t. Each canvas was both an output and a test of his own perception. He looked at what he’d made, something landed or it didn’t, and that result shaped the next mark. His doing, as a painter, was a long-running encounter with his own emerging work, repeated for years before anyone else ever saw a single canvas.

This means the beholder’s share doesn’t start with the viewer. The painting a viewer eventually stands in front of is already the compressed trace of someone else’s interpretive loop — Rothko interpreting his own marks, being changed by what he saw, adjusting, and doing it again. What the viewer receives is interpretation downstream of interpretation, not a raw stimulus meeting a beholder for the first time.

At first glance, a language model can run a version of this loop. It can generate-evaluate-adjust; it can produce an output, receive a reward signal, and update its weights. Reinforcement Learning from Human Feedback (RLHF) is structurally a loop of exactly that shape. If Rothko’s process is just “generate something, see what it does, adjust,” why doesn’t a sufficiently long training loop count as the same thing?

The distinction lies in what the loop is optimizing against.

An AI optimization loop is guided by an external reward function—it measures its output against a statistical distribution of human preferences or external data. It is trying to match an outsourced definition of success.

Rothko’s loop, by contrast, was an alignment check against an internal, somatic state. He modified the canvas because a specific juxtaposition of colors caused a physical or emotional shift within his own living frame. He was navigating his own horizon of meaning, adjusting the paint until it resonated with an internal register.

This is where frame revision and genuine surprise come from: they are products of deep personal commitment that cannot be outsourced. The model updates its parameters to satisfy an external rule; the artist transforms the canvas to resolve an internal encounter.

Why This Isn’t Just About Art

The same gap — having without doing, information without encounter — shows up wherever language models are deployed on anything that matters to a person.

A model can produce a clinically sound summary of a medical history. It can draft a legally coherent argument. It can generate a response to a difficult email that sounds like it came from someone who cares. In every case, the having is excellent and arriving fast. But the doing — the part where a particular person, in their particular situation, meets the material and is changed by what they find — never happens inside the model. It happens, if it happens at all, in the human reading the output.

This is not a complaint about accuracy. The outputs can be right. It’s a claim about what kind of act is taking place. Producing a correct account of something is not the same act as interpreting it, even when the two produce identical text.

What This Doesn’t Settle

I want to resist ending with the easy move — “so AI can’t really understand anything, humans are still special.” That’s too fast, and it skips past harder questions.

What does it actually take for an encounter to count as doing rather than completion? I’ve gestured at it — being fooled, being moved, having something land — but I haven’t given a clean criterion, and I’m not sure one exists yet. It’s also not obvious that every human encounter with art clears this bar either; plenty of gallery visits are pure having, a checklist of works seen, no doing involved at all. The having/doing/being distinction may turn out to be a matter of degree rather than a hard line, in which case the question becomes which conditions tip an encounter from one register into the other — for humans and for anything else.

What I think is solid: the beholder’s share was never just about producing accurate accounts of stimuli. It was about an encounter that changes the one having it. When a human is transformed by an artwork, that transformation ripples outward into the continuity of their life history—altering how they leave the museum, how they speak to their family, how they face their own mortality. A model’s update remains mathematically isolated to its performance on a task distribution. It has no continuous life for the transformation to ripple into.

Until a system can be the kind of thing that’s changed by what it processes — not just updated, but transformed in the way that sustained attention transforms a person — completion and interpretation will keep producing the same words while doing two different things.

This essay is a follow-up to Sense-making and the Beholder’s Share and draws on the framework from Having, Doing, Being in the Age of Machine Minds.