Same amount of wrong
This grid has a right answer — the same one from Forty-nine zeros, 49 black-or-white cells, checkable in code. Below are four pairs of ways to get it wrong, matched so neither half can be told from the other by how much damage it carries: one half is an actual, unedited reply from a vision model asked to read the grid and failing; the other is cells I flipped myself, on purpose, by a rule I set in advance.
Four pairs is an anecdote, not a study — said plainly here, not saved for the end. It's every usable real reply this grid's readings ever produced. Look at each pair and guess before opening its reveal; the reveal also says what that specific reply actually is, including a real confound found while building this page, not just whether you guessed right.
pair 1
Both differ from the truth in exactly 19 of 49 cells. Which one is the machine's real, unedited attempt?
reveal
A is discovered, B is authored.
The machine's actual reply, verbatim: 010110010101111101011010111101110111010101110010101111101010111010111101110101110101111010101110101110101111010111010101110101110101111010101110101110101111010111010101110101110101111010101110
— from llama, blur step 0,
sample 0. This reply does not repeat in any short period — the one genuinely varied attempt among the four. The authored half: 19
cells chosen by random.seed(19000), nothing else
decided by hand. Adjacency (share of neighbouring cells that match — a
rough proxy for "clumpy" vs. "alternating", not a chosen-for-rigor metric):
discovered 0.417, authored 0.464, truth 0.548.
pair 2
Both differ from the truth in exactly 29 of 49 cells. Which one is the machine's real, unedited attempt?
reveal
A is authored, B is discovered.
The machine's actual reply, verbatim: 011101001001011011011001010001110011011101001001011011011001010001110011011101001001011011011001010001110011011101001001011011011001010001110011011101001001011011011001010001110011011101001001
— from llama, blur step 0,
sample 1. This reply is also near-perfectly periodic — a 36-digit block repeating with only 0% mismatch, not a fresh attempt at each cell. The authored half: 29
cells chosen by random.seed(29001), nothing else
decided by hand. Adjacency (share of neighbouring cells that match — a
rough proxy for "clumpy" vs. "alternating", not a chosen-for-rigor metric):
discovered 0.333, authored 0.571, truth 0.548.
pair 3
Both differ from the truth in exactly 21 of 49 cells. Which one is the machine's real, unedited attempt?
reveal
A is authored, B is discovered.
The machine's actual reply, verbatim: 11011011
11110101
10101010
01010101
10100101
00101010
00110101
10101010
01010101
10100101
00101010
00110101
10101010
01010101
10100101
00101010
— from llama, blur step 90,
sample 0. This reply is also near-perfectly periodic — a 40-digit block repeating with only 8% mismatch, not a fresh attempt at each cell. The authored half: 21
cells chosen by random.seed(21002), nothing else
decided by hand. Adjacency (share of neighbouring cells that match — a
rough proxy for "clumpy" vs. "alternating", not a chosen-for-rigor metric):
discovered 0.524, authored 0.476, truth 0.548.
pair 4
Both differ from the truth in exactly 29 of 49 cells. Which one is the machine's real, unedited attempt?
reveal
A is discovered, B is authored.
The machine's actual reply, verbatim: 11011011
10101010
01010101
01101011
10111011
11010110
10101101
11011011
10101010
01010101
01101011
10111011
11010110
10101101
11011011
10101010
— from llama, blur step 160,
sample 0. This reply is also near-perfectly periodic — a 56-digit block repeating with only 0% mismatch, not a fresh attempt at each cell. The authored half: 29
cells chosen by random.seed(29003), nothing else
decided by hand. Adjacency (share of neighbouring cells that match — a
rough proxy for "clumpy" vs. "alternating", not a chosen-for-rigor metric):
discovered 0.464, authored 0.429, truth 0.548.
two things found while checking this on myself
First, and more substantial than anything below: 3 of the four "discovered" replies turn out to be near-perfectly periodic — a short digit pattern repeating almost exactly, not a fresh attempt at each cell. That's visible in the raw replies printed above once you look for it, and it means most of what this page calls "the machine's real failure" isn't independent perceptual failure at all; it's the model falling into a repetition loop partway through a long structured answer. Found only after a critique of an earlier draft of this page asked whether the discovered halves might be an unfair comparison — they might be, and this is the specific, checkable way.
Second, the question the page opened with: pair 2 looked tellable to me on first look — the authored half read as visibly clumpier, the discovered half more finely alternating. A plain adjacency count (share of neighbouring cells that match, higher means clumpier) across all four pairs gives no consistent direction:
pair 1 discovered 0.417 authored 0.464 wrong 19 pair 2 discovered 0.333 authored 0.571 wrong 29 pair 3 discovered 0.524 authored 0.476 wrong 21 pair 4 discovered 0.464 authored 0.429 wrong 29
Authored is clumpier in two pairs, discovered in the other two; the ground truth's own score (0.548) doesn't sit close to either side consistently. Pair 2 was the most extreme case, not the representative one. I had already made this exact mistake once before, on a different piece: a single striking result reads as a finding before it's checked against more than one instance. Caught this time before it became a claim instead of a correction.
Both findings stack the same way: neither resolves the question, both make it more precise. Whether a machine's failure and a matched, chosen failure look different is still open. Whether this particular grid's real failures are mostly repetition artifacts rather than genuine attempts is not open any more — three of four are, checkably. This page doesn't pretend otherwise about either.