← Journal
NOTE 24·Results

The confirmatory test stopped at its first run: not interpretable

Protocol 023 was to test the plaids pilot on six fresh network instances of the cat circuit model. Its first run finished, but a declared integrity check found one of 3,600 input cells firing differently from the reference, and the declared early stop ended the protocol. The declared outcome is NOT INTERPRETABLE, and no confirmatory reading was made. An exploratory diagnosis afterwards found that exact repeats of the run's inputs did not reproduce the difference, which points to an execution fault on the laptop; its cause is untested and its rate close to unknown. The plaids question over instances is still open.

Protocol 023 was the confirmatory test that note 22 promised. It asked whether the plaids pilot's departure from a relay holds on average over six fresh network instances of the same 2 mm cat circuit model, instances that had never been built. Note 23 moved the model to an 8 GB laptop after our research server was lost, and published the SHA-256 of 023's declared text before any of its runs.

The protocol has ended. Its first run finished normally. A declared integrity check then refused that run's output, and the declared early stop ended the queue. The declared outcome is NOT INTERPRETABLE. No confirmatory reading was made. No other network instance was built.

The analysis wrote this text, fixed in the declared design before any run:

Protocol 023 cannot be read as declared: the early stop: the server check refused the export of pynn_seed 7 right after its run, for a fault taken to be shared by every instance, and the local check refused it too (the LGN lead-in identity: X_OFF 1799 of 1800 trains identical). No confirmatory reading is made. The data and the failure are kept; a rerun needs a new declared revision.

The design asks that one limit stand beside any outcome of this protocol, because its other outcome texts call the instances ones that "had never been built": "the server half of the seed search ran with round 3's script on 3 November 2026 at 00:29:38Z, the final script ran only on the Mac, and the server's files can no longer be read."

An exploratory diagnosis followed the outcome. It changes nothing in it. It is about an execution fault on the laptop that stopped a confirmatory simulation protocol, and it says nothing new about cat cortex or about plaids.

Spike raster titled 'Cell 2738 fired extra spikes in s7 until 14.3 s'. Four rows of vertical ticks over 0 to 15 seconds: the pilot reference, with three spikes in the first half second only, then s7, d5 and rate-r1. The d5 and rate-r1 rows are identical. The s7 row has the same bursts as d5 plus extra spikes, 41 of them marked in violet, spread from the start to about 14 seconds. Numbers above count s7's spikes minus d5's in each second: between +1 and +7 in each of the first 14 seconds, then +0. A dashed line at 14,344.6 ms marks where the s7 train becomes identical to d5's. A lower left panel shows the first 504 ms: the reference, d5 and rate-r1 have spikes at about 73, 120 and 271 ms, and s7 has five, at about 73, 77, 116, 153 and 270 ms. A lower right panel shows 13.9 to 14.7 seconds, where the three rows are identical after the dashed line.
Figure 1. Spike times of one LGN OFF cell, number 2738, in protocol 023's run s7 and in two later exact repeats of its inputs, d5 and rate-r1, over the first 15 s of the 35 s stimulus sequence. The top row is the pilot's lead-in reference, which holds only the first 504 ms. Violet, taller ticks are the 41 s7 spikes with no d5 spike within 5 ms; the numbers above give s7's spikes minus d5's in each second. The dashed line marks 14,344.6 ms, from which s7's train equals d5's to the end of the sequence. Lower left: the lead-in that the protocol's check compared. Lower right: the trains around the point where they become equal again. The figure shows that this one cell fired differently in s7 than in repeats of the same inputs, for 14.3 s. It does not show the cause, or anything about cat cortex. It is exploratory and changes nothing in the declared outcome. Plotting code · SVG · Spike times

What 023 asked

The pilot, protocol 022, ran one network instance and found that the model circuit did not answer two gratings at once with the sum of its answers to each alone. Its declared outcome was GO, for three measures: suppression (XOS), dominance (Dc) and readout shift (ds). One instance is one set of cell positions and connections. Protocol 023 asked whether the departure holds on average over new ones.

  • Instances. Six slots, PyNN seeds 7 to 12, and two reserves, 13 and 14. Each seed gives new cell positions and connections on the model's one orientation map. Each instance had its own declared stimulus order.
  • Measures. The pilot's three, with the same definitions and the same margin of 0.10.
  • The test. For each measure, a one-sided t-test of the six instance estimates in the pilot's direction, with Holm's correction over the three measures at a family-wise 0.025.
  • Outcomes. CONFIRMED, NOT CONFIRMED or NOT INTERPRETABLE, each with its text fixed in advance.
  • Integrity checks. After each run, a blind checker reads only the run's records, the 30 % single gratings and the LGN lead-in. It computes no plaid quantity. It refuses an export that is not the declared run, whose LGN firing is outside a band around the pilot's, or whose LGN lead-in differs from the pilot's.
  • The early stop. The first refusal ends the protocol at once, read as a fault shared by every instance, so that a systematic fault costs one run and not the whole queue.

It was declared on 7 November 2026 at 20:25:30 UTC. The declared text is now published in protocol-023-public, as note 23 said it would be. That copy cannot hash to the value note 23 published: four of its lines held six copies of the laptop's home folder path, which this site never publishes, and they read <home> there. The frozen manifest, published byte for byte, pins the declared text at that value, the queue's launch record names that manifest, and the original is kept unchanged in our private workspace. The package's README says how to check this.

What ran

The design was frozen at 00:46:40 UTC on 8 November, with every test passing, and the queue started 41 seconds later. Its first slot was pynn_seed 7.

Run s7 Value
Time (UTC) 00:47:21 to 01:53:28, 1 h 06 min
Exit 0
Memory peak 3.186 GiB, under the 3.7 GiB cap
Connections 14,712,684
Export 24.5 MB, the same SHA-256 on the laptop and after the fetch

Why the early stop fired

The lead-in is the 504 ms of grey before the first grating. The design gives two grounds for checking it. The LGN's random noise is the same stream in every instance, cell by cell. And before the declaration, two earlier instances and the pilot already had identical LGN trains in the lead-in on the server, as did the laptop's repeat of the pilot in note 23. The trains diverge once a grating is shown, because the cell positions differ. So each of the 3,600 LGN cells should fire in the lead-in exactly as in the pilot, at the export's 0.01 ms precision.

Check Role s7
The export is the declared run refusal passed
LGN firing at 30 % contrast, within 0.90 to 1.10 of the pilot's refusal passed: 0.987 (ON), 1.007 (OFF)
Every LGN spike in the lead-in equals the pilot's refusal refused: 1,800 of 1,800 ON cells, 1,799 of 1,800 OFF cells
Layer 4 response at 30 %, within 0.80 to 1.25 of the pilot's reported only 1.085 (45° cells), 1.282 (135° cells)

The one cell that differed is OFF cell 2738. In the lead-in it fired at 72.7, 77.2, 116.4, 152.5 and 270.2 ms. The reference has 72.9, 119.5 and 270.6 ms.

The check ran twice with the same frozen code: right after the run (the "server check", a name kept from the server design; it now runs on the laptop) and after the fetch (the local check). Both refused, with identical check values. The layer 4 values have no decision role, and they come from a run whose input was faulty in one cell.

By the declared rule, that was the end. In the design's words, "the first refusal of the server check is taken as systematic: the protocol is NOT INTERPRETABLE with that refusal as its condition, once the local check of the same export, after the fetch, has refused it too." The queue stopped at 01:53:34 UTC, the local check refused at 01:55:30 UTC, and the analysis wrote the outcome at 01:55:52 UTC.

The design knew the cost. Its risk 14 reads: "A fault of that one instance (an export damaged as it was written, say) would then end the protocol where the replacement rule could have replaced it; the rule accepts that cost for the server time it saves when the fault is shared."

What an exploratory diagnosis found, after the outcome

The diagnosis began four minutes after the outcome was written. Every claim below is labelled: measured (read from a file or a run), inferred (reasoned from measurements) or untested. A skeptical review then read it, with the verdict "holds with corrections". The corrections are applied here and in its public summary.

First, a procedural gap. The design's own fences say: "Any exploratory reading of 023 data gets its own manifest, frozen before it reads any 023 export." The diagnosis had no such manifest. It read s7's export, its output folder and the queue's records without one. Its records say it computed no XOS, Dc or ds, and the outcome was already written. That limits what the gap can have touched. It does not close it, and a new declared revision will have to say how it treats this diagnosis.

  • The repeats (measured). The laptop ran s7's exact inputs again: d1, the first 5 steps, and d5 and rate-r1, the whole sequence. All three gave the reference lead-in. d5 and rate-r1 agree on all 18,600 neurons' spike trains over all 35,133 ms. One more repeat, rate-r2, stopped at the disk floor after 18 minutes, because another program filled the laptop's disk. It is not a model result.
  • The difference (measured). Against d5, s7 differs in one LGN cell only, 2738. It fired 47 extra spikes, more in every second, until 14,344.6 ms. From there to the end of the sequence its last 327 spikes are identical to d5's (Figure 1). Every cortical spike train differs over the sequence; the first cortical difference is at 75.2 ms, after the cell's first differing spike at 72.7 ms. The cortex differs because of that cell, the only input that differs (inferred).
  • Not the inputs (inferred). The same declared inputs gave the reference three times, so s7's difference was not a fixed result of seed 7, its stimulus order, the files or the recorded environment. A recomputation outside the simulator found that the lead-in input is the same for every cell, under both seeds and both stimulus orders (measured). One limit: s7's receipt does not record the shell's search path, so its equality with the repeats is not verified.
  • What fits (inferred, found after the fact). The window matches one 16 KiB memory page, 2,048 input values, at the start of one cell's input buffer. The data fit cell 2738's luminance input reading as zero for those values, about +60 pA of extra drive.
  • The cause (untested). Three places could have held the zeros and fit equally well: the cell's long-lived input buffer, two temporary arrays of the same size in the same computation, or a pointer moved by one page. Candidate mechanisms are macOS's handling of freed memory, under a setting our memory wrapper adds, and memory compression or swap. A hardware error is unlikely, because a whole page of exact zeros looks like a virtual memory event, not a flipped bit. A test that zeroes that input on purpose is designed and has not run.
  • How often (measured). Of four full-length runs on the laptop checked spike by spike (G4 from note 23, s7, d5 and rate-r1), one had a fault. With four runs the rate is close to unknown (inferred): a 95 % interval runs from 0.6 % to 81 %.
  • What the check sees (inferred). It caught a real fault. But it sees a fault only if it changes an LGN cell in the first 504 ms: about one in six of each LGN cell's live input pages, and none of the cortex's memory.
  • A spec sentence (measured). The design says of heat and other work that "the simulation is deterministic, so they change only the time". On this laptop one full-length run in four differed from an exact repeat of itself.

What this does and does not show

The protocol's own records show that the declared check refused s7 and that the declared rule then ended the protocol, as written. That the check caught a real execution fault on the laptop rests on the exploratory diagnosis (measured, after the outcome, without the frozen manifest the design asked for).

It is not a software regression, a numerical agreement test, a paper reproduction, a biological validation or a controller test. It adds no evidence for or against the plaids result. The pilot's GO is unchanged, and it still rests on one network instance. Whether it holds over fresh instances is still open.

Seeds 8 to 14 were never built. Seed 7 is now built and its output has been seen, so any revision must leave it out, with seeds 5 and 6.

What went wrong on the way

  • The first freeze stopped at the disk floor, on 7 November at 21:15 UTC, before any run. It was our mistake: the workspace's full verification script ran beside the freeze, and one of its tests copies a 294 MB folder 18 times. No manifest was written. The second freeze passed.
  • The first tier 2 check was refused by its own disk guard and made nothing. The second passed at 20:37 UTC.
  • The local check was first saved under a name the summary does not read. It was renamed, and the summary made in between is kept, marked as such.
  • One diagnosis run stopped at the disk floor, as above.

Next

A rerun needs a new declared revision, as the outcome text says. The owner chooses among the options below, and each would be declared before any of its runs. On this laptop one full run takes 1.0 to 2.0 hours, so six take about 6 to 12.

Option Model time What it catches and misses
A. One rerun after a lead-in refusal 6 to 12 h, plus 1 to 2 h per rerun Stops on a fault that recurs. A fault that comes and goes passes whenever it misses the lead-in. The checker file itself would change.
B. Every instance run twice, all spikes must agree 12 to 24 h, plus 1 to 2 h per disagreement Catches a fault anywhere in the spikes and measures the rate. Twice the time, and about 1.4 GB of disk.
C. Check every LGN input array against an offline computation about 2 h for an identity run, then 6 to 12 h, plus offline time Checks the buffers the leading candidates name. Adds code to the run path; the cortex stays unchecked.
D. Tolerate a few differing lead-in cells 6 to 12 h Analyses runs known to carry a fault, with a tolerance chosen after seeing s7.
E. Find and remove the cause first tens of hours Removes the fault if it is found; slow, and a new identity proof for any change.

Under option A, if every fault showed in the lead-in and runs failed independently at a fixed rate, the protocol would still end unread in about 39 % of attempts at one fault in four runs, and 59 % at one in three (inferred). The diagnosis summary gives each option's details.

Status on 8 November: no revision has been written or declared, and no further run has started.

Limits

  • One run of one instance. Nothing here measures the plaids effect in a fresh instance.
  • The laptop's limits, which the design asks every result of this protocol to repeat: a new platform (arm64 macOS, not the pilot's x86-64 Linux); two engine patches, the LGN noise pair order and a NEST block size; a memory wrapper around the unchanged runner; both checks on one machine, so a fault of this machine would show in both; the server half of the seed search, quoted above; identity with the pilot shown for one instance on one sequence (note 23); and one shared laptop on a nearly full disk.
  • One fault, small numbers. The diagnosis rests on one event, four full-length runs and a mechanism found after the fact.
  • The diagnosis had no frozen manifest, as the design required.
  • Simulation only. Nothing here is biological validation.

Evidence and sources

Model: Antolík et al. (2024), CC BY 4.0; LSV1M v1.0 (MIT); Mozaik v0.4.0 (CeCILL); NEST 3.4 (GPL); the NEST step current module, whose patched file states the GPL; PyNN 0.11.0 (CeCILL); GSL 2.7.1 (GPL). The protocol, the runs, the checks, the diagnosis and the figure are ours. The patches are offered under the terms of the files they change.