← Journal
NOTE 23·Methods

The model moved to a laptop: all 18,600 spike trains matched

The research server was lost, so the 2 mm model moved to an 8 GB laptop, under a memory cap of a little under half the machine. Small fixes to its random numbers and arithmetic came first. The plaids pilot's intact run, repeated on the laptop, then reproduced all 18,600 neurons' spike trains over 35 seconds of model time, and the frozen analysis gave the same three measures. This is numerical agreement on one network instance. It is not new biology, and it is not a confirmation of the pilot.

The research server that ran this journal's full-model and plaids simulations has been lost. The next planned step, a confirmatory test of the plaids pilot, still needs somewhere to run. The machine we have is an 8 GB laptop that also has other work to do.

So the model moved to the laptop under a rule set first: at most 3.7 GiB of memory, a little under half the machine, with a watchdog that stops the run above that. The pilot's intact run had peaked at 5.26 GiB on the server, so it could not simply be copied across.

Two questions followed. Does it fit? And if it fits, is it still the same simulation?

The headline: in the one run we compared, it fits, and every spike in the 35,133 ms sequence is the same.

Every spike in the sequence matched. The laptop repeated the pilot's intact network with the same arguments, seed and stimulus sequence. All 18,600 neurons' spike trains equal the server's over the whole 35,133 ms sequence, at the export's 0.01 ms precision. That is 7,948,521 spikes.

The three measures came out the same. The frozen analysis gave the pilot's XOS, Dc and ds in every stored digit.

It fitted. Memory peaked at 3.208 GiB, and the run took 2.00 hours. On the server it took 2 h 54 min.

This is numerical agreement on one run. It is not new biology, and it says nothing about cat cortex. It is not a confirmation of the pilot either: that is protocol 023, a separate test on fresh network instances, which has been declared and has not been run.

Line chart titled 'The full run stayed under the memory cap'. The x axis is hours since the run started, 0 to 2. The y axis is physical footprint in gibibytes, 0 to 5. The violet line climbs to about 2.7 GiB in the first 10 minutes, steps up to about 2.9 and 3.05 GiB by one hour, falls back to about 2.9, rises slowly to about 3.0, climbs to a peak of 3.208 GiB at 1.93 hours, dips to about 3.0 and rises to about 3.15, then drops to about 0.5 GiB when the run ends at 2.00 hours. Three horizontal lines sit above it: the gate at 3.515 GiB, the cap at 3.7 GiB, and the server pilot's peak at 5.26 GiB, labelled as a different meter. The curve stays below all three.
Figure 1. Memory of the plaids pilot's intact run on the 8 GB laptop, as the physical footprint of the run's processes. The line is the log, one row about every 16 s, thinned. The dot is the peak from the 2 s samples, 3.208 GiB at 1.93 hours, which falls between log rows. The dashed line is the 3.7 GiB cap, and the dotted line the 3.515 GiB pass gate written before the run. The top line is the same run's peak on the server, 5.26 GiB, from a different meter on a different system, so it shows scale and not an exact saving. Plotting code · SVG · Samples

Why moving a simulation can change every spike

A simulation like this one looks repeatable because its seeds are fixed. But a seed fixes only a stream of raw random numbers. What happens to those numbers next can depend on the machine.

The model's LGN, the stage between the retina and the cortex, gets noisy input drawn while the simulation runs. It is the only run-time random input we found in the model. NEST, the simulator, takes its normally distributed numbers from the C++ standard library. The server's library and the laptop's library use the same method, which turns two uniform random numbers into a pair of normal numbers. They then hand out each pair in opposite order.

So from the same seed, every LGN cell got the same noise with neighbouring values exchanged. The statistics do not change. Every cell's actual input does, and the cortex downstream follows from it.

The smallest test we have showed this at once. It is the regression from note 7: 1.275 seconds of a tiny model, against the reference data its authors supply.

Against the reference Server Laptop, first build Laptop, fixed
Spike trains equal 1,656 of 1,656 15 of 1,656 1,656 of 1,656
Largest voltage difference 4.26e-14 mV 25.5 mV 6.34e-9 mV

The fix had two parts, and they went in together, so this test does not separate them.

  • A small patch to the model's LGN cell hands each pair back in the server's order. It uses the random stream exactly as before.
  • The GSL numerical library was rebuilt at the server's version, 2.7.1, without fused multiply-add, an instruction that merges a multiplication and an addition and so rounds differently.

The size of the first difference points to the pair order as the main cause. LGN voltages that had differed by up to 3.2 mV in the first milliseconds became exactly equal.

Look at the last row of the table again. The laptop's arithmetic is still not the server's: its largest voltage difference is about 150,000 times the server's own. Spike trains stay equal only while such differences do not push a threshold crossing into another 0.1 ms time step. Our plan listed as a risk that the cortical spike trains would not be equal.

Fitting in 3.7 GiB

We estimated the unchanged run at 4.36 GiB on the laptop, over the cap, even with the first change in the table below. A memory study then found where the memory went. Much of it was copies that nothing read again.

Change Saving (GiB) Basis
A macOS allocator setting that returns freed memory at once 0.8 to 1.2 Estimate
Drop PyNN's copy of the connection lists once NEST holds them 0.439 Estimate
Clear the LGN input cache once NEST has its copy 0.330 Estimate
Free the LGN cells' arrays early 0.44, during the LGN step Estimate
Store NEST's lists in blocks of 64 elements, not 1,024 0.49 Estimate
Stop a leak in NEST's Python layer 0.894, on a short run Measured

Each saving applies at its own moment in the run, so they do not add up to the gap between the two peaks.

The last row corrected us. The study first read about 0.76 GiB of the server's memory as freed memory that the allocator had kept. Most of it was probably a leak. Each time PyNN picks neurons out of a NEST collection with an array of positions, NEST's Python layer keeps an object it should release, and PyNN does this once per target neuron and projection. A census of the heap in a running 2 mm job on the laptop found about 0.84 GiB of them. Our fix asks for the same collection by a route that does release it.

Of these changes, only the block size touches NEST's source, as a one-line patch. The rest sit in one small wrapper, run_controller_mem.py, which runs the unchanged run_controller.py, so that file's recorded hash stays true.

Each change had to leave the simulation as it was. We tested that before the long run, on a 0.6 mm sheet, in three steps: the allocator and Python changes together, then the block size, then the leak fix.

  • At every step, the spike trains in all 24 segments and the export, apart from the run time and command line it records, were identical to an unchanged reference run.
  • With the block size and with the leak fix, so were the LGN input and every projection's source, target, weight, delay and final state.
  • A control run with a different network seed differed, which shows the comparison can fail.

A run of 5 stimulus steps on the real 2 mm sheet then matched the pilot in all 18,600 neurons for 2,289 ms.

The test

The test was the pilot's own intact run, on 7 November from 06:08:32 to 08:08:48 UTC, 2.00 hours in all. Its pass conditions were written down after the short run and before this one: the file's last change is 11 seconds before the start.

Condition Result
The run exits normally and the cap does not stop it Exit 0
Peak footprint at most 3.515 GiB, 5 % under the cap 3.208 GiB, at 1.93 hours
All 3,600 LGN spike trains equal the pilot's 3,600 of 3,600
The frozen analysis accepts the export, and each measure lies within 3 pilot standard errors Accepted, all three differences 0.0

Cortical identity was not a condition. The gate only asked us to report it, because rounding differences could still have changed spikes later in the run. No spike train differed anywhere in the sequence.

Population Spike trains identical Spikes, on both machines
LGN ON cells 1,800 of 1,800 1,895,288
LGN OFF cells 1,800 of 1,800 1,235,659
Layer 4 excitatory 6,000 of 6,000 1,078,093
Layer 4 inhibitory 1,500 of 1,500 1,021,663
Layer 2/3 excitatory 6,000 of 6,000 1,287,141
Layer 2/3 inhibitory 1,500 of 1,500 1,430,677

Neuron positions, orientation preferences, resolved parameters and the count of 14,740,479 connections are equal too.

The measures follow. XOS is 0.5843923122251847, Dc 0.7128815055003441 and ds 0.3424811433221438 on both machines. Of the 6,284 numbers that the two results files both hold, one differs: the recorded run time.

Equal spikes should give this, though it was not guaranteed: numpy and scipy on the laptop are different builds from the server's. The same input went through the same frozen code, so these are the pilot's numbers again, and not a second measurement of anything.

Before the run we predicted a peak of 3.38 GiB, likely between 3.2 and 3.5, and 2.5 to 3 hours. The peak landed at the low edge of that range, and the run was shorter than predicted.

Two cautions on the comparison with the server:

  • Different meters. The laptop's 3.208 GiB is the physical footprint of the run's processes, which counts compressed memory at full size. Its resident memory peaked at 1.906 GiB. The server's 5.26 GiB is the peak Linux's service manager reported for the run. Part of the gap is the system and the meter, and not our changes. Swap did not grow: it stood at 423 MiB at the start, and that was its highest value.
  • Different machines and loads. The two have different processors, and the laptop was not idle. 2.00 hours against 2 h 54 min is two single runs, not a benchmark.

What went wrong on the way

  • Leftover C++ header folders on the laptop hid the real headers, and the build's own checks failed quietly, which left five settings undefined. A compiler flag fixed it.
  • The first small run stopped before simulating. macOS starts Python's worker processes in a different way from Linux, and the model's connection code needs the Linux way.
  • Our first sketch for dropping the connection lists freed nothing. An empty slice of an array still holds the whole array in memory. A copy of the slice does not.
  • The numerical library behind numpy starts one thread per core by default: 8 here. The pilot's receipt does not record the setting, but on the 4-CPU server the same default gives 4. In small test runs on the busy laptop, the LGN step took 1,536 s at 8 threads against 77 s at 4. The faster run also had the memory changes, so the factor of about 20 is not a clean measurement, but a profile of a run stuck in that step at 8 threads put 99 % of its samples in the library's thread synchronisation. Setting it to 4 changed no output.

Each of these is in the deviations list.

What this does and does not show

It shows numerical agreement between two builds of the same software, on one run. For that run's sequence, the laptop build stands in for the server build, spike for spike.

It is not a reproduction of the paper, and it is not biological validation: no cat recording enters it. It is not a controller test. And it adds no evidence for or against the plaids result, because it repeats the same network with the same stimulus. Whether that result holds in other network instances is protocol 023's question.

What was lost

The larger files of run 011, the full-model run in note 15, were kept only on the server. There were 39 files and 3.10 GB: the run's datastore and its voltage and conductance exports. They are gone for good.

Their hashes and sizes stay published, so the record of what existed stands, but those files can no longer be read again. That run's published results, its receipts and all its spike times, 12 MB, are still on this site. The planned confirmatory test does not read the lost files.

The built software went with the server too. It has been rebuilt here from the pinned sources, with the differences listed.

The server can make no new runs, so any further comparison with it is limited to server exports we still hold.

Limits

One network instance and one stimulus sequence. Agreement was shown between one server build and one laptop build. Another seed, another stimulus or a longer run could diverge.

Spike times only, at 0.01 ms. That is finer than the model's 0.1 ms time step, but the 2 mm run recorded no voltages, so differences below the level of a spike are unmeasured there. In the tiny model they reach 6.34e-9 mV. One threshold crossing landing in another time step would start a real divergence in the cortex. The LGN gets no cortical input, so it could not spread there.

The sequence only. The comparison covers 0 to 35,133 ms. The closing blank, to 35,583.1 ms of model time, was not compared.

Not whole files. The two export files differ, and each records its own run time. We compared spike times, neuron tables, parameters, steps and the connection count, not every byte.

The intact run only. The plain lesion and the connection export were not repeated, so nothing here tests them. The frozen analysis ran with the lesion marked as failed, which is how it accepts one run. No lesion run was attempted on the laptop.

The memory changes were tested in three steps, not one at a time, and on small runs. The full run is their only test at full length, and it ran once. Repeats of a run on the laptop were shown only on small models.

The gate is a local record. The pass conditions were written before the run, but no hash of them was published first, so unlike a declared protocol the order rests on the laptop's own file times.

Peaks are sampled every 2 seconds, so a shorter spike could be missed.

This platform, as it stands. The laptop's system libraries are part of the runtime, so a macOS update would change it. Python, numpy, scipy and other parts also differ from the server's in version or in compiled code, as the deviations list records.

Simulation only. Nothing here is biological validation.

Next

Protocol 023, the confirmatory test that note 22 promised, has been declared and has not been run. Anything specific to that test belongs in its own declared text.

Status on 7 November: protocol 023 was declared on 7 November 2026 at 20:25:30 UTC, before any of its runs. The SHA-256 of its declared text is 1e20a4a90a474d2a2f4348f095133a820120f0a4e85dc1095e6b32ec00cffac3. None of its runs has started. The declared text itself is not published yet. It will be published with the result, so that this hash can be checked against it then.

Evidence and sources

Model: Antolík et al. (2024), CC BY 4.0; LSV1M v1.0 (MIT); Mozaik v0.4.0 (CeCILL); NEST 3.4 (GPL); the NEST step current module, whose patched file states the GPL; PyNN 0.11.0 (CeCILL); GSL 2.7.1 (GPL). The reduction, the runs, the analysis, the figures, the two patches, the wrapper and the memory cap are ours. The patches are offered under the terms of the files they change.