← Journal
NOTE 10·Software verification

Checking the analysis on saved data

All 48 numerical comparisons agree with the original analysis on retained small-model recordings. A timing detail explains why a nominal 10 ms bin can span 10.5 ms.

Our measurement helpers now agree with the original numerical analysis on saved simulation data. All 48 comparisons passed, with a largest absolute difference of about 7.1 × 10⁻¹⁵. The source datastore's file hashes remained unchanged.

This is an analysis check using the small cat visual-cortex model run already reported in note 07. We did not run a new neural simulation. The result establishes agreement between two calculations on the same inputs; full-model reproduction and independent biological validation remain separate, unfinished tasks.

What we compared

We used the retained 105 ms spontaneous segment from each of four cortical populations: excitatory and inhibitory neurons in layers 4 and 2/3. For each population, we compared the mean and standard deviation of six measurements: firing rate, interspike-interval variability, pairwise spike-count correlation, membrane voltage, excitatory conductance and inhibitory conductance. Six metrics × two summaries × four populations gives 48 comparisons.

Analysis diagram. Saved 105 millisecond segments from four cortical populations enter two calculation paths: the pinned original framework and Figure 4 numerical statements, and Felisyn's independent metric helpers. Neuron identities and signal timing are checked. All 48 comparisons of metric means and standard deviations agree under an absolute tolerance of 1e-10 plus relative tolerance of 1e-12. This verifies analysis on a small saved run, not biology.
Figure 1. The implemented comparison, drawn from its saved result receipt. “Original analysis” refers to the required framework analyses and unchanged numerical statements for Figure 4, rather than regeneration of the complete paper figure. All 48 results · Integration code.

The reference path invokes the pinned Mozaik analyses and extracts the numerical prelude of SpontStatisticsOverview.subplot from LSV1M v1.0 without changing those statements. The second path uses our independent helpers. Before comparing values, the harness checks recorded neuron IDs against the raw spike-train order and correlation-matrix IDs. It checks segment windows, analog timing and sampling intervals, preserves signed voltage, and converts conductance from µS to nS.

The comparison tolerance was set before inspecting results: absolute 1e-10 plus relative 1e-12, applied in each metric's reported units. This is a floating-point agreement criterion. It is not a biological equivalence margin or a tolerance derived from the paper's error bars. All 48 comparisons contained finite values.

Which recordings contributed

Population Recorded spike trains Eligible cells Unique eligible pairs Analog neurons per variable
L4 excitatory 107 107 5,671 10
L4 inhibitory 19 19 171 4
L2/3 excitatory 116 116 6,670 10
L2/3 inhibitory 24 24 276 6
Total 266 266 12,788 30

All 266 recorded spike trains meet the original eligibility threshold of at least seven spikes. Each within-population pair is counted once, with self-pairs excluded. These 12,788 pairs share neurons; they are not independent experimental replicates. No pair had an undefined correlation in this fixture.

For each analog variable, the 30 recorded neurons contribute 105 samples each, from 0 through 104 ms at 1 ms intervals. That is 3,150 samples per variable. The short, active software-test configuration is not intended to match the full model's published spontaneous physiology. It also does not exercise every edge case: silent cells, excluded cells and undefined correlations are covered by separate synthetic tests.

The 105 ms binning detail

Our earlier helper required the recording duration to divide evenly into 10 ms bins. The actual small run lasts 105 ms. Inspecting the original histogram helper showed that it rounds the requested bin count, then divides the entire time range into that number of equal bins. In the tested runtime, 105 / 10 rounds to ten bins, each 10.5 ms wide.

Time-bin diagram with a common millisecond axis. The saved 105 millisecond spontaneous segment is divided into ten bins of 10.5 milliseconds by the original helper. A saved 210 millisecond grating segment is divided into twenty-one bins of exactly 10 milliseconds. No activity values are drawn. The intended 40,320 millisecond full protocol would divide evenly into 4,032 ten-millisecond bins.
Figure 2. Bin edges derived from the original helper and retained segment durations. The rectangles show time intervals, not invented neural activity. Pinned source · Versioned helper · Drawing code.

The original PSTH retains a nominal 10 ms sampling label and uses that nominal width for rate normalization. Here we compare Pearson correlations, which are unchanged by a common positive scaling of the binned values. This does not mean bin edges or timing labels are irrelevant to other analyses.

The new helper keeps strict divisibility as its default. An explicit upstream_range option reproduces the original full-range behavior and reports the nominal width, effective width and bin count. We preserve all 105 ms rather than trim the segment. The previous helper remains archived in its original evidence package.

As a control, we inspected the actual upstream histograms for all eight retained cortical grating segments. Each lasts 210 ms and has 21 bins of exactly 10 ms. The planned 40,320 ms full-model segment would likewise divide evenly into 4,032 bins. That longer experiment has not yet been run.

Failures and reproducibility

The first integration attempt failed on the test-loader import path. The second failed because a datastore view retained a snapshot of its analysis-result list before new results were added. Re-querying the full datastore at the required points fixed the harness. These were setup errors, not measured neural discrepancies; both logs remain alongside the successful third attempt.

The versioned helper passes 12 synthetic software tests, including strict rejection and explicit handling of the 105 ms case. The portable recording export preserves the actual selected spikes and analog arrays as JSON. The replay script lets readers recalculate the helper outputs without loading simulator-specific pickle files. The full two-path comparison still requires the pinned environment and original datastore documented in the evidence package.

What this enables next

Together with the configuration audit, this gives us a measurement path that has been exercised on real simulation output. The next work is to resolve or explicitly declare the remaining historical compatibility choices, freeze the full-model run contract and comparison policy, and assess a bounded larger run.

For the eventual controller, later tests must show that specified inputs change neural outputs and that disrupting the circuit changes the requested actions. Today's analysis agreement is a prerequisite for trustworthy measurements in those experiments; it is not evidence of autonomous behavior or token control.

Evidence and sources

Follow-up on 13 October 2026

Note 11 documents the historical connector behavior and freezes the full-model experiment contract. The full run remains pending.