# Felisyn full-model experiment package 007

22 October 2026 (Europe/Vienna). A frozen, versioned follow-up to protocol-006 for running the full model on the existing research host. No full-length simulation has been performed with this package.

## Why a new protocol

Two findings change how the declared experiment can be run. Protocol-006 is kept unchanged and was not executed.

1. **Recording recipe.** [recipe-check-009](https://www.felisyn.com/research/recipe-check-009/README.md) showed two things. First, the released `param_spont` recorders reproduce the paper's S5 Table spontaneous counts exactly. Second, Figure 4G's smallest densities need more recorded neurons than `param/` provides. Protocol-006 declared `param/`. This protocol uses the released `param_spont/defaults`, which the model README uses for the spontaneous protocol. A `diff` of the two released trees shows three kinds of difference: self-references, recorder definitions, and the position of an unchanged `'native_nest': False` entry within the `cell` block of `l4_cortex_inh`, `l23_cortex_exc` and `l23_cortex_inh` (same key and value). Otherwise, network, input and simulation values are identical. recipe-check-009 described the difference as recorders only and missed the reordered entry. That package is not edited.
2. **Compute.** The owner decided on 22 October not to rent new servers. [feasibility-010](https://www.felisyn.com/research/feasibility-010/README.md) ran the full network on the existing 4-vCPU, ~7.8 GiB host with a 64 GiB swap file, using 16 virtual processes as 1 MPI rank × 16 threads. Construction took 3.0 h. A 0.52 s run peaked at 7.5 GiB memory plus 24.0 GiB swap. NEST's simulation loop ran at about 760 s per simulated second during the last 150 ms.

## What changes from protocol-006

| Item | Protocol-006 | Protocol-007 |
| --- | --- | --- |
| Parameter tree | `param/defaults` | `param_spont/defaults` (released, unchanged) |
| Spike-recorded neurons (L4 exc / L4 inh / L2/3 exc / L2/3 inh) | 838 / 152 / 862 / 173 | 12,128 / 3,467 / 12,176 / 3,487 |
| Voltage- and conductance-recorded neurons | 62 / 22 / 80 / 26 | 400 / 399 / 400 / 398 |
| 16 virtual processes | 16 MPI ranks × 1 thread | 1 MPI rank × 16 OpenMP threads, passive waiting |
| Host | proposed 16 vCPU, 128 GiB rental | existing 4 vCPU, ~7.8 GiB host, 64 GiB swap file |
| Simulation timeout | 3 h, then 60 s grace | 72 h, then 120 s grace |
| Wrapper | saves raw data, defers plotting | the same, plus NEST timer receipts after each run step |
| Analysis export | JSON lists | the same metrics; traces as NumPy `.npz` arrays |
| Secondary analysis | none | spike measures on protocol-006's sample: 836 / 152 / 856 / 173 neurons, descriptive |

Unchanged:

- the 37 packaged model files and their hashes, the five source commits, the Python 3.9.20 runtime and package versions, and NEST 3.4;
- PyNN seed 5 and effective Mozaik seed 513, trial 0, one replicate;
- `create_experiments_spont`, the 0.1 ms step and 1 ms analog sampling;
- the six metrics, the 48 numerical parity comparisons with their tolerances, and the descriptive comparison with the 24 printed Figure 4 means.

## The frozen experiment

`run-contract.json` is the machine-readable specification. The model is the unchanged LSV1M v1.0 tree in `model/` (MIT license). One 40,320 ms spontaneous segment (`InternalStimulus`) is measured in four cortical populations. The workflow adds a 150 ms blank before it, a 150 ms blank reset, and a 150 ms terminal blank. The expected PyNN run requests are therefore 150, 40,320, 150 and 150 ms. PyNN 0.11.0's NEST backend adds one `min_delay` (0.1 ms) to its first run (`pyNN/nest/simulator.py`, `_State.run`), so NEST's clock should read 150.1, 40,470.1, 40,620.1 and 40,770.1 ms after the four calls. The feasibility probe logged requests of 150, 70, 150 and 150 ms and ended at 520.1 ms, consistent with this. Spike-count correlation uses 4,032 bins of 10 ms. Do not trim an additional burn-in from the analysis window.

The six measurements are firing rate, ISI CV, within-population spike-count correlation, signed membrane voltage, excitatory conductance and inhibitory conductance. Each is summarised as a population mean and SD with the original conventions. Retain eligible-cell and pair counts; pairs that share neurons are not independent replicates.

**Decomposition caveat.**

- The released runs used 16 processes.
- NEST's per-virtual-process random streams depend on the number of virtual processes, not on how they are split into ranks and threads, according to NEST's documentation (not retained here).
- Mozaik draws connection seeds per target neuron from one stream shared by all processes (retained sources), so connectivity should not depend on the split either.
- Evidence (feasibility-010):
  - On the tiny upstream model, splits with equal virtual-process counts gave identical spike trains (0 of 2,208 differed). Traces differed at rounding level, up to 4.3 × 10⁻⁹ for 16 × 1 versus 1 × 16. In some segments the first recorded voltage sample differed by up to 10 mV, for reasons not investigated. A 2-versus-16 control differed in 2,205 of 2,208 spike trains.
  - For the full network, neuron positions and recorder selections under 1 × 16 equal those of recipe-check-009's 16 × 1 build.
- The upstream README warns that runs outside Slurm "might slightly differ" from the published results.
- A long recurrent run can amplify rounding differences into different spike times. The results are therefore those of the 1 × 16 split, not a bitwise copy of a 16 × 1 run.

## Host preparation and cleanup (as root)

The run depends on the temporary swap file `/swapfile-felisyn` (64 GiB, not in `/etc/fstab`). Before launching, confirm that `swapon --show` lists it. A reboot deactivates it. A reboot, a lost swap file, a full disk, an out-of-memory kill or an external stop each ends the attempt as failed. Keep that attempt's directory and receipts. Keep the swap file active until the analysis programs below have finished and their receipts are retained off the host. Then run `swapoff /swapfile-felisyn && rm /swapfile-felisyn` to return the disk space.

Unpack this package in `/opt/felisyn/protocol-007`, then `chown -R felisyn:felisyn /opt/felisyn/protocol-007` (`felisyn` is the unprivileged research user). Check the package against its manifest:

```bash
cd /opt/felisyn/protocol-007 && python3 -c 'import hashlib,json;from pathlib import Path;m=json.loads(Path("manifest.json").read_text());print([f["path"] for f in m["files"] if hashlib.sha256(Path(f["path"]).read_bytes()).hexdigest()!=f["sha256"]] or "manifest OK")'
```

## Execution on the existing host

As the research user:

```bash
source /opt/felisyn/venv/bin/activate
source /opt/felisyn/venv/bin/nest_vars.sh
export MPLBACKEND=Agg PYTHONDONTWRITEBYTECODE=1
cd /opt/felisyn/protocol-007
python -m unittest -v test_contract.py test_metrics.py
mkdir -p /opt/felisyn/protocol-007-checks /opt/felisyn/full-runs
python verify_runtime.py --sources /opt/felisyn/sources --output /opt/felisyn/protocol-007-checks/runtime-check.json
python check_compatibility.py --model /opt/felisyn/protocol-007/model --output /opt/felisyn/protocol-007-checks/compatibility-check.json
python launch.py --output /opt/felisyn/full-runs
```

Use absolute output paths: `check_compatibility.py` changes into the model directory. The default `launch.py` call is a read-only preflight. It exits 2 if the host is below a declared gate: 4 CPUs; 7.5 GiB memory and 6.5 GiB available; 60 GiB swap in total and 55 GiB free; 20 GiB free disk. It also exits 2 if another Felisyn simulation or analysis is running, if another protocol-007 launch holds the lock, if a package file differs from the manifest, if a model file differs from its contract hash, or if a source checkout is not at its pinned commit with a clean tree. The gates are engineering choices derived from the probe, not measured minimum requirements.

Then, **as root**, start the run as a systemd unit that runs as the research user, so that it survives SSH disconnections:

```bash
systemd-run --unit=felisyn-full-007 --uid=felisyn --gid=felisyn -p MemoryHigh=7000M -p RuntimeMaxSec=270000 \
  -p OOMPolicy=continue -p WorkingDirectory=/opt/felisyn/protocol-007 /bin/bash -c 'source /opt/felisyn/venv/bin/activate && \
  source /opt/felisyn/venv/bin/nest_vars.sh && python launch.py --output /opt/felisyn/full-runs --execute \
  >> /opt/felisyn/full-runs/launch-007.log 2>&1'
```

`MemoryHigh` throttles the unit's memory so that other services keep some RAM, which is intended to keep the host reachable. Swap use is not limited. `OOMPolicy=continue` keeps the launcher alive to record an out-of-memory kill of the model. Before a permitted retry, run `systemctl reset-failed felisyn-full-007`.

`--execute` repeats the preflight while holding an exclusive lock in the output directory. It then creates a fresh run directory and runs the runtime and parameter-schema checks there. A failed check is recorded as `precheck_failed`. It then runs `run_model.py nest 16 param_spont/defaults` under GNU `timeout` (72 h, then a 120 s grace period), with the thread and library settings from the contract. `launch.json` records:

- the command, contract and manifest hashes, the effective environment, and the unit's cgroup memory limits and swap devices;
- at the end, a status: `simulation_saved`, `timed_out`, `failed`, `precheck_failed` or `interrupted`;
- the return code, the elapsed time and the saved marker. These are also written after a stop signal or an exception.

Host memory, swap, the model process's resident and swapped memory, load and swap traffic are sampled every 15 s into `resources.csv`. The run passes execution only if all of the following hold:

- the process exits 0;
- the saved marker reports raw data saved and no explosion;
- the four run steps match the expected requests and NEST times, with 16 threads and 16 virtual processes.

The basis is the receipts that `simulate_receipts.py` writes after each run step. If those are missing, it falls back to Mozaik's log lines plus NEST's final status in the marker. The receipt wrapper sits on PyNN's state class, because NEST 3.4 blocks attribute assignment on its module. Mozaik's firing-rate guard is evaluated after a segment; it is not a continuous early-abort monitor.

The probe projects about 12.6 h, excluding steps it could not measure. Retrieving, converting and saving 40 s of recordings under swap is not measured. With 70–150 ms segments, the probe spent 13–15 min per presentation outside what Mozaik counts as simulation time. Any change in speed later in the run is also not measured. The 72 h timeout is a stop rule, not a forecast. On the existing host a longer limit costs nothing, while a timeout forfeits the run.

**Stop and retry rules.**

- Every attempt is retained and reported.
- A failed or timed-out simulation may be repeated unchanged under this protocol only after a documented infrastructure failure: a host reboot, loss of swap, a full disk, an out-of-memory kill or an external stop. At most two attempts are allowed in total.
- Any other rerun, or any changed setting, is a new versioned protocol.
- The unchanged analysis programs may be rerun on the same saved datastore with different resource limits. Any change to analysis code is a new version.

## Analyze a completed run

First inspect `launch.json`, `simulation.log` (including the `FELISYN-SIMULATE` lines), the saved parameters and the runtime receipts. Confirm that the declared recipe, split and seeds actually ran. `DATASTORE` is the directory inside the run directory that contains `datastore.recordings.pickle`; `RUN_ID` is the run directory's name. The swap file must still be active. Then, as root, run the analysis as a unit:

```bash
systemd-run --unit=felisyn-analysis-007 --uid=felisyn --gid=felisyn -p MemoryHigh=7000M -p OOMPolicy=continue \
  -p WorkingDirectory=/opt/felisyn/protocol-007 /bin/bash -c 'source /opt/felisyn/venv/bin/activate && \
  source /opt/felisyn/venv/bin/nest_vars.sh && export MPLBACKEND=Agg PYTHONDONTWRITEBYTECODE=1 && \
  python validate_datastore.py --run-label RUN_ID --datastore "DATASTORE" --figure-source model/visualization_functions.py \
    --framework /opt/felisyn/sources/mozaik --output RUN_DIRECTORY/analysis >> RUN_DIRECTORY/analysis.log 2>&1; \
  python compare_reference.py --receipt RUN_DIRECTORY/analysis/analysis-parity.json \
    --output RUN_DIRECTORY/analysis/printed-reference-comparison.json >> RUN_DIRECTORY/analysis.log 2>&1; \
  python secondary_subset.py --datastore "DATASTORE" --framework /opt/felisyn/sources/mozaik \
    --primary RUN_DIRECTORY/analysis/analysis-parity.json --output RUN_DIRECTORY/analysis >> RUN_DIRECTORY/analysis.log 2>&1'
```

`validate_datastore.py` is the protocol-006 program with one declared change. Analog traces stay NumPy arrays and are exported per population and variable as compressed `.npz` files, with float64 values unchanged. Spikes are exported to `exported-spikes.json.gz`. With JSON lists, the roughly 190 million voltage and conductance samples of the spontaneous recording set would need several GiB of Python objects. Definitions, the AST-extracted upstream Figure 4 prelude, tolerances (atol 1e-10, rtol 1e-12) and comparisons are unchanged. On the probe's recordings, both versions gave identical results, and the exports held identical values (feasibility-010). `compare_reference.py`, `metrics.py`, `reference-means.csv`, `verify_runtime.py` and `test_metrics.py` are byte-identical copies from protocol-006.

Expect the analysis to exceed physical memory and rely on swap. Measured parts: 5.6 GiB peak for the whole program on the 70 ms probe recordings, and 3.2–3.6 GiB for each correlation step on synthetic data of full size and length. The combined peak at full length has not been measured.

The reference comparison is descriptive only. It reports 24 signed mean differences against the printed Figure 4 labels and flags those within half a printed unit. This is **not** a biological equivalence threshold, a paper-reproduction pass or biological validation.

### Declared secondary analysis

[Note 13](https://www.felisyn.com/journal/matching-the-published-recording-sets) said that a comparison on protocol-006's sample of neurons stays possible. `secondary_subset.py` computes the independent spike-based measures for the neurons that both recipes spike-record: firing rate, ISI CV and correlation, with `metrics.spike_metrics` and the primary window and bins. That is 836, 152, 856 and 173 of protocol-006's 838, 152, 862 and 173 neurons. Their indices and IDs were computed before the run under the 1 × 16 split and frozen in `secondary/param-spike-subset.json`. The positions and selections behind them equal recipe-check-009's 16 × 1 receipts (feasibility-010). The program first checks that the run's neuron positions hash to the frozen values and that the IDs match. It then reports the subset values next to the primary values and the printed Figure 4 means. It is descriptive, has no upstream parity check and is not a pass criterion. Voltage and conductance are not repeated: note 13 found that most of protocol-006's small analog sample is not recorded under `param_spont`.

## Limits

- One run, one seed pair, one host. Timing and memory are single observations.
- Retrieval and saving of 40 s of recordings, the analysis's combined memory peak and its run time at full length are not measured.
- The released archive's exact runtime remains unknown (see protocol-006). Ubuntu repositories are not snapshotted.
- Swapping changes speed, not arithmetic. Results still depend on the 1 × 16 split as described above.
- A passing run and analysis establish execution and numerical agreement between two analysis paths. They do not validate the model biologically and do not connect it to any controller.

## Evidence and reuse

- `manifest.json` records SHA-256 and size for every file except itself and the zip; `protocol-evidence.zip` contains the manifested files.
- `test_contract.py` contains synthetic tests only; its fixtures are not neural evidence.
- `checks/smoke_receipts.py` and its receipts show the receipt wrapper on the pinned PyNN and NEST: two calls with 16 threads, requests of 150 and 70 ms, and NEST times of 150.1 and 220.1 ms.
- `secondary/param-spike-subset.json` comes from feasibility-010's `receipts/subset-1x16/subset-indices.json`, whose SHA-256 it records.
- The contract is generated from protocol-006's contract plus the declared changes.

Sources and licences:

- Model: Antolík J, Cagnol R, Rózsa T, Monier C, Frégnac Y, Davison AP (2024), *A comprehensive data-driven model of cat primary visual cortex*, [PLOS Computational Biology](https://doi.org/10.1371/journal.pcbi.1012342), CC BY 4.0. `reference-means.csv` is our transcription of the printed Figure 4 labels (via protocol-006 and benchmark-004).
- [LSV1M v1.0](https://github.com/CSNG-MFF/mozaik-models/tree/04bf11b72ff5f78e66f28e037016e8e33b09c12e/LSV1M), MIT; licence in `model/LICENSE`.
- Mozaik (CeCILL; licence text in `sources/mozaik-LICENSE`): `sources/` holds `connectors/modular.py` from commit f5c09a0e (v0.4.0), its parent d2ea60ce and commit deca176f (June 2023), with retrieval records in `sources/provenance.json`.
- PyNN 0.11.0, NEST 3.4 and the step-current module, as pinned in protocol-006.
