"""Protocol 023: the blind integrity check of one instance export, on the server and on the Mac; the early stop; and the
replacement rule with the queue's one record of it.

  python3 check_instance_023.py server CONTRACT.json K EXPORT OUT.json
        on the server, run by run_queue_023.sh right after each run that exits 0 (from the server's copy of the
        frozen package): the checks below on the new export; exit 0 passed, 3 refused (OUT written once); any other
        exit means no check was made (no record)
  python3 check_instance_023.py check CONTRACT.json K FETCHED_DIR OUT.json [--not-frozen]
        on the Mac, the local check of FETCHED_DIR/exports/sK.json.gz as fetch_023.sh saved it (OUT written once)
  python3 check_instance_023.py summary CONTRACT.json FETCHED_DIR CHECKS_DIR OUT.json [--not-frozen]
        the instances to analyse, from the queue's receipts (FETCHED_DIR/receipts) and the local check records

Spec: docs/specs/2026-10-29-protocol-023-confirm.md ("Integrity checks per instance", "The replacement rule"). It
computes no plaid quantity: it reads only the 45@30 and 135@30 singles (LGN and layer 4 groups in window G), the LGN
lead-in and the export's own records.

Before it reads an export it stops, writing no record: on the Mac unless the running checker and the code it imports
are the frozen manifest's (manifest.json next to CONTRACT.json; --not-frozen only for synthetic tests), and the export
is a regular local file (not an iCloud placeholder) whose SHA-256 is the host's, as fetch_023.sh recorded it in
FETCHED_DIR/data-sha256.txt; on the server unless the running code is the code the contract was made from. A read
failure is never a refusal: only a check of the export's content writes one. An export that cannot be parsed is read
twice, and refused as 'unreadable' only when its bytes read cleanly and still have the checked SHA-256.

Refusals, each of which makes the instance invalid, each with its kind:
  1. the export is not the declared run ('run'), or the host's bytes cannot be parsed ('unreadable'): its run name must
     be m23-sK-intact-a1 or -a2 for the instance's pynn_seed K, its argv the declared command, its resolved weights and
     seeds 017's intact-A entry with pynn_seed K, its NEST kernel rng_seed 42 with 4 threads; the steps must be the
     instance's declared sequence file, with protocol 022's timing, one presentation and one sequence segment per sheet,
     the LGN cache disabled, and well-formed neuron tables and spike trains (protocol 022's checks);
  2. the LGN band ('lgn_band'): the X_ON and X_OFF mean counts per cell in G at 30 % (the 45 and 135 deg singles)
     within 0.90 to 1.10 of protocol 017's 8.06 and 5.73 (plaid_stats.validation, unchanged);
  3. the LGN lead-in identity ('leadin_identity'): every X_ON and X_OFF cell's spike times in the 504 ms lead-in equal
     the pilot intact export's (leadin-reference-023.json), with the same cells.
Reported only, never a refusal: the layer 4 band (the 30 % own singles of the 45 and 135 deg groups against 017's
2.97 and 2.18 counts, band 0.80 to 1.25), the group sizes and the NEST kernel's other records.

The early stop (early_stop()): the queue stops at the first export its server check refuses (runs/early-stop.txt).
Every refusal kind is one that a fault shared by every instance would produce (the same frozen files, engine,
environment and LGN noise stream), so that refusal ends the protocol (NOT INTERPRETABLE) once the local check of the
same export refuses it too; a local check that passes it stops the summary (the two checks disagree).

The replacement rule (replacement()): a slot is invalid if its run failed twice or was killed for memory, or the local
check refused its export. Each invalid slot gets the next unused reserve (13, then 14); a reserve that is itself
invalid is followed by the next unused reserve for the same slot; at most two replacements. The queue's one record,
runs/replacements.txt, says in time order which reserve filled which slot and why ("reserve R for slot S: CAUSE", or
"no reserve left for slot S: CAUSE"); the summary reads it, checks every line against the rule, the receipts and the
check records, and stops on any disagreement. The analysed instances are the valid slots and the valid reserves that
filled the others, in pynn_seed order. As soon as more slots need a reserve than are unused, the protocol is NOT
INTERPRETABLE (fewer than six valid instances).
Simulation only.
"""
import hashlib, json, os, re, stat, sys, time
sys.dont_write_bytecode = True                  # before any project or frozen import: no __pycache__ beside them
from pathlib import Path

HERE = Path(__file__).resolve().parent
sys.path.insert(0, str(HERE))
import numpy as np                              # noqa: E402
import analyze_plaids as ap                     # noqa: E402  (protocol 022's driver, byte-identical; imports 017's code)
import plaid_stats as ps                        # noqa: E402
import confirm_023 as cf                        # noqa: E402

a = ap.a                                        # protocol 017's analyze_decisions (frozen, unchanged)
RUN_NAME = re.compile(r'^m23-s(\d+)-intact-a([12])$')   # the declared names' form; make_contract_023.py checks them
LGN = ('X_ON', 'X_OFF')
LEAD_MS = ap.LEAD_FRAMES * ap.FRAME_MS          # 504 ms
SINGLES_30 = ('45@30', '135@30')
QUEUE_STATUS = ('done', 'failed twice', 'killed for memory')
KINDS = ('run', 'unreadable', 'lgn_band', 'leadin_identity')
SF_DATALESS = getattr(stat, 'SF_DATALESS', 0x40000000)   # macOS: an iCloud placeholder whose data is not local
FROZEN_FILES = ('contract.json', 'controller/check_instance_023.py', 'controller/confirm_023.py',
                'controller/plaid_stats.py', 'controller/analyze_plaids.py', 'controller/frozen017.py',
                'leadin-reference-023.json')
RUNNING = {'controller/check_instance_023.py': Path(__file__), 'controller/confirm_023.py': Path(cf.__file__),
           'controller/plaid_stats.py': Path(ps.__file__), 'controller/analyze_plaids.py': Path(ap.__file__),
           'controller/frozen017.py': Path(ap.frozen017.__file__)}
REPLACEMENT_LINE = re.compile(r'^(\S+) (?:reserve (\d+)|no reserve left) for slot (\d+): (.+)$')
RUN_CAUSE = re.compile(r'^(?:the run|reserve (\d+)) (failed twice|killed for memory)$')
REFUSED_CAUSE = re.compile(r'^(?:reserve (\d+) )?refused by the blind checker \(check record SHA-256 ([0-9a-f]{64})\)$')
EARLY_STOP_LINE = re.compile(r'^(\S+) early stop: pynn_seed (\d+) refused by the server check \(record SHA-256 '
                             r'([0-9a-f]{64})\)$')
SERVER_PASSED, SERVER_REFUSED = 0, 3            # the server check's exit codes; any other: no check was made
PEAK_RSS = re.compile(r'Maximum resident set size \(kbytes\): (\d+)')


class Refusal(Exception):
    """The export is not the declared one, or fails a declared band: the instance is invalid."""

    def __init__(self, message, kind='run'):
        super().__init__(message)
        self.kind = kind


def refuse(message, kind='run'):
    raise Refusal(message, kind)


def stop(message):
    """Not an integrity refusal: nothing is written, and the check can be made again correctly."""
    raise SystemExit(f'refused: {message}')


def sha256(path):
    h = hashlib.sha256()
    with open(path, 'rb') as fh:
        for block in iter(lambda: fh.read(1 << 20), b''):
            h.update(block)
    return h.hexdigest()


def dataless(path):
    """True for an iCloud placeholder (macOS SF_DATALESS): reading it would start a download and can stall."""
    try:
        return bool(os.lstat(path).st_flags & SF_DATALESS)
    except (AttributeError, OSError):
        return False


# ---------------------------------------------------------------- the frozen code and the fetched files

def check_frozen(base, contract, allow_unfrozen=False):
    """In the frozen package (manifest.json next to the contract): the contract, the lead-in reference, every declared
    sequence file and this checker's code are the manifest's, the running code is that code, the manifest records
    that every test passed at the freeze, and protocol 017's package and pins match. Returns the record."""
    manifest = base / 'manifest.json'
    if not manifest.exists():
        if not allow_unfrozen:
            stop('no manifest.json next to the contract (use --not-frozen only for synthetic tests)')
        return {'manifest': False}
    try:
        m = json.loads(manifest.read_text())
        listed = {f['path']: f['sha256'] for f in m['files']}
        tests = m.get('tests_at_freeze') or {}
    except (OSError, ValueError, KeyError, TypeError) as exc:
        stop(f'manifest.json cannot be read ({type(exc).__name__}: {exc})')
    if tests.get('run') is not True or tests.get('all_passed') is not True:
        stop('the frozen manifest does not record that every test passed at the freeze (a --skip-tests freeze is for '
             'tests only)')
    sequences = sorted(rel for rel in listed if rel.startswith('sequences/'))
    for rel in FROZEN_FILES + tuple(sequences):
        if rel not in listed or not (base / rel).is_file() or sha256(base / rel) != listed[rel]:
            stop(f'{rel} differs from the frozen manifest')
    for rel, path in RUNNING.items():
        if sha256(path) != listed[rel]:
            stop(f'the running {rel} is not the frozen one')
    _, _, p017 = ap.check_017(contract)
    return {'manifest': True, 'manifest_sha256': sha256(manifest), 'files_checked': list(FROZEN_FILES) + sequences,
            'protocol_017': p017}


def host_hashes(fetched):
    """{export file name: SHA-256} from FETCHED_DIR/data-sha256.txt, the host's hashes as fetch_023.sh recorded them."""
    path = Path(fetched) / 'data-sha256.txt'
    if not path.is_file() or dataless(path):
        stop(f'{path} is missing or an iCloud placeholder (fetch_023.sh writes it)')
    out = {}
    for line in path.read_text().splitlines():
        if not line.strip():
            continue
        parts = line.split()
        if len(parts) != 3 or not re.fullmatch(r'[0-9a-f]{64}', parts[0]) or not re.fullmatch(r's\d+\.json\.gz', parts[2]):
            stop(f'{path} holds a line that is not "SHA-256  bytes  sK.json.gz": {line!r}')
        out[parts[2]] = parts[0]
    return out


def local_export(fetched, k, hosts):
    """The fetched export of instance K: a regular local file, not a placeholder, whose SHA-256 is the host's."""
    name = f's{k}.json.gz'
    path = Path(fetched) / 'exports' / name
    try:
        st = os.lstat(path)
    except OSError as exc:
        stop(f'no fetched export at {path} ({type(exc).__name__}); fetch it with fetch_023.sh')
    if not stat.S_ISREG(st.st_mode):
        stop(f'{path} is not a regular file')
    if dataless(path):
        stop(f'{path} is an iCloud placeholder; download it first')
    if name not in hosts:
        stop(f'{name} has no host SHA-256 in data-sha256.txt')
    try:
        digest = sha256(path)
    except OSError as exc:
        stop(f'{path} cannot be read on this Mac ({type(exc).__name__}: {exc})')
    if digest != hosts[name]:
        stop(f"{path} is not the host's file (SHA-256 {digest}, the host's {hosts[name]}); fetch it again")
    return path, digest


# ---------------------------------------------------------------- the declared run

def declaration(contract, k):
    """The contract's entry for instance K, or a refusal."""
    entry = (contract.get('instances') or {}).get(str(k))
    if not isinstance(entry, dict) or entry.get('pynn_seed') != k:
        refuse(f'pynn_seed {k!r} is not a declared instance of the contract')
    return entry


def check_identity(ex, decl, contract):
    """The run is the declared one: run name, command line, resolved weights and seeds, NEST kernel. Returns the run
    name."""
    k = decl['pynn_seed']
    argv = ex.get('argv') if isinstance(ex.get('argv'), list) else []
    run_name = argv[-1] if argv else None
    if not isinstance(run_name, str) or run_name not in decl['run_names']:      # the contract's names are the authority
        refuse(f'run name {run_name!r} is not one of the declared {decl["run_names"]}')
    if argv != decl['argv'][run_name]:
        refuse(f'command line {argv} is not the declared {decl["argv"][run_name]}')
    if ex.get('resolved') != decl['resolved']:
        refuse(f"resolved weights or seeds {ex.get('resolved')} differ from the declared entry for pynn_seed {k}")
    nest = ex.get('nest') if isinstance(ex.get('nest'), dict) else {}
    want = contract['engine']['nest_kernel']
    got = {key: nest.get(key) for key in want}
    if got != want:
        refuse(f'the NEST kernel recorded {got}, not the declared {want}')
    return run_name


def check_structure(name, ex, chunks, shown):
    """Protocol 022's checks of the step period and layout (analyze_plaids.check_run_plaids without its run-identity
    part, which check_identity replaces): no separate warm-up, the LGN cache disabled, steps of 30 + 21 frames of 7 ms,
    one presentation with one sequence segment per sheet and well-formed tables and trains, a 72-frame lead-in, 0.8 cpd
    and 2 Hz, the declared steps, and the presentation's duration. Returns (period, layout)."""
    if any(s['stimulus']['name'] == 'InternalStimulus' for s in ex['segments']):
        refuse(f'{name}: a separate warm-up (InternalStimulus) was run; the declared design has none')
    cache = ex.get('lgn_cache')
    if not isinstance(cache, dict) or cache.get('enabled') is not False:
        refuse(f'{name}: the LGN cache is not recorded as disabled ({cache!r})')
    if [ex.get('step_frames'), ex.get('gap_frames')] != [ap.STEP_FRAMES, ap.GAP_FRAMES]:
        refuse(f"{name}: step and gap frames {ex.get('step_frames')}, {ex.get('gap_frames')} are not {ap.STEP_FRAMES}, "
               f'{ap.GAP_FRAMES}')
    if len(chunks) != 1:
        refuse(f'{name}: {len(chunks)} presentations; the declared design has one')
    chunk = chunks[0]
    missing = [sh for sh in ap.SHEETS if sh not in chunk]
    if missing:
        refuse(f'{name}: no sequence segment for {missing}')
    per_sheet = {sh: sum(1 for s in ex['segments'] if s.get('sheet') == sh and s['stimulus']['name'] == a.SEQUENCE_STIMULUS)
                 for sh in ap.SHEETS}
    if any(n != 1 for n in per_sheet.values()):
        refuse(f'{name}: sequence segments per sheet {per_sheet}; the declared export has one per sheet')
    try:
        ap.check_tables(name, ex)
        ap.check_segments(name, ex, chunk)
    except ap.Refusal as exc:
        refuse(str(exc))
    stims = [chunk[sh]['stimulus'] for sh in ap.SHEETS]
    if any(st != stims[0] for st in stims):
        refuse(f"{name}: the sheets' stimulus records differ")
    st = stims[0]
    if [st.get('step_frames'), st.get('gap_frames'), st.get('frame_duration')] != [ap.STEP_FRAMES, ap.GAP_FRAMES, ap.FRAME_MS]:
        refuse(f'{name}: step timing differs from the declared {ap.STEP_FRAMES}, {ap.GAP_FRAMES} and {ap.FRAME_MS} ms frames')
    if st.get('lead_frames') != ap.LEAD_FRAMES:
        refuse(f"{name}: lead-in of {st.get('lead_frames')} frames, not the declared {ap.LEAD_FRAMES}")
    if st.get('first_step', 0) != 0:
        refuse(f"{name}: the presentation starts at step {st.get('first_step')}, not 0")
    if [st.get('spatial_frequency'), st.get('temporal_frequency')] != [ap.SPATIAL_FREQUENCY_CPD, ap.TEMPORAL_FREQUENCY_HZ]:
        refuse(f"{name}: gratings of {st.get('spatial_frequency')} cpd and {st.get('temporal_frequency')} Hz, not the "
               f'declared {ap.SPATIAL_FREQUENCY_CPD} cpd and {ap.TEMPORAL_FREQUENCY_HZ} Hz')
    try:
        presented = json.loads(st['sequence']) if isinstance(st.get('sequence'), str) else None
    except ValueError:
        presented = None
    if presented != shown:
        refuse(f'{name}: presented sequence differs from the declared sequence')
    expected = a.chunk_bounds(len(shown), 0, ap.LEAD_FRAMES)
    found = [(int(st.get('first_step', 0)), len(presented), st.get('lead_frames', 0.0))]
    if found != [(f, last - f, float(lead)) for f, last, lead in expected]:
        refuse(f'{name}: presentation {found} is not the declared {expected}')
    period = (ap.STEP_FRAMES + ap.GAP_FRAMES) * ap.FRAME_MS
    layout = [(f, last - f, lead * ap.FRAME_MS) for f, last, lead in expected]
    duration = layout[0][2] + len(shown) * period
    if st.get('duration_ms') != duration:
        refuse(f"{name}: presentation of {st.get('duration_ms')} ms, not the declared {duration}")
    for sh in ap.SHEETS:
        t = chunk[sh].get('t_stop_ms')
        if not (isinstance(t, (int, float)) and not isinstance(t, bool) and abs(t - duration) < 1.0):
            refuse(f"{name}: the {sh} sequence segment ends at {t!r} ms, not at the presentation's {duration} ms")
    return period, layout


def read_sequence(base, decl):
    """The instance's declared steps and conditions: its sequence file must match the contract's SHA-256 and pass
    protocol 022's sequence check (analyze_plaids.check_sequence)."""
    seq_file = base / decl['sequence']['file']
    if not seq_file.is_file():
        refuse(f'the sequence file {decl["sequence"]["file"]} is not in the package')
    if sha256(seq_file) != decl['sequence']['sha256']:
        refuse(f'{seq_file.name} does not match its SHA-256 in the contract')
    try:
        return ap.check_sequence(json.loads(seq_file.read_text()))
    except ValueError as exc:
        refuse(f'{seq_file.name} cannot be read ({type(exc).__name__}: {exc})')
    except ap.Refusal as exc:
        refuse(f'{seq_file.name}: {exc}')


# ---------------------------------------------------------------- the bands and the lead-in

def single_means(ex, chunks, layout, period, cond):
    """plaid_stats.Means of the four summaries the bands read (the 45 and 135 deg layer 4 excitatory groups and all
    X_ON and X_OFF cells, in window G), filled only for the 45@30 and 135@30 singles; every other condition is nan.
    017's counting returns every step's counts; only the rows of those two singles' steps are read."""
    names = ['G|L4E|45', 'G|L4E|135', 'G|X_ON|all', 'G|X_OFF|all']
    M = np.full((ps.N_CONDITIONS, len(names)), np.nan)
    rows = {lab: np.where(cond == ps.INDEX[lab])[0] + 1 for lab in SINGLES_30}     # + 1: the warm-up is step 0
    win = list(ps.WINDOWS_MS['G'])
    prefs = np.asarray(ex['neurons']['V1_Exc_L4']['orientation_pref'], float)
    l4 = [('V1_Exc_L4', i) for i in ex['neurons']['V1_Exc_L4']['ids']]
    c4 = a.counts_searchsorted(chunks, layout, l4, period, win)
    cells = {}
    for col, g in enumerate((45, 135)):
        mask = ps.group_mask(prefs, g)
        cells[str(g)] = int(mask.sum())
        for lab, r in rows.items():
            M[ps.INDEX[lab], col] = float(c4[r][:, mask].mean(axis=1).mean())
    for col, sh in enumerate(LGN, 2):
        c = a.counts_searchsorted(chunks, layout, [(sh, i) for i in ex['neurons'][sh]['ids']], period, win)
        for lab, r in rows.items():
            M[ps.INDEX[lab], col] = float(c[r].mean(axis=1).mean())
    cells['90'] = int(ps.group_mask(prefs, 90).sum())
    return ps.Means(M, np.zeros((ps.N_CONDITIONS, 0), complex), names, []), cells


def bands(m):
    """plaid_stats.validation (022's 30 % check, unchanged), split: its LGN checks are refusals here, its layer 4
    checks are reported only."""
    v = ps.validation(m, 'intact', lgn=True)
    lgn = [c for c in v['checks'] if c['what'].startswith('X_')]
    l4 = [c for c in v['checks'] if c['what'].startswith('L4E')]
    return {'lgn': {'checks': lgn, 'passes': all(c['passes'] for c in lgn), 'band': list(ps.VALIDATION['lgn_ratio']),
                    'role': 'refusal'},
            'layer4': {'checks': l4, 'inside_band': all(c['passes'] for c in l4), 'band': list(ps.VALIDATION['ratio']),
                       'role': 'reported only, never a refusal or a replacement'}}


def leadin_trains(chunk, lead_ms=LEAD_MS):
    """{sheet: {id: [spike times before lead_ms]}}, as make_leadin_reference_023.py extracts them."""
    out = {}
    for sh in LGN:
        spikes = chunk[sh]['spikes']
        out[sh] = {i: [float(t) for t in spikes[i] if float(t) < lead_ms] for i in sorted(spikes, key=int)}
    return out


def leadin_identity(trains, reference):
    """Per LGN sheet: the cells compared, how many lead-in trains are identical, and the first ten that differ."""
    out = {}
    for sh in LGN:
        mine, ref = trains[sh], reference['trains'][sh]
        same_cells = set(mine) == set(ref)
        common = sorted(set(mine) & set(ref), key=int)
        differ = [i for i in common if mine[i] != ref[i]]
        out[sh] = {'cells': len(mine), 'reference_cells': len(ref), 'same_cells': same_cells,
                   'identical': len(common) - len(differ), 'differing': len(differ), 'first_differing': differ[:10],
                   'passes': same_cells and not differ}
    out['passes'] = all(out[sh]['passes'] for sh in LGN)
    return out


# ---------------------------------------------------------------- one export

def check_loaded(contract, base, k, ex, chunks, reference, rec):
    """Every check of a loaded export (ex, chunks from 017's load_run) into the record rec; raises Refusal for the
    first identity or structure fault, and records band and lead-in failures as refusals with their kinds."""
    decl = declaration(contract, k)
    rec['role'] = decl['role']
    shown, cond = read_sequence(base, decl)
    try:
        rec['run_name'] = check_identity(ex, decl, contract)
        period, layout = check_structure(f'instance {k}', ex, chunks, shown)
    except (OverflowError, RecursionError, ValueError, TypeError, KeyError, AttributeError) as exc:
        refuse(f'the export is malformed ({type(exc).__name__}: {exc})')
    rec['checks']['run'] = {'passes': True, 'run_name': rec['run_name'], 'argv': ex['argv'], 'resolved': ex['resolved'],
                            'nest': ex.get('nest'), 'presentations': layout, 'step_period_ms': period}
    m, cells = single_means(ex, chunks, layout, period, cond)
    b = bands(m)
    rec['checks']['lgn_band'] = b['lgn']
    rec['report']['layer4_band'] = b['layer4']
    rec['report']['group_cells_L4E'] = cells
    ident = leadin_identity(leadin_trains(chunks[0]), reference)
    rec['checks']['leadin_identity'] = ident
    rec['report']['run_seconds'] = ex.get('run_seconds')
    rec['report']['nest_kernel'] = ex.get('nest')
    if not b['lgn']['passes']:
        rec['refusals'].append('the LGN band: ' + '; '.join(
            f"{c['what']} is {c['value']:.3f} counts per cell, {c['ratio']:.3f} of {c['reference']:.2f}, outside "
            f"{c['band'][0]:.2f} to {c['band'][1]:.2f}" for c in b['lgn']['checks'] if not c['passes']))
        rec['kinds'].append('lgn_band')
    if not ident['passes']:
        rec['refusals'].append('the LGN lead-in identity: ' + '; '.join(
            f"{sh} {ident[sh]['identical']} of {ident[sh]['reference_cells']} trains identical"
            f"{'' if ident[sh]['same_cells'] else ', other cells'}" for sh in LGN if not ident[sh]['passes']))
        rec['kinds'].append('leadin_identity')


def new_record(k, export):
    return {'scope': 'protocol 023: blind integrity check of one instance export; it computes no plaid quantity; '
                     'simulation only',
            'pynn_seed': k, 'export': Path(export).name, 'refusals': [], 'kinds': [], 'checks': {}, 'report': {}}


def check_export(contract, base, k, export, reference, digest=None):
    """The blind check of instance K's export: a record with 'valid', the refusals, their kinds and the reported
    values. digest: the SHA-256 the caller verified (local_export, or the server check's own); the export must still
    have it after the check, or the check stops (the file changed: not a property of the run). A read failure here is
    never a refusal: an export 017's loader cannot read is read a second time, and refused as 'unreadable' only if it
    fails again while its bytes read without an error and still have the checked SHA-256, so that the fault lies in the
    export's content (the host's bytes), not in reading them here."""
    rec = new_record(k, export)
    rec['export_sha256'] = digest if digest is not None else sha256(export)
    try:
        declaration(contract, k)
        try:
            ex, chunks = ap.load(f'instance {k}', export)
        except ap.Refusal:
            same_bytes(export, rec['export_sha256'])
            try:                                # a second read: a fault of the content fails the same way twice
                ex, chunks = ap.load(f'instance {k}', export)
            except ap.Refusal as exc:
                same_bytes(export, rec['export_sha256'])
                refuse(str(exc), 'unreadable')
        check_loaded(contract, base, k, ex, chunks, reference, rec)
        del ex, chunks
    except Refusal as exc:
        rec['refusals'].append(str(exc))
        rec['kinds'].append(exc.kind)
    same_bytes(export, rec['export_sha256'])
    rec['valid'] = not rec['refusals']
    return rec


def same_bytes(export, digest):
    """Stop, writing nothing, unless the export's bytes read without an error and have the checked SHA-256: a read
    fault here, or a file that changed while it was checked, is never an integrity refusal."""
    try:
        now = sha256(export)
    except OSError as exc:
        stop(f'{export} cannot be read here ({type(exc).__name__}: {exc}); nothing is written')
    if now != digest:
        stop(f'{export} changed while it was checked; nothing is written')


# ---------------------------------------------------------------- the server check

def made_from(contract, rel):
    """The SHA-256 the contract records for a file it was made from, by its path in the package (rel)."""
    hits = [h for key, h in (contract.get('made_from') or {}).items() if key == rel or key.endswith('/' + rel)]
    if len(hits) != 1:
        stop(f'the contract records no single SHA-256 for {rel}')
    return hits[0]


def check_running(contract):
    """On the server: the running checker and the code it imports are the files the contract was made from (the
    frozen ones), and protocol 017's package and pins match (analyze_plaids.check_017). Returns the record."""
    for rel, path in RUNNING.items():
        if sha256(path) != made_from(contract, rel):
            stop(f'the running {rel} is not the one the contract was made from')
    _, _, p017 = ap.check_017(contract)
    return {'checked_against': "the contract's made_from", 'files': sorted(RUNNING), 'protocol_017': p017}


def server_check(contract_path, k, export, out):
    """The server check (run_queue_023.sh, right after each run that exits 0): the blind checker's checks of the new
    export on the server, by the server's copy of the frozen package (in the launch list the queue checks before every
    attempt). Writes OUT once and returns SERVER_PASSED or SERVER_REFUSED; it stops (exit 1, no record) when the check
    cannot be made. If OUT already is this checker's record of this export for this contract (the queue was stopped
    after the check), it returns that record's verdict and checks nothing again."""
    if not re.fullmatch(r'\d+', k):
        stop(f'{k!r} is not a pynn_seed')
    k = int(k)
    contract, base = load_contract(contract_path)
    try:
        declaration(contract, k)
    except Refusal as exc:
        stop(str(exc))
    running = check_running(contract)
    path = Path(export)
    if path.name != f's{k}.json.gz':
        stop(f'{path} is not the export of pynn_seed {k} (s{k}.json.gz)')
    try:
        st = os.lstat(path)
    except OSError as exc:
        stop(f'no export at {path} ({type(exc).__name__})')
    if not stat.S_ISREG(st.st_mode):
        stop(f'{path} is not a regular file')
    try:
        digest = sha256(path)
    except OSError as exc:
        stop(f'{path} cannot be read ({type(exc).__name__}: {exc})')
    contract_sha, checker_sha = sha256(contract_path), sha256(__file__)
    if Path(out).exists():
        rec = read_record(Path(out), k, contract_sha, checker_sha, {path.name: digest}, where='server')
        print(f"pynn_seed {k}: already checked on the server: {'PASSED' if rec['valid'] else 'REFUSED'}")
        return SERVER_PASSED if rec['valid'] else SERVER_REFUSED
    reference = load_reference(contract, base)
    rec = check_export(contract, base, k, path, reference, digest)
    rec.update(where='server', checker_sha256=checker_sha, contract_sha256=contract_sha, running_code=running,
               utc=time.strftime('%Y-%m-%dT%H:%M:%SZ', time.gmtime()))
    write_once(out, rec)
    print(f"pynn_seed {k}: server check {'PASSED' if rec['valid'] else 'REFUSED'}" +
          ''.join(f'\n  refused ({kd}): {r}' for r, kd in zip(rec['refusals'], rec['kinds'])))
    return SERVER_PASSED if rec['valid'] else SERVER_REFUSED


# ---------------------------------------------------------------- the replacement rule and its one record

def statuses(queue, records, seeds=None):
    """Each seed's status for the rule: 'valid', 'invalid: <cause>', 'pending' (done, not checked) or 'not run'."""
    out = {}
    for k in seeds or list(cf.SLOTS) + list(cf.RESERVES):
        q = queue[k]
        if q == 'not run':
            out[k] = 'not run'
        elif q != 'done':
            out[k] = f'invalid: the run {q}'
        elif k not in records:
            out[k] = 'pending'
        elif records[k]['valid']:
            out[k] = 'valid'
        else:
            out[k] = 'invalid: refused by the blind check (' + '; '.join(records[k]['refusals']) + ')'
    return out


def parse_lines(text, pattern, name):
    """The lines of one of the queue's records, each matched to its declared form; and the lines that are not."""
    out, problems = [], []
    for x in text.splitlines():
        if not x.strip():
            continue
        m = pattern.match(x)
        if m:
            out.append(m.groups())
        else:
            problems.append(f'{name}: a line the queue does not write: {x!r}')
    return out, problems


def replacement_lines(text):
    """The queue's one replacement record (replacements.txt), in time order: [{'utc', 'reserve' (None for 'no reserve
    left'), 'slot', 'cause'}], and the lines that are not of its form."""
    rows, problems = parse_lines(text, REPLACEMENT_LINE, 'replacements.txt')
    return [{'utc': u, 'reserve': int(r) if r else None, 'slot': int(s), 'cause': c} for u, r, s, c in rows], problems


def cause_problem(who, slot, cause, queue, records):
    """Why a replacement line's cause is not what made `who` (the slot itself, or the reserve last given to it)
    invalid; None if it is."""
    m = RUN_CAUSE.match(cause) or REFUSED_CAUSE.match(cause)
    if not m:
        return f'a cause the queue does not write: {cause!r}'
    named = int(m.group(1)) if m.group(1) else slot
    if named != who or (m.group(1) is None) != (who == slot):
        return (f"the cause is about {'the slot' if m.group(1) is None else 'reserve ' + m.group(1)}, but the seed it "
                f'replaces is {who}')
    if m.re is RUN_CAUSE:
        if queue.get(who) != m.group(2):
            return f'pynn_seed {who} is {queue.get(who)}, not {m.group(2)}'
        return None
    rec = records.get(who)
    if rec is None or rec['valid'] or rec['sha256'] != m.group(2):
        return f'the check record {m.group(2)} of pynn_seed {who} is not a refusal among the check records'
    return None


def replacement(status, queue, records, lines, slots=None, reserves=None):
    """The declared replacement rule, read with the queue's one record of it (lines: replacements.txt, in time order).
    A slot is invalid if its run failed twice or was killed for memory, or the blind checker refused its export. Each
    invalid slot gets the next unused reserve (13, then 14); a reserve that is itself invalid is followed by the next
    unused reserve for the same slot; at most two replacements. Each line must be the rule's: the next unused reserve
    (so no reserve fills two slots), for a slot whose own run, or whose last reserve, is invalid for the cause the line
    gives ('no reserve left' only once both are used); every reserve that ran is in the record. Returns the state:
    'six valid instances' (with the analysed instances in pynn_seed order and the slot each reserve filled), 'fewer
    than six valid instances' (as soon as more slots need a reserve than are unused), or 'pending'; and the record's
    problems, on which the summary stops."""
    slots = list(cf.SLOTS if slots is None else slots)
    reserves = list(cf.RESERVES if reserves is None else reserves)
    problems, used, chain, exhausted = [], [], {s: [] for s in slots}, []
    for i, ln in enumerate(lines, 1):
        s, r, cause = ln['slot'], ln['reserve'], ln['cause']
        where = f"replacements.txt line {i} ({'reserve ' + str(r) if r is not None else 'no reserve left'} for slot {s})"
        if s not in chain:
            problems.append(f'{where}: {s} is not a slot')
            continue
        if s in exhausted:
            problems.append(f'{where}: slot {s} already had no reserve left')
            continue
        who = chain[s][-1] if chain[s] else s
        why = cause_problem(who, s, cause, queue, records)
        if why:
            problems.append(f'{where}: {why}')
        if r is None:
            if len(used) < len(reserves):
                problems.append(f'{where}: reserve {reserves[len(used)]} was still unused')
            exhausted.append(s)
            continue
        nxt = reserves[len(used)] if len(used) < len(reserves) else None
        if r != nxt:
            problems.append(f"{where}: {r} is not the next unused reserve ({nxt if nxt else 'none was left'})")
            continue
        used.append(r)
        chain[s].append(r)
    for r in reserves:
        if queue.get(r, 'not run') != 'not run' and r not in used:
            problems.append(f'reserve {r} ran, but replacements.txt gives it to no slot')
    analysed, filled_by, need, pending = [], {}, [], []
    for s in slots:
        who = chain[s][-1] if chain[s] else s
        st = status.get(who, 'not run')
        if st == 'valid':
            analysed.append(who)
            if who != s:
                filled_by[str(s)] = who
        elif st in ('pending', 'not run'):
            pending.append(who)
        else:
            need.append(s)
    unused = [r for r in reserves if r not in used]
    out = {'analysed': None, 'filled_by': filled_by,
           'invalid_slots': [s for s in slots if str(status.get(s)).startswith('invalid')],
           'reserves_used': [{'reserve': r, 'slot': s, 'status': status.get(r, 'not run')}
                             for s in slots for r in chain[s]],
           'pending': sorted(pending), 'needing_a_reserve': need, 'unused_reserves': unused, 'problems': problems}
    if len(need) > len(unused):
        out['state'] = 'fewer than six valid instances'
        out['why'] = (f'fewer than six valid instances after the declared replacements: '
                      f'{count(len(need), "slot")} without a valid instance and {count(len(unused), "unused reserve")} ('
                      + '; '.join(f'slot {s}: ' + '; '.join(f'{x} {status.get(x)}' for x in [s] + chain[s]) for s in need)
                      + ')')
    elif pending or need:
        out['state'] = 'pending'
        out['why'] = 'waiting for ' + '; '.join(
            [f'pynn_seed {x} (not run or not checked)' for x in sorted(pending)] +
            [f'the next reserve for slot {s} (its queue or launch_023.sh reserve)' for s in need])
    else:
        out['state'] = 'six valid instances'
        out['analysed'] = sorted(analysed)
    return out


def count(n, noun):
    return f'{n} {noun}' + ('' if n == 1 else 's')


# ---------------------------------------------------------------- the early stop

def early_stop(lines, queue, records, server, start_utc):
    """The early stop, from the queue's early-stop.txt (lines), the server check record of every done seed (server),
    the local check records and each done seed's start time. The queue stops at the first export its server check
    refuses; every refusal kind is one that a fault shared by every instance would produce, so that refusal ends the
    protocol (NOT INTERPRETABLE), once the local check of the same export has refused it too. Returns {'state':
    'none' | 'stopped' | 'pending', ...} with the problems, on which the summary stops: a done seed without a server
    record, a refusal on the server without an early stop or with an attempt started after it, a stop the server
    records do not give, and a local check that passed what the server check refused."""
    problems = []
    refused = sorted(k for k, rec in server.items() if rec is not None and not rec['valid'])
    for k in sorted(server):
        if server[k] is None:
            problems.append(f'pynn_seed {k} is done, but has no server check record (the queue marks a run done only '
                            'after its server check)')
    if not lines:
        for k in refused:
            problems.append(f'the server check refused pynn_seed {k}, but the queue did not stop (no early-stop.txt)')
        return {'state': 'none', 'problems': problems}
    if len(lines) > 1:
        problems.append(f'early-stop.txt holds {len(lines)} lines; the queue writes one')
    utc, seed, digest = lines[0]
    seed = int(seed)
    rec = server.get(seed)
    out = {'state': 'stopped', 'utc': utc, 'pynn_seed': seed, 'server_record_sha256': digest, 'problems': problems}
    if queue.get(seed) != 'done' or rec is None or rec['valid'] or rec['sha256'] != digest:
        problems.append(f'early-stop.txt: the queue stopped for pynn_seed {seed} with record {digest}, which is not a '
                        "refusal among the fetched server check records")
        return out
    out.update(refusals=list(rec['refusals']), kinds=list(rec['kinds']))
    for k in refused:
        if k != seed:
            problems.append(f'the server check also refused pynn_seed {k}: the queue stops at the first refusal')
    for k, t in sorted(start_utc.items()):
        if k != seed and t > utc:
            problems.append(f'pynn_seed {k} started at {t}, after the early stop at {utc}')
    local = records.get(seed)
    if local is None:
        out['state'] = 'pending'
        out['why'] = f'the local check of pynn_seed {seed}, which the server check refused, has not been made'
    elif local['valid']:
        problems.append(f'the local check passed pynn_seed {seed}, which the server check refused: the two checks of the '
                        'same export disagree')
    return out


def decide(status, queue, records, lines, early):
    """The early stop first, then the replacement rule (whose record is checked either way)."""
    rule = replacement(status, queue, records, lines)
    if early['state'] == 'stopped':
        rule.update(state='stopped by the early check', analysed=None,
                    why=(f"the early stop: the server check refused the export of pynn_seed {early['pynn_seed']} right "
                         'after its run, for a fault taken to be shared by every instance, and the local check refused it '
                         'too (' + '; '.join(early.get('refusals') or []) + ')'))
    elif early['state'] == 'pending':
        rule.update(state='pending', analysed=None, why=early['why'])
    return rule


# ---------------------------------------------------------------- the summary

def queue_status(receipts, k):
    """A seed's state in the fetched queue receipts: 'done', 'failed twice', 'killed for memory' or 'not run'."""
    f = Path(receipts) / f's{k}-status.txt'
    if not f.is_file():
        return 'not run'
    s = f.read_text().strip()
    if s not in QUEUE_STATUS:
        stop(f'{f} holds {s!r}, not one of {QUEUE_STATUS}')
    return s


def done_receipts(receipts, k):
    """A done attempt's own receipts: exit 0 and the server files' hashes the same at its start and end (the queue
    never marks done an attempt during which they changed). Returns its start time and its peak resident size in kB
    (time -v's maximum resident set size; MemoryPeak is the unit's running maximum), or None."""
    r = Path(receipts)
    exit_file, start, end = r / f's{k}-exit.txt', r / f's{k}-sha256-start.txt', r / f's{k}-sha256-end.txt'
    if not exit_file.is_file() or exit_file.read_text().strip() != '0':
        stop(f's{k} is done, but its receipt s{k}-exit.txt does not hold 0')
    if not (start.is_file() and end.is_file()) or start.read_bytes() != end.read_bytes() or not start.read_bytes().strip():
        stop(f's{k} is done, but its server-file hashes at start and end are missing or differ')
    utc = r / f's{k}-start-utc.txt'
    if not utc.is_file() or not re.fullmatch(r'\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}Z', utc.read_text().strip()):
        stop(f's{k} is done, but its receipt s{k}-start-utc.txt holds no UTC time')
    tv = r / f's{k}-time-v.txt'
    m = PEAK_RSS.search(tv.read_text()) if tv.is_file() else None
    return utc.read_text().strip(), (int(m.group(1)) if m else None)


def read_record(path, k, contract_sha, checker_sha, hosts, where='local'):
    """A check record (where: 'local', made on the Mac, or 'server', made by the queue), or a stop unless it is the
    frozen checker's record of instance K's host export for this contract."""
    try:
        rec = json.loads(Path(path).read_text())
    except (OSError, ValueError) as exc:
        stop(f'{path} cannot be read ({type(exc).__name__}: {exc})')
    problems = []
    if not isinstance(rec, dict):
        stop(f'{path} is not a check record')
    if rec.get('where') != where:
        problems.append(f"it is a {rec.get('where')!r} record, not a {where!r} one")
    if rec.get('pynn_seed') != k:
        problems.append(f"it is the check of pynn_seed {rec.get('pynn_seed')!r}")
    if rec.get('contract_sha256') != contract_sha:
        problems.append('it was made for another contract')
    if rec.get('checker_sha256') != checker_sha:
        problems.append('it was made by another checker than this frozen one')
    if rec.get('export') != f's{k}.json.gz' or rec.get('export_sha256') != hosts.get(f's{k}.json.gz'):
        problems.append("its export is not the host's file in data-sha256.txt")
    refusals, kinds = rec.get('refusals'), rec.get('kinds')
    if (not isinstance(rec.get('valid'), bool) or not isinstance(refusals, list) or not isinstance(kinds, list)
            or len(refusals) != len(kinds) or any(kd not in KINDS for kd in kinds) or rec['valid'] != (not refusals)):
        problems.append('its verdict, refusals and kinds are not a check record\'s')
    if problems:
        stop(f'{path}: ' + '; '.join(problems))
    return {'file': Path(path).name, 'sha256': sha256(path), 'where': where, 'valid': rec['valid'], 'refusals': refusals,
            'kinds': kinds, 'export': rec['export'], 'export_sha256': rec['export_sha256'],
            'contract_sha256': rec['contract_sha256'], 'checker_sha256': rec['checker_sha256']}


def read_text(path):
    return Path(path).read_text() if Path(path).is_file() else ''


def summary(contract_path, fetched, checks):
    """The status of every slot and reserve from the fetched receipts and the local check records; the server check
    of every done run; the early stop; and the replacement rule read with the queue's one record of it. Stops, writing
    nothing, on any record that is not the declared procedure's."""
    receipts = Path(fetched) / 'receipts'
    if not receipts.is_dir():
        stop(f'{receipts} does not exist (fetch_023.sh writes it)')
    hosts = host_hashes(fetched)
    contract_sha, checker_sha = sha256(contract_path), sha256(__file__)
    seeds = list(cf.SLOTS) + list(cf.RESERVES)
    queue, records, server, peaks, start_utc = {}, {}, {}, {}, {}
    for k in seeds:
        queue[k] = queue_status(receipts, k)
        local = Path(checks) / f'check-s{k}.json'
        if queue[k] == 'done':
            start_utc[k], peaks[k] = done_receipts(receipts, k)
            srv = receipts / f's{k}-server-check.json'
            server[k] = (read_record(srv, k, contract_sha, checker_sha, hosts, where='server') if srv.is_file()
                         else None)
            if local.is_file():
                records[k] = read_record(local, k, contract_sha, checker_sha, hosts)
        elif local.exists():
            stop(f'check-s{k}.json exists, but the queue did not mark s{k} done')
    status = statuses(queue, records, seeds)
    lines, problems = replacement_lines(read_text(receipts / 'replacements.txt'))
    stops_, more = parse_lines(read_text(receipts / 'early-stop.txt'), EARLY_STOP_LINE, 'early-stop.txt')
    problems += more
    early = early_stop(stops_, queue, records, server, start_utc)
    rule = decide(status, queue, records, lines, early)
    problems += early['problems'] + rule['problems']
    if problems:
        stop("the queue's records do not follow the declared procedure; nothing is written:\n  - " + '\n  - '.join(problems))
    return {'scope': 'protocol 023: the instances to analyse, from the queue receipts and the blind checks; simulation only',
            'queue_status': {str(k): v for k, v in queue.items()},
            'status': {str(k): v for k, v in status.items()},
            'check_records': {str(k): v for k, v in records.items()},
            'server_checks': {str(k): v for k, v in server.items()},
            'start_utc': {str(k): v for k, v in start_utc.items()},
            'replacement_record': lines, 'early_stop_record': [list(x) for x in stops_],
            'early_stop': early, 'rule': rule,
            'attempt_peak_rss_kb': {str(k): v for k, v in peaks.items()},
            'data_sha256': {'file': 'data-sha256.txt', 'sha256': sha256(Path(fetched) / 'data-sha256.txt')},
            'utc': time.strftime('%Y-%m-%dT%H:%M:%SZ', time.gmtime())}


def write_once(path, obj):
    text = json.dumps(ap.clean(obj), indent=1, allow_nan=False) + '\n'
    try:
        with open(path, 'x', encoding='utf-8') as fh:
            fh.write(text)
    except FileExistsError:
        stop(f'{path} exists; a check record is never overwritten')


def load_contract(path):
    try:
        contract = json.loads(Path(path).read_text())
    except (OSError, ValueError) as exc:
        stop(f'the contract {path} cannot be read ({type(exc).__name__}: {exc})')
    return contract, Path(path).resolve().parent


def load_reference(contract, base):
    ref = contract['integrity']['leadin_identity']
    path = base / ref['file']
    if sha256(path) != ref['sha256']:
        stop(f'{path} does not match its SHA-256 in the contract')
    return json.loads(path.read_text())


def main(args):
    allow_unfrozen = '--not-frozen' in args
    args = [x for x in args if x != '--not-frozen']
    if len(args) == 5 and args[0] == 'check':
        _, contract_path, k, fetched, out = args
        if not re.fullmatch(r'\d+', k):
            stop(f'{k!r} is not a pynn_seed')
        k = int(k)
        contract, base = load_contract(contract_path)
        if Path(out).exists():
            stop(f'{out} exists; a check record is never overwritten')
        frozen = check_frozen(base, contract, allow_unfrozen)
        reference = load_reference(contract, base)
        try:
            declaration(contract, k)
        except Refusal as exc:
            stop(str(exc))
        export, digest = local_export(fetched, k, host_hashes(fetched))
        rec = check_export(contract, base, k, export, reference, digest)
        rec.update(where='local', checker_sha256=sha256(__file__), contract_sha256=sha256(contract_path),
                   frozen_check=frozen, utc=time.strftime('%Y-%m-%dT%H:%M:%SZ', time.gmtime()))
        write_once(out, rec)
        print(f"pynn_seed {k}: {'VALID' if rec['valid'] else 'REFUSED'}" +
              ''.join(f'\n  refused ({kd}): {r}' for r, kd in zip(rec['refusals'], rec['kinds'])))
        return 0 if rec['valid'] else 1
    if len(args) == 5 and args[0] == 'server' and not allow_unfrozen:
        _, contract_path, k, export, out = args
        return server_check(contract_path, k, export, out)
    if len(args) == 5 and args[0] == 'summary':
        _, contract_path, fetched, checks, out = args
        contract, base = load_contract(contract_path)
        if Path(out).exists():
            stop(f'{out} exists; a summary is never overwritten')
        frozen = check_frozen(base, contract, allow_unfrozen)
        res = summary(contract_path, fetched, checks)
        res.update(checker_sha256=sha256(__file__), contract_sha256=sha256(contract_path), frozen_check=frozen)
        write_once(out, res)
        rule = res['rule']
        print(rule['state'] + (f": {rule.get('why')}" if rule.get('why') else '') +
              (f"; analysed {rule['analysed']}" if rule['analysed'] else ''))
        return 0
    raise SystemExit(__doc__)


if __name__ == '__main__':
    sys.exit(main(sys.argv[1:]))
