Benchmarking channel montages against each other

ssvep.analysis.montage_benchmark compares calibration-free decoding across named channel subsets of one dense recording. It exists because a standard occipital montage — the toolbox's own P1 PO3 O1 POz Oz P2 PO4 O2 (docs/MONTAGE_1010.md) among them — is a strict subset of any 10-10 cap that includes those sites: "how much does an 8-channel montage give up against a full cap?" does not need two recordings, only one recorded on the denser montage, with the smaller montages derived offline by selecting channels from it.

This module is deliberately not a new decoder. Every number it reports comes from ssvep.analysis.pipeline.run_by_condition (which calls run_analysis once per condition) — the same functions the shipped offline pipeline runs per recording — or from the existing cross-validated greedy channel search (ssvep.analysis.channel_search.search). It only decides which channels each arm decodes and which reference each arm sees, and makes sure every arm is otherwise decoded identically.

Four design points, all load-bearing

The reference answers a different question depending on what device is being modelled — pick reference_mode accordingly. benchmark_montages re-references each named montage one of two ways:

• "device" (the default) re-references a montage to the reference that montage's own device declares. A montage is either a plain channel-label list (no declared reference) or {"channels": [...], "reference": ...}, where reference is a channel label or label list present in the dense recording — e.g. the toolbox's own 8-channel occipital montages put their Cyton SRB reference on a single left earlobe (docs/MONTAGE_1010.md, docs/MONTAGE_FREE_ELECTRODE.md); a dense 10-10 cap has no earlobe channel of its own, so a linked-mastoid pair, ["TP9", "TP10"], approximates it — it is not the same reference. The pair is bilateral where the shipped device's is one-sided, sits on scalp that itself carries more EEG (temporal, posterior alpha) than an earlobe does, and every arm scored against it is only as good as those two contacts' quality on the dense cap. A single "TP9" alone is the closer one-sided analogue of the real reference; the pair is more stable but answers a related, not identical, question. reference can also be "car" (this device genuinely has no separate reference electrode, only a common average over its own inputs — a real device some reference-free amps are built as, refused under 3 channels as degenerate) or "none" (this montage's channels are already true-referenced on their own). A montage with no declared reference raises when the dense recording itself has no true hardware reference (full_reference in ssvep.spatial._UNREFERENCED) — there is no safe default to fall back to: a common average over the montage's own (often compact, occipital) cluster would subtract the mean of exactly the channels carrying the SSVEP, scoring a signal montage at chance while looking like a legitimate result. An explicit "none" declaration raises under the same condition and for the same reason — on an unreferenced dense recording, a montage's channels cannot already be "true-referenced on their own" by construction, so accepting the declaration would silently reproduce the exact chance-level escape hatch the raise exists to close. A declared reference that names one of the montage's own channels is refused too — a device cannot use one of its own inputs as its reference and also record it as a signal channel. This is the adopter's question: "how much does this montage give up as a standalone device?"
• "shared" instead re-references once, over every channel of the dense recording, and only then subsets to each montage's channels (a montage's declared reference is ignored in this mode). This leaks the excluded channels' average into every arm — a montage with zero true signal on any of its electrodes can decode near-perfectly, because the number then describes the shared reference, not the montage. It answers a different, narrower question ("how does the dense recording's own analysis look restricted to these channels afterward") and is kept for that comparison, never as a device estimate.

full_occipital_roi and full_all_channels describe the dense recording itself, not a hypothetical smaller device, so they always use the shared/dense reference (ssvep.spatial.occipital_decode_plan, from the recording's own declared full_reference) regardless of reference_mode.

The full-recording ceiling is reported at its own best configuration, not as a bare all-channel decode. Calibration-free CCA/FBCCA has no channel-selection mechanism of its own (ssvep.spatial's module docstring), so handing it every channel of a dense montage lets it overfit its reference sinusoids to noise on non-occipital channels. benchmark_montages returns full_occipital_roi (the full recording restricted to its own occipital ROI, re-referenced per occipital_decode_plan — common-average only when the recording has no true hardware reference, otherwise as recorded — the fair ceiling to compare montages against) and full_all_channels (the unrestricted decode) as two clearly separate keys, so a maintainer comparing a small montage against "the full recording" cannot silently pick the flattering, classifier-property number by accident.

The dense recording's own montage (full_montage) is checked against its real channels, the same as a named montage. A caller-supplied full_montage that names a site the recording doesn't have raises (or, under strict=False, is dropped and recorded as full_montage_missing_channels/full_montage_duplicate_channels) rather than failing open — silently shrinking the occipital ROI without saying so, or, if nothing matches at all, silently expanding to every channel while still being reported as "the occipital ROI." roi_source and the reported decode_channels always describe what was actually decoded.

Decoding happens per condition, and a contrast-reversing design must say so. Every arm is decoded with ssvep.analysis.pipeline.run_by_condition, exactly as the shipped pipeline (ssvep.analysis.batch) does: a recording that mixes conditions (e.g. two stimulation bands) is scored at each condition's own chance level, never pooled into one candidate set, and pattern_reversal (forwarded exactly as run_analysis documents it) moves every reference to the response frequency for a contrast-reversing design. Leaving it unset on a reversal recording is the single most likely way to get a spurious "montage doesn't matter" result. channel_count_curve follows the same rule: it decodes per condition and accepts pattern_reversal too.

Use

from ssvep.analysis import montage_benchmark as mb

result = mb.benchmark_montages(
    epochs,                                   # a dense recording's Epochs
    {
        # a plain channel list is fine when the dense recording already has a true hardware
        # reference (nothing needs declaring); on an unreferenced recording it must declare one:
        "standard_10_10": {"channels": [...], "reference": ["TP9", "TP10"]},
        "alternative_8": {"channels": [...], "reference": ["TP9", "TP10"]},
    },
    full_reference="none",                    # the recording's actual hardware reference
    reference_mode="device",                  # the default: each montage as its own device
)
result["reference_mode"]                      # "device" (or "shared")
result["reref"]                               # the scheme full_occipital_roi/full_all_channels used
result["roi_source"]                          # how the occipital ROI was actually obtained
result["channel_search"]                      # {condition: search-result}, computed once
result["full_occipital_roi"]                  # {"reref", "decode_channels", "by_condition"}
result["full_all_channels"]                   # {"reref", "by_condition"} — the labelled counter-example
result["montages"]["standard_10_10"]          # {"reference_mode", "reref", "reference", "ref_labels", "by_condition"}
result["montages"]["standard_10_10"]["by_condition"][None]   # a run_analysis() result
result["config"]                              # every input this result was produced from

Each by_condition entry maps a condition (None for a single-condition recording) to exactly run_analysis's own return value — accuracy/ITR at every decision window, SNR, TRCA, the confusion matrix — so any metric the shipped pipeline already reports is available for every montage and condition, without this module summarising anything away.

A named montage's channel labels are checked against the recording's own (strict=True by default): a missing label (a case slip, or a 10-20/10-10 alias like T5/T6 for the 10-10 P7/P8) or a duplicated one raises, naming it, rather than silently scoring a smaller or duplicated channel set under the montage's name. Pass strict=False to drop offending labels instead (logged as a warning, and recorded in the result as missing_channels/ duplicate_channels); a montage that resolves to zero known channels always raises, regardless of strict. windows defaults to None (every arm resolves its own usable-length sweep, exactly as run_analysis does with none of its own); an explicit windows containing a value longer than the usable trial length remaining after the onset_offset_s trim raises, naming the offending window(s) — the same rule channel_count_curve's own window_s follows below — rather than silently returning an empty window_sweep for every arm.

channel_count_curve(epochs, ks) reports best-k accuracy across a channel-count sweep (e.g. k = 1, 2, 4, 8, 16, 32, 64), reusing channel_search.search's cross-validated greedy selection at each k rather than a new search. It is put through the same plan benchmark_montages uses for its shared/dense arms, in the same order run_analysis applies them (onset-lead-in trim, then pattern_reversal's relabelling, then the reference occipital_decode_plan recommends for full_reference), so the curve and benchmark_montages are decoding the same thing and can be read side by side. window_s defaults to None — the full usable trial length remaining after the onset trim — rather than a fixed 4 s, which used to silently score every trial as too short once a real onset offset was applied (tot == 0 returning an unflagged 0.0); a window_s longer than that usable length raises instead. Returns {"by_condition": {condition: [row-per-k, ...]}, "reref", "config"} — one row per k in ks, per condition (never pooled, and pattern_reversal is accepted the same way benchmark_montages takes it), each row carrying k and reref alongside channel_search.search's own fields. Per-recording greedy selection is head-specific and optimistic even with its honest split-half correction — this curve is a diagnostic of where returns flatten with channel count on one recording, not a montage to adopt from it; validating any resulting montage change means running this across participants.

Scope

This module takes named channel-label montages and a recording from its caller — it never infers a montage from data, per the manifest spine's rule that design facts come from the manifest, not reverse-engineered from a recording. Which montages to compare, and the recording to compare them on, are deployment- and study-specific: this module only supplies the reusable comparison mechanics. Recruiting participants and running a session that would produce a recording answering "does our default montage hold up against a full cap" is a separate, ethics-board-gated experiment tracked on its own issue; nothing in this module depends on that approval, and its own tests run only against synthetic data (tests/test_montage_benchmark.py).