Skip to content

Consent-aware export — sharing recordings off the acquisition PC

There was no export path anywhere in the tree that read the per-session consent flags (the consent sidecar block's future_use, public_share, observer_only_delete keys) — an operator copying data out by hand had to remember them (#17). CLAUDE.md section 4 requires honouring them on every export/share, mechanically, not by someone remembering. Code: src/ssvep/io/export.py; UI: the Export… row on the Analyze tab of ssvep-toolbox.

This is a different operation from the auto-push described in docs/PUSH.md: push mirrors a finished run to the lab's own Canadian server as internal storage (the sanctioned local → server → compute chain, COMPLIANCE R1/R3); export copies a consent-filtered subset of the dataset out, for reuse or public sharing, to wherever the operator points the destination folder. The two paths share the same discipline — re-read consent from the sidecar on disk, never trust a caller-supplied list — but gate on different flags, because they answer different questions.

1. The flag → purpose mapping

Two purposes, in a dropdown on the Analyze tab:

Purpose (dropdown label) export module constant Gated by (consent block of the session sidecar)
Future use (internal reuse) export.FUTURE_USE consent.future_use
Public share export.PUBLIC_SHARE consent.public_share

Regardless of purpose, a session with consent.observer_only_delete: true is never exported — COMPLIANCE R5 says that data must be purged at session end and never leave the machine under any purpose, the same rule ssvep.runtime.push.consent_block_reason enforces on the push-to-server path. This is checked before the purpose-specific flag, so an observer-only session is held back the same way for both dropdown choices.

Missing or unreadable is always "no", never "yes" — R5's "default to the most restrictive interpretation when a flag is missing" applies to every flag read here, not only the delete one:

Situation Exported?
consent.<purpose flag>: true, observer_only_delete: false ✅ yes
consent.<purpose flag>: false ❌ no — not granted for this purpose
consent.<purpose flag> absent ❌ held back — R5: most restrictive when a flag is missing
consent.observer_only_delete: true (any run in the session) ❌ the whole session is held back, under either purpose
consent.observer_only_delete absent ❌ held back — same rule as push
no *.session.json sidecar, or an unreadable one ❌ held back — consent unknown

The gate is per-session, not per-subject: a subject with two sessions, one consented for public share and one not, exports only the consented one. participants.tsv in the export then lists only subjects with at least one exported session — see §3.

export.export_dataset(bids_root, destination, purpose) copies, into destination:

  1. The de-identified, dataset-level scaffold, verbatim — dataset_description.json, README, CHANGES, participants.json, every task-*_events.json, channels.json, .bidsignore. These are exactly the files docs/BIDS.md/CLAUDE.md §4 describe as the static, tracked scaffold: no participant data, generated from constants, safe to hand out whole.
  2. participants.tsv, filtered — not copied verbatim, because unlike the files above it grows a row per participant and is git-ignored precisely because it does (CLAUDE.md §4). Only rows for subjects with at least one session eligible for the chosen purpose are kept.
  3. Per eligible session, path-for-path exactly as it sits locally (same "mirror 1:1" rule docs/PUSH.md uses for the push):
  4. sourcedata/sub-XXX/ses-YYY/ — the raw XDF + its *.session.json (embedded manifest, provenance, consent, impedance) + *_impedance.json snapshots.
  5. sub-XXX/ses-YYY/ — the derived BIDS-standard eeg/ (BrainVision + sidecars), if it has been derived yet (run the Analyze tab or code/01_xdf_to_bids.py first if not — an export never derives anything itself).

Deliberately out of scope for this first version: BIDS/derivatives/ssvep-analysis/ (the analysis outputs — reports, metrics.json). The issue this closes is about the raw/BIDS data export gap; extending the same gate to derivatives is a separate, small change if it's ever wanted — raise it with Aaron rather than assuming.

Nothing is ever read from outside bids_root. Every path the export touches is built by joining bids_root with a relative sub-path, so nothing that isn't already part of the de-identified BIDS tree can leak into what gets shared — no config/, no Reference/, no .ssh, nothing from the operator's own filesystem. A symlink anywhere in the tree (a file or a directory) is skipped rather than followed, for the same reason: a link could otherwise point outside bids_root and have its target copied.

Re-exporting into a destination that already holds an earlier export reconciles it, never just adds to it. Copying only ever overwrites matching paths — nothing is wiped wholesale first — but before anything new is copied in, any sub-XXX/ses-YYY already under destination that is not currently eligible for the chosen purpose is removed. Otherwise a session whose consent was revoked (or that vanished from the source) between two exports into the same folder would linger there, and an operator has every reason to treat that folder as fully consent-filtered.

3. Why participants.tsv and the rest are filtered

De-identification is enforced on shape, not convention (docs/DATA_GOVERNANCE.md); at Dalhousie that binds to docs/COMPLIANCE.md R2, which limits participants.tsv to sub-XXX + coarse demographics — no names/DOB/contact/session dates. That is already the shape of the file on disk (nothing in this tree invents richer demographics), so filtering it is mostly about rows: a row is still a fact about a specific coded participant, so a subject excluded from an export (no eligible session under that purpose) must not have so much as a coarse-demographics row appear in it. export.py reads the source tsv, keeps only rows whose participant_id has at least one included session, and writes that subset.

It also whitelists columns to exactly the documented set (participant_id, species, group, age, sex, hand, weight, height; see BIDS/participants.json), dropping anything else — not because the source file is expected to carry more (nothing in this tree writes a richer one), but because this is the one mechanically-enforced gate standing between a hand-edited or future participants.tsv and a shared destination, and R2 is exactly the rule it would be there to catch.

Every other scaffold file (§2.1) carries no participant-scoped rows at all — dataset_description. json, README, task-*_events.json, etc. describe the dataset, not a participant — so they are copied whole, unfiltered.

4. Using it — the Analyze tab

Export… sits below the Analyze/Open report/Open index row, with a purpose dropdown:

  1. Pick Future use (internal reuse) or Public share.
  2. Click Export…. The tool re-scans every session under the BIDS root's sourcedata/ and re-reads each one's consent sidecar right then — not from whatever the tree above happens to have ticked (that selection is for analysing, a different operation; export always considers the whole dataset for the chosen purpose).
  3. If nothing is eligible, you're told so (and shown why, per session) and nothing further happens.
  4. Otherwise you see a count of what will be exported and what will be held back (with reasons), and confirm before anything is copied.
  5. Choose a destination folder via the file dialog (default under the repo, never a relative path resolving against wherever the app happened to launch from — ui._paths.repo_dir, the same fix every other dialog default in this codebase uses).
  6. The copy runs on a worker thread (a full dataset can be large) and the status label + a summary dialog report what was exported and what was held back when it finishes.

Nothing here decides when to export or who receives the folder afterwards — that remains a human call, same as it is for the push-to-server path. This tool's job ends at "here is the consent-correct subset, on disk, ready to hand off"; carrying it further (e.g. off the lab's Canadian machines) is still bound by COMPLIANCE R3 like any other transfer.

5. Compliance notes

No PII added. The export is a filtered copy of an already de-identified tree (R1/R2); it invents nothing and reads nothing from outside bids_root.

Public share still means Canada+USA per R3 until the data is actually released publicly — R3 limits anonymized data access to Canada+USA IPs; a "Public share" export is a step toward a release, not itself an open publication, and the destination folder is subject to the same residency rule as anything else until a release decision (outside this tool) actually publishes it.

No new dependency. The copy is shutil.copy2 + stdlib csv/json/pathlib — nothing in environment.yml needed to change.

Testing. tests/test_export.py covers the consent gate as a pure function (eligible_sessions/excluded_sessions/consent_gate_reason) independent of any file copy or GUI, plus the file-copy behaviour of export_dataset (participants.tsv filtering, scaffold verbatim-copy, nothing pulled from outside bids_root). tests/test_analyze_ui.py covers the panel wiring — the purpose dropdown, that "nothing eligible" and "cancelled" never reach the folder picker, and that a finished/failed result folds back into the status label — without starting the real worker thread, the same house style the batch-analysis tests already use.