CAN events and recording manifests¶
Reviewed manifests retain explicit roles, hashes, clock units and provenance. The event data module preserves every row and batches independent chronological streams. See event architecture.
recordings ¶
Explicit, content-checked recording splits; usable without torch.
audit_catalog ¶
Inventory all requested datasets without assigning trusted split roles.
Source code in graphids/data/recordings.py
iter_rows ¶
Yield every row in physical order; validate the entire recording on exhaustion.
Source code in graphids/data/recordings.py
read_rows ¶
events ¶
Lossless frame caches and chronological source-only event chunks.
Caches preserve byte values, lengths, numeric CAN IDs, FP64 recording clocks, row order (including ties), and unknown labels. Chunks never merge or drop rows.
CANEventDataModule ¶
CANEventDataModule(*, manifest_path: str | Path, datasets: list[str], cache_root: str | Path, representation_cfg: CANEventRepresentationConfig | None = None, chunk_events: int = defaults.CHUNK_EVENTS, stream_batch_size: int = defaults.STREAM_BATCH_SIZE, effective_batch_events: int = defaults.EFFECTIVE_BATCH_EVENTS, max_microbatch_events: int = defaults.MAX_MICROBATCH_EVENTS, num_workers: int = defaults.NUM_WORKERS, pin_memory: bool = defaults.PIN_MEMORY, prefetch_factor: int = defaults.PREFETCH_FACTOR, persistent_workers: bool = defaults.PERSISTENT_WORKERS, cuda_prefetch: bool = defaults.CUDA_PREFETCH)
Bases: LightningDataModule
Explicit source-only learning path with optional causal observation context.
Continuous chronological training and evaluation on one device. Each optimizer window declares independent objective target totals.
Source code in graphids/data/events.py
EventLocalContext
dataclass
¶
Bases: _TensorFields
Preceding same-ID timing; first-seen rows have seen=False.
EventStreamWindows ¶
EventStreamWindows(recordings, chunk_events, streams=defaults.STREAM_BATCH_SIZE, effective_events=defaults.EFFECTIVE_BATCH_EVENTS)
Bases: Dataset
Schedule independent live recordings; group tails by exact length.
Replacement starts only after a slot's recording ends. No fake frames, warmup replay or padding. Metadata only is retained by the scheduler.
Source code in graphids/data/events.py
EventTrainingLoader ¶
EventTrainingWindow
dataclass
¶
EventTrainingWindow(groups: list[list[CANEventBatch]], classification_targets: int, payload_targets: int)
One optimizer step, with CPU target totals for exact accumulation.
materialize_events ¶
Two bounded-memory CSV passes; publish only fully written caches.
Source code in graphids/data/events.py
config ¶
Continuous event stream batching and pooled CAN data configuration.
CANEventDataConfig ¶
Bases: BaseModel
Data settings; representation schema 2 enables causal per-ID context.
context ¶
Causal observation indices and training-only timing statistics.
fit_timing_statistics ¶
Fit only verified-normal train-query rows, never selection/target traffic.
Deterministic evenly spaced rows cap sampling at 65,536 per recording. ID support counts refer to these sampled, observed predecessor pairs. Rows without a preceding bus/ID observation are excluded from fitting.
Source code in graphids/data/context.py
previous_rows ¶
Previous physical row of each ID, independent of chunk boundaries.