Research code style¶
GraphIDS favors code a researcher can read from configuration through data, model, loss, and evaluation. Use the existing Python, PyTorch, Lightning, Pydantic, and Ruff conventions; keep the algorithm visible.
Module organization¶
- Use shallow domain packages:
data,models,exp, andcli. Shared evaluation, metrics, and callbacks live directly undergraphids. - Keep data/model configuration in the owning package's
config.py, with descriptiveConfigclass names. Experiment composition belongs inexp. - Import from the owning module. Avoid catch-all facades, generic config aliases, and wrapper factories that only call a config constructor.
Interfaces and validation¶
- Validate configuration with the domain-owned Pydantic models. Direct constructors and checkpoint reconstruction use those same definitions. Do not maintain a second set of handwritten range and cross-field checks.
- Validate external recordings, caches, checkpoints, and stream transitions at their owning boundaries. Internal helpers trust the resulting objects.
- Retain checks for provenance, temporal continuity, scored-row coverage, checkpoint compatibility, and capacities that could otherwise lose messages. Tests cannot detect changed inputs during a future experiment.
- Prefer explicit arguments, stable return types, and direct method calls. Normalize supported external forms once, rather than probing for methods or accepting several internal representations throughout a pipeline.
- Optional hooks belong at integration boundaries. For example, a custom class-path model may omit event-data preparation; event-model internals can rely on a prepared event datamodule.
Readable scientific logic¶
- Separate file parsing and artifact publication from numerical calculations. Use functions on tensors/arrays and existing dataclasses for research state.
- Give functions one clear job. Prefer a short sequence of named operations to nested conditionals, compressed expressions, or a configurable framework.
- Annotate interfaces and explain tensor shapes, scientific assumptions, and non-obvious mathematics. Use paper notation when it clarifies an equation.
- Initialize optional state explicitly instead of repeatedly using
getattrwith fallback values. Avoid adapters and aliases without an active consumer. - Let unexpected exceptions propagate with their original traceback. Do not
catch a builder's
TypeErrorand retry with different arguments. - Catch errors for actual recovery, cleanup, useful input context, or optional
telemetry isolation. Keep the protected block small. Use context managers or
finallyfor cleanup; do not replace required validation withassert. - Preserve spawn-based CUDA multiprocessing, model training mode, numerical stability, and bounded kernel workspaces when simplifying control flow.
Tests and tools¶
- Test distinct scientific or workflow behavior. Keep independent numerical oracles, gradient comparisons, causality, masking, coverage, and resume tests.
- Remove tests of upstream framework guarantees and obsolete interfaces. Reduce duplicate parameter combinations; do not combine unrelated behaviors just to lower the reported count.
- Share synthetic inputs through
tests/event_fixtures.py; never import helpers from another test module. Mock external services at their boundaries. - Synthetic CPU cases are unmarked. Mark training cases
slurm, and CUDA cases bothgpuandslurm. On HPC, run every pytest command inside an allocation. Usepython -m pytest -m 'not gpu and not slurm'for the CPU development suite. - Run
ruff check graphids tests scriptsandruff format --checkon changed Python files. Avoid blanket lint suppressions and complexity/count quotas.
Sources¶
These conventions adapt the Scientific Python guides on design and testing, and the Google Python exception guidance. They are project conventions, not a claim that ML research has one universal style.