domains / families / science / datalad

datalad

separate

Retrieves, versions, and publishes scientific datasets with DataLad and git-annex, and captures computational provenance with datalad run, rerun, and containers-run. Use when cloning or fetching data from OpenNeuro, DANDI, datasets.datalad.org, or any DataLad dataset; when a file in a dataset reads as a broken symlink or a small pointer instead of real data; when an analysis needs a machine-readable record of how each output was produced so it can be re-executed; or when publishing a dataset to siblings such as a GitHub repository plus a storage remote. Also use to decide between DataLad and plain Git for a data-carrying repository.

When to use

• Files are too large for Git to handle comfortably, or the total exceeds what every
• Data lives in more than one place (a lab server, a cluster scratch, S3, a supercomputer)
• The analysis must be re-executable, and a plain commit message is not enough evidence.
• You are consuming published datasets from OpenNeuro, DANDI, or `datasets.datalad.org`,
• The project nests other datasets inside it and you want each one to keep its own

When not to use

• repositories that are code and text only where everything fits comfortably in Git
• scenarios where nobody needs partial checkouts

Skill page

Source