File schemas¶

This page contains auto-generated JSON schemas [1] for the text files used or generated by Nipoppy.

Global configuration file¶

Tip

See the Quickstart guide for an example config file.

How to read the schema table

Read the table from top to bottom, left to right
Required properties are in bold
The type of each property is in italics, to the right of a cell labelled “type”
- Unless if that property is an object described by another schema, in which case it is a link to that schema
Default values are shown for optional properties (None if empty)
See the JSON schema docs more details about keyword meanings

Below is the schema used for the global configuration JSON file.

Config¶

Schema for dataset configuration.
type	object
properties
DATASET_NAME	Dataset Name
	Name of the dataset
	type	string
VISIT_IDS	Visit Ids
	List of visits available in the study. A visit ID is an identifier for a data collection event, not restricted to imaging data.
	type	array
	items	type	string
SESSION_IDS	Session Ids
	List of BIDS-compliant sessions available in the study, prefixed with “ses-” (inferred from VISITS if not given)
	default	null
	anyOf	type	array
		items	type	string
		type	null
DICOM_DIR_MAP_FILE	Dicom Dir Map File
	Path to a TSV file mapping participant IDs to DICOM directories, to be used in the DICOM reorg step. Note: this field and DICOM_DIR_PARTICIPANT_FIRST cannot both be specified. The TSV file should have three columns: “participant_id” , “session_id”, and “participant_dicom_dir”
	default	null
	anyOf	type	string
		format	path
		type	null
DICOM_DIR_PARTICIPANT_FIRST	Dicom Dir Participant First
	Whether subdirectories under <DATASET_ROOT>/sourcedata/imaging/pre_reorg) follow the pattern <PARTICIPANT>/<SESSION> (default) or <SESSION>/<PARTICIPANT>. Note: this field and DICOM_DIR_MAP_FILE cannot both be specified
	default	null
	anyOf	type	boolean
		type	null
SUBSTITUTIONS	Substitutions
	Top-level mapping for replacing placeholder expressions in the rest of the config file. Note: the replacement only happens if the config is loaded from a file with `nipoppy.config.main.Config.load()`
	type	object
	default
	additionalProperties	type	string
BIDS_PIPELINES	Bids Pipelines
	Configurations for BIDS conversion, if applicable
	type	array
	default
	items	BidsPipelineConfig
PROC_PIPELINES	Proc Pipelines
	Configurations for processing pipelines
	type	array
	items	ProcPipelineConfig
EXTRACTION_PIPELINES	Extraction Pipelines
	Configurations for extraction pipelines
	type	array
	default
	items	ExtractionPipelineConfig
CUSTOM	Custom
	Free field that can be used for any purpose
	type	object
	default
	additionalProperties	True
CONTAINER_CONFIG	Configuration for running a container
	default	COMMAND	apptainer
		ARGS
		ENV_VARS
		INHERIT	True
	ContainerConfig
additionalProperties	False

AnalysisLevelType¶

Pipeline step types.
type	string
enum	participant_session, participant, session, group

BidsPipelineConfig¶

Schema for BIDS pipeline configuration.
type	object
properties
CONTAINER_CONFIG	Configuration for running a container
	default	COMMAND	apptainer
		ARGS
		ENV_VARS
		INHERIT	True
	ContainerConfig
NAME	Name
	Name of the pipeline
	type	string
VERSION	Version
	Version of the pipeline
	type	string
DESCRIPTION	Description
	Free description field
	default	null
	anyOf	type	string
		type	null
CONTAINER_INFO	Information about the container image file
	default	FILE	null
		URI	null
	ContainerInfo
STEPS	Steps
	List of pipeline step configurations
	type	array
	default
	items	BidsPipelineStepConfig
additionalProperties	False

BidsPipelineStepConfig¶

Schema for BIDS pipeline step configuration.
type	object
properties
CONTAINER_CONFIG	Configuration for running a container
	default	COMMAND	apptainer
		ARGS
		ENV_VARS
		INHERIT	True
	ContainerConfig
NAME	Name
	Step name. Required if the pipeline has multiple steps
	type	string
	default	default
DESCRIPTOR_FILE	Descriptor File
	Path to the JSON descriptor file. Only needed for custom pipelines
	default	null
	anyOf	type	string
		format	path
		type	null
INVOCATION_FILE	Invocation File
	Path to the JSON invocation file
	default	null
	anyOf	type	string
		format	path
		type	null
ANALYSIS_LEVEL	Analysis level of the pipeline step. This controls the granularity of the loop over subjects and sessions. By default, pipeline runners will loop over all subjects and sessions, but this field field can be set to “AnalysisLevelType.participant” to loop over subjects only, “AnalysisLevelType.session” to loop over sessions only, and AnalysisLevelType.group to only run the pipeline a single time.
	default	participant_session
	AnalysisLevelType
UPDATE_STATUS	Update Status
	Whether or not the in_bids column in the curation status file should be updated
	default	False
	anyOf	type	boolean
		type	null
additionalProperties	False

ContainerConfig¶

Schema for container configuration. Does not include information about the container image.
type	object
properties
COMMAND	Command
	Name of or path to Apptainer/Singularity executable
	type	string
	default	apptainer
ARGS	Args
	Arguments for Apptainer/Singularity call (to be appended after the subcommand)
	type	array
	default
	items	type	string
ENV_VARS	Env Vars
	Environment variables that should be available inside the container
	type	object
	default
	additionalProperties	type	string
INHERIT	Inherit
	Whether this config should inherit from higher-lever container configs. If false, will ignore higher-level configs
	type	boolean
	default	True
additionalProperties	False

ContainerInfo¶

Schema for container image (i.e., file) information.
type	object
properties
FILE	File
	Path to the container associated with the pipeline, relative to the root directory of the dataset
	default	null
	anyOf	type	string
		format	path
		type	null
URI	Uri
	The Docker or Apptainer/Singularity URI for the container
	default	null
	anyOf	type	string
		type	null
additionalProperties	False

ExtractionPipelineConfig¶

Schema for extraction pipeline configuration.
type	object
properties
CONTAINER_CONFIG	Configuration for running a container
	default	COMMAND	apptainer
		ARGS
		ENV_VARS
		INHERIT	True
	ContainerConfig
NAME	Name
	Name of the pipeline
	type	string
VERSION	Version
	Version of the pipeline
	type	string
DESCRIPTION	Description
	Free description field
	default	null
	anyOf	type	string
		type	null
CONTAINER_INFO	Information about the container image file
	default	FILE	null
		URI	null
	ContainerInfo
STEPS	Steps
	List of pipeline step configurations
	type	array
	default
	items	ExtractionPipelineStepConfig
PROC_DEPENDENCIES	Proc Dependencies
	List of processing pipeline(s) (including step names) whose output the extraction pipeline depends on
	type	array
	items	PipelineInfo
additionalProperties	False

ExtractionPipelineStepConfig¶

Schema for extraction pipeline step configuration.
type	object
properties
CONTAINER_CONFIG	Configuration for running a container
	default	COMMAND	apptainer
		ARGS
		ENV_VARS
		INHERIT	True
	ContainerConfig
NAME	Name
	Step name. Required if the pipeline has multiple steps
	type	string
	default	default
DESCRIPTOR_FILE	Descriptor File
	Path to the JSON descriptor file. Only needed for custom pipelines
	default	null
	anyOf	type	string
		format	path
		type	null
INVOCATION_FILE	Invocation File
	Path to the JSON invocation file
	default	null
	anyOf	type	string
		format	path
		type	null
ANALYSIS_LEVEL	Analysis level of the pipeline step. This controls the granularity of the loop over subjects and sessions. By default, pipeline runners will loop over all subjects and sessions, but this field field can be set to “AnalysisLevelType.participant” to loop over subjects only, “AnalysisLevelType.session” to loop over sessions only, and AnalysisLevelType.group to only run the pipeline a single time.
	default	participant_session
	AnalysisLevelType
additionalProperties	False

PipelineInfo¶

Schema for pipeline information.
type	object
properties
NAME	Name
	Name of the pipeline
	type	string
VERSION	Version
	Version of the pipeline
	type	string
STEP	Step
	Name of the pipeline step
	type	string
	default	default
additionalProperties	False

ProcPipelineConfig¶

Schema for processing pipeline configuration.
type	object
properties
CONTAINER_CONFIG	Configuration for running a container
	default	COMMAND	apptainer
		ARGS
		ENV_VARS
		INHERIT	True
	ContainerConfig
NAME	Name
	Name of the pipeline
	type	string
VERSION	Version
	Version of the pipeline
	type	string
DESCRIPTION	Description
	Free description field
	default	null
	anyOf	type	string
		type	null
CONTAINER_INFO	Information about the container image file
	default	FILE	null
		URI	null
	ContainerInfo
STEPS	Steps
	List of pipeline step configurations
	type	array
	default
	items	ProcPipelineStepConfig
TRACKER_CONFIG_FILE	Tracker Config File
	Path to the tracker configuration file associated with the pipeline. This file must contain a list of tracker configurations, each of which must be a dictionary with a NAME field (string) and a PATHS field (non-empty list of strings)
	default	null
	anyOf	type	string
		format	path
		type	null
additionalProperties	False

ProcPipelineStepConfig¶

Schema for processing pipeline step configuration.
type	object
properties
CONTAINER_CONFIG	Configuration for running a container
	default	COMMAND	apptainer
		ARGS
		ENV_VARS
		INHERIT	True
	ContainerConfig
NAME	Name
	Step name. Required if the pipeline has multiple steps
	type	string
	default	default
DESCRIPTOR_FILE	Descriptor File
	Path to the JSON descriptor file. Only needed for custom pipelines
	default	null
	anyOf	type	string
		format	path
		type	null
INVOCATION_FILE	Invocation File
	Path to the JSON invocation file
	default	null
	anyOf	type	string
		format	path
		type	null
ANALYSIS_LEVEL	Analysis level of the pipeline step. This controls the granularity of the loop over subjects and sessions. By default, pipeline runners will loop over all subjects and sessions, but this field field can be set to “AnalysisLevelType.participant” to loop over subjects only, “AnalysisLevelType.session” to loop over sessions only, and AnalysisLevelType.group to only run the pipeline a single time.
	default	participant_session
	AnalysisLevelType
TRACKER_CONFIG_FILE	Tracker Config File
	Path to the tracker configuration file associated with the pipeline step
	default	null
	anyOf	type	string
		format	path
		type	null
PYBIDS_IGNORE_FILE	Pybids Ignore File
	Path to file containing a list of regex patterns (strings) to ignore when building the PyBIDS layout
	default	null
	anyOf	type	string
		format	path
		type	null
GENERATE_PYBIDS_DATABASE	Generate Pybids Database
	Whether or not to generate a PyBIDS database as part of the pipeline step (default: true)
	default	True
	anyOf	type	boolean
		type	null
additionalProperties	False

Tracker configuration file¶

How to read the schema table

Read the table from top to bottom, left to right
Required properties are in bold
The type of each property is in italics, to the right of a cell labelled “type”
- Unless if that property is an object described by another schema, in which case it is a link to that schema
Default values are shown for optional properties (None if empty)
See the JSON schema docs more details about keyword meanings

Here is the schema for a tracker configuration file:

TrackerConfig
Schema for tracker configuration.
type	object
properties
PATHS	Paths
	List of at least one path to track. A path can include template strings for participant/session IDs and/or glob expressions
	type	array
	items	type	string
		format	path
PARTICIPANT_SESSION_DIR	Participant Session Dir
	Path to the directory where participant-session results are expected
	default	null
	anyOf	type	string
		format	path
		type	null
additionalProperties	False

Manifest file¶

How to read the schema table

Read the table from top to bottom, left to right
Required properties are in bold
The type of each property is in italics, to the right of a cell labelled “type”
- Unless if that property is an object described by another schema, in which case it is a link to that schema
Default values are shown for optional properties (None if empty)
See the JSON schema docs more details about keyword meanings

Here is the schema for the manifest file:

ManifestModel
A user-provided listing of participant and visits available in the dataset. Note: This class is called “model” to be consistent with Pydantic nomenclature, but it can be thought of as a schema for each row in the manifest file.
type	object
properties
participant_id	Participant ID
	Participant identifier, without the BIDS prefix
	type	string
visit_id	Visit Id
	Visit identifier
	type	string
session_id	Session Id
	Imaging session identifier, without the BIDS prefix
	anyOf	type	string
		type	null
datatype	Datatype
	Imaging datatype, as recognized by BIDS (see https://bids-specification.readthedocs.io/en/stable/common-principles.html)
	anyOf	type	array
		items	type	string
		type	null
additionalProperties	True

Curation status file¶

How to read the schema table

Read the table from top to bottom, left to right
Required properties are in bold
The type of each property is in italics, to the right of a cell labelled “type”
- Unless if that property is an object described by another schema, in which case it is a link to that schema
Default values are shown for optional properties (None if empty)
See the JSON schema docs more details about keyword meanings

Here is the schema for the curation status file:

CurationStatusModel
An internally- or user-generated file to keep track of the BIDSification process. Should contain exactly the same data as the manifest, with some additional columns. Note: This class is called “model” to be consistent with Pydantic nomenclature, but it can be thought of as a schema for each row in the curation status file.
type	object
properties
participant_id	Participant ID
	Participant identifier, without the BIDS prefix
	type	string
visit_id	Visit Id
	Visit identifier
	type	string
session_id	Session Id
	Imaging session identifier, without the BIDS prefix
	anyOf	type	string
		type	null
datatype	Datatype
	Imaging datatype, as recognized by BIDS (see https://bids-specification.readthedocs.io/en/stable/common-principles.html)
	anyOf	type	array
		items	type	string
		type	null
participant_dicom_dir	Participant DICOM directory
	Path to the directory containing raw DICOMs (in potentially messy tree structure) for the participant-visit pair, relative to the raw data directory
	type	string
in_pre_reorg	In Pre Reorg
	Whether files are available on disk
	type	boolean
in_post_reorg	In Post Reorg
	Whether files have been organized in the sourcedata directory
	type	boolean
in_bids	BIDSified
	Whether files have been converted to BIDS
	type	boolean
additionalProperties	True

Processing status file¶

How to read the schema table

Read the table from top to bottom, left to right
Required properties are in bold
The type of each property is in italics, to the right of a cell labelled “type”
- Unless if that property is an object described by another schema, in which case it is a link to that schema
Default values are shown for optional properties (None if empty)
See the JSON schema docs more details about keyword meanings

Here is the schema for the processing status file:

ProcessingStatusModel
A file generated by the trackers. Contains processing statuses for image processing pipelines. Note: This class is called “model” to be consistent with Pydantic nomenclature, but it can be thought of as a schema for each row in the processing status file.
type	object
properties
participant_id	Participant ID
	Participant identifier, without the BIDS prefix (as in the manifest)
	type	string
bids_participant_id	BIDS participant ID
	Participant identifier with BIDS prefix (e.g., sub-01)
	type	string
session_id	Session Id
	Imaging session identifier, without the BIDS prefix
	type	string
bids_session_id	Bids Session Id
	Imaging session identifier with BIDS prefix (e.g., ses-01)
	type	string
pipeline_name	Pipeline Name
	The name of the pipeline being tracked
	type	string
pipeline_version	Pipeline Version
	The version of the pipeline being tracked
	type	string
pipeline_step	Pipeline Step
	The name of the pipeline step being tracked
	type	string
status	Status
	The status of the pipeline run for this participant-visit pair
	type	string

DICOM directory mapping file¶

How to read the schema table

Read the table from top to bottom, left to right
Required properties are in bold
The type of each property is in italics, to the right of a cell labelled “type”
- Unless if that property is an object described by another schema, in which case it is a link to that schema
Default values are shown for optional properties (None if empty)
See the JSON schema docs more details about keyword meanings

Here is the schema for the DICOM directory mapping file:

DicomDirMapModel
A file for mapping participant IDs to DICOM directories. Note: This class is called “model” to be consistent with Pydantic nomenclature, but it can be thought of as a schema for each row in the mapping file.
type	object
properties
participant_id	Participant ID
	Participant identifier, without the BIDS prefix
	type	string
session_id	Session Id
	Imaging session identifier, without the BIDS prefix
	type	string
participant_dicom_dir	Participant’s raw DICOM directory
	Path to the participant’s source DICOM directory, relative to <DATASET_ROOT>/sourcedata/imaging/pre_reorg
	type	string

Layout configuration¶

How to read the schema table

Read the table from top to bottom, left to right
Required properties are in bold
The type of each property is in italics, to the right of a cell labelled “type”
- Unless if that property is an object described by another schema, in which case it is a link to that schema
Default values are shown for optional properties (None if empty)
See the JSON schema docs more details about keyword meanings

The default Nipoppy layout

The default Nipoppy layout uses the following configuration:

{
    "dpath_bids": {
        "path": "bids",
        "description": "This directory contains the raw dataset converted to the BIDS standard."
    },
    "dpath_derivatives": {
        "path": "derivatives",
        "description": "This directory contains the outputs of image processing pipelines. Outputs of different pipelines are stored in `<pipeline_name>/<pipeline_version>` subdirectories. Each subdirectory has two additional subdirectories: one for the actual output (under `output`) and one for temporary files created during processing (under `work`)."
    },
    "dpath_sourcedata": {
        "path": "sourcedata",
        "description": "This directory is for the unorganized or arbitrarily organized source data (imaging and tabular)."
    },
    "dpath_src_tabular": {
        "path": "sourcedata/tabular",
        "description": "This directory is for the unorganized or arbitrarily organized tabular data."
    },
    "dpath_src_imaging": {
        "path": "sourcedata/imaging",
        "description": "This directory is for the unorganized or arbitrarily organized imaging data (files and archives)."
    },
    "dpath_downloads": {
        "path": "sourcedata/imaging/downloads",
        "description": "This directory is for downloaded data archives (e.g., tarballs or zipped files)."
    },
    "dpath_pre_reorg": {
        "path": "sourcedata/imaging/pre_reorg",
        "description": "This directory is for arbitrarily organized source imaging files."
    },
    "dpath_post_reorg": {
        "path": "sourcedata/imaging/post_reorg",
        "description": "This directory is for the organized source data that is ready for BIDS conversion."
    },
    "dpath_code": {
        "path": "code",
        "description": "This directory is for custom code."
    },
    "dpath_containers": {
        "path": "containers",
        "description": "This directory is for storing the container image files for the BIDS conversion/processing pipelines."
    },
    "dpath_pipelines": {
        "path": "pipelines",
        "description": "This directory is for storing the Nipoppy configuration information and custom code."
    },
    "dpath_scratch": {
        "path": "scratch",
        "description": "This directory is for temporary/working files."
    },
    "dpath_pybids_db": {
        "path": "scratch/pybids_db",
        "description": "This directory is for PyBIDS databases generated/used during processing."
    },
    "dpath_work": {
        "path": "scratch/work",
        "description": "This directory is for temporary/working files generated/used during processing."
    },
    "dpath_logs": {
        "path": "logs",
        "description": "This directory contains log files."
    },
    "dpath_tabular": {
        "path": "tabular",
        "description": "This directory is for tabular/phenotypic data."
    },
    "dpath_assessments": {
        "path": "tabular/assessments",
        "description": "This directory is for assessments data."
    },
    "fpath_config": {
        "path": "global_config.json",
        "description": "The configuration file for a dataset is where all data processing is configured."
    },
    "fpath_curation_status": {
        "path": "sourcedata/imaging/curation_status.tsv",
        "description": "The curation status file is generated/updated internally and is used to keep track of the raw data organization and BIDSification process."
    },
    "fpath_manifest": {
        "path": "manifest.tsv",
        "description": "The manifest file lists all expected study participants and visits."
    },
    "fpath_processing_status": {
        "path": "derivatives/processing_status.tsv",
        "description": "The processing status file lists availability of derived imaging data and pipeline processing statuses."
    },
    "fpath_demographics": {
        "path": "tabular/demographics.tsv",
        "description": "The demographics file contains demographic information for study participants."
    }
}

Here is the schema used for JSON files that define the layout of the dataset:

LayoutConfig
Relative paths for the dataset layout.
type	object
properties
dpath_bids	Directory for raw imaging data in BIDS
	DpathInfo
dpath_derivatives	Directory for imaging derivatives
	DpathInfo
dpath_sourcedata	Directory for source imaging and tabular data
	DpathInfo
dpath_src_tabular	Directory for tabular data source files (e.g., downloaded CSVs, Excel files, RedCAP reports)
	DpathInfo
dpath_src_imaging	Directory for non-BIDS imaging data files and archives
	DpathInfo
dpath_downloads	Directory for downloaded data
	DpathInfo
dpath_pre_reorg	Directory for unorganized source imaging files
	DpathInfo
dpath_post_reorg	Directory for imaging data that is organized but not yet in BIDS
	DpathInfo
dpath_code	Directory for code and scripts
	DpathInfo
dpath_pipelines	Directory for configurations or other files needed to run pipelines
	DpathInfo
dpath_containers	Directory for storing container images
	DpathInfo
dpath_scratch	Directory for temporary files
	DpathInfo
dpath_pybids_db	Directory for PyBIDS databases
	DpathInfo
dpath_work	Directory for temporary/working files generated during pipeline runs
	DpathInfo
dpath_logs	Directory for logs generated by Nipoppy
	DpathInfo
dpath_tabular	Directory for tabular data
	DpathInfo
dpath_assessments	Directory for tabular assessment data
	DpathInfo
fpath_config	Path to the configuration file
	FpathInfo
fpath_manifest	Path to the manifest file
	FpathInfo
fpath_curation_status	Path to the curation status file (for tracking the BIDSification process)
	OptionalFpathInfo
fpath_processing_status	Path to the processing status file (for tracking imaging derivative availability at the participant level)
	OptionalFpathInfo
fpath_demographics	Path to the study’s demographics data file
	OptionalFpathInfo
additionalProperties	False

DpathInfo
Relative path and description for a directory.
type	object
properties
path	Path
	Relative path to the file or directory
	type	string
	format	path
description	Description
	Description of the function of the file or directory
	default	null
	anyOf	type	string
		type	null

FpathInfo
Relative path and description for a file.
type	object
properties
path	Path
	Relative path to the file or directory
	type	string
	format	path
description	Description
	Description of the function of the file or directory
	default	null
	anyOf	type	string
		type	null

OptionalFpathInfo
Relative path and description for a file that is optional.
type	object
properties
path	Path
	Relative path to the file or directory
	type	string
	format	path
description	Description
	Description of the function of the file or directory
	default	null
	anyOf	type	string
		type	null

Custom Nipoppy configurations in a Boutiques descriptor¶

How to read the schema table

Read the table from top to bottom, left to right
Required properties are in bold
The type of each property is in italics, to the right of a cell labelled “type”
- Unless if that property is an object described by another schema, in which case it is a link to that schema
Default values are shown for optional properties (None if empty)
See the JSON schema docs more details about keyword meanings

Nipoppy can read additional configurations from a Boutiques descriptor, under the custom\nipoppy key. The JSON object is expected to comply with the following schema:

BoutiquesConfig
Schema for custom configuration within a Boutiques descriptor.
type	object
properties
CONTAINER_SUBCOMMAND	Container Subcommand
	Subcommand for Apptainer/Singularity call
	type	string
	default	run
CONTAINER_CONFIG	Configuration for running a container
	default	COMMAND	apptainer
		ARGS
		ENV_VARS
		INHERIT	True
	ContainerConfig
additionalProperties	False

ContainerConfig
Schema for container configuration. Does not include information about the container image.
type	object
properties
COMMAND	Command
	Name of or path to Apptainer/Singularity executable
	type	string
	default	apptainer
ARGS	Args
	Arguments for Apptainer/Singularity call (to be appended after the subcommand)
	type	array
	default
	items	type	string
ENV_VARS	Env Vars
	Environment variables that should be available inside the container
	type	object
	default
	additionalProperties	type	string
INHERIT	Inherit
	Whether this config should inherit from higher-lever container configs. If false, will ignore higher-level configs
	type	boolean
	default	True
additionalProperties	False