Convert your fMRI data into BIDS format¶

TODO: [TIM] add figures. It would definitely be nice to show a full tree of an example repostitory, and how it changes at each step of the way.

BIDS standards¶

To organize our fMRI dataset, we follow the BIDS Specification.

If you are not familiar with the BIDS Specification, the BIDS Starter Kit provides all the information needed to get started, along with example BIDS datasets, Talks and Slides, and most importantly Tutorials.

It is crucial that you get familiar with BIDS folders/files naming convention and structure. Most, if not all, the tools we are going to use in the next steps are BIDS Apps, and they rely on data organized following the BIDS Specification. Following this structure will make it easier to use these tools, share your code and data, and communicate with other scientists.

The BIDS Specification provides guidelines on how to organize all your data formats, including (f/d)MRI, EEG, eye-tracking, Task events associated with Neuro-Imaging recordings or not, and Derivatives (e.g., pre-processed files, Regions Of Interest mask files, GLM files, etc.).

At any moment, you can check your dataset for BIDS compliance. To do so, you can use the BIDS dataset validator.

BIDS Conversion Overview¶

After scanning participants, you'll obtain data from two primary sources:

The scanner: Functional and structural outputs (.nii files), alongside potential dicom files
The stimulus presentation computer: Behavioural outputs (mainly log files and .mat files) and potentially eye-tracking data

Your first step is to organize these files in a sourcedata folder. Follow the structure outlined in How to store raw data. Once your data is properly arranged, you can proceed to anonymize it and convert it to BIDS format.

Here's a high-level overview of the steps involved in arranging your data in a BIDS-compatible way. While this provides a general understanding, most of these steps should be performed using the code provided in each sub-section to minimize errors.

First and foremost, make sure your DICOM / nifti file names do not contain subject identificative information, such as the subject's name. This is particularly relevant for our pipeline, because that's exactly what our scanner does. You can either manually rename your files, or run the small Python utility (available here) in your terminal. More information on how to use it can be found in the docstring inside the script and in the relevant section below.
Create events.tsv files (more info on required and optional columns can be found here):
- Make one file per functional run
- Add them to the ./BIDS/sub-xx/func folder
- Optional: Create an events.json sidecar file to describe extra columns in event files
Example:
```
└─ sub-01/
└─ func/
  ├─ sub-01_task-exp_events.tsv 
  └─ sub-01_task-exp_events.json 
```
Rename functional nifti files:
- Follow BIDS naming format: sub-<label>_task-<label>_run-<label>_bold.nii
- Add them to the ./BIDS/sub-xx/func folder
Rename structural nifti files:
- Follow BIDS naming format: sub-<label>_T1W.nii
- Add them to the ./BIDS/sub-xx/func folder
Optional: Process dicom files (if you got DICOM files from the scanner):
- Anonymize bold dicom files for your first participant using anonymize_dicm
- Convert anonymized dicom files using dicm2nii to get .json sidecar files for your anatomical or function scans.
JSON files

Each nii file must have a sidecar JSON file. However, if your fMRI protocol did not change, all the important JSON fields are going to be the same across different scanning sessions, and therefore JSON files can be re-used across subjects. This will save you some time, since getting DICOM files from the scanner can be quite time-consuming.
Update .json sidecar files:
- Complete the PhaseEncodingDirection and SliceTiming fields (see Missing fields in JSON files for more information)
- Duplicate and rename sidecar files to accompany each bold.nii file
Create essential modality agnostic BIDS files:
- dataset_description.json
- participants.tsv and participants.json
- task-<taskname>_bold.json
Set up additional components:
- Create a derivatives folder for future outputs
- Optional: Include a .bidsignore file if needed
Validate your BIDS structure:
- Use the BIDS validator

By following these steps systematically, you'll ensure your data is properly organized in BIDS format, facilitating easier analysis and collaboration.

Step-by-step instructions¶

Here we provide more detailed instructions to perform each of the steps mentioned above.

Folder Structure

All the steps and scripts below assume a specific folder structure and file naming convention. They will not work otherwise. Ensure you strictly follow the instructions in the How to store raw data page.

TODO: [ANDREA] in the how to store raw data page, create a folders tree that includes all the relevant folders and subfolder. The current tree is not complete.

Anonymize raw scanner data¶

Expected project structure¶

This small utility tool renames files within a BIDS-like directory structure, specifically targeting files containing _WIP_ in their names.

The script operates on the following project structure:

Project_Name/
├── sourcedata/
│   └── sub-xx/
│       ├── dicom/
│       ├── dicom_anon/
│       ├── bh/
│       ├── et/
│       └── nifti/
└── BIDS/
    ├── derivatives/
    └── sub-xx/
        ├── anat/
        └── func/

The script processes files within the 'sourcedata' directory.

Execute the script from the 'sourcedata' directory:

Open a terminal or command prompt.
Navigate to the project's root:

cd /path/to/Project_Name

Change to the 'sourcedata' directory:

cd sourcedata

Command-line Arguments¶

--level {group,participant}: Process all subjects (group) or individual subjects (participant).
--confirm {True,False}: Ask for confirmation before renaming (default: True).
--dry_run: Preview changes without renaming.
--sub [SUB ...]: Specify subject IDs to process.

Examples¶

Dry run for all subjects:

python /path/to/anon_nii_filename.py --level group --dry_run

Rename files for subjects 01 and 02 with confirmation:

python /path/to/anon_nii_filename.py --level participant --sub 01 02 --confirm True

Rename files for all subjects without confirmation:

python /path/to/anon_nii_filename.py --level group --confirm False

Output filename:

sub-01/nifti/sub-01_WIP_T1w_20240101141322.nii

Caution

Always backup your data before running renaming operations.

How It Works¶

The script traverses the 'sourcedata' directory structure.
It identifies files containing 'WIP' in their names.
New names are generated based on the subject ID and the part of the filename after 'WIP'.
Depending on the options, it either renames the files or shows the proposed changes.

Important Notes¶

Only files containing 'WIP' are processed. Others are ignored.
To process all files, modify the rename_files_in_directory function:

if '_WIP_' in file:

to:

if True:  # Caution: processes all files

Modifying the Script

Processing all files may lead to unintended renaming. Always use --dry_run first and review proposed changes carefully.

Tips¶

Use --dry_run to preview changes before actual renaming.
Process subjects in smaller batches for large datasets.
Regularly check BIDS specifications for naming convention compliance.

Troubleshooting¶

If issues occur:

Ensure you're in the 'sourcedata' directory.
Check permissions for renaming files in 'sourcedata'.
Verify all required Python dependencies are installed.
For unprocessed files, check if they contain 'WIP'.

Creating Event Files¶

Event files are crucial for analyzing fMRI data. They contain information about the timing and nature of stimuli or tasks during the scan. To create your event files manually:

Navigate to your sourcedata/sub-xx/bh/ folder.
Locate the behavioral output files (.mat or .log) for each run.
Create a corresponding events.tsv file for each run in the BIDS/sub-xx/func/ folder.

Each events.tsv file should contain at least three columns: onset, duration, and trial_type. Additional columns can be included as needed for your specific analysis.

TODO: [ANDREA] Add script for automatically converting behavioral data to BIDS-compliant event files.

TODO: [TIM] Include information about event files. Mention how they should ideally be created directly by the behavioural task script. Add a link to the task template to show how that can be done.

TODO: [TIM] Add information about making an events.json file and the advantages of it.

TODO: [TIM] Add information about using event files to make contrasts in the SPM step. Having a 'condition' + column for instance might be quite useful.

TODO: [TIM] Give information about which columns will be useful to include in such files, why, and at what + future step they will become important.

Converting DICOM files (Optional)¶

If you have DICOM files from the scanner:

Navigate to your sourcedata/sub-xx/dicom/ folder.
Use the anonymize_dicm script to anonymize the DICOM files.
Use the dicm2nii script to convert the anonymized DICOM files to NIfTI.

TODO: [TIM] Give information on how to use the anonymization and dicom to nifti scripts, and what the results should be like. Give links to the scripts.

Creating JSON Sidecar Files¶

Locate the JSON sidecar files in sourcedata/sub-xx/dicom_converted/.
Open each JSON file and update the PhaseEncodingDirection and SliceTiming fields.
Copy the updated JSON files to accompany each NIfTI file in the BIDS folder.

Each nii file must have a corresponding JSON sidecar file. If your fMRI protocol didn't change, you can reuse JSON files across subjects.

TODO: [TIM] Explain how to get the two missing fields and why it's important. Link to the fmri-general section about it.

TODO: [TIM] Explain how to duplicate and rename the sidecar file.

TODO: [ANDREA] fill this out with more in depth info about the JSON etc.

Renaming and Moving NIfTI Files¶

Navigate to your sourcedata/sub-xx/dicom_converted/ folder.
Identify the functional and structural NIfTI files.
Rename the files following BIDS conventions:
- Functional: sub-<label>_task-<label>_run-<label>_bold.nii
- Structural: sub-<label>_T1w.nii
Move the renamed files to their respective folders in BIDS/sub-xx/:
- Functional files go to BIDS/sub-xx/func/
- Structural files go to BIDS/sub-xx/anat/

TODO: [ANDREA] is dicom converted and nii the same folder? from which folder should we get the final nifti files to move? this folder needs to be consistent across different workflows (e.g., dicom conversion or just nifti files)

TODO: [TIM] Give instructions on how to rename files, both functional and structural, including what happens in case of several scan sessions and the added ses label.

Rename and move automatically

A MATLAB script that can do this automatically can be found here. Remember to change the input and output folders, run names and subjects numbers at the top of the script according to your needs. The script expects as input your nifti files, along with the JSON sidecar template files.

TODO: [ANDREA] the script should first check whether the JSON files are already availabe in the folder and, if that's the case, it should choose these files over the templates. Also, we need to add more info about these template files in this page!

Creating Essential BIDS Files¶

Create the following modality agnostic BIDS files files in your BIDS/ root folder:
- dataset_description.json
- participants.tsv
- participants.json
- task-<taskname>_bold.json
Fill in the required information for each file according to the BIDS specification.

Modality agnostic templates

In the modality agnostic BIDS files page, you can find templates and examples.

Setting Up Additional Components¶

Create a derivatives/ folder in your BIDS/ directory.
If needed, create a .bidsignore file in your BIDS/ root folder to exclude any non-BIDS compliant files.

Why should I use a .bidsignore file?

A `.bidsignore' file is useful to communicate to the BIDS validator which files should not be indexed, because they are not part of the standard BIDS structure. More information can be found here.

Validating Your BIDS Structure¶

Use the online BIDS Validator to check your BIDS structure.
Upload your entire BIDS/ folder and review any errors or warnings.
Make necessary corrections based on the validator's output.

By following these detailed steps, you'll ensure your data is properly organized in BIDS format, facilitating easier analysis and collaboration.

Now that you have your data in BIDS format, we can proceed to data pre-processing and quality assessment. See the next guide for instructions. → Pre-processing and QA