Neuroimaging Analysis Kit - Progress and challenges forstandardized fMRI processing

Pierre Bellecpierre.bellec@criugm.qc.ca

Feindel Brain Imaging Lecture - BIC/MNI/McGill 2016/10

P. Bellec Mining brain networks Montreal 2016 1 / 34

What’s NIAK

The Neuroimaging Analysis Kit (NIAK):a software package for data miningof brain networks in large fMRI datasets.

◮ A catalogue of complete workflows.

◮ Scales for large datasets / analyses.

◮ Incorporates robust, valid methods.

◮ Free and open-source (MIT).

P. Bellec Mining brain networks Montreal 2016 2 / 34

Available pipelines

Check http://niak.simexp-lab.org for more info.P. Bellec Mining brain networks Montreal 2016 3 / 34

fMRI preprocessing: flowchart

P. Bellec Mining brain networks Montreal 2016 4 / 34

Structural non-linear coregistration

P. Bellec Mining brain networks Montreal 2016 5 / 34

Structural processing: main outputs

raw T1 volume

non-uniformity corrected volume

brain mask

linear MNI152 template

non-linear MNI152 template

linear (lsq9) coregistered volume

non-linear coregistered volume

The main outputs of the T1 (CIVET) processing pipeline.

P. Bellec Mining brain networks Montreal 2016 6 / 34

T1 to BOLD image registrationC

orr

ect

to z

ero

mean, sm

ooth

, ere

sam

ple

Corre

ct to

zero

mean, sm

ooth

iteration 1 : large smoothing

iteration 5 : small smoothing

5 iterations progressive fit : LSQ6 coregistration - MINCTRACC (mutual information)

iteration 1 : large smoothing

iteration 5 : small smoothing

T1 volume & brain mask fMRI volume & brain mask

smooth : 8,4,8,4,3step : 4,4,4,2,1simplex : 8,4,2,2,1

smooth : 8,4,8,4,3step : 4,4,4,2,1simplex : 8,4,2,2,1

P. Bellec Mining brain networks Montreal 2016 7 / 34

Estimation of rigid-body motion over time

within-run motion parameters :3 rotations

3 translationsfor each volume

volume of reference (median)

time

time

P. Bellec Mining brain networks Montreal 2016 8 / 34

Scrubbing: frame displacement

Frame displacement is the sum of absolute displacements in translation and rotation motion

parameters. Frames with excessive FD (FD> 0.5) and neighbours are eliminated (Power et al.

Neuroimage 2012).

P. Bellec Mining brain networks Montreal 2016 9 / 34

Regression of confounds

P. Bellec Mining brain networks Montreal 2016 10 / 34

How does it work?

Main use case is based on Matlab/Octave. Tutorial available at http://niak.simexp-lab.org/niak_tutorial_fmri_preprocessing.html,also as a jupyter notebook.

P. Bellec Mining brain networks Montreal 2016 11 / 34

fMRI preprocessing: update

Notable features in upcoming NIAK “COG” release

◮ Dual support for MINC and NIFTI file formats.

◮ Minimally preprocessed data: resampled data is available along withall available confounds, for further analysis in SPM, FSL, etc.

◮ Interactive fMRI preprocessing report.

◮ Simplified guidelines for quality control

P. Bellec Mining brain networks Montreal 2016 12 / 34

Interactive report

Reports can be consulted offline or online. Live demo athttps://simexp.github.io/qc_cobre/.

P. Bellec Mining brain networks Montreal 2016 13 / 34

Guidelines for quality control of brain registration

Simplified guildelines for quality control as well as a collection of images torate are available on zooniverse https://www.zooniverse.org/projects/simexp/brain-mri-registration-quality-control

P. Bellec Mining brain networks Montreal 2016 14 / 34

One pipeline, many jobs...

Jobs and dependencies for fMRI preprocessing of two subjects with two functional runs each.

P. Bellec Mining brain networks Montreal 2016 15 / 34

The pipeline system for Octave and Matlab

NIAK is powered by PSOM, an open-source library for scripting pipelinesusing Octave or Matlab (Bellec et al., Frontiers in Neuroinformatics,2012).

◮ Parallel computing: Detection and execution of parallel componentsin the pipeline. The same code can run in a variety of executionenvironments (local, multi-core, cluster).

◮ Provenance tracking: Generation of a comprehensive record of thepipeline stages and the history of execution.

◮ Fault tolerance: Multiple attempts will be made to run each jobbefore it is considered as failed. Failed jobs can be automaticallyre-started.

◮ Smart updates: When an analysis is started multiple times, only theparts of the pipeline that need to be reprocessed are executed.

http://psom.simexp-lab.org

P. Bellec Mining brain networks Montreal 2016 16 / 34

PSOM 1.0

PSOM 1.0 follows a basic submission design, where each available job getssubmitted for execution, with a maximum of N concurrent processes. Running J

jobs requires J submissions, regardless of N. Works inefficiently for large J andshort jobs in systems with large overhead for submission.

P. Bellec Mining brain networks Montreal 2016 17 / 34

Benchmark PSOM 1.0

Adapted from Bellec et al. Front. in NeuroInf. 2012.

◮ Dataset Cambridge, 198 subjects with T1/fMRI.

◮ 5153 jobs / 7.7 G raw input / 21 G output / 8348unique input/output files.

◮ peuplier: single machine (i7, 4 cores / 8 threads),local file system.

◮ magma: single machine (AMD, 24 cores), NFS filesystem.

◮ guillimin: supercomputer (Xeon, 14k cores on 2011),infiniband parallel file system.

◮ Efficiency went down from 90% on peuplier-8 to 60%on guillimin-200.

P. Bellec Mining brain networks Montreal 2016 18 / 34

PSOM 2.0

PSOM 2.0, currently in beta, features an agent-based execution model. RunningN parallel processes requires N + 1 submissions, regardless of number of jobs J.Works efficiently in systems with large overhead for submission, regardless of J

and job duration (1s+).

P. Bellec Mining brain networks Montreal 2016 19 / 34

Benchmark PSOM 2.0

◮ Dataset Human Connectome Project, 875 subjects with T1 + 7mutliband fMRI task runs.

◮ 123k jobs / 3.4 T raw input / 3.8 T output / 173k uniqueinput/output files.

◮ guillimin: supercomputer (Xeon, 20k+ cores on 2016), infinibandparallel file system.

◮ Up to 300 concurrent processes allowed.

◮ Serial time: 17.9k hours / 746.87 days. Parallel time: 70 hrs.Parallelization efficiency: 85%

◮ deviation from 100% efficiency mostly attributable to queuing delaysin order to access resources.

P. Bellec Mining brain networks Montreal 2016 20 / 34

Integration with CBRAIN - frontend

The NIAK fMRI preprocessing pipeline, and soon the whole NIAK pipelinecatalogue, are integrated in CBRAIN.

P. Bellec Mining brain networks Montreal 2016 21 / 34

NIAK deployment using Docker

◮ The NIAK now ships as a docker container, available in docker hubhttps://hub.docker.com/, as well as singularityhttp://singularity.lbl.gov/, designed for high-performancecomputing infrastructures.

◮ The container includes all dependencies (MINC-toolkit, Octave,PSOM, NIAK, Brain Connectivity Toolbox, Jupyter).

◮ This facilitates installation on all platforms, CBRAIN, Linux, Mac,Windowshttp://niak.simexp-lab.org/niak_installation.html

P. Bellec Mining brain networks Montreal 2016 22 / 34

Integration with CBRAIN - backend

Lavoie-Courchesne et al., Journal of Physics: Conference Series 2011; Glatard et al., HBM 2016; Glatard et al., submitted toFuture Generation Computer Systems

P. Bellec Mining brain networks Montreal 2016 23 / 34

Integration with CBRAIN - BIDS support

Datasets fed into niak need to be organized using the Brain Imaging DataStructure (BIDS) http://bids.neuroimaging.io/. Proper organizationcan be validated on the CBRAIN portal.

Gorgolewski et al., Scientific Data 2016

P. Bellec Mining brain networks Montreal 2016 24 / 34

Challenge: pipeline development

A software package is a living organism.

Courtesy of Sam and Max.

Developping scientific workflows islike signing for a game of whack amole (Courtesy of Alan Evans).

P. Bellec Mining brain networks Montreal 2016 25 / 34

Challenge: pipeline development

A worked example:

Courtesy of Sam and Max.

◮ Increase image resolution.GOOD.

◮ Better inter-subject brainalignment. GOOD.

◮ Thus, less spatial blurring isneeded. GOOD.

◮ Hence, random field theorybreaks down. FAIL.

P. Bellec Mining brain networks Montreal 2016 26 / 34

Challenge 2: OS/numerical instability

DICE coefficient between matched components from an ICAdecomposition across two runs executed on the same system.(1) automatic detection of the number of components; (2) same randomseeds; (3) different system libraries (libmath). From Glatard et al., Fontiersin Neuroinformatics 2015. Numerical noise creeps up in pipeline analysis.

P. Bellec Mining brain networks Montreal 2016 27 / 34

Challenge 2: OS/numerical instability

P. Bellec Mining brain networks Montreal 2016 28 / 34

(partial) solution: continuous integration tests

NIAK continuous integration tests running onhttps://circleci.com/gh/SIMEXP/niak

P. Bellec Mining brain networks Montreal 2016 29 / 34

(partial) solution: continuous integration tests

Each change in NIAK triggers a comparison between current results and afixed, target version, across all available pipelines. Quantitative reportsshow which stage of the pipeline has changed, and by how much.

P. Bellec Mining brain networks Montreal 2016 30 / 34

Future solution: large-scale validation at release

Future NIAK releases will systematically replicate a number of keylarge-scale validation experiments and compare results across versions.This will be made possible by the docker container technology.

Between-group comparisons in resting-state connectivity across threepopulations. See Bellec et al., Orban, Neuroimage 2015.

P. Bellec Mining brain networks Montreal 2016 31 / 34

Conclusions

The NIAK:

◮ A catalogue of complete workflows. Preprocessing is mature, otherpipelines to follow. CBRAIN interface is beta, feedback welcome.

◮ Scales for large datasets / analyses. Latest version of NIAK is able tohandle smoothly HCP sample.

◮ Incorporates robust, valid methods. Quality control guidelines,continuous tests, large-scale validation tests.

◮ Free and open-source (MIT).

Recent progresses:

◮ Pipeline research dashboard for fMRI preprocessing. More to come.

◮ PSOM 2 scales better than PSOM 1.

◮ Generic, efficient integration in CBRAIN.

P. Bellec Mining brain networks Montreal 2016 32 / 34

Perspectives

◮ More stable methods.

◮ Large-scale tests for each release.

◮ NIAK course in 02/2017.

◮ Looking for beta testers for NIAK@CBRAIN.

◮ Get in touch for feature requests on the preprocessing pipeline.

P. Bellec Mining brain networks Montreal 2016 33 / 34

Acknowledgements

◮ Funding: Brain Canada CBRAIN, UdM, UNF/CRIUGM, CCNA (CIHR),Courtois foundation, Lemaire foundation.

◮ Computing: Compute Canada, Guillimin supercomputer, UNF servers.

◮ NIAK@CBRAIN: Pierre-Olivier Quirion, Tristan Glatard, SebastienLavoie-Courchesne.

◮ QC@NIAK: Yassine Benhajali, Sebastian Urchs, AmanPreet Badhwar,Perrine Ferre, Angela Tam, Christian Dansereau.

◮ validation@NIAK: Pierre Orban, Yassine Benhajali, Felix Carbonell,Christian Dansereau, Genevieve Albouy, Maxime Pelland, CameronCraddock, Olivier Collignon, Julien Doyon, Emmanuel Stip.

◮ NIAK: Pierre-Olivier Quirion, Angela Tam, Sebastian Urchs, YassineBenhajali, Christian Dansereau, Felix Carbonell, Jussi Tohka. Many indirectcontributions

◮ CBRAIN: Alan Evans, Marc-Etienne Rousseau, Pierre Rioux, Reza Adalat,and the CBRAIN team.

P. Bellec Mining brain networks Montreal 2016 34 / 34