Repeatability and reproducibility of FreeSurfer, FSL-SIENAX and SPM · PDF file 2019. 1....

Click here to load reader

  • date post

  • Category


  • view

  • download


Embed Size (px)

Transcript of Repeatability and reproducibility of FreeSurfer, FSL-SIENAX and SPM · PDF file 2019. 1....


    Repeatability and reproducibility of FreeSurfer, FSL-SIENAX and SPM brain volumetric measurements and the effect of lesion filling in multiple sclerosis

    Chunjie Guo1,2 & Daniel Ferreira1 & Katarina Fink3,4 & Eric Westman1 & Tobias Granberg3,5

    Received: 10 April 2018 /Revised: 14 July 2018 /Accepted: 7 August 2018 /Published online: 21 September 2018 # The Author(s) 2018

    Abstract Objectives To compare the cross-sectional robustness of commonly used volumetric software and effects of lesion filling in multiple sclerosis (MS). Methods NineMS patients (six females; age 38±13 years, disease duration 7.3±5.2 years) were scanned twice with repositioning on three MRI scanners (Siemens Aera 1.5T, Avanto 1.5T, Trio 3.0T) the same day. Volumetric T1-weighted images were processed with FreeSurfer, FSL-SIENAX, SPM and SPM-CAT before and after 3D FLAIR lesion filling with LST. The whole-brain, grey matter (GM) and white matter (WM) volumes were calculated with and without normalisation to the intra- cranial volume or FSL-SIENAX scaling factor. Robustness was assessed using the coefficient of variation (CoV). Results Variability in volumetrics was lower within than between scanners (CoV 0.17–0.96% vs. 0.65–5.0%, p

  • Keywords Multiple sclerosis .Magnetic resonance imaging . Brain . Reproducibility of results . Imaging, three-dimensional

    Abbreviations CoV Coefficient of variation CSF Cerebrospinal fluid FLAIR Fluid-attenuated inversion recovery FSL Functional magnetic resonance imaging of the

    brain software library GM Grey matter LST Lesion segmentation toolbox MS Multiple sclerosis SIENAX Structural image evaluation with normalisation of

    atrophy cross-sectional SPM Statistical parametric mapping WM White matter


    Multiple sclerosis (MS) is a common chronic neuroinflammatory and neurodegenerative disease [1]. Demyelinating lesions in the brain and spinal cord are the pathological hallmarks of MS, which are detectable in vivo with magnetic resonance imaging (MRI). MRI has therefore become an essential tool for the diag- nosis and monitoring of disease activity in MS [1, 2]. In MS, the lesion volume reflects the inflammatory burden while atrophy measures quantify neurodegenerative aspects of the disease, which play an important role in all disease stages [3]. Volumetry is therefore commonly used as a secondary endpoint in clinical trials [4]. Furthermore, volumetry can be helpful in improving our understanding of the disease since atrophy pat- terns have been shown to be different in MS compared to other demyelinating disorders [5].

    Obtaining robust imaging biomarkers in MS for assess- ment of the inflammatory and neurodegenerative burden of disease is, however, challenging [3]. Brain volumetry is influ- enced by several subject-related factors such as hydration sta- tus, inflammation and clinical therapy [6]. MS lesions can specifically affect tissue segmentations since white matter (WM) lesions can be misclassified as grey matter (GM) or cerebrospinal fluid (CSF) [7, 8]. Brain volumetry is also im- pacted by technical factors such as MRI field strength and scanner model, as well as post-processing related issues [8–10]. Understanding the effect and magnitude of technical factors is important when planning MRI studies [8].

    There are several freely available tools for automated brain volumetry that are commonly applied in MS. Popular choices include FreeSurfer [11], Structural Image Evaluation with Normalisation of Atrophy Cross-sectional (SIENAX) [12] and Statistical Parametric Mapping (SPM) [13]. These soft- ware can automatically pre-process and segment T1-weighted images of the brain. FreeSurfer is computationally demanding

    and is based on a combined volumetric- and surface-based segmentation aimed to reduce partial volume effects from the convoluted shape of the cortical ribbon [11]. FreeSurfer uses a template-driven approach to provide a detailed parcellation and segmentation of the cortex and subcortical structures. SIENAX, part of the FMRIB Software Library (FSL), is computationally less demanding but only provides measurements of the gross tissue volumes (WM, GM and CSF) [12]. FSL-SIENAX relies on registration to the Montreal Neurological Institute 152 template for skull strip- ping and then performs intensity-based segmentation; the tem- plate registration step provides a scaling factor that can be used for normalisation. SPM is based on non-linear registra- tion of the brain to a template and segments brain tissues by assigning tissue probabilities per voxel [13]. Computational Anatomy Toolbox (SPM-CAT) is an extension for SPM that provides segmentations with a different segmentation ap- proach based on spatial interpolation, denoising, additional affine registration steps, local intensity correction, adaptive segmentation and partial volume segmentation [14]. Like FSL-SIENAX, the SPM-based methods are less computation- ally demanding, relative to FreeSurfer, and only provide gross brain tissue volumes.

    The primary purpose of this study was to compare the repeatability on the same scanner and the reproducibility on different scanners for brain tissue segmentations in FreeSurfer, FSL-SIENAX, SPM and SPM-CAT. A secondary aim was to study the effect of automated lesion filling to reduce MS lesion-related brain tissue segmentation bias.

    Materials and methods


    Nine MS patients (six females, three males; mean age 38±13 years; mean disease duration 7.3±5.2 years) diagnosed ac- cording to the McDonald 2010 diagnostic criteria [15], were prospectively recruited from the outpatient clinic at the Department of Neurology, Karolinska University Hospital in Huddinge, Stockholm, Sweden, among consecutive patients referred for a clinical MRI. The participants were representa- tive of the MS population in Sweden, with all subtypes repre- sented in proportion to their frequency in clinical practice: six relapsing-remitting (RR), two secondary progressive, one pri- mary progressive [16]. Exclusion criteria were contraindica- tions to MRI, neurological co-morbidities or a history of head trauma (none were excluded). The physical disability of the patients was assessed according to the Expanded Disability Status Scale [17] by an MS-experienced neurologist (K.F.).

    1356 Eur Radiol (2019) 29:1355–1364

  • The median physical disability score was 2.0 (range 1.0–5.5). The study was approved by the local ethics committee and written informed consent was obtained from all participants.

    MRI protocol

    All participants were scanned twice on the same day on all three clinical MRI systems used in the study: Siemens Aera (1.5 T), Avanto (1.5 T) and Trio (3.0 T) (Siemens Healthcare, Erlangen, Germany). A 3D T1-weighted magnetisation- prepared rapid gradient-echo (MPRAGE) sequence was ac- quired twice with repositioning in between, resulting in a total of six T1-weighted volumes per participant. A representative example of the MPRAGE acquisitions is illustrated in Fig. 1. One 3D T2-weighted Fluid-Attenuated Inversion Recovery (FLAIR) was additionally acquired on each scanner for lesion segmentation. The MRI acquisition parameters are detailed in Table 1.

    Image analysis

    Each of the six 3DT1-weighted volumes from each participant was analysed cross-sectionally and processed in FreeSurfer, FSL-SIENAX, SPM and SPM-CAT. No additional pre- processing or manual intervention was performed to avoid introducing biases in the tissue segmentations. All input and output underwent visual quality assurance by an experienced

    rater (T.G.) and were found to be of satisfactory quality. Examples of the volumetric output are presented in Fig. 2.

    FreeSurfer FreeSurfer 6.0.0 (http://surfer.nmr.mgh.harvard. edu, Harvard University, Boston, MA, USA) was used to perform automatic processing as previously described [11, 18]. FreeSurfer was run with the options ‘-mprage’ and for the 3.0 T data also ‘-3T’, as recommended by its developers. The variable ‘Brain Segmentation Volume Without Ventricles from Surf’ was used as the FreeSurfer estimation of the brain volume, which excludes the brainstem. The variable ‘Total grey matter volume’ was used as the estimation of the GM volume. The WM volume was assessed by summing the ‘cerebral WM’, ‘cerebellar WM’, ‘brainstem’ and ‘corpus callosum’ FreeSurfer variables. It is notable that FreeSurfer specifically segments white matter hypointensities. For normalisation purposes, the brain volume, GM volume and WM volume were divided by the ‘Estimated Total Intracranial Volume’.

    FSL-SIENAX The SIENAX method implemented in FSL 5.0 (, Oxford University, Oxford, UK) was used to obtain an automated quantification of the brain volume, GM volume and WM volume with automatic normalisation for head size with a subject-specific scaling factor, as previously described [19]. For this study, FSL-SIENAXwas run with the optimised brain

    Fig. 1 Example of mid-axial slices from the six 3D T1-weight- ed acquisitions in a 35-year-old male (referred to as MS1 in Online Supplementary Table 1) with relapsing-remitting multiple sclerosis and an Expanded Disability Status Scale score of 2.0. For each of the three scanners (Siemens Aera 1.5 T, Siemens Avanto 1.5 T and Siemens Trio 3.0 T) two acquisitions were made with repositioning in between

    Eur Radiol (2019) 29:1355–1364 1357

  • extraction parameters ‘-B -f 0.1’, in accordance with previous recommendations for MS studies [20].

    SPM Statistical Parametric Mapping, SPM12, (http://