COMPUTED TOMOGRAPHY

Comparing five different iterative reconstruction algorithmsfor computed tomography in an ROC study

Kristin Jensen & Anne Catrine T. Martinsen &

Anders Tingberg & Trond Mogens Aaløkken & Erik Fosse

Received: 15 January 2014 /Revised: 1 July 2014 /Accepted: 8 July 2014# European Society of Radiology 2014

AbstractObjectives The purpose of this study was to evaluate lesionconspicuity achieved with five different iterative reconstruc-tion techniques from four CT vendors at three different doselevels. Comparisons were made of iterative algorithm andfiltered back projection (FBP) among and within systems.Methods An anthropomorphic liver phantom was examinedwith four CT systems, each from a different vendor. CTDIvollevels of 5 mGy, 10 mGy and 15 mGy were chosen. Imageswere reconstructed with FBP and the iterative algorithm onthe system. Images were interpreted independently by fourobservers, and the areas under the ROC curve (AUCs) werecalculated. Noise and contrast-to-noise ratios (CNR) weremeasured.Results One iterative algorithm increased AUC (0.79, 0.95,and 0.97) compared to FBP (0.70, 0.86, and 0.93) at all doselevels (p<0.001 and p=0.047). Another algorithm increasedAUC from 0.78 with FBP to 0.84 (p=0.007) at 5 mGy.Differences at 10 and 15 mGy were not significant (p-values:

0.084–0.883). Three algorithms showed no difference in AUCcompared to FBP (p-values: 0.008–1.000). All of the algo-rithms decreased noise (10–71 %) and improved CNR.Conclusions Only two algorithms improved lesion detection,even though noise reduction was shown with all algorithms.Key Points• Iterative reconstruction algorithms affected lesion detectiondifferently at different dose levels.

• One iterative algorithm improved lesion detectability com-pared to filtered back projection.

• Three algorithms did not significantly improve lesiondetectability.

• One algorithm improved lesion detectability at the lowestdose level.

Keywords Computed tomography . Image reconstruction .

Radiological phantom . Liver

Introduction

In order to improve visualization of pathology without in-creasing radiation exposure to the patient, CT vendors havedeveloped new reconstruction techniques such as iterativereconstruction algorithms. According to the vendors, iterativereconstruction improves image quality, and thereby radiationdose can be reduced compared to the standard reconstructiontechnique, filtered back projection (FBP), [1–4]. The newreconstruction techniques may have an effect on image textureand diagnostic image quality [5–8], however, and there maybe inter-vendor differences. Therefore, it is important to testnew techniques before implementing them in clinical routine.

FBP has been the primary image reconstruction techniquein CT [9, 10]. Simplifications in technique have improvedspeed and reduced power consumption, but artefacts and

K. Jensen (*) :A. C. T. Martinsen : E. FosseThe Intervention Centre, Rikshospitalet, Postboks 4950, Nydalen,0424 Oslo, Norwaye-mail: uxjekr@ous-hf.no

K. Jensen :A. C. T. Martinsenlnstitute of Physics, University of Oslo, 0027 Oslo, Norway

A. TingbergDepartment of Medical Radiation Physics, Lund University, SkåneUniversity Hospital, 205 02 Malmö, Sweden

T. M. AaløkkenDepartment of Radiology and Nuclear Medicine, Rikshospitalet,Postboks 4950, Nydalen, 0424 Oslo, Norway

E. Fosselnstitute of Clinical Medicine, University of Oslo, 0027 Oslo,Norway

Eur RadiolDOI 10.1007/s00330-014-3333-4

increased image noise degrade the image quality, especially atlow dose levels [11, 12].

The iterative reconstruction process is a loop where theforward projection of an image estimate is compared to themeasured data. Depending upon the complexity of the itera-tive reconstruction, corrections are made and the process isrepeated [13]. The algorithms are often divided into hybridand pure iterative reconstruction. The hybrid iterative recon-struction algorithms involve blending with FBP to keep thenoise characteristics and image texture that the radiologists areaccustomed to. The pure iterative reconstruction does notblend with FBP.

When image noise is decreased, lesion detectability may beimproved. However, if image texture changes and new arte-facts appear, lesion conspicuity can be degraded, the impactsof which have thus far not been studied. Different strengths ofalgorithms are offered, providing many choices of procedure.Some promise improved contrast and spatial resolution and areduction in artefacts. The number of choices makes it difficultto determine the procedures and patients for which thesealgorithms should be used.

Several iterative reconstruction algorithms are commercial-ly available. Different algorithms introduced by the vendorsare variations and combinations of statistical methods andmodel-based methods [14].

The purpose of this study was to evaluate lesion conspicu-ity achieved with five different iterative reconstruction tech-niques from four CT vendors at three different dose levels.Comparisons were made of iterative reconstruction and FBPamong and within systems.

Materials and methods

Systems

The following CT systems with FBP and iterative algorithmswere tested in this study: GE CT750 HD (64-slice) withAdaptive Statistical Iterative Reconstruction (ASiR) andVeo; Philips Ingenuity (64-slice) with iDose4; Siemens Defi-nition Flash (128-slice) with Sinogram Affirmed IterativeReconstruction (SAFIRE); and Toshiba Aquillion One (320-slice) with Adaptive Iterative Dose Reduction (AIDR 3D).The machines were in two different hospitals but at fourdifferent locations.

Phantom

The study used a customized anthropomorphic liver phantom(St. Bartholomew’s Hospital, Clinical Physics Group, LondonEC1A 7BE, UK) (Fig. 1) designed for receiver operatingcharacteristic (ROC) analysis [15, 16]. The phantom is35 cm in the lateral direction, 27 in the anterior posterior

direction, and 6 cm deep (length axis). The phantom is anupper abdomen phantom, containing tissue equivalents ofliver, pancreas, and kidneys. The Hounsfield units (HU) ofthe liver were between 48 HU and 58 HU, depending upon thescanner. In the liver, there are four cylindrical inserts dividedinto eight sectors each (at 12.00, 13.30, 15.00, 16.30, 18.00,19.30, 21.00, and 22.30), for a total of 32 sectors. This isshown schematically to the right in Figure 1. In 16 of thesectors, there are drilled holes with diameters ranging from2 mm to 7 mm. Consequently, there were 32 sectors in eachimage to be evaluated, and there were 16 lesions at differentlocations in each image. The inserts were switched and rotatedfor each imaging model and for each dose level (12 differentpatterns in total) to avoid learning bias, as readers may recog-nize the test patterns. In this study, the holes were filled withwater, resulting in an attenuation difference of about 50 HUcompared to the liver tissue equivalent. This contrast differ-ence is higher compared to many liver lesions in vivo.

Imaging

Imaging parameters were based on the standard liver pro-tocol on each system, but minor adjustments were neces-sary to produce comparable images for all systems. Theparameters were as follows: 3-4 cm detector collimation;thinnest possible slice thickness; pitch as close to 1 aspossible; 120 kVp; abdominal reconstruction filter recom-mended by the vendors for CT liver examinations; recon-structed slice thickness, 3 mm; display field of view,350 mm; and rotation time between 0.50-0.74 s. Tubecurrent was adjusted so that the CTDIvol was 5 mGy,10 mGy, and 15 mGy, respectively. The fixed dose levelswere chosen so that they were lower than the nationaldiagnostic reference level for abdominal CT examinationsof 18 mGy [17]. When imaging the phantom with thestandard liver protocol and automatic dose modulation,the CTDIvol is between 10-18 mGy, depending upon thehospital’s liver protocol. Imaging parameters are shown inTable 1.

Reconstruction

Adaptive Statistical Iterative Reconstruction, ASiR

Information from the FBP reconstruction is used as a referencefor the initial image. Statistical methods and regularization areused to reduce noise and preserve edges, and the raw data isrecalculated. When the iterations are terminated, the resultingimage is blended with the FBP image corresponding to thechosen percentage ASiR. For example, ASiR 30 % is areconstruction blending of 30%ASiR and 70% FBP [18, 19].

Eur Radiol

iDose4

The noisiest projections in the raw data are denoised using aPoisson denoising algorithm, which is accomplished by aniterative process in the raw data domain, reducing noise andstreak artefacts [2]. In the image space, new iterations areperformed to reduce the noise and preserve the edges by usingan anatomical model [2, 20]. Seven different levels of iterationcan be chosen. iDose 1 is the mildest and iDose 7, correspond-ingly, is the strongest.

Sinogram Affirmed Iterative Reconstruction, SAFIRE

Corrections in the FBP image are performed in a loop betweenraw data and the image domain to reduce artefacts and de-crease noise. In addition, a loop in the image domain is

executed to decrease noise through a statistical optimizationprocess, while edges are preserved. The number of iterationsdepends upon the needs of the specific scan [3, 21]. SAFIREhas five levels of iterative reconstruction, with SAFIRE 1 themildest and SAFIRE 5 the strongest.

AIDR 3D

A statistical model in combination with an imaging model isused on the raw data to reduce noise and artefacts. Theiterations are executed in image space only, where edge pres-ervation and smoothing are performed. The corrected image isblended with the initial image (from the raw data) to keep thenoise granularity [4]. There are three different levels of itera-tion with different iterative strengths: mild, standard, andstrong.

Fig. 1 Left: A customizedanthropomorphic liver phantom.The imitated liver has four insertswith drilled holes which can befilled with liquids with a differentdensity than the liver. Theseinserts can be rotated and theirpositions can be switched. Right:Schematic example of one of theinserts with four holes of differentsize

Table 1 Imaging parameters for the four CT systems GE CT750 HD, Philips Ingenuity, Siemens Definition Flash, and Toshiba Aquillion One

Scanner Reconstructionmethod

Tubevoltage(kV)

mAs Rotationtime (s)

Pitch Collimation Reconstructedslicethickness

Reconstructionfilter

Reconstructedfield of view

CTDI(mGy)

GE CT750HD

FBP 120 81, 158and 241

0.7 0.98 64x0.625 3 Std 350 5.0, 9.9and 15.2ASiR 30 %

ASiR 40 %

ASiR 50 %

Veo

PhilipsIngenuity

FBP 120 89, 189and 277

0.74 1.014 64x0.625 3 B 350 5.8, 12.4and 18.1iDose 1

iDose 3

iDose 6

SiemensDefinitionFlash

FBP 120 74, 149and 223

0.5 1 128x0.6 3 B30f/I30f 350 5.0, 10.1and 15.0SAFIRE 1

SAFIRE 3

SAFIRE 5

ToshibaAquillionOne

FBP 120 45, 85and 130

0.5 0.813 80x0.5 3 FC13 350 5.2, 9.9and 15.2AIDR 3D mild

AIDR 3D std

AIDR 3D str

Eur Radiol

Veo

Veo is a pure iterative reconstruction algorithm that correctsthe initial estimation of the image with geometric models(focus, voxel, and detector size) as well as the statisticalmodels. In addition to reducing noise and artefacts, spatialresolution can be improved. Physical models and medicalimage characteristics are also utilized [22].

All images were reconstructed with the vendors’standard FBP in addition to iterative reconstruction foreach scanner. Different strengths of iterative reconstruc-tion were applied to the scans. The GE images werereconstructed with Veo, ASiR 30 %, 40 %, and 50 %;Philips with iDose 1, 3, and 6; Siemens with SAFIRE1, 3, and 5; and Toshiba with AIDR 3D mild, standard,and strong (Table 1). To make comparison easier, algo-rithms were divided into three groups: strong iterativereconstruction (ASiR 50 %, iDose 6, SAFIRE 5, AIDR3D strong, and Veo); medium (ASiR 40 %, iDose 3,SAFIRE 3, AIDR 3D standard, and Veo); and mild(ASiR 30 %, iDose 1, SAFIRE 1, AIDR 3D mild, andVeo). The strength of the algorithms is not necessarilyequal. Since Veo does not have a degree of iterativestrength, it is compared to all strengths of the otheriterative algorithms.

ASiR 50 % was chosen rather than the higher blend ofASiR because of the reported pixilated appearance of theimages at higher blends [11, 23, 24]. For the same reason,iDose 6 was chosen instead of iDose 7.

Reconstruction times for an upper abdomen with Veo arebetween 20 and 40 minutes, depending upon data size [25].ASiR, iDose, SAFIRE, and AIDR have reconstruction timesof 20-32 images per second [2, 26, 27].

Image interpretation

Single images from the middle of the phantom wereinterpreted from each scan series. The images wereinterpreted independently by three experienced radiolo-gists (more than 20 years’ experience) and one experi-enced CT physicist (six years’ experience). The readerswere blinded to all imaging data, and the images wereinterpreted in randomised order on the same PACSworkstation (Sectra PACS, 1 megapixel colour LCD,Eizo RadiForce R11). The readers knew how manysectors they were to interpret and the location of thesectors. They evaluated all sectors on a four-point scale.Lesion conspicuity was graded as follows: 1, no lesion;2, possibly no lesion; 3, possibly lesion; 4, lesion. Thereaders did not how many lesions there were to bedetected. Six of the first seven images were used fortraining of the observers and were excluded in thecomparison, but used for evaluation of intra-observer

variation. The 5 mGy FBP of Philips Ingenuity was leftout of the interpretation. Due to the risk of introducinguncontrolled bias, we decided not to evaluate this imageat a later time point. The ambient light was subdued.None of the readers were involved in the imagingprocess.

Fifteen images with different iterative reconstruction tech-niques and dose levels were reviewed two or three times toevaluate possible intra-observer variation.

Data and statistics

The null hypothesis of the study proposes that all com-mercially available iterative reconstruction algorithmsused in CT improve lesion detection in the liver, and thatthere are no difference in lesion detection among thevendors. A significance level of 0.05 was used, and thiswas adjusted to control for family-wise error rate using aBonferroni-Holm correction [28] when multiple compari-sons were made.

For each image, the reviews of the 32 sectors from each ofthe four readers were combined to provide a data set of 128sectors for each image. True-positive findings were plottedagainst false-positive findings (sensitivity against 1-specificity), and area under the curve (AUC) and confidenceintervals were calculated using nonparametric receiver oper-ating characteristics (ROC) methodology [29, 30] with thestatistical software Analyse-it (Version 2.03, Analyse-it Soft-ware Ltd, United Kingdom).

The data was checked for bias from multiple observers bycomparing AUC with AUC calculated in two other ways: 1)mean of AUC from ROC curves from each observer, and 2)AUC from ROC curve resulting after averaging the scoresfrom the observers for each sector.

Bias due to multiple readings was checked by intra-observer analysis to ensure that there was no difference be-tween results of images interpreted early in the reading andthose interpreted late in the reading.

Noise, contrast-to-noise ratio (CNR), and detection ratewith lesion size

Noise and HU values were automatically measured in fourdifferent regions of interest (ROIs) in the liver using a com-mercial software package (MATLAB 7.14 (R2012a), TheMathWorks Inc., Natick, MA, 2000). The mean of these fourROIs was calculated together with the CNR.

Lesions were divided into three groups according to lesionsize. Group 1 consisted of lesion sizes of 2-3 mm, group 2 oflesion sizes of 3.5-5 mm, and group 3 of lesion sizes of 6-7 mm. For each image, the number of detected lesions in eachgroup was counted, and a mean of the numbers for the fourreaders was calculated.

Eur Radiol

Results

Generally, the AUCs increased and confidence intervalsnarrowed with higher dose levels. Differences among vendorswere higher than differences with and without iterative recon-struction within one system (Fig. 2). Only significant differ-ences after correction for family-wise error rate are included inthe discussion that follows. Figure 3 shows the phantom withdifferent reconstruction algorithms at 5 and 10 mGy.

Veo

Veo resulted in larger AUCs (0.79, 0.95, and 0.97) comparedto FBP (0.70, 0.86, and 0.93) at all dose levels (p<0.001 at5 mGy, p<0.001 at 10 mGy, and p=0.047 at 15 mGy [notsignificant after correction for family-wise error rate])(Tables 2 and 3 and Fig. 2). At 5 mGy, Veo resulted in a largerAUC than ASiR and AIDR (p-values<0.004). At 10 mGy,Veo resulted in a larger AUC than all other reconstructionswith the exception of SAFIRE 5 (p-values<0.003). At15 mGy, Veo resulted in a larger AUC than iDose and ASiR(p-values of 0.001-0.008) (the difference between Veo and

ASiR 30%was not significant after correcting for family-wiseerror rate) (Tables 2 and 4, and Fig. 2).

ASiR

ASiR did not result in larger AUCs compared to FBP at any ofthe three dose levels (p-values of 0.031-0.496). No significantdifferences among AUCs from ASiR 30 %, ASiR 40 %, andASiR 50 % were detected (Table 3). The AUCs were also inthe lower range compared to the other reconstruction algo-rithms (0.71–0.73 at 5 mGy, 0.87–0.88 at 10 mGy, and 0.89–0.92 at 15 mGy) (Table 2 and Fig. 2).

iDose

iDose did not result in larger AUCs compared to FBP at 10and 15 mGy (p-values of 0.037–1.000) (Tables 2 and 3). Nosignificant differences among AUCs from iDose 1, iDose 3and iDose 6were detected (p-values of 0.037–0.980) (Table 3).Compared to the other reconstruction algorithms, iDose 6resulted in one of the largest AUCs at 5 mGy (0.79), whileat 10 mGy and 15 mGy the AUCs were two of the smallest(0.86–0.91) (Tables 2 and 4, and Fig. 2).

Fig. 2 Areas under the ROC curves (marked) with 95 % confidence interval (error bars) for filtered back projection (FBP) and iterative reconstructionwith GE CT750 HD, Philips Ingenuity, Siemens Definition Flash, and Toshiba Aquillion One. From left: 5 mGy, 10 mGy, and 15 mGy

Eur Radiol

SAFIRE

SAFIRE 3 resulted in a higher AUC (0.84) compared to FBP(0.78) at 5 mGy (p=0.007). At the other dose levels, SAFIREdid not result in a larger AUC compared to FBP (p-values of0.084–0.883). No differences among AUCs from SAFIRE 1,SAFIRE 3, and SAFIRE 5 were detected (Tables 2 and 3).Compared to the other reconstruction algorithms, SAFIREresulted in larger AUCs at all dose levels (0.80–0.84 at5 mGy, 0.88–0.92 at 10 mGy, and 0.93–0.97 at 15 mGy)(Table 2 and Fig. 2).

AIDR

AIDR did not result in larger AUCs compared to FBP at anyof the dose levels (p-values of 0.008–0.837). No differencesbetween AUCs from AIDR mild, AIDR standard, and AIDRstrong were detected (p-values of 0.148–0.969) (Tables 2 and3). Compared to the other reconstruction algorithms, AIDRresulted in the smallest or one of the smallest AUCs at 5 mGy(0.68–0.70) and 10 mGy (0.84–0.85). At 15 mGy, AIDRresulted in one of the largest AUCs (0.94–0.95) (Tables 2and 4, and Fig. 2).

Fig. 3 Images of the phantom at 5 mGy (left) and 10 mGy (right). Fromleft to right and down for each dose level: Filtered back projection (FBP)GE CT750 HD, ASiR 40 %, FBP GE CT750 HD, Veo, FBP Philips

Ingenuity, iDose 3, FBP Siemens Definition Flash, SAFIRE 3, FBPToshibaAquillion One andAIDR 3D standard. The inserts are at differentpositions and rotated at the different images

Eur Radiol

FBP

The AUCs from FBP with GE CT750 HD, Philips Ingenuity,Siemens Definition Flash, and Toshiba Aquillion One were

0.70, no result for Ingenuity, 0.78, and 0.68 at 5 mGy; 0.86,0.87, 0.88, and 0.81 at 10 mGy; and 0.93, 0.90, 0.95, and0.92 at 15 mGy (Table 2 and Fig. 2). At 5 mGy, DefinitionFlash resulted in a significantly larger AUC than CT750 HD

Table 2 Areas under the ROC curve (AUC) and 95 % confidence interval (CI) with different dose levels, vendors, filtered back projection (FBP) anditerative reconstruction.

GE AUC 95 % CI Philips AUC 95 % CI Siemens AUC 95 % CI Toshiba AUC 95 % CI

5 mGy FBP 0.70 0.63-0.77 FBP FBP 0.78 0.71-0.84 FBP 0.68 0.60-0.75

ASiR 30 % 0.73 0.66-0.80 iDose 1 0.75 0.69-0.82 SAFIRE 1 0.80 0.73-0.86 AIDR 3D mild 0.70 0.62-0.78

ASiR 40 % 0.71 0.64-0.79 iDose 3 0.75 0.68-0.82 SAFIRE 3 0.84 0.78-0.90 AIDR 3D standard 0.68 0.61-0.76

ASiR 50 % 0.73 0.66-0.80 iDose 6 0.79 0.72-0.85 SAFIRE 5 0.80 0.74-0.86 AIDR 3D strong 0.70 0.63-0.77

Veo 0.79 0.73-0.86

10 mGy FBP 0.86 0.80-0.91 FBP 0.87 0.82-0.93 FBP 0.88 0.83-0.94 FBP 0.81 0.74-0.88

ASiR 30 % 0.88 0.83-0.94 iDose 1 0.87 0.82-0.93 SAFIRE 1 0.89 0.83-0.94 AIDR 3D mild 0.84 0.78-0.90

ASiR 40 % 0.87 0.81-0.93 iDose 3 0.84 0.78-0.89 SAFIRE 3 0.88 0.83-0.94 AIDR 3D standard 0.85 0.79-0.91

ASiR 50 % 0.88 0.83-0.93 iDose 6 0.86 0.80-0.91 SAFIRE 5 0.92 0.87-0.97 AIDR 3D strong 0.84 0.78-0.91

Veo 0.95 0.91-0.99

15 mGy FBP 0.93 0.88-0.97 FBP 0.90 0.85-0.95 FBP 0.95 0.91-0.99 FBP 0.92 0.87-0.96

ASiR 30 % 0.92 0.87-0.96 iDose 1 0.89 0.84-0.94 SAFIRE 1 0.97 0.94-1.00 AIDR 3D mild 0.94 0.90-0.98

ASiR 40 % 0.89 0.84-0.94 iDose 3 0.90 0.85-0.95 SAFIRE 3 0.93 0.89-0.97 AIDR 3D standard 0.95 0.91-0.99

ASiR 50 % 0.91 0.86-0.96 iDose 6 0.91 0.86-0.95 SAFIRE 5 0.93 0.89-0.98 AIDR 3D strong 0.95 0.91-0.98

Veo 0.97 0.94-1.00

Table 3 p-values. Significant results are marked with an asterisk (*) and significant results after family wise error rate correction are marked with twoasterisks (**)

FBP ASiR 30 % ASiR 40 % ASiR 50 % FBP iDose 1 iDose 3

5 mGy ASiR 30 % 0.038* iDose 1

ASiR 40 % 0.150 iDose 3 0.980

ASiR 50 % 0.038* 0.962 0.248 iDose 6 0.093 0.106

Veo 0.000** 0.004** 0.001** 0.002**

10 mGy ASiR 30 % 0.204 iDose 1 1.000

ASiR 40 % 0.463 0.781 iDose 3 0.037* 0.037*

ASiR 50 % 0.148 0.352 0.627 iDose 6 0.172 0.337 0.128

Veo 0.000** 0.000** 0.001** 0.002**

15 mGy ASiR 30 % 0.496 iDose 1 0.466

ASiR 40 % 0.031* 0.071 iDose 3 0.728 0.205

ASiR 50 % 0.303 0.549 0.257 iDose 6 0.317 0.254 0.676

Veo 0.047* 0.008* 0.001** 0.003**

FBP SAFIRE 1 SAFIRE 3 FBP AIDR 3D mild AIDR 3D standard

5 mGy SAFIRE 1 0.156 AIDR 3D mild 0.265

SAFIRE 3 0.007** 0.036* AIDR 3D standard 0.837 0.182

SAFIRE 5 0.070 0.756 0.084 AIDR 3D strong 0.122 0.969 0.1482

10 mGy SAFIRE 1 0.843 AIDR 3D mild 0.008*

SAFIRE 3 0.883 0.676 AIDR 3D standard 0.012* 0.672

SAFIRE 5 0.084 0.097 0.056 AIDR 3D strong 0.016* 0.880 0.762

15 mGy SAFIRE 1 0.247 AIDR 3D mild 0.058

SAFIRE 3 0.338 0.019* AIDR 3D standard 0.045* 0.151

SAFIRE 5 0.472 0.029* 0.066 AIDR 3D strong 0.178 0.878 0.925

Eur Radiol

Tab

le4

p-values.A

lgorith

msweredividedintothreegroups:m

ild(Veo,A

SiR30

%,iDose1,SAFIRE1,andAID

R3D

Mild

),medium(Veo,A

SiR

40%,iDose3,SAFIRE3,andAID

R3D

Standard),

andstrong

(Veo,A

SiR50

%,iDose6,SA

FIRE5andAID

R3D

Strong).Significantresults

aremarkedwith

anasterisk

(*)and

significantresultsafterfam

ily-w

iseerrorratecorrectio

naremarkedwith

two

asterisks(**)

FBP

Mild

Medium

Strong

Ingenuity

Definition

Flash

Aquillion

One

ASiR

30%

iDose1

SAFIRE

1AID

R3D

Mild

ASiR

40%

iDose3

SAFIRE

3AID

R3D

Standard

ASiR

50%

iDose6

SAFIRE

5AID

R3D

Strong

5mGy

Veo

0.004**

0.036*

0.819

0.000**

0.001**

0.041*

0.041*

0.000**

0.002**

0.669

0.676

0.000**

CT750HD/

ASiR

0.000**

0.284

0.131

0.003**

0.088

0.031*

0.000**

0.076

0.010**

0.001**

0.137

Ingenuity

/iDose

0.017*

0.010*

0.000**

0.001**

0.360

0.002**

Definition

Flash/

Safire

0.000**

0.000**

0.000**

0.000**

10mGy

Veo

0.002**

0.001**

0.003**

0.000**

0.001**

0.000**

0.002**

0.000**

0.002**

0.000**

0.098

0.000**

CT750HD/

ASiR

0.096

0.247

0.006**

0.575

0.751

0.085

0.125

0.257

0.185

0.075

0.034

0.052

Ingenuity

/iDose

0.578

0.001**

0.240

0.119

0.033*

0.456

0.004**

0.471

Definition

Flash/

Safire

0.005**

0.018*

0.042*

0.001**

15mGy

Veo

0.008*

0.001**

0.198

0.047*

0.001**

0.002**

0.023

0.090

0.003**

0.003**

0.036

0.107

CT750HD/

ASiR

0.070

0.214

0.541

0.049*

0.007*

0.078

0.248

0.031

0.006**

0.964

0.037

0.045

Ingenuity

/iDose

0.027*

0.339

0.001**

0.008*

0.077

0.014

0.062

0.021

Definition

Flash/

Safire

0.142

0.042*

0.274

0.546

Eur Radiol

and Aquillion One (p<0.001). At 10 mGy, Aquillion Oneresulted in a significantly smaller AUC than CT750 HD,Ingenuity, and Definition Flash (p=0.006, p=0.001, and p=0.005). At 15 mGy, no differences were significant (Table 4).

Inter- and intra-observer differences and multiple correlations

Interobserver analysis showed little difference among ob-servers. ROC curves from the medium iterative reconstructionat 10 mGy for each observer are shown in Figure 4 as arepresentative example. P-values are shown in Table 5.Intra-observer analysis showed little differences among dif-ferent interpretations (Table 6).

The check for bias from multiple observers showed nodifference in AUC, and no additional corrections were made.

Noise and CNR

Table 7 shows noise, noise reduction, and CNR for all recon-struction techniques and dose levels. All of the iterative re-construction algorithms reduced noise compared to FBP at all

Fig. 4 Interobserver differences. ROC curves from medium-strength iterative reconstruction at 10 mGy

Table 5 Interobserver differences. p-values after comparing withDelong-Delong-Clarke Pearson method [30], medium-strength iterativereconstruction at 10 mGy. Significant differences (p>0.05) are markedwith an asterisk (*). Dash (-) indicates no difference

Observer 2 Observer 3 Observer 4

ASiR 40% Observer 1 0.390 0.074 0.039*

Observer 2 0.317 0.597

Observer 3 0.337

iDose 3 Observer 1 0.317 0.317 0.063

Observer 2 - 0.143

Observer 3 0.143

SAFIRE 3 Observer 1 0.955 0.378 0.177

Observer 2 0.816 0.496

Observer 3 0.283

AIDR standard Observer 1 0.561 0.545 0.545

Observer 2 0.816 0.816

Observer 3 -

Veo Observer 1 0.561 0.545 0.545

Observer 2 0.816 0.816

Observer 3 -

Eur Radiol

dose levels, and the higher strength of each algorithm reducedthe noise more than the lower-strength algorithms. The reduc-tion in noise was 14–30%with ASiR, 61–71%with Veo, 10–46 % with iDose, 11–51 % with SAFIRE, and 29–67 % withAIDR. CNR from 5 mGy, 10 mGy, and 15 mGy ranged from1.5–2.0, 2.3–2.6, and 2.9–3.2 for ASiR; 4.7, 5.0, and 6.5 forVeo; 1.5–2.2, 3.1–4.4, and 3.3–4.9 for iDose; 2.1–4.1, 3.3–4.8, and 4.1–6.2 for SAFIRE; and 2.0–2.5, 2.5–3.3, and 3.7–5.2 for AIDR.

Detection rate with lesion size

All of the largest lesions (6–7 mm) were detected with allreconstruction algorithms with the exception of Aquillion Oneat 5 mGy. The middle-sized lesions (3.5–5 mm) had detectionrates of 50–69 % at 5 mGy, 75 %-100 % at 10 mGy, and86 %-100 % at 15 mGy. The smallest lesions (2–3 mm) haddetection rates of 0–35 % at 5 mGy, 20–80 % at 10 mGy, and45–90 % at 15 mGy (Table 8).

Discussion

In this study, we demonstrated that some iterative reconstruc-tion techniques improved lesion conspicuity in a liver phan-tom while others did not, depending upon iterative techniqueand dose level. As the image quality with FBP was high at15 mGy, the iterative reconstruction did not improve lesiondetectability compared to FBP. The larger AUCs for iterativetechniques at 5 mGy (only SAFIRE 3 and Veo, however, wassignificantly larger) indicated that iterative reconstructionsmoothed noise and/or decreased artefacts at low doses, asexpected.

Many studies have shown quantitative improvementin image quality with iterative reconstruction comparedto FBP [31–35], and that with reduced noise, lesiondetection and lesion conspicuity can be improved. Thisstudy, however, shows that this is not necessarily true.The iterative reconstruction algorithms that improvedlesion detectability at 5 mGy had noise reduction of71 % (Veo) and 30 % (SAFIRE 3) compared to FBP.AIDR standard showed noise reduction of 64 % com-pared to FBP, resulting in an absolute noise value com-parable to that of SAFIRE 3. CNR was also compara-ble. However, no improvement in lesion detection wasfound, suggesting that quantitative measures are notadequate for evaluating the effectiveness of iterativereconstruction algorithms.

Image texture may change and artefacts can be introduced,and this may affect lesion detection. The larger AUCs withFBP and SAFIRE 1 at 15 mGy compared to SAFIRE 3 and 5(not significant) may be related to unfamiliar image textureintroduced by strong iterative reconstructions at already-low-noise images. The readers commented that it appeared that theimages rotated because they were more blurry. More studiesare now focused on diagnostic effectiveness rather than quan-titative measures, and our results are consistent with otherstudies [21, 36–38].

The image quality experienced can be affected by theimage quality level that the readers are accustomed to. Thereaders in this study work in a large hospital with nationalassignments in additional areas, and are therefore accustomedto images from all vendors, although images with AIDR andASiR (not Veo) would be more common as these are thealgorithms at the location. These algorithms did not score highin this study.

Table 6 Intra-observer differences at 10 mGy. p-values from comparison after 2 or 3 interpretations of images with Delong-Delong-Clarke Pearsonmethod [30]. No significant differences (p>0.05) were found. Dash (-) indicates no difference

Observer 1 Observer 2 Observer 3 Observer 4

Interpretation no 1 2 1 2 1 2 1 2

ASiR 30 % 2 0.480 0.317 0.289 0.548

3 0.145 0.109 0.143 0.317 0.401 0.860 0.330 0.217

Veo 2 0.480 0.361 0.350 -

iDose 6 2 0.401 0.317 0.317 0.143

SAFIRE 3 2 0.373 - 0.636 0.317

3 0.774 0.778 - - - 0.636 0.208 0.496

SAFIRE 5 2 0.350 0.333 0.448 0.317

AIDR 3D strong 2 0.220 - 0.590 -

3 0.050 0.661 - - 0.220 0.143 - -

Aquillion One FBP 2 0.598 0.358 0.695 0.249

AIDR 3D standard 2 0.867 0.816 0.317 -

3 0.412 0.317 0.441 0.330 0.152 0.084 - -

Eur Radiol

Tab

le7

Noise

andcontrast-to-noiseratio

(CNR)

GE

Noise

Noise

reduction

CNR

Philips

Noise

Noise

reduction

CNR

Siemens

Noise

Noise

reduction

CNR

Toshiba

Noise

Noise

reduction

CNR

5mGy

FBP

32,1

1,5

FBP

28,2

1,4

FBP

22,1

1,6

FBP

42,5

0,8

ASiR30

%27,7

14%

1,5

iDose1

24,6

13%

1,5

SAFIRE1

19,1

13%

2,1

AID

R3D

mild

18,2

57%

2,0

ASiR40

%24,6

23%

1,6

iDose3

21,7

23%

1,8

SAFIRE3

15,4

30%

3,4

AID

R3D

standard

15,4

64%

2,4

ASiR50

%22,8

29%

2,0

iDose6

16,0

43%

2,2

SAFIRE5

10,8

51%

4,1

AID

R3D

strong

13,8

67%

2,5

Veo

9,5

71%

4,7

10mGy

FBP

23,1

1,9

FBP

17,3

2,7

FBP

18,7

2,2

FBP

25,6

1,7

ASiR30

%19,0

18%

2,3

iDose1

15,3

12%

3,1

SAFIRE1

13,2

29%

3,3

AID

R3D

mild

16,2

37%

2,5

ASiR40

%16,4

29%

2,4

iDose3

13,3

23%

3,8

SAFIRE3

10,0

47%

4,5

AID

R3D

standard

14,0

46%

2,8

ASiR50

%17,7

23%

2,6

iDose6

9,4

46%

4,4

SAFIRE5

9,8

48%

4,8

AID

R3D

strong

12,2

52%

3,3

Veo

8,5

63%

5,0

15mGy

FBP

18,5

2,3

FBP

14,1

2,9

FBP

12,7

3,9

FBP

18,4

2,3

ASiR30

%15,0

19%

2,9

iDose1

12,7

10%

3,3

SAFIRE1

11,2

11%

4,3

AID

R3D

mild

13,1

29%

3,7

ASiR40

%14,0

24%

3,2

iDose3

11,1

21%

3,6

SAFIRE3

9,9

22%

4,1

AID

R3D

standard

11,8

36%

4,3

ASiR50

%12,9

30%

2,9

iDose6

8,1

43%

4,9

SAFIRE5

7,0

45%

6,2

AID

R3D

strong

9,8

47%

5,2

Veo

7,3

61%

6,5

Eur Radiol

Tab

le8

Percentage

ofdetected

lesionsingroups

with

differentsize;group1:2-3mm

,group

2:3.5-5mm

andgroup3:6-7mm

.Ingroup1therewere5lesions,ingroup2therewere7lesionsandin

group3therewere4lesions

Ø(m

m)

Detectedlesions

GE

2-3

3.5-5

6-7

Philips

2-3

3.5-5

6-7

Siemens

2-3

3.5-5

6-7

Toshiba

2-3

3.5-5

6-7

5mGy

FBP

0%

54%

100%

FBP

30%

54%

100%

FBP

5%

57%

88%

ASiR30

%0%

61%

100%

iDose1

10%

61%

100%

SAFIRE1

10%

79%

100%

AID

R3D

mild

10%

57%

95%

ASiR40

%5%

50%

100%

iDose3

5%

68%

100%

SAFIRE3

35%

75%

100%

AID

R3D

standard

5%

57%

88%

ASiR50

%5%

61%

100%

iDose6

30%

64%

100%

SAFIRE5

5%

79%

100%

AID

R3D

strong

0%

57%

88%

Veo

15%

75%

100%

10mGy

FBP

50%

75%

100%

FBP

30%

93%

100%

FBP

40%

93%

100%

FBP

25%

82%

100%

ASiR30

%55

%86

%100%

iDose1

35%

89%

100%

SAFIRE1

50%

89%

100%

AID

R3D

mild

20%

89%

100%

ASiR40

%60

%86

%95

%iDose3

15%

86%

100%

SAFIRE3

45%

93%

100%

AID

R3D

standard

25%

93%

100%

ASiR50

%50

%82

%100%

iDose6

25%

89%

100%

SAFIRE5

55%

100%

100%

AID

R3D

strong

35%

86%

100%

Veo

80%

93%

100%

15mGy

FBP

65%

93%

100%

FBP

45%

93%

100%

FBP

90%

89%

100%

FBP

60%

96%

100%

ASiR30

%60

%93

%100%

iDose1

45%

93%

100%

SAFIRE1

80%

100%

100%

AID

R3D

mild

75%

96%

100%

ASiR40

%55

%86

%100%

iDose3

55%

89%

100%

SAFIRE3

80%

86%

100%

AID

R3D

standard

75%

96%

100%

ASiR50

%70

%86

%100%

iDose6

55%

89%

100%

SAFIRE5

80%

86%

100%

AID

R3D

strong

70%

96%

100%

Veo

80%

100%

100%

Eur Radiol

The FBP images at 5 mGy from GE CT750 HD andToshiba Aquillion One resulted in poor lesion conspicuity.The initial amount of noise may have influenced the effective-ness of ASiR and AIDR 3D, since these had small AUCs at5 mGy compared to the other algorithms. The better detectionwith AIDR at 15 mGy indicates that the noise was on a levelthat was favourable with AIDR.

Iterative algorithms that improve lesion detection at lowdoses, as with Veo and SAFIRE, can help to reduce theradiation dose to the patients. However, the protocols mustbe optimised. An aggressive dose reduction can reduce lesiondetection even with Veo, as Pickhardt et al. [36] demonstrated.

Our study had certain limitations. We used an anthropo-morphic liver phantom, showing that iterative reconstructionimproved lesion detectability with fixed lesion sizes and con-trast and when lesions were known to be in a predefined area.The phantom has a fixed size, and the results do not show howsizes among patients affect lesion conspicuity with the differ-ent algorithms. To fully assess iterative reconstruction tech-niques, clinical trials must be performed for specific proce-dures. One of the observers was familiar with the phantom.However, because this was a relative-value study and thereconstructions were compared with each other, this shouldnot have affected the results. We tested only one CT systemfrom each vendor. In the future, a larger number of systemsshould be tested to confirm that our CTunits are representativeof the systems. Since only ASiR 50% and iDose 6 were testedin this study, better detectability could have been achievedwith higher strengths of ASiR and iDose.

In conclusion, different iterative reconstruction techniqueshad different impacts on lesion conspicuity in a liver phantom.Iterative reconstruction improved image quality at some doselevels, but the differences among algorithms influenced theeffectiveness and the improvement. General improvement inlesion conspicuity was observed only with Veo.

Acknowledgments Thanks to Erlend Andersen, Joanna FennKristiansen, Wendy Garborg, and Rima Seputytë for help with phantomscanning, and thanks to Per Kristian Hol and Kristin Forså for imageevaluation.

The scientific guarantor of this publication is Anne Catrine Martinsen.The authors of this manuscript declare no relationships with any companieswhose products or services may be related to the subject matter of thearticle. The authors state that this work has not received any funding. KyrreEmblem kindly provided statistical advice for this manuscript. InstitutionalReviewBoard approval was not required because this was a phantom study.Methodology: diagnostic or prognostic multicenter study.

References

1. Thibault JB (2011) A model-based paradigm: A new frontier inimage reconstruction. In: GE Healthcare CT publication GEHealthcare. http://www.gehealthcare.com/usen/ct/products/docs/CT_Clarity_062411_pg63-65.pdf. Accessed 9 Jan 2014

2. Scibelli A (2011) iDose4 iterative reconstruction technique.Healthcare, Philips

3. Grant K, Raupach R (2012) SAFIRE: Sinogram Affirmed IterativeReconstruction. usa.healthcare.siemens.com

4. Irwan R, Nakanishi S, Blum A (2011) AIDR 3D - Reduces dose andsimultaneously improves image quality. Toshiba Med Syst 1–8.www.toshiba-medical.eu/upload/TMSE_CT/White%20Papers/Toshiba_White%20paper%20CT_nov11.pdf. Accessed 14 Jul 2013

5. Miéville F, Gudinchet F, Brunelle F, Bochud F, Verdun F (2012) Iterativereconstruction methods in two different MDCT scanners: physical met-rics and 4-alternative forced-choice detectability experiments - A phan-tom approach. Phys Med. doi:10.1016/j.ejmp.2011.12.004

6. Noël PB, Fingerle AA, Renger B, Münzel D, Rummeny EJ, DobritzM (2011) Initial performance characterization of a clinical noise-suppressing reconstruction algorithm for MDCT. AJR Am JRoentgenol 197:1404–1409

7. Moscariello A, Takx RAP, Schoepf J et al (2011) Coronary CTangiography: image quality, diagnostic accuracy and potential forradiation dose reduction using a novel iterative image reconstructiontechnique - comparison with traditional filtered back projection. EurRadiol 21:2130–2138

8. Gervaise A, Osemont B, Lecocq S et al (2012) CT image qualityimprovement using adaptive iterative dose reduction with wide-volume acquisition on 320-detector CT. Eur Radiol 22:295–301

9. Prakash P, Kalra MK, Kambadakone AK et al (2010) Reducingabdominal CT radiation dose with adaptive statistical iterative recon-struction technique. Investig Radiol 45:202–210

10. Fleischmann D, Boas FE (2011) Computed tomography - old ideasand new technology. Eur Radiol 21:510–517

11. Sagara Y, Hara AK, Pavlicek W, Silva AC, Paden RG, Wu Q (2010)Abdominal CT: comparison of low-dose CTwith adaptive statisticaliterative reconstruction and routine-dose CT with filtered back pro-jection in 53 patients. AJR Am J Roentgenol 195:713–719

12. Nelson RC, Feuerlein S, Boll DT (2011) New iterative reconstructiontechniques for cardiovascular computed tomography: how do theywork, and what are the advantages and disadvantages? J CardiovascComput Tomogr 5:286–292

13. Nuyts J, De Man B, Fessler JA, Zbijewski W, Beekman FJ (2013)Modelling the physics in the iterative reconstruction for transmissioncomputed tomography. Phys Med Biol 58:R63–R96

14. Beister M, Kolditz D, Kalender WA (2012) Iterative reconstructionmethods in x-ray CT. Phys Med 28:94–108

15. Olerud HM, Skretting A (1999) An anthropomorphic phantom forreceiver operating characteristic studies in CT imaging of liver le-sions. Br J Radiol 72:35–43

16. Martinsen AC, Sæther HK, Hol PK, Olsen DR, Skaane P (2011)Iterative reconstruction reduces abdominal CT dose. Eur J Radiol 81:1483–1487

17. Authority NRP (2012) Reviderte og nye nasjonale referanseverdierfor røntgendiagnostiske undersøkelser per 2012. StrålevernInfo 2–10

18. Silva AC, Lawder HJ, Hara A, Kujak J, Pavlicek W (2010)Innovations in CT dose reduction strategy: application of the adap-tive statistical iterative reconstruction algorithm. AJR Am JRoentgenol 194:191–199

19. Mueck FG, Körner M, Scherr M et al (2012) Upgrade to iterativeimage reconstruction (IR) in abdominal MDCT imaging: a clinicalstudy for detailed parameter optimization beyond vendor recommen-dations using the adaptive statistical iterative reconstruction environ-ment (ASIR) . Fortschr Röntgenstr 184:229–238

20. Leipsic J, Heilbron BG, Hague C (2011) Iterative reconstruction forcoronary CTangiography: finding its way. Int J Cardiovasc Imaging.doi:10.1007/s10554-011-9832-3

21. Baker ME, Dong F, Primak A et al (2012) Contrast-to-noise ratio andlow-contrast object resolution on full- and low-doseMDCT: SAFIREversus filtered back projection in an low-contrast object phantom andin the liver. AJR Am J Roentgenol 199:8–18

Eur Radiol

22. GE Healthcare (2010) Veo - For Discovery CT750 HD. In: GEHealthcare, Manual

23. Wirth S, Mück F, Körner M et al (2011) B-856: Image quality anddose saving aspects of adaptive statistical iterative reconstruction(ASIR) in 64-row abdominal CT imaging. European Congress ofRadiology, ECR, Vienna

24. Singh S, Kalra MK, Hsieh J et al (2010) Abdominal CT: Comparisonof Adaptive Statistical Iterative and filtered back projection recon-struction techniques. Radiology 257:373–383

25. de Mey J (2011) Implementing ultra-low dose CT with Veo atUniversity Hospital, Brussels. www.gehealthcare.com/CT,November 2011

26. Ghetti C, Palleri F, Serreli G, Ortenzia O, Ruffini L (2013) Physicalcharacterization of a new CT iterative reconstruction method operat-ing in sinogram space. J Appl Clin Med Phys 14:263–271

27. Angel E (2012) AIDR 3D iterative reconstruction. Toshiba Med Syst1–10. www.medical.toshiba.com/downloads/rxl-wp-aidr-3d-iterative-reconstruction. Accessed 14 Jul 2014

28. Holm S (1979) A simple sequentially rejective multiple test proce-dure. Scand J Statist 6:65–70

29. Metz CE (1978) Basic principles of ROC analysis. Semin Nucl Med8:283–298

30. DeLong ER, DeLong DM, Clarke-Pearson DL (1988) Comparingthe areas under two or more correlated receiver operating character-istic curves: A nonparametric approach. Biometrics 44:837–845

31. ChangW, Lee JM, Lee K et al (2013) Assessment of a model-based ,iterative reconstruction algorithm (MBIR) regarding image qualityand dose reduction in liver computed tomography. Invest Radiol48(8):598–606

32. Yoon MA, Kim SH, Lee JM et al (2012) Adaptive statistical iterativereconstruction and Veo: assessment of image quality and diagnosticperformance in CT colonography at various radiation doses. JComput Assist Tomogr 36:596–601

33. Olsson ML, Norrgren K (2012) An investigation of the iterativereconstruction method iDose4 on a Philips CT Brilliance 64 using aCatphan 600 phantom. SPIE Proc

34. Fletcher JG, Grant K, Fidler JL et al (2012) Validation of dual-source single-tube reconstruction as a method to obtain half-doseimages to evaluate radiation dose and noise reduction: Phantomand human assessment using CT colonography and sinogram-affirmed iterative reconstruction (SAFIRE) . J Comput AssistTomogr 36:560–569

35. Ohno Y, Takenaka D, Kanda T et al (2012) Adaptive iterativedose reduction using 3D processing for reduced- and low-dosepulmonary CT: Comparison with standard-dose CT for imagenoise reduction and radiological f indings. AJR Am JRoentgenol 199:W477–W485

36. Pickhardt PJ, Lubner MG, Kim DH et al (2012) Abdominal CTwithModel-Based Iterative reconstruction (MBIR): initial results of aprospective trial comparing ultralow-dose with standard-dose imag-ing. AJR Am J Roentgenol 199:1–9

37. Dobeli K, Lewis S, Meikle S, Thiele D, Brennan P (2013) Noise-reducing algorithms do not necessarily provide superior dose optimi-sation for hepatic lesion detection with multidetector CT. Br J Radiol.doi:10.1259/bjr.20120500

38. Schindera ST, Odedra D, Raza SA et al (2013) Iterative reconstruc-tion algorithm for CT: Can radiation dose be decreased while low-contrast detectability is preserved? Radiology 269:511–518

Eur Radiol