Using belief function theory to deal with uncertainty andimprecision in image processing

Benoît Lelandais1, Isabelle Gardin1,2, Laurent Mouchard1, Pierre Vera1,2, Su Ruan1

1LITIS EA 4108 - QuantIF, University of Rouen22 bd Gambetta, 76183 Rouen Cedex, France

2Department of nuclear medicine, Henri-Becquerel center,1 rue d’Amiens, 76038 Rouen Cedex 1, France

May 10 2012

Introduction Method Application to PET image fusion Discussion - Conclusion References

1 IntroductionPET imagingObjectives

2 Method for reducing uncertainty and imprecision in image processingBBA estimation for each imageInformation fusion for reducing uncertaintyInformation fusion for reducing imprecisionIllustration of the method on two simulated images sources

3 Application to PET image fusionFrame of discernmentBBA estimationFusion with an a priori knowledgeMulti-modal fusion

4 Discussion - Conclusion

- 2/25

Introduction Method Application to PET image fusion Discussion - Conclusion References

IntroductionContext

Both 3D anatomical and functional medical images are obtained during the sameacquisition

Anatomical imaging: CT (Computed Tomography).

Functional imaging: PET (Positron Emission Tomography) with FDG(Fluoro-Deoxy-Glucose). The FDG is an indicator of tumor glucose metabolism.

(a) PET/CT tomograph (b) Anatomical imagewith a tumoral region

(c) Functional image witha high FDG uptake in tu-mor cells

Transverse slices for a single patient suffering from lung cancer. The area of interest (tumorlesion) is located in the rectangle.

- 3/25

Introduction Method Application to PET image fusion Discussion - Conclusion References

IntroductionContext

Principle of treating tumors by radiation therapy

Segmentation of tumoral volume:From anatomical imaging: Computed Tomography imaging (Fig. (a)).From functional imaging: Positron Emission Tomography (PET) with FDGFluoro-Deoxy-Glucose (Fig. (b)).Integration of this information by the radiation oncologist to determine the targetvolume to be irradiated preserving organs at risk (Fig. (c)).

(a) CT segmentation (b) PET segmentation (c) Treatment planning.Irradiation everyday during ∼ 35 days.

- 4/25

Introduction Method Application to PET image fusion Discussion - Conclusion References

IntroductionPET imaging

PET imaging using three radiotracers

Radiotracer injection according to the biological function to study.FDG (Fluoro-Deoxy-Glucose): Indicator of glucose metabolism.FLT (FLuoro-Thymidine): Indicator of cell proliferation [Yang et al., 2010].FMiso (Fluoro-Misonidazole): Indicator of lack of oxygen (hypoxia).

⇒ cell radioresistance [Chang et al., 2009].

Complementary images: Relevance of information fusion.

(a) PET FDG (b) PET FLT (c) PET FMiso

Transverse slices for a single patient suffering from lung cancer. The time between eachacquisition is smaller than 72 hours. The area of interest (tumor lesion) is located in therectangle.

- 5/25

Introduction Method Application to PET image fusion Discussion - Conclusion References

IntroductionPET imaging

Imperfections in PET imaging

Low contrast.

Noise:Relative to the accuracy of a sensor with respect to reality.Induced by the statistical nature of the signal and by the reconstruction algorithm.

⇒ Information is uncertain.

Partial Volume Effect (PVE) (contamination of neighboring structures):Induced by the low spatial resolution of the acquisition system.

⇒ Information is imprecise (lack of knowledge) and is mainly localized at the transitionbetween regions.

(a) PET phantom (b) Histogram (c) PET phantom (d) Profile

Fig. (b) shows a histogram calculated from the theoretically uniform region of Fig. (a). Fig. (d) showsthe profile selected in Fig. (c) presenting fuzzy transitions between regions.

- 6/25

Introduction Method Application to PET image fusion Discussion - Conclusion References

IntroductionThéorie des fonctions de croyance

Belief function theory [Dempster, 1967; Shafer, 1976; Smets, 1990]

Frame of discernment (C classes): Ω = ω1, ω2, . . . , ωC.Multiple hypotheses are considered: 2Ω = ∅,ω1,ω2,ω1, ω2, . . . ,Ω.

Belief masses mΩ, also called BBA (Basic Belief Assignment), defined on thesehypotheses (∑A⊆Ω mΩ

(A) = 1).Uncertainty and imprecision modeling.

Belief revision:Discounting: manage reliability of a source.Disjunctive combination: fusion of multiple distinct sources whose one is reliable.Conjunctive combination: fusion of multiple distinct and reliable sources.

- 7/25

Introduction Method Application to PET image fusion Discussion - Conclusion References

IntroductionObjectives

Objectives

Dealing with imperfections in image processing:Reducing uncertainty and imprecision.Efficiency in the case of low contrasts.

Construction of parametric images to help the radiation oncologist in thedelineation of lesions for achieving a therapy, with potentially:

An increase of radiation session frequency (from one to two per day) forhigh-proliferative lesions.A radiation dose escalation for hypoxic lesions (due to their radioresistance).

- 8/25

Introduction Method Application to PET image fusion Discussion - Conclusion References

Method for reducing uncertainty and imprecision in image processingproposed approach

Method principle - 3 steps

1 BBA estimation for each image.Measure the membership degrees of each voxel according to two classes using FuzzyC-Means (FCM) [Bezdek, 1981].Integration of a neighboring information fusion using disjunctive rule in the FCMiterative process.

Conversion of membership degrees into BBA by transferring a part of belief masses forimperfect data to disjunctions.Centroid updating only from perfect data.

2 Fusion of neighboring information using Dempster’s rule [Capelle-Laize et al.,2004; Zhang et al., 2007].

Uncertainty reduction inside noisy regions.Highlighting of the boundary between regions (due to PVE problems).

3 Fusion of multiple knowledges using conjunctive rule.Imprecision reduction.

- 9/25

Introduction Method Application to PET image fusion Discussion - Conclusion References

Method for reducing uncertainty and imprecision in image processingproposed approach

Frame of discernment

Two classes:Ω = ω1,ω2

2Ω = ∅,ω1,ω2,ω1, ω2

Neighborhood contribution

Vc : the current voxel.

Vi : a neighbor of Vc .

αi : a weighting coefficient associated to Vi depending on the distance separatingVi from Vc .

Example of αi on PET images:

αi = exp(−12(Vc −Vi )2 /σ2)

with σ = FWHM2√

2 log 2and FWHM = 6 mm the Full Width at Half Maximum of PET

images.

- 10/25

Introduction Method Application to PET image fusion Discussion - Conclusion References

Method for reducing uncertainty and imprecision in image processing1 - BBA estimation for each image

Fuzzy C-Means (FCM) clustering algorithm

+ Unsupervised estimation of membership degrees towards the two classes.

+ Efficiency in the case of low contrast.

– Impossible to deal simultaneously with uncertainty and imprecision at the sametime.

Use of belief function theory to improve FCM algorithm.Combination using disjunctive rule of each voxel with its weighted neighborhood ateach iteration.

⇒ Transfer the belief masses towards hypothesis ω1, ω2 for voxels spatiallyambiguous (noise, PVE).

⇒ Update the class centroids from only certain and precise data.

- 11/25

Introduction Method Application to PET image fusion Discussion - Conclusion References

Method for reducing uncertainty and imprecision in image processing1 - BBA estimation for each image

Weighting

Each neighbor Vi of Vc is ponderated using:

m′Vi(A) = αimVi (A) ∀A ≠ ∅

m′Vi(∅) = 1 − αi (1 −mVi (∅))

The further away from Vc the voxel Vi is, the lower its contribution to thecomputation will be.

⇒ Reduction of the influence of distant voxels before fusing them.

Disjunctive combination

Combination of each voxel with its weighted neighborhood:

MVc (.) = ∪Vi ∈Φ(Vc)

m′Vi(.)

⇒ The belief for ambiguous data is transfered on hypothesis ω1, ω2.⇒ The centroid updating inside FCM algorithm is done using only certain and

precise data.

- 12/25

Introduction Method Application to PET image fusion Discussion - Conclusion References

Method for reducing uncertainty and imprecision in image processing2 - Information fusion for reducing uncertainty

Conjunctive fusion of neighboring information

Objective: Take advantage of neighborhood to remove uncertainty.

Fusion of each voxel with its discounted neighborhood using Dempster’s rule.

Discounting

Each neighbor Vi of Vc is discounted using:

m′Vi(A) = αimVi (A) ∀A ≠ Ω

m′Vi(Ω) = 1 − αi (1 −mVi (Ω))

⇒ Reduction of the influence of distant voxels before fusing them.

Conjunctive combination (Dempster’s rule).

Combination of each voxel with its discounted neighborhood:

M′

Vc (.) = ⊕Vi ∈Φ(Vc)

MVi (.)

⇒ Removing uncertainty due to noise.

⇒ Highlighting imprecision at the boundary between regions.- 13/25

Introduction Method Application to PET image fusion Discussion - Conclusion References

Method for reducing uncertainty and imprecision in image processing3 - Information fusion for reducing imprecision

Fusion of multiple knowledge

Reducing the imprecision using multiple knowledge.Combination of one source with a learned knowledge using the conjunctive rule.Combination of multiple sources of information using the conjunctive rule.

- 14/25

Introduction Method Application to PET image fusion Discussion - Conclusion References

Method for reducing uncertainty and imprecision in image processingIllustration of the method on two simulated images sources

1 BBA estimation for each image.2 Information fusion for reducing uncertainty.3 Information fusion for reducing imprecision.

Source 1Gaussian blur

& Gaussian noise

Source 2salt and

pepper noise

Ô⇒1

Ô⇒

1

ω1 ω2 ω1, ω2 ω1, ω2 ω2 ω1

Ô⇒2

Ô⇒

2

ω1 ω2 ω1, ω2 ω1, ω2 ω2 ω1

Ô⇒3

Ô⇒

3

ω1 ω2 ω1, ω2- 15/25

Introduction Method Application to PET image fusion Discussion - Conclusion References

Application to PET image fusionFrame of discernment

Frame of discernment

Five exclusive classes Ω = N,M,P,H,FN: Normal (without any high-uptake).M: High-glucose Metabolism.P: High-glucose Metabolism + High cell Proliferation.H: High-glucose Metabolism + Hypoxia.F : Full (High-Métabolisme glucidique + High cell Proliferation + Hypoxia).

Modality FDG FLT FMisoBackground N N,M,H N,M,PHigh-uptake M,P,H,F P,F H,F

(a) PET FDG (b) PET FLT (c) PET FMiso

- 16/25

Introduction Method Application to PET image fusion Discussion - Conclusion References

Application to PET image fusionBBA estimation

BBA estimation of each voxel of each image using FCM and neighborhood fusion:

Ô⇒1

Ô⇒2

FDG N M,P,H,F Ω N M,P,H,F Ω

Ô⇒1

Ô⇒2

FLT N,M,H P,F Ω N,M,H P,F Ω

Ô⇒1

Ô⇒2

FMiso N,M,P H,F Ω N,M,P H,F Ω

Background Uptake Background Uptake

- 17/25

Introduction Method Application to PET image fusion Discussion - Conclusion References

Application to PET image fusionFusion with an a priori knowledge

Problem in PET imaging

Low spatial resolution.

⇒ Partial Volume Effect:Spill over effect: contamination of neighboring structures.Spill out effect: underestimation of the tracer concentration in small structures.

⇒ Fuzzy transition between regions which is particularly important for small structuresand for low contrasts.

Fusion of each voxel with an a priori knowledge

Objective: Take advantage of an a priori knowledge, learned from phantom data,to remove a part of imprecision.

- 18/25

Introduction Method Application to PET image fusion Discussion - Conclusion References

Application to PET image fusionFusion with an a priori knowledge

Learning

Estimation of PVE from phantom data.

Learning of a function, β(V ,C) ∈ [0,1], varying according to both volume andcontrast of spheres and corresponding to the part of imprecision.

PET phantom images(several contrasts).

- 19/25

Introduction Method Application to PET image fusion Discussion - Conclusion References

Application to PET image fusionFusion with an a priori knowledge

Inclusion of a priori knowledge

Determination of a value β(V ,C) by measuring the volume corresponding toHigh Uptake (HU) and the contrast.

Conversion of β(V ,C) in a simple mass function:

mext(HU) = β(V ,C)mext(Ω) = 1 − β(V ,C)

Fusion of each voxel with mext using Dempster’s rule.

⇒ Reduction of imprecision due to partial volume effect.

- 20/25

Introduction Method Application to PET image fusion Discussion - Conclusion References

Application to PET image fusionFusion with an a priori knowledge

For each modality, application of the fusion with the a priori knowledge:Imprecision reduction:

Ô⇒1,2

Ô⇒3

FDG N M,P,H,F Ω N M,P,H,F Ω

Ô⇒1,2

Ô⇒3

FLT N,M,H P,F Ω N,M,H P,F Ω

Ô⇒1,2

Ô⇒3

FMiso N,M,P H,F Ω N,M,P H,F Ω

Background Uptake Background Uptake

⇒ Possibility to fuse the three PET images.- 21/25

Introduction Method Application to PET image fusion Discussion - Conclusion References

Application to PET image fusionMulti-modal fusion

Multi-modal fusion

Conjunctive combination of information contained in the three modalities.Imprecision reduction.Identification of conflicting regions:

High-Proliferative (P) or Hypoxics (H) regions presenting a low glucoseMetabolism (M).

Conversion of belief masses into plausibility:Parametric image creation presenting regions requiring a radiotherapy treatment:

classical (M).with an increased frequency of sessions of radiation (P).with an increased irradiation dose on hypoxic lesions (H).with an increase of both frequency of sessions and irradiation dose (F).

- 22/25

Introduction Method Application to PET image fusion Discussion - Conclusion References

Application to PET image fusionMulti-modal fusion

Parametric images obtained helping the radiation-oncologist in the radiotherapytreatment planning.

FDGÔ⇒1,2,3

Ô⇒Pl(M) Pl(H)

FLTÔ⇒1,2,3

Ô⇒multi-modale

Fusion Pl(P) Pl(F)

FMisoÔ⇒1,2,3 Ô⇒

Conflict maxPl

- 23/25

Introduction Method Application to PET image fusion Discussion - Conclusion References

Discussion - Conclusion

Discussion

Disjunctive combination (inside FCM algorithm) followed by a conjunctivecombination of voxels for each modality:

Can be applied whatever the distribution of initial belief masses is.Imprecision due to partial volume effect modeling and noise reduction.

Conjunctive combination of multiple sources (multiple images or learnedknowledges):

Imprecision due to partial volume effect reduction.

Conclusion

Using Belief Function theory and spatial information for taking into account bothimprecision and uncertainty.

It offers a great help for radiation oncologist in order to segment lesions frommulti-tracer functional images (FDG, FLT, FMISO).

Future works

Test our method on a larger database to assess its robustness.

Test our method on other images to confirm its genericity.

- 24/25

Introduction Method Application to PET image fusion Discussion - Conclusion References

Bibliography I

J. C. Bezdek. Pattern Recognition with Fuzzy Objective Function Algorithms. Kluwer AcademicPublishers, Norwell, MA, USA, 1981. ISBN 0306406713.

A. S. Capelle-Laize, O. Colot, and C. Femandez-Maloigne. Evidential segmentation scheme ofmulti-echo mr images for the detection of brain tumors using neighborhood information. InformationFusion, 5(3):203–216, 2004.

J. Chang, B. Wen, P. Kazanzides, P. Zanzonico, R. D. Finn, G. Fichtinger, and C. C. Ling. A roboticsystem for 18F-FMISO PET-guided intratumoral pO2 measurements. Med Phys, 36:5301–5309, Nov2009.

A. Dempster. Upper and lower probabilities induced by a multivalued mapping. Annals of MathematicalStatistics, 38(2):325–339, 1967.

G. Shafer. A mathematical theory of evidence. Princeton university press, 1976.

P. Smets. The combination of evidence in the transferable belief model. IEEE Trans. Pattern Anal.Mach. Intell., 12(5):447–458, 1990.

W. Yang, Y. Zhang, Z. Fu, J. Yu, X. Sun, D. Mu, and A. Han. Imaging of proliferation with 18F-FLTPET/CT versus 18F-FDG PET/CT in non-small-cell lung cancer. Eur. J. Nucl. Med. Mol. Imaging,37:1291–1299, Jul 2010.

P. Zhang, I. Gardin, and P. Vannoorenberghe. Information fusion using evidence theory for segmentationof medical images. In International Colloquium on Information Fusion, volume 1, pages 265–272,2007.

- 25/25