Precise Image-Based Measurements through Irregular...
Transcript of Precise Image-Based Measurements through Irregular...
ACTAUNIVERSITATIS
UPSALIENSISUPPSALA
2019
Digital Comprehensive Summaries of Uppsala Dissertationsfrom the Faculty of Science and Technology 1869
Precise Image-BasedMeasurements through IrregularSampling
TEO ASPLUND
ISSN 1651-6214ISBN 978-91-513-0783-1urn:nbn:se:uu:diva-395205
Dissertation presented at Uppsala University to be publicly examined in Room 2446, ITC,Lägerhyddsvägen 2, Uppsala, Friday, 6 December 2019 at 13:00 for the degree of Doctorof Philosophy. The examination will be conducted in English. Faculty examiner: ProfessorHugues Talbot (Université Paris-Saclay).
AbstractAsplund, T. 2019. Precise Image-Based Measurements through Irregular Sampling.(Noggranna bildbaserade mätningar via irreguljär sampling). Digital ComprehensiveSummaries of Uppsala Dissertations from the Faculty of Science and Technology 1869. 63 pp.Uppsala: Acta Universitatis Upsaliensis. ISBN 978-91-513-0783-1.
Mathematical morphology is a theory that is applicable broadly in signal processing, but inthis thesis we focus mainly on image data. Fundamental concepts of morphology include thestructuring element and the four operators: dilation, erosion, closing, and opening. One wayof thinking about the role of the structuring element is as a probe, which traverses the signal(e.g. the image) systematically and inspects how well it "fits" in a certain sense that dependson the operator.
Although morphology is defined in the discrete as well as in the continuous domain, oftenonly the discrete case is considered in practice. However, commonly digital images are arepresentation of continuous reality and thus it is of interest to maintain a correspondencebetween mathematical morphology operating in the discrete and in the continuous domain.Therefore, much of this thesis investigates how to better approximate continuous morphologyin the discrete domain. We present a number of issues relating to this goal when applyingmorphology in the regular, discrete case, and show that allowing for irregularly sampled signalscan improve this approximation, since moving to irregularly sampled signals frees us fromconstraints (namely those imposed by the sampling lattice) that harm the correspondence inthe regular case. The thesis develops a framework for applying morphology in the irregularcase, using a wide range of structuring elements, including non-flat structuring elements (orstructuring functions) and adaptive morphology. This proposed framework is then shown tobetter approximate continuous morphology than its regular, discrete counterpart.
Additionally, the thesis contains work dealing with regularly sampled images using regular,discrete morphology and weighting to improve results. However, these cases can be interpretedas specific instances of irregularly sampled signals, thus naturally connecting them to theoverarching theme of irregular sampling, precise measurements, and mathematical morphology.
Keywords: image analysis, image processing, mathematical morphology, irregular sampling,adaptive morphology, missing samples, continuous morphology, path opening.
Teo Asplund, Department of Information Technology, Division of Visual Information andInteraction, Box 337, Uppsala University, SE-751 05 Uppsala, Sweden. Department ofInformation Technology, Computerized Image Analysis and Human-Computer Interaction,Box 337, Uppsala University, SE-75105 Uppsala, Sweden.
© Teo Asplund 2019
ISSN 1651-6214ISBN 978-91-513-0783-1urn:nbn:se:uu:diva-395205 (http://urn.kb.se/resolve?urn=urn:nbn:se:uu:diva-395205)
Dedicated to my familyand my friends
List of papers
This thesis is based on the following papers, which are referred to in the text
by their Roman numerals.
I Asplund, T., Luengo Hendriks, C. L., Thurley, M., and Strand, R.
(2017) Mathematical Morphology on Irregularly Sampled Data in OneDimension, In Mathematical Morphology - Theory and Applications
2.1, pp. 1-24.
II Asplund, T., Luengo Hendriks, C. L., Thurley, M., and Strand, R.
(2017) Mathematical Morphology on Irregularly Sampled Signals, In
Computer Vision - ACCV 2016 Workshops. LNCS, vol 10117,
pp. 506-520
III Asplund, T., Serna, A., Marcotegui, B., Strand, R., and Luengo
Hendriks, C. L. (2019) Mathematical Morphology on IrregularlySampled Data Applied to Segmentation of 3D Point Clouds of UrbanScenes, In Mathematical Morphology and Its Applications to Signal
and Image Processing. ISMM 2019. LNCS, vol 11564, pp. 375-387
IV Asplund, T., Luengo Hendriks, C. L., Thurley, M., and Strand, R.
(2019) Adaptive Mathematical Morphology on Irregularly SampledSignals in Two Dimensions, (submitted)
V Asplund, T., Luengo Hendriks, C. L., Thurley, M., and Strand, R.
(2019) Estimating the Gradient for Images with Missing Samples UsingElliptical Structuring Elements, (submitted)
VI Asplund, T., and Luengo Hendriks, C. L. (2016) A Faster, UnbiasedPath Opening by Upper Skeletonization and Weighted AdjacencyGraphs, In IEEE Transactions on Image Processing 25.12,
pp. 5589-5600.
©2016 IEEE. Reprinted, with permission from the authors.
Reprints were made with permission from the publishers.
Related work
During the research for this thesis the author has also contributed to the
following publications.
I Asplund, T., Luengo Hendriks, C. L., Thurley, M., and Strand, R.
(2016) A New Approach to Mathematical Morphology on OneDimensional Sampled Signals, In Proceedings of 23rd International
Conference on Pattern Recognition, IEEE pp. 3904-3909.
II Asplund, T., Luengo Hendriks, C. L., Thurley, M., and Strand, R.
(2017) Approximating Continuous One-Dimensional Morphology byIrregular Sampling, In Proceedings of the Swedish Symposium on
Image Analysis (SSBA 2017)
III Asplund, T., Bengtsson Bernander, K., and Breznik, E. (2019) CNNson Graphs: A New Pooling Approach and Similarities to MathematicalMorphology, In Proceedings of the Swedish Symposium on Deep
Learning (SSDL 2019)
Contents
1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
1.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
1.2 Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
1.3 Thesis Outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
2 Mathematical Morphology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
2.1 Basic Operators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
2.1.1 Dilation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
2.1.2 Erosion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
2.1.3 Hit-and-Miss Transform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
2.1.4 Closing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
2.1.5 Opening . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
2.2 Umbra and Top . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
2.3 Continuous Morphology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
2.4 Skeletonization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
2.4.1 Desirable Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
2.4.2 Digital Skeleton . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
2.5 Geodesic Morphology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
2.5.1 Reconstruction by Dilation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
2.6 Granulometry . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
2.7 Morphological Gradient and Laplacian . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
2.8 Tophat . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
2.9 Adaptive Morphology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
2.10 Path Opening . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
2.10.1 Constrained Path Opening . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
2.10.2 Approximating Path Opening . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
2.11 Estimating the Gradient for Images with Missing Pixels using
Elliptical Structuring Elements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
2.12 Continuous and Discrete Morphology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
2.12.1 Morphological Dilations as Partial Differential
Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
3 Mathematical Morphology on Irregularly Sampled Signals . . . . . . . . . . . . . . . . . . . 35
3.1 Approximating Continuous Morphology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
3.1.1 Morphological Operators Introduce Higher Frequency
Content . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
3.1.2 Effect of Shifting the Sampling Grid . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
3.1.3 Effect of the Sampling Grid on the Structuring
Element . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
3.2 Point Clouds . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
3.3 Irregular Morphology in 1D . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
3.4 Irregular Morphology in 2D . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
3.5 Adaptive Morphology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
3.5.1 Elliptical Adaptive Structuring Elements . . . . . . . . . . . . . . . . . . . . . . . 43
3.5.2 Adapting the Elliptical Adaptive Structuring Elements
to Irregularly Sampled Signals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
4 Conclusions and Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
4.1 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
4.2 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
5 Brief Summaries of the Included Papers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
6 Summary in Swedish . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
7 Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
List of Abbreviations
1D One dimension, one-dimensional
2D Two dimensions, two-dimensional
nD n dimensions, n-dimensional
LST Local structure tensor
MM Mathematical morphology
NDC Normalized differential convolution
PO Path opening
PPO Parsimonious path opening
SE Structuring element
USPO Upper skeleton path opening
1. Introduction
1.1 Motivation
Mathematical morphology (MM) is a widely used approach for processing
signals (e.g. images). Linear filters are not affected by the sampling grid
according to Nyquist-Shannon sampling theory [47, 57]. However, for mor-
phological filters, which are non-linear, this is not the case. As a result, for
example, an object being imaged will yield different results depending on the
exact location of said object within the field of view of the camera. This behav-
ior is generally undesirable, since one is usually interested in properties of the
imaged object itself, not the image. Therefore we wish to address this issue by
reducing the dependence on a sampling grid. By implementing morphology
on irregularly sampled signals, we take a step in this direction. This leads to
increased precision, allowing for sub-pixel measurements.
Let us consider the connection between the discrete and the continuous
domain. Given a band-limited image, for example an image projected through
a system of lenses [29], which has then been sampled correctly (i.e. at a
sampling rate more than twice the highest frequency according to the Nyquist-
Shannon threshold [47, 57], although in practice the sampled signal has a finite
extent, which breaks the bandlimitedness), such as the image captured by the
camera in the example above, one may reconstruct the continuous signal from
the discrete one. However, applying a non-linear filter, such as a morphological
filter, will generally introduce higher frequency content into the signal, thus
breaking the correspondence between the continuous, filtered signal and the
filtered discrete signal, since the filtered signal cannot be sampled correctly on
the sampling grid. This motivates the move to irregular sampling.
Additionally, allowing for irregularly sampled signals makes it possible to
deal with a large amount of data of interest which is irregularly sampled to
begin with, for example data obtained from range cameras. Typically, such data
is dealt with by resampling onto a regular grid when applying morphology.
This resampling can result in undesirable artifacts that negatively affect results.
The work presented in this thesis applies to irregularly sampled data, so this
resampling step is avoided.
13
1.2 Objectives
The aim of this thesis is to improve the precision of image-based measurements
through irregular sampling, specifically by introducing a framework for dealing
with mathematical morphology on irregularly sampled signals as part of the
processing. In paper VI both the input and the output signals are regularly
sampled, however in an intermediate processing step a graph representation is
constructed whose nodes represent samples at certain (not regular) positions
in the original image and edges indicate neighbor relationships.
In papers I, and II, the focus is, mainly, on regularly sampled input signals,
but with irregularly sampled output (although the methods developed are also
applicable to irregularly sampled input as is shown) while paper V deals with
the special case where the input is irregularly sampled, but all samples are
taken at grid positions (similarly to paper VI). Paper IV extends the work of
paper II dealing with regularly and irregularly sampled input and allows for a
wider set of filtering approaches than the previous papers in this thesis.
Finally, paper III, mainly deals with irregularly sampled input and output,
namely point clouds of urban scenes. What follows is a list of objectives:
• Develop a framework for mathematical morphology on irregularly sam-
pled signals.
• Develop better approximations of continuous morphology in the discrete
domain via irregular sampling or weighted measurements.
• Apply the framework for MM on irregularly sampled signals to irregu-
larly sampled input data (e.g. 3D point clouds).
• Consider the special case of irregularly sampled signals where the avail-
able samples fall on grid points, but there are missing samples.
• Develop adaptive and non-flat morphology for the irregular case.
1.3 Thesis Outline
This thesis consists of seven chapters including the introduction, the summary
in Swedish, and acknowledgments. The second chapter (i.e. following the in-
troduction) introduces mathematical morphology, focusing mostly on discrete,
regularly sampled, signals (e.g. grayscale images). The third chapter takes
these concepts, generalizes them to irregularly sampled signals, and describes
why this is useful, in particular when approximating continuous morphology
in the discrete domain, but also in cases where the input signal is irregularly
sampled. The fourth chapter presents the main contributions of the thesis and
indicates some future directions of interest. The fifth chapter gives a brief
summary of each paper included in the thesis. The sixth chapter presents a
summary of the thesis in Swedish, and the final chapter, i.e. chapter seven, is
dedicated to acknowledgments.
14
2. Mathematical Morphology
Here I give a brief introduction to mathematical morphology to give a basis
for the following chapters (see, for example the book edited by Talbot and
Najman [46], for further reading). Introduced by Matheron and Serra in the
1960s [41, 56], mathematical morphology is widely used in image analysis,
often used as a pre- or postprocessing step. Morphological operators are
conceptually similar to convolutions, but unlike convolutions, they are non-
linear.
Initially applicable to binary images (seen as sets of foreground points)
using set-theoretical concepts, morphology has been extended to several other
domains, including grayscale [56, 64] and color images [12, 67, 76], graphs [73,
19, 45], and point clouds [13, 37], among others. The main structure underlying
morphology is the complete lattice [5].
Definition 1. A complete lattice is a pair (L,�), a set, L, equipped with apartial order � where every subset X ⊂ L has a supremum and an infimumin L.
Some typical examples of complete lattices include binary sets with set
inclusion ⊂ as the partial order, where the supremum and infimum are the
union and intersection, or grayscale images with pixel-wise comparison of
intensities as the partial order and the pixel-wise supremum and infimum. For
details see the papers by Heijmans and Ronse [25, 53].
In this thesis we will mostly be concerned with signals that are defined by
sets of tuples of three values (x, y, z) (or functions z(x, y)). E.g. grayscale
images, where x, y ∈ Z indicate pixel position and z ∈ R̄, where R̄ is the
real numbers equipped with −∞ and +∞, indicates graylevel, or irregularly
sampled signals with position x, y ∈ R and value/height z ∈ R̄. In the
case of grayscale images, a partial order can easily be defined by pointwise
comparison of images, using the natural ordering of pixels. However, for
irregularly sampled signals, pointwise comparisons is not generally possible,
since the positions of samples are not restricted to a grid, meaning there is no
obvious relationship between pairs of points in two signals. In paper I, this
issue is dealt with by associating a continuous signal, with infinite support,
to each sampled signal (via a kind of interpolation) and then comparing these
continuous signals.
The basic operations of mathematical morphology are dilations, erosions,closings, and openings [56]. Commonly a set B ⊂ E, called a structuring
15
element (SE), is used to probe the data, where E is some set, e.g. R2 or Z2.
In some cases a structuring function B : E → R̄ is used instead. In the first
case, the SE is called flat, since it is a 2D image without height. In the case
of a structuring function, the probe may have a height that varies, meaning
the structuring element is non-flat. Note that the flat case is a special case of
non-flat morphology, namely using the structuring function
B(x) ={0, if x ∈ B
−∞, otherwise(2.1)
The structuring element (or structuring function) is used to probe the data
at each point, yielding a new signal where the output is based, in some sense,
on how well the probe (the SE) fits. Thus, depending on structures of interest,
one may choose an appropriately shaped structuring element.
More specifically, a morphological transformation depends on the signal Xand the structuring element B, where B is a point set where the positions are
given relative to some origin (note that the origin is not necessarily contained
within the SE). Conceptually, applying the transformation to X is done by
systematically sliding B across the entire signal outputting a new value at
every position based on some relationship between X and the translated SE,
B.
In this thesis, the terms “structuring element” and “structuring function” are
used somewhat interchangeably. However, when using the term structuring
function, the context is usually non-flat morphology (but as pointed out above,
flat morphology is just a special case, see 2.1). Conversely, when using the
term structuring element, the context is normally flat morphology, although
this distinction may sometimes get muddied when discussing generalizing from
flat to non-flat morphology.
2.1 Basic Operators
There are four fundamental morphological operators called dilations, erosions,openings, and closings. In this section we briefly describe these operators.
2.1.1 Dilation
The dilation of X by B is often notated as X ⊕B and is defined as:
[X ⊕B](x) =∨b∈B
X(x− b) (2.2)
That is, conceptually, placing B at x, consider all points in X “hit” by B and
take the supremal value. In the more general case of non-flat morphology,
16
where B is a structuring function, dilation is:
[X ⊕ B](x) =∨b∈E
(X(x− b) + B(b)
)(2.3)
Meaning the supremum is weighted based on B.
2.1.2 Erosion
The (non-flat) erosion of X by B is notated X � B and is defined as:
[X � B](x) =∧b∈E
(X(x+ b)− B(b)
)(2.4)
Note the differences compared to dilation, namely, infimum instead of supre-
mum, and offset x by b instead of −b. As a result, we have the useful property
that dilation and erosion are dual:
X ⊕ B = [X� � B̆]� (2.5)
Where � denotes the complement and B̆(x) = B(−x) is the reflected structur-
ing function. In other words, dilation of the foreground is the complement of
the erosion of the background with the reflected structuring function.
2.1.3 Hit-and-Miss Transform
The hit-and-miss transform (also called hit-or-miss transform), is used to
match patterns in an image. In this case, a compound structuring element
B = (B1, B2) is used. The hit-and-miss transform of X by B, denoted
X ⊗B can be defined in terms of ⊕ and �:
X ⊗B = (X �B1) \ (X ⊕B2) (2.6)
Here B1 describes a foreground pattern and B2 a background pattern. Nor-
mally the hit-and-miss transform is applied to binary images using binary (flat)
structuring elements.
2.1.4 Closing
The closing of X by B, denoted X • B can be defined in terms of ⊕ and �:
X • B = (X ⊕ B)� B (2.7)
Applying a closing can be thought of as sliding the structuring element along
the underside of the signal interpreted as a surface, as snugly as possible, where
the value at each point in the output is the height of the SE at that point. This
can be used to break apart connections in an image or smooth out contours, for
example.
17
Figure 2.1. From left to right: Original image, erosion, dilation, opening, and closing
using a (flat) disk shaped structuring element.
2.1.5 Opening
Similarly to the relationship between dilations and erosions, closings and open-
ings are dual. The opening of X by B is denoted X ◦ B where
X ◦ B = (X � B)⊕ B (2.8)
and
X • B = [X� ◦ B̆]� (2.9)
Figure 2.1 shows examples of these basic operators (except for the hit-and-
miss transform).
2.2 Umbra and TopOne way of interpreting grayscale morphology, is by recasting the problem
as a case of binary morphology by transforming the grayscale function into a
binary one. For a given function X from R2 → R̄, its umbra [64, 52], U(X)is defined as:
U(X) = {(x, y) ∈ R2 × R̄ | y ≤ X(x)} (2.10)
Associated with every umbra V is its top T :
T (V )(x) =∨{y | (x, y) ∈ V } (2.11)
Figure 2.2 illustrates this concept. Note that T (U(X)) = X . Using the umbra
approach we have that
X ⊕ B = T [U(X)⊕ U(B)] (2.12)
X � B = T [U(X)� U(B)] (2.13)
where the dilation and erosion on the left-hand side is the gray-scale version,
while on the right they are the binary versions, also known as Minkowski
addition and subtraction [42, 22]. This interpretation of grayscale morphology
as a form of binary morphology can be a useful way of looking at morphological
operations, and helped inspire, to some extent, the approach proposed for MM
on irregularly sampled signals in papers I, II, III, and IV. There are some
technicalities that are not discussed here, particularly regarding continuous
graylevels. See the paper by Ronse for details [52].
18
(a)
(b)
Figure 2.2. (a) A function and its umbra. (b) An umbra and its top.
2.3 Continuous Morphology
Usually, morphology as applied in image processing is only considered in the
discrete domain. For practical reasons this is sensible, since one normally deals
with regularly sampled, digital images. However, there is nothing preventing us
from choosing E = R2, if we so desire. If we are interested in the continuous
signal that may be underlying the sampled signal (e.g. the signal from the
real world that has been sampled in the digital image), we should consider
the correspondence between the sampled signal and the continuous one. In
other words, if—as is usually the case—we are interested in the imaged objectitself, not the image data, we should take care that the sampled signal is a good
representation of the object.
For a properly sampled, band-limited signal, there is a correspondence
between the continuous signal and its discrete representation according to the
Nyquist-Shannon sampling theorem [47, 57]. Here, “properly sampled” means
that the sampling rate is more than twice the highest frequency.
For linear filters, such as convolutions, this correspondence is preserved
even after their application. However, in the case of morphology, the filters are
not linear, (because of the supremum/infimum). In practice this means higher
frequency content can be introduced into the transformed signal, meaning
the correspondence between discrete and continuous domain is broken (see
Fig 2.3).
19
SE
Figure 2.3. Blue: 1D, continuous sinusoid signal. Brown: Continuous dilation of the
signal using the structuring element shown in the bottom right. The sine-wave can
be represented using slightly more than two samples per period. However, the dilated
signal introduces higher frequency content into the signal (note the cusps generated at
the valleys of the sine).
Figure 2.4. Propagation of a wave from the border of X . The skeleton S(X) will be
the “y” shape that is beginning to take form in the center of the object.
2.4 Skeletonization
The purpose of skeletonization is to simplify an object resulting in a thin,
abstract representation, called a skeleton. This skeleton should preserve the
topological and geometric properties of the original object. The idea was first
introduced in 1967 by Blum [6] for binary images. Assume a point setX ⊂ R2
from the border of which we propagate a wave inwards at constant speed from
each point. The skeleton S(X) is the set of points where two or more waves
meet. See Fig. 2.4.
A more formal description of a skeletonization follows: Given a point set
X ⊂ E, let
B(x, r) = {y ∈ E | d(x, y) ≤ r} (2.14)
where d(x, y) is the distance between x and y, and x ∈ X is the center of a
ball with radius r ≥ 0.
20
Definition 2. A ball B(x, r) ⊂ X is said to be maximal, if there is no otherball B(x, r′) such that B(x, r) � B(x, r′) and B(x, r′) ⊂ X .
The skeleton by maximal balls is the set of pointsS(X) containing all centers
of maximal balls inside X . This definition, though quite intuitive, does not
guarantee that the connectivity of the original set is preserved (e.g. the skeleton
by maximal balls of two balls touching at a point is the set containing the two,
disjoint center points of the balls), which is undesirable. More precisely, the
skeleton by maximal balls is not a homotopic skeleton (i.e. does not preserve
homotopy).
Informally, two binary images X and Y are homotopic if one can be de-
formed, continuously, into the other.
2.4.1 Desirable Properties
The following four properties are desirable for a skeleton [48]:
1. Subset of the original object
2. Thin
3. Allows reconstruction of the original object
4. Topologically equivalent to the original object
In general, all desirable properties cannot simultaneously be satisfied.
2.4.2 Digital Skeleton
In addition to not being homotopic, the skeleton by maximal balls does not
guarantee a thin (i.e. one pixel wide) skeleton in the discrete case. Instead,
commonly, a sequential thinning approach is used in the discrete case. Such
an approach may guarantee thin, homotopic skeletons.
ThinningThe thinning of an image X using a composite SE B = (B1, B2), X B is
defined as
X B = X \X ⊗B (2.15)
Skeletonization by thinning is performed by iterating thinnings using a
sequence of composite SEs. Commonly used sequences are the Golay alpha-bet [21, 56] on a given raster. Iterating these sequential thinnings converges to
some set of points, which is the skeleton by thinning.
Saha et al. [54] present a good overview of different skeletonization algo-
rithms.
21
2.5 Geodesic Morphology
The geodesic distance dX(x, y) is the length of the shortest path between xand y contained within the set X (possibly +∞, if no path exists) [33]. The
geodesic ball BX(x, r) is the subset of the ball with radius r located at x which
is entirely contained within X:
BX(x, r) = {x′ ∈ X | dX(x, x′) ≤ r} (2.16)
This leads to the definition of geodesic dilation [74, 34] of size r of some set
Y , within X , δ(r)X (Y ):
δ(r)X (Y ) =
⋃y∈Y
BX(y, r) (2.17)
2.5.1 Reconstruction by Dilation
Morphological reconstruction [74] is a useful tool to recover the shape of some
set of marked objects. In the binary case, we can obtain the reconstruction by
dilation ρX(Y ) for a marker Y (i.e. a set of points that indicate the objects of
interest) and a mask image X by successive geodesic dilations of Y inside X:
ρX(Y ) = limn→∞ δ
(n)X (Y ) (2.18)
i.e., apply larger and larger geodesic dilations until idempotency is reached.
In the grayscale case, a corresponding operation can be performed by it-
eratively dilating the marker Y (which is in this case a grayscale image) and
taking the infimum of the dilated image and the mask image X (which is
also a grayscale image). That is, an elementary grayscale geodesic dilation
δ̂(1)X (Y ) = (Y ⊕B)∧X , whereB is the unit ball, is used to define the grayscale
geodesic dilation of size n:
δ̂(n)X (Y ) = δ̂
(1)X ◦ δ̂(1)X · · · ◦ δ̂(1)X (Y ) (2.19)
meaning δ̂(n)X is the composition of n elementary grayscale geodesic dilations
δ̂(1)X . Then, analogously, grayscale reconstruction is computed by successive
grayscale geodesic dilations until reaching idempotency.
The result being that peaks of X marked by Y are extracted by the recon-
struction (see Fig. 2.5).
2.6 Granulometry
Granulometry [61] is a tool of mathematical morphology, which can be used
to find size distributions of objects in an image X , by repeatedly applying
22
Figure 2.5. The reconstruction (dashed blue) of the marker (red) using a mask (green).
The marker indicates some peaks of interest that are extracted by the reconstuction by
dilation.
openings (or closings) of increasing size and summing up the pixel values for
each result. That is, for some family of SEs {Bi}i∈{0,...,N}, a corresponding
set of openings is computed:
γi(X) = X ◦Bi (2.20)
giving the granulometric function
V (X, i) = |γi(X)| (2.21)
where | · | denotes some kind of measure, e.g., number of pixels, volume, or
sum of pixel values. From this, one may compute the size distribution
P (X, i) = V (X, i)− V (X, i+ 1) (2.22)
The shape of the structuring elementsBi should be chosen based on the objects
of interest. For example, in paper VI path openings are used to estimate length
distributions of line-like structures (i.e., paths).
Figure 2.6 shows an example of applying granulometry (with a disk-shaped
SE) to an image containing coins of different sizes.
A benefit of this approach is that there is no need to segment the objects of
interest before measuring sizes.
2.7 Morphological Gradient and Laplacian
The morphological gradient [3, 51] G(X) of an image X is normally defined
as
G(X) = (X ⊕B)− (X �B) (2.23)
23
55 60 65 70Radius
0
2
4
6
8
Inte
nsity
diffe
renc
e 105 Size distribution
Figure 2.6. Left: A photo of some coins. This image is preprocessed slightly by
inverting it (so that the coins are brighter than the background) and applying a small
closing, to remove some darker details inside the coins. Right: Part of the size
distribution yielded by the granulometric function on the preprocessed coin image.
The three peaks around 56, 61, and 68 correspond to the three different coin sizes in
the image.
where B is a unit ball SE. I.e., the morphological gradient of a signal X is
the difference between its dilation and its erosion with a unit ball. There are
a number of variations on this theme, for example the external and internal
gradient:
Ge(X) = (X ⊕B)−X (external) (2.24)
Gi(X) = X − (X �B) (internal) (2.25)
These operators yield positive values at each pixel in an image X and can be
used to estimate the gradient magnitude of the signal. A notable application of
the internal and external gradients is their use in computing the morphological
Laplacian,Δm [70] which is given by the difference between the external and
internal gradient:
Δm(X) = Ge(X)−Gi(X) (2.26)
To yield directional estimates, one may use non-isotropic structuring ele-
ments that are oriented appropriately [3] as is done in Paper V for images with
missing samples.
Figure 2.7 shows examples of the morphological gradient and Laplacian.
2.8 TophatThe tophat, TB
h , is a morphological operator that enhances parts of an image
where the SE,B does not fit. This can be used to enhance objects of interest by
suppressing the background. The tophat is the difference between the original
image and its opening:
TBh (X) = X −X ◦B (2.27)
24
Figure 2.7. Left: Morphological gradient. Right: Morphological Laplacian. The SE
used is a disk of radius 2. Note the diamond shaped artifacts (e.g. the bright diamond
in the top-left or the one near the neck of the man) in both images. These are a result of
the discretization of the disk SE and is one of the issues with approximating continuous
morphology in the regular, discrete domain.
Figure 2.8 shows an example where the tophat is used in order to enhance
linear features of an image.
2.9 Adaptive Morphology
In paper IV the proposed morphology on irregularly sampled signals is gen-
eralized in a number of ways, including allowing for adaptive morphology,
meaning the SE is no longer rigid, and may instead adapt based on position or
image content, for example. In some cases any particular structuring element
is not suited for the entire image. In such cases, adaptive morphology can be
used, shaping the SE appropriately. Because of the variety of different image
attributes that could be chosen to adapt the SE, there are many papers dealing
with adaptive morphology. This section contains a brief overview. See the
surveys by Ćurić et al. [20] and Maragos and Vachier [40] for further reading.
An early example of adaptive morphology was presented by Beucher etal. [4] who used a structuring element dependent on position to deal with
perspective. Another early example is presented in a 1993 paper by Verly
and Delanoy [71], developing adaptive MM in the context of range images,
adjusting the SE based on the distance, since a large object far away may be
similar in size to a small object close to the camera. Other early examples
include the work by Cheng and Venetsanopoulos [16, 17], and Chen et al. [15].
More recently, so called morphological amoebas were introduced [35],
adapting the shape of structuring elements based on the content of the im-
25
Figure 2.8. Left: Input image (one of the Brodatz texture images [10]), middle:
opening of the image using a small disk shaped SE, right: the difference between the
original image and its opening, i.e. the tophat. Here the tophat is used to enhance the
linear features of the input image.
age, trying to keep a high homogeneity of pixels covered by the SE at any
given point, by considering the local similarity between neighboring pixels.
Another relatively recent method was presented by Landström and Thur-
ley [32]. The main idea being to shape elliptical structuring elements based
on the eigenvectors of the local structure tensor (LST) [28] which along with
the eigenvalues give information about the orientation (and its strength) of the
data. In a later paper this approach is extended by using non-flat quadratic
structuring functions [31]. A more thorough description of this approach [32]
can be found in Section 3.5.1, where an adaptation of their approach to the
case of irregularly sampled signals is presented. For further details see also
paper IV.
2.10 Path OpeningThe path opening (PO) is a morphological filter that addresses the problem of
enhancing long, thin structures [11]. This is of interest in a number of applica-
tions, such as detecting rivers or roads in remote sensing applications [66, 26],
vessel segmentation [77, 58], identifying tracks of dust devils on Mars [63],
etc. A straightforward approach to the problem of detecting such structures
would be to apply openings with line segments at several different orientations
and taking the supremum [62]. This approach is slow, however, and though it
may work well in cases where the features of interest consist of straight line
segments, if instead the features are curved, this approach gives unsatisfac-
tory results. The path opening was first proposed in 2000 by Buckley and
Talbot [11] and deals with this problem.
Conceptually the path opening uses many flexible line-like structuring ele-
ments of a given length, L to open an image, followed by taking the supremum
of these openings by all these structuring elements. The number of structuring
elements, in a naive implementation, grows exponentially with L, however
26
Figure 2.9. Adjacency graphs used by the original path opening. (a) South-north
adjacencies, (b) southwest-northeast, (c) west-east, (d) northwest-southeast. For a
given adjacency graph, an allowed path is built by starting at some node, following
one of the arrows, and repeating L times, where L is the length of the opening. (See
paper VI, © 2016 IEEE)
in practice it can be computed in linear time with respect to L [24]. The
structuring elements are defined by a set of adjacency graphs which constrain
paths to follow some general direction. Commonly, the adjacency graphs used
describe 90◦ cones whose main axis point in one of four directions namely,
south-north, southwest-northeast, west-east, or northwest-southeast. These
adjacency graphs are illustrated in Fig. 2.9.
2.10.1 Constrained Path Opening
The path opening has a problem of overestimating lengths of line-like objects
that have some thickness, stemming from the possibility for paths to “zig-
zag” inside the object. The constrained path opening [38] is a variant of the
path opening that attempts to deal with this issue by restricting the possible
paths such that this behavior is minimized, thus yielding less biased length
measurements.
2.10.2 Approximating Path Opening
In order to speed up the computation of the path opening, approximate path
openings have been proposed. The main idea is to preselect a limited number
of paths through the image followed by applying an opening only to these
preselected paths and finally reconstructing the result using the opened image
as a marker and the original image as a mask (see Sec. 2.5.1).
Parsimonious Path OpeningThe parsimonious path opening (PPO) [44] is an approximate path opening
which has a time complexity independent of path length L. This is achieved
by preselecting 1D paths from one border of the image to the opposing border.
In other words, there is one path selected per border pixel. After this, the
27
preselected paths are opened using a 1D opening, which can be done in O(1)per pixel [43], i.e., independent of the length of the opening. In this way, the
time complexity of the parsimonious path opening is independent of L and
proportional to the number of pixels in the image.
A problem with PPO is the fact that the preselection of paths leaves blind
regions. These blind regions are affected by the content of the image. The
preselection strategy pulls paths towards bright structures, which may lead to
particular structures of interest in the middle of the image being occluded by
bright structures toward an edge of the image, since the selected paths start at
the edge of the image and are constrained within a 90◦ cone.
Another issue is the fact that the length measurements of PPO are biased
(this is also the case for the regular path openings, in fact even more so), where
paths that are mostly horizontal, vertical, or diagonal are correctly measured,
however the lengths of paths that generally follow an angle that is not a multiple
of π/4 tend to be overestimated.
Proposed path opening: Faster, Unbiased Path Opening bySkeletonization and Weighted GraphsIn paper VI, a new approximate path opening algorithm is presented, the upperskeleton path opening (USPO). This paper takes inspiration from the PPO,
preselecting paths, applying an opening of length L to the preselected paths,
followed by a reconstruction. However, the proposed approach addresses the
two problems of the PPO mentioned above. Firstly, the preselected paths do
not suffer from the problem of occlusion. This is achieved by selecting paths
by skeletonizing the original image using the so called upper skeleton [72],
yielding a grayscale skeleton which is taken as the selected paths. These paths
follow the bright ridges of the image, thus selecting the most important paths,
since we are interested in bright, long, thin structures. A problem with this
approach is that it precludes us from applying the fast 1D opening [43] used
by the PPO, since the skeleton is not easily broken up into 1D paths. Instead,
we propose a way of constructing a sparse graph out of the skeleton, upon
which a graph based path opening is applied. Empirically, this yields at least
comparable speeds to the PPO.
The second main contribution of paper VI deals with the biased length
measurements of the PPO (and the regular PO for that matter). There are three
factors that contribute to the reduction in measurement bias, namely:
• The skeleton is thin. This prevents paths from zig-zagging inside elon-
gated structures, by essentially following the main axis of objects. This
benefit is shared with PPO.
• The adjacency graphs are weighted using weights that minimize the
relative error for digital lines of arbitrary orientation [50]. The regular
path opening formulation measures diagonal and horizontal/vertical steps
as equal in length (both have a weight of one), however, realistically, a
diagonal step is longer than a horizontal/vertical one. The PPO weights
28
Figure 2.10. Top: Input image, from an electron microscope, of a DNA molecule [24].
(a) Traditional path opening, (b) constrained path opening, (c) parsimonious path
opening, and (d) upper skeleton path opening, all using the same length threshold
L = 50. (See paper VI, © 2016 IEEE)
diagonal steps as√2 and horizontal/vertical are weighted as 1. This
yields accurate measurements for angles that are multiples of π/4, but
will generally overestimate lengths for other angles and do not minimize
the relative error for arbitrary orientations. The weights proposed in
paper VI greatly improve the results of length measurements.
• The USPO does not suffer from the problem of occlusions. The skele-
tonization is not adversely affected by the presence of bright structures
closer to the edges of the image, unlike the path selection approach taken
by the PPO.
Figure 2.10 shows the traditional path opening, as well as the constrained path
opening and the approximate variants described above applied to an image to
enhance a thin, sinuous structure.
2.11 Estimating the Gradient for Images with MissingPixels using Elliptical Structuring Elements
In paper V a way of estimating the gradient for image with missing pixels
is presented. This is useful in a number of applications, for example, the
gradient magnitude can be used to enhance edges, while the components of
the gradient gives information about the orientation of said edges. A common
way of estimating the gradient for an image, using mathematical morphology,
is to apply the morphological gradient [51]. However, this only yields an
29
estimate of the gradient magnitude, not the components themselves, which are
important, for example, when computing the local structure tensor [28]. One
way to estimate directionality is to use linear structuring elements to estimate
horizontal and vertical derivatives, however this does not work well in the case
of missing samples, since the linear SE is likely to miss many pixels (since it
is thin and, depending on the percentage of dropped pixels, there are only few
nearby pixels at any point).
The proposed approach instead uses half-ellipses as structuring elements.
That is, starting from a structuring element with elliptical support:
B =
{(x, y)
∣∣∣∣ x2
a2+
y2
b2≤ 1
}(2.28)
we derive several structuring elements as the intersection of ellipses and four
different half-planes:
B−h = Bh ∩H− B+
h = Bh ∩H+ (2.29)
B−v = Bv ∩ V − B+
v = Bv ∩ V + (2.30)
Where Bh is an ellipse where a = a0 and b = b0 for some a0, b0 ∈ R, Bv is
an ellipse where a = b0 and b = a0, and
H− = {(x, y) | x < −ε} H+ = {(x, y) | x > ε} (2.31)
V − = {(x, y) | y < −ε} V + = {(x, y) | y > ε} (2.32)
These SEs can then be used to estimate the horizontal and vertical component
of the gradient:
Ix = (I ⊕B−h )− (I ⊕B+
h ) Iy = (I ⊕B−v )− (I ⊕B+
v ), (2.33)
This approach turns out to give results that yield angle estimates biased
towards multiples of π/2. However, by using the same approach outlined
above, except rotating the ellipse and half-planes for a number of angles, it is
possible to estimate directional derivatives Dθ along several angles θ. I.e., the
differences estimate
∇vθI = ∇I · vθ ≈ Dθ (2.34)
where vθ is the unit vector (cos(θ), sin(θ)). Using this, a system of equations
can be set up: ⎡⎢⎢⎢⎣vθ0
vθ1...
vθn
⎤⎥⎥⎥⎦(∂I
∂x
∂I
∂y
)T
=
⎡⎢⎢⎢⎣Dθ0
Dθ1...
Dθn
⎤⎥⎥⎥⎦ (2.35)
30
-3 -2 -1 0 1 2 3Ground truth gradient direction
-3
-2
-1
0
1
2
3Horizontal and vertical SEs
Ideal estimateMorphological w = 5 l = 9
-3 -2 -1 0 1 2 3Ground truth gradient direction
-3
-2
-1
0
1
2
3Horizontal, vertical, and diagonal SEsIdeal estimateMorphological w = 5 l = 9
Figure 2.11. In these graphs, the estimated angle for a disk image (with no missing
pixels) is plotted vs the true angle. In the ideal case, the graph should therefore be a line
with slope 1 (shown in blue). When using only the horizontal and vertical estimates,
there is a bias. However, when also including the diagonal estimates this bias almost
completely disappears. Here w and l indicate the length of the minor and major axis
of the SE respectively.
Solving for(
∂I∂x
∂I∂y
)T
gives:
(∂I
∂x
∂I
∂y
)T
=
⎡⎢⎢⎢⎣vθ0
vθ1...
vθn
⎤⎥⎥⎥⎦+ ⎡⎢⎢⎢⎣Dθ0
Dθ1...
Dθn
⎤⎥⎥⎥⎦ (2.36)
where the + indicates the pseudoinverse. This approach is similar to one taken
by Hassouna and Farag [23]. Figure 2.11 shows the angle estimate versus
the true angle at the edge of a disk with a Gaussian profile when using only
horizontal and vertical estimates as well as when diagonal estimates are also
included and shows the bias being significantly reduced.
Although the approach, as presented here, is specific to regularly sampled
images with missing samples, it can easily be applied to irregularly sampled
signals, by simply replacing the regular, discrete operators with their proposed
counterparts for irregular morphology. This approach is taken in paper IV
where the gradient estimate is used to compute the local structure tensor [28]
used to shape SEs for adaptive morphology. The paper also proposes a num-
ber of additional improvements including combining the estimates yielded by
(2.33) with the estimates yielded by replacing the dilations in (2.33) with ero-
sions, as well as making use of non-flat structuring elements. Those variants
are compared against other gradient estimation methods and shown to perform
favorably under certain conditions.
31
-4 -2 0 2 40
0.2
0.4
0.6
0.8
1
-4 -2 0 2 40
0.2
0.4
0.6
0.8
1
-4 -2 0 2 40
0.2
0.4
0.6
0.8
1
Figure 2.12. A small shift of the sampling grid causes the maximal sample to either
fall close to the extremum or completely miss it.
2.12 Continuous and Discrete Morphology
In this section we specify three main issues when applying morphology in the
regular, discrete domain, if the goal is to approximate continuous morphology.
The motivation for considering the approximation of the continuous case is,
as previously stated, the fact that one is usually interested in the subject being
imaged, not the image data itself. I.e., not the sampled set of pixels.
First, recall the Nyquist-Shannon sampling theorem:
Given a function f , with a maximal frequency of B, a sampling of f witha sampling rate greater than 2B completely determines f . [47, 57]
In other words, a continuous function f can be reconstructed from its samples,
if the sampling rate is greater than twice the highest frequency of f .
There are a number of issues to consider regarding the correpondence be-
tween regular, discrete morphology and the continuous counterpart. First:
Applying a morphological operator to a signal generally introduces higher
frequency content (see Fig. 2.3). Thus, for a band-limited signal which can
be represented by some regular sampling at a given frequency, its transformed
counterpart may no longer be properly representable at the same sampling rate.
However, regular, discrete morphology does not account for this, meaning the
correspondence between the transformed discrete signal and the continuous
one is broken.
Secondly: The regular, discrete MM operators depend on the position of the
sampling grid. In other words, shifting the grid (or equivalently the signal) will
lead to different results, since the operators depend on extrema in the signal,
which may be missed if the sampling grid is shifted even a small amount
(see Fig. 2.12). However, since the original sampled signal, assuming correct
sampling, can reconstruct the band-limited continuous signal, the information
about the extrema exists in the discrete signal, and could be taken into account.
In practice, this means the result depends on the position of an object being
imaged, which is not desirable.
32
Figure 2.13. A discretization of the continuous disk.
Thirdly: The structuring element depends on the sampling grid. Since the
SE is a subset of the image domain, the sampling grid causes discretization
problems [39]. For example, if one wants to use a disk-shaped SE, it has to be
discretized first (see Fig. 2.13), meaning a small disk (relative to the sampling
grid) may end up looking like a diamond.
These issues cause the approximation of continuous morphology, and there-
fore the correspondence to the objects being imaged, to suffer. In the following
section a brief overview of related morphological approaches that deal with
morphology and the continuous domain in some capacity is presented.
2.12.1 Morphological Dilations as Partial Differential Equations
In a 1992 paper by Brockett and Maragos [8], extended in 1994 [9], continuous-
space morphological erosions, dilations, openings, and closings are modeled
as nonlinear partial differential equations whose evolution can compute con-
tinuous morphology in the discrete domain.
For some function f : Rn → R representing a continuous nD signal, and a
continuous structuring function g : B → R, the multiscale dilation of f by gat scale s ≥ 0 is defined as
δ(x, s) = f ⊕ gs(x) =∨
b∈sB
f(x− b) + sg(b/s) (2.37)
where s is a scale parameter and gs is a scaled version of g by s. The goal is
then to study the evolution equation
∂δ
∂s(x, s) = lim
r→0+
δ(x, s+ r)− δ(x, s)
r(2.38)
In particular, when B is a unit disk, the PDE for multiscale dilations becomes
∂δ
∂s=
√∣∣∣∣∂δ∂x∣∣∣∣2
+
∣∣∣∣∂δ∂y∣∣∣∣2
(2.39)
As s increases, the PDE-based operators can create discontinuities in the spatial
derivatives, so one has to take special care at these points. The authors propose
33
replacing the conventional derivatives with the morphological counterpart, i.e.
M(f)(x) = limr→0+
(∨|v|≤r f(x+ v)
)− f(x)
r(2.40)
which is essentially the external morphological gradient (see Eqn. 2.24). See
the papers [8, 9] for details, including equations for several other SE shapes.
Similar equations can also be written down for erosion, openings, and
closings.
There are several papers built on similar ideas [69, 68, 55, 7], however, in
general the result is represented on a regular grid and therefore suffers from
the problems previously described. Additionally, the PDE based approaches
have a tendency to blur edges and are slow compared to regular, discrete mor-
phology [75].
34
3. Mathematical Morphology on IrregularlySampled Signals
In this chapter, an approach to morphology for irregularly sampled signals is
developed. In his thesis [65], Thurley develops morphology for irregularly
sampled input, however the positions of samples of the transformed signal is
the same as those in the input, which means most of the problems with approx-
imating continuous morphology remain. Calderon and Boubekeur develop
binary 3D morphology on point clouds [13], however this approach is limited
to binary signals, and must explicitly estimate the underlying surface. Also
relevant to this problem is morphology on graphs. Najman and Cousty [45]
provide a good survey of current research on morphology on graphs, however
the proposed approach does not explicitly work on any graph structure, al-
though it may be possible to achieve faster computation by incorporating some
such underlying structure.
I will also show how the idea of irregular sampling can be used to improve
the approximation of the continuous case in the discrete domain. There are
two main reasons for examining morphology in the case of irregular sampling.
First: as previously described, there are a number of issues with approximating
continuous morphology using regular, discrete morphology. These issues
can be alleviated, yielding a better approximation of the continuous case, by
allowing for irregular sampling. Secondly: there is an abundance of data
that is irregularly sampled to begin with, meaning applying regular, discrete
morphology requires some preprocessing of the data to make it amenable,
often by resampling onto a regular grid, which normally causes interpolation
artifacts and problems where the original sampling had a hole in the data.
3.1 Approximating Continuous MorphologyPreviously I described three issues regarding the approximation of continuous
MM using regular, discrete morphology. Here I will motivate how these issues
are addressed by allowing for irregular sampling.
3.1.1 Morphological Operators Introduce Higher FrequencyContent
After applying a morphological operator, thereby generally introducing higher
frequency content into a signal and breaking the correspondence between the
35
-50 0 50 100 150 200 250 3000
50
100
150
200
250
300
350Regular, discrete dilationProposedOriginal Signal
Figure 3.1. Dilation of the black signal using regular, discrete morphology (blue)
and the proposed approach (red). Note the decreased number of samples on plateaus
compared to parts of the signal that fluctuate more wildly.
discrete and the continuous domain, one solution would be to sample the sig-
nal more densely to deal with this problem. However, increasing the sampling
density everywhere is expensive, and potentially unnecessary. The operators
usually generate a signal with many plateaus (see Fig. 2.3). Accurately rep-
resenting these parts of the signal should not require many samples (in the
extreme case, two samples per plateau is sufficient, i.e., one at each end point).
On the other hand, the operators also generate cusps, where the sampling den-
sity should be increased to better capture the behavior of the quickly changing
signal. This leads to the conclusion that representing the transformed signal
as a set of irregularly taken samples could benefit the approximation of the
continuous case (without exploding the number of samples) by increasing the
sampling density near “interesting” parts of the signal (e.g. the cusps), while
decreasing the density on very smooth parts (e.g. the plateaus). Figure 3.1
shows an example of the proposed approach from paper I applied to a 1D-
signal showing the described behavior of adjusting the sampling density based
on the smoothness of the signal.
3.1.2 Effect of Shifting the Sampling Grid
As shown in Figure 2.12, shifting the sampling grid may result in extrema
being missed. Thus, even though the sampling density is the same, and the
band-limited signal is correctly sampled, the result of applying a MM operator
will differ based on a small shift of the sampling grid. However, if we allow for
irregularly sampled input, we could, in principle reconstruct the extrema of the
continuous signal and insert only those points into the input before applying the
morpological operator, thus achieving a result that is not affected by the small
36
Figure 3.2. The continuous dilation (top right) of the input (top left) is 1.0 everywhere.
However, the discrete dilation of the sampled signal (bottom left) differs in the middle
(in the vicinity of (0, 0)), since the maximum at (0, 0) was missed when sampling.
By inserting a single sample at (0, 0) (thereby yielding an irregularly sampled signal),
this issue can be dealt with (bottom right). The red points indicate the location of the
samples of the discrete signal.
shift of the grid. Figure 3.2 shows an example of a continuous signal and its
dilation, as well as the dilation of the same sampled signal where the sampling
grid is deliberately placed such that an extremum is missed. The discrete
dilation therefore differs from the continuous one, because dilation depends
on maxima in the signal. However, adding in just a single point (namely that
extremum), thus yielding an irregularly sampled signal, the proposed approach
now corresponds well with the continuous case.
3.1.3 Effect of the Sampling Grid on the Structuring Element
Finally, as previously discussed, the structuring element is affected by the
sampling grid (see Fig. 2.13). However, in the proposed approach, there is no
sampling grid, and therefore no restriction on the sampling of the SE. Figure 3.3
shows an example of dilating an image using a disk shaped SE in the case of an
37
Figure 3.3. From left to right: input image, regular dilation on sampled input, irregular
dilation on sampled input, and regular dilation on non-sampled input. The red square
indicates the zoomed in area being displayed.
image that has been subsampled (after low-pass filtering). The original image
is also dilated with a disk-shaped SE of corresponding size. Because of the
subsampling, the size of the SE in relation to the grid is very small, so the
discretized disk ends up diamond shaped. However, the approach proposed in
paper III does not require a sampling grid, so the edge of the continuous SE
can be sampled with arbitrarily many samples at the exact position of the edge.
Therefore, the proposed approach yields a dilation that better approximates the
continuous case (or in this example, the dilation of the original image before
subsampling, which acts as a stand-in for the continuous case).
3.2 Point Clouds
It is worth noting that, as an additional benefit of developing MM for irregularly
sampled signals, an abundance of data that is already irregularly sampled to
begin with can be dealt with immediately, i.e., without requiring resampling.
Other work on morphology for irregularly sampled signals includes work by
Thurley [65] and more recent work on point clouds [13, 49, 1]. Also related is
the work on mathematical morphology on graphs [73, 19, 45], since a natural
structure for representing irregularly sampled data is often a graph.
In paper III a variant of the proposed approach is used to segment 3D point
clouds of urban scenes, where the height of samples is taken as their value,
as a demonstration of its applicability to processing signals that are irregularly
sampled to begin with.
3.3 Irregular Morphology in 1D
What follows is a brief description of the proposed approach to irregular
morphology in 1D. For details see paper I (an earlier paper from 2016 by
Asplund et al. may also be of interest [2]).
Let
S = P(Z× R̄), (3.1)
38
be the set of possible samples (i.e., position and value), where P denotes the
power set. Then a (possibly irregularly) sampled continuous function is an
element from the set
S = {A ∈ S | for all (xi, yi), (xj , yj) ∈ A, xi = xj ⇒ yi = yj}. (3.2)
It is possible to define a partial order � on S, such that (S,�) forms a
complete lattice. Since two irregularly sampled signals cannot, in general,
be compared pointwise, a function T : S → R̄R is defined which takes
an irregularly sampled signal as input and yields a continuous signal with
a support equal to R. This top function is used to define the partial order:
U � V ⇐⇒ T (U) ≤ T (V ), for U, V ∈ S (3.3)
where T (U) ≤ T (V ) iff T (U)(x) ≤ T (V )(x) for all x ∈ R.
The pseudocode for the algorithm to dilate a 1D, irregularly sampled signal
is shown in Alg. 1. Roughly, the algorithm performs these four steps:
1. Select a sample and translate the SE such that its origin coincides with
the sample position.
2. Make two copies of the sample.
3. Shift the copies towards the endpoints of the SE.
4. If a copy would end up under the SE placed at a sample of greater value,
stop shifting it.
There is a parameter ε > 0 that controls the minimal margin between two
neighboring output samples. This parameter cannot be 0, since this could yield
output where two samples at the same position take different values, therefore
being part of a sampling that is not an element of S. Using a self-balancing
binary search tree [27], the time complexity of the algorithm is O(N logN),where N is the number of samples in the input.
Erosions, closings, and openings are easily performed using the same algo-
rithm by making use of the duality of dilation and erosion and composition.
That is, by multiplying the signal with −1 to invert it, applying a dilation,
followed by multiplying the result by −1, one may perform an erosion.
3.4 Irregular Morphology in 2D
In papers II, III, and IV a similar approach to the 1D approach presented
above is developed, but for the case of 2D signals (e.g. grayscale images).
Paper II presents a first attempt at generalizing to 2D, which suffers from some
problems dealt with in the later papers III and IV (mainly an issue of fuzzy
edges in the output, as well as strict requirements on the shape of the SE).
39
Algorithm 1: Duplicate-and-shift dilation pseudocode.
Data: A list of input samples, NODES, which contains positions and values
for each sample. The left, and right edges, SE−, and SE+, of the SE as
offsets from the origin.
Result: A list of output samples DNODES which contains positions and
values for each sample in the dilated signal.
1 Function Dilate–Irregular (NODES, SE−, SE+)
2 let DNODES be an empty array
3 sort NODES according to y-value in descending order
4 for each node i ∈ NODES do5 let i− and i+ be duplicates of i6 let pos(i−) = pos(i) + SE−
7 let pos(i+) = pos(i) + SE+
8 let NODES− be the list of nodes that precede i9 let j− be the nearest neighbor of i in NODES−, s.t. pos(j−) < pos(i)
10 let j+ be the nearest neighbor of i in NODES−, s.t. pos(j+) > pos(i)11 //If j− does not exist, let pos(j−) = −∞12 //If j+ does not exist, let pos(j+) = +∞13 if pos(i−) ≤ pos(j−) + SE+ then14 pos(i−) = pos(j−) + SE+ + ε
15 if pos(i+) ≥ pos(j+) + SE− then16 pos(i−) = pos(j−) + SE− − ε17 if pos(i−) ≤ pos(i+) then18 if �n ∈ DNODES : pos(i−) = pos(n) then19 insert i− into DNODES
20 if �n ∈ DNODES : pos(i+) = pos(n) then21 insert i+ into DNODES
22 if pos(i−) < pos(i) < pos(i+) then23 insert i into DNODES
24 else25 drop nodes i−, i, and i+
26 return DNODES
However, paper II also deals with questions about how to sample the output
signal to avoid a dense sampling everywhere without adversely affecting the
result. Here we give an overview of the approach in paper IV which is a more
general case of the algorithm in paper III allowing for more varied structuring
elements.
The following are necessary:
• A signed scalar field f(x, y, xc, yc, v, r) that describes a structuring ele-
ment centered at position (xc, yc) that depends on some value v (usually
derived from the input signal), and whose size is determined by r. Here
f < 0 indicates the inside of a SE, f > 0 the outside, and 0 the edge.
40
}Figure 3.4. Illustration of the steps for processing one sample (the blue “x”) as viewed
from the side (therefore the flat SEs look like line segments). The blue samples are
taken on the border of a SE placed on top of the sample currently being processed. The
red samples are taken on the border of a slightly larger SE. The samples that fall in the
shadow of a SE are discarded (the blue “x” and the red and blue stars). The remaining
samples from the larger SE are dropped downward until they intersect a SE at a lower
value, or fall through to −∞ (in which case they are discarded). These steps are then
repeated until all input samples have been processed.
• A way of sampling the zero-set f = 0, which we denote ∂f(xc, yc, v, r).
For the sake of simplicity, let us consider a function f(x, y, xc, yc, r) that is
independent of v, and its zero-set ∂f(xc, yc, r). To perform a dilation of size
r on an input signal
I = {(xi, yi, zi)}i∈{1,2,...,N} (3.4)
of N samples, where xi and yi is the sample position, and zi is its value, do
the following (see Fig. 3.4):
1. Let I0δ = ∅, in subsequent iterations, we will fill out sets Iiδ with samples
from the dilation.
2. Select an unprocessed sample (xc, yc, zc) ∈ I and mark it as processed
(blue “x” In Fig. 3.4).
3. Sample ∂f(xc, yc, r). Denote this set of samples B (shown in blue in
Fig. 3.4).
4. Sample ∂f(xc, yc, r + ε), where ε > 0 is some small real number. We
call this set of samples Bε (shown in red).
5. For each sample (x′, y′) ∈ B, and each sample (xi, yi, zi) ∈ I \{xc, yc, zc},such that zi ≥ zc, compute fi = f(x′, y′, xi, yi, r+ε). If fi ≤ 0, discard
(x′, y′) (see the blue star and blue “x” in Fig. 3.4). Let B̂ denote the
remaining set of samples (blue circle). This step discards samples from
41
the smaller SE that end up in the shadow (umbra) of a SE placed at a
higher sample.
6. Repeat the procedure in 5. except replacing B with Bε. Let B̂ε denote
the remaining samples (shown as the red circle at the top of the arrow in
Fig. 3.4). Figure 3.4 shows one such sample being discarded (red star).
7. Let
H(x, y) = {zi | (xi, yi, zi) ∈ I∧zi < zc∧f(x, y, xi, yi, r) ≤ 0} (3.5)
and let
zh(x, y) = max (H(x, y) ∪ −∞) (3.6)
Then, for each sample (x′ε, y
′ε) in B̂ε, compute zh(x
′ε, y
′ε). This means,
zh is the value of a given sample of the bigger SE (B̂ε) after being
projected downward until hitting a SE at a lower value (or falling through
to −∞). This is illustrated in Fig. 3.4 by the red circle.
8. Let
B̂c = {(x, y, zc) | (x, y) ∈ B̂}, (3.7)
B̂hε = {(x, y, zh(x, y)) | (x, y) ∈ B̂ε}, and (3.8)
Ii+1δ = Iiδ ∪ B̂c ∪ B̂h
ε (3.9)
Then INδ is the dilated signal.
In essence, two structuring elements are centered on top of each input sample
(x, y, z), one slightly larger than the other, and then their borders are sampled
(steps 2-4). After this, all newly created samples are checked and those that
fall in the shadow of a SE placed at a neighboring sample at a higher position
are removed (steps 5-6). The samples from the larger SE are projected down
until they hit a SE at a lower level, or fall through (step 7). The union of all the
sampled structuring elements is the dilated signal.
Figure 3.5 shows an example of applying the procedure described above
to a point cloud to compute a tophat of a scene containing a signpost. The
opening is computed by composing an erosion and a dilation, the erosion can
be computed using the same procedure, by duality. In order to subtract the
opening from the original signal, the opening is interpolated back onto the
original sampling positions using linear interpolation.
Since the structuring elements are defined as signed scalar fields it is possible
to combine several simple scalar fields to generate more complicated ones.
Note especially that taking the supremum/infimum of a pair of scalar fields
is like taking the intersection/union of the SEs that they represent. I.e., for
a pair of structuring elements B1 and B2 represented by the scalar fields f1and f2, the scalar field f1 ∨ f2 will represent the SE B1 ∩ B2, and similarly
f1∧f2 represents the SE B1∪B2. This can be used to construct intricate SEs.
42
0-0.5
43-131
43.5
-1-131.5
44
Signpost
-132 -1.5
44.5
-132.5
45
45.5
46
0-0.5
43-131
43.5
-1-131.5
44
Eroded signpost
-132 -1.5
44.5
-132.5
45
45.5
46
0-0.5
43-131
43.5
-1-131.5
44
Open
-132 -1.5
44.5
-132.5
45
45.5
46
00 -0.5-131
0.5
-1-131.5
1
Tophat of signpost
-132
1.5
-1.5-132.5
2
2.5
3
Figure 3.5. Examples of irregular morphology applied to a simple point cloud of a
signpost. Here, the SE is a disk with radius 0.016m.
Figure 3.6 shows some examples. Moreover a SE can be defined by sampling
a function and interpolating values to generate the scalar field.
3.5 Adaptive Morphology
So far we have only looked at the case where the SE is fixed. However,
there are cases where one may want to change the shape of the SE based
on, for example, its position (location-adaptive) or some local property of the
image (input-adaptive). This concept of adaptive morphology was previously
described in the regularly sampled case in Section 2.9. In this section the
work on MM on irregularly sampled signals is extended to allow for adaptive
morphology in the irregular case.
Returning to the idea of SEs as signed scalar fields, f(x, y, xc, yc, v, r),one may perform the same eight steps presented in the previous section even
if the shape of f depends on (xc, yc) or some value v. As an example,
we adapt the work by Landström and Thurley [32], which presents adaptive
morphology using different elliptical structuring elements based on the local
structure tensor [28] for regularly sampled signals.
3.5.1 Elliptical Adaptive Structuring Elements
In this section a brief description of the work by Landström and Thurley [32]
on adaptive morphology using elliptical SEs is presented. See also the thesis
by Landström [30]. First a brief description of the local structure tensor [28]
43
Figure 3.6. The top four images show the result of taking the minimum of a p-norm
disk with four smaller translated 2-norm disks. The bottom four images show the
minimum of the four pairs of maxima of the p-norm disk and each of the four smaller
2-norm disks. Combining signed scalar functions using ∨ and ∧ (essentially working
as intersection and union) allows for a wide variety of structuring elements.
44
Figure 3.7. The elliptical SE is shaped by the eigenvectors and their associated
eigenvalues. The semi-major axis (a) is aligned with the eigenvector e2, and the semi-
minor axis (b) aligns with the eigenvector e1. The length of these axes are decided by
the eigenvalues.
Local Structure TensorConsider a function I(x) that maps tuples x = (x1, x2) ∈ R2 to R (e.g. a
grayscale image). The local structure tensor is defined as:
LST (I)(x) = Gσ ∗(∇I(x)∇T I(x)
), (3.10)
where∇ =(
∂∂x1
, ∂∂x2
)T
andGσ is a Gaussian filter with standard deviationσ.
I.e. each x is mapped to a 2× 2–matrix. This matrix contains local directional
information. Computing the eigenvalues λ1(x) ≥ λ2(x), the following cases
apply:
1. λ1 ≈ λ2 � 0: A crossing or a point.
2. λ1 � λ2 ≈ 0: A dominant direction.
3. λ1 ≈ λ2 ≈ 0: No edge.
The eigenvectors e1(x) and e2(x) associated with λ1(x) and λ2(x) point in
the direction of largest and smallest variation respectively [14].
Adaptive Morphology Using Elliptical SEsThe approach proposed by Landström and Thurley computes the LST and
shapes an elliptical SE at each position x such that the major axis depends
on e2 and the minor axis on e1, rotating the ellipse accordingly, as shown
in Fig. 3.7. The size of the ellipse is determined by the eigenvalues. This
approach can be used to preserve strong edges when filtering, because the
ellipses near edges will be thin and oriented along the edge. In a later paper,
Landström extends this approach, using non-flat morphology [31].
45
-20 0 20 40
-130-120-110-100-90-80-70-60
Input
-20 0 20 40
Angles
-20 0 20 40
Interpolated adaptive erosion
Figure 3.8. The top image shows the irregularly sampled input (a pile of rocks). The
figure on the bottom left illustrates the angles used to align the elliptical SEs, based on
the eigenvectors of the LST. The bottom right shows an example of an erosion using
the adaptive, elliptical SEs.
3.5.2 Adapting the Elliptical Adaptive Structuring Elements toIrregularly Sampled Signals
To allow for information about the LST to affect the scalar fields, it is passed as
a parameter: f(x, y, xc, yc, LST (I)(xc, yc), r). To compute the LST for the
irregularly sampled signal a morphological approach is used to estimate the
gradient. In paper V an approach for estimating the components of the gradient
in cases of regularly sampled signals with missing samples is presented. The
same approach can be applied in the case of irregularly sampled signals (in
fact, regularly sampled signals with missing pixels can be seen as a special
case of irregularly sampled signals) exchanging regular, discrete morphological
operators for their irregular counterparts. Figure 3.8 shows this approach
applied to an irregularly sampled rock pile image to adaptively erode the
image. Figure 3.9 shows an example of applying this approach, illustrating
several of the steps, to a regularly sampled signal where 50% of samples are
46
first discarded, in order to get an irregularly sampled image. This is followed
by an adaptive dilation (as described above), and finally an interpolation onto
the regular grid for visualization purposes.
47
Figu
re3.
9.T
his
figure
illu
stra
tes
seve
ral
step
sof
the
irre
gula
rap
pro
ach
for
elliptica
lad
aptive
stru
cturi
ng
elem
ents
.Fro
mle
ftto
right:
The
regula
rly
sam
ple
din
putim
age.
The
inputim
age
afte
rdis
card
ing
50%
of
pix
els
(show
nin
red),
thus
yie
ldin
gan
irre
gula
rly
sam
ple
dim
age,
and
dra
win
gso
me
of
the
elliptica
lSE
sth
atar
ead
apte
dbas
edon
the
LST.
The
resu
ltof
dilat
ing
the
irre
gula
rly
sam
ple
dim
age
usi
ng
the
adap
ted
elliptica
lSE
s.Fin
ally
,th
ela
stim
age
show
sth
edilat
ion
afte
rin
terp
ola
ting
bac
konto
the
regula
rgri
dof
the
ori
gin
alim
age.
This
step
isnot
nec
essa
ryfo
rfu
rther
morp
holo
gic
alpro
cess
ing,butai
ds
vis
ual
izat
ion.
48
4. Conclusions and Future Work
4.1 Contributions
The major contributions of this thesis are:
• Developing a framework for mathematical morphology on irregularly
sampled signals (papers I, II, III, and IV)
• Improving the approximation of continuous morphology in the discrete
domain (papers I, II, III, and IV) by allowing for irregularly sampled
signals as output for morphology in the discrete domain, thus decoupling
from the sampling grid, which enables working around problems, such
as those stemming from non-linear filters introducing higher frequency
content into the filtered signal, thereby breaking the correspondence
between a band-limited, correctly sampled signal and its continuous
counterpart after applying a filter.
• Proposing a faster, approximate path opening (paper VI) by taking inspi-
ration from the parsimonious path opening, preselecting paths of interest
and applying an opening only to the preselected paths. The speed of
the proposed algorithm is shown to be comparable to the parsimonious
path opening. However, the proposed path opening does not suffer from
problems of occlusion present in the case of the PPO. Additionally, we
propose less biased weights for path lengths and show that this translates
to less biased length measurements for the proposed path opening than
those of the regular path opening or the PPO. We also apply the pro-
posed morphology on irregularly sampled signals to find a more precise
approximate parsimonious path opening (paper I), since the preselected
paths are essentially irregularly sampled 1D signals.
• Applying the framework for morphology on irregularly sampled signals
to the problem of segmenting 3D point clouds of urban scenes (paper III).
Since the proposed framework allows for irregularly sampled signals as
input, this allows us to apply morphology on the original data without the
need for resampling onto a regular grid, which is the common approach.
• Developing a way of estimating the components of the gradient for ir-
regularly sampled images, or images with missing samples (paper V).
The foundation of the proposed approach is morphology using elliptical
structuring elements. Regularly sampled images with missing samples
can be seen as a special case of irregularly sampled signals. A common
way of dealing with this type of problem is to use normalized differen-tial convolutions. However, NDC is not as easily applied to the general
49
case of irregularly sampled signals. The proposed approach shows com-
petitive performance and is applicable in the general case of irregular
sampling, since at its core the method uses morphology, meaning the
previously developed framework for MM on irregularly sampled data is
applicable.
• Developing adaptive morphology for irregularly sampled signals with
SEs described by scalar fields, using the local structure tensor to shape
elliptical SEs (paper IV). The necessary gradient estimation is performed
using the previously proposed method.
4.2 Future Work
There are several avenues to explore in the future, extending the current work.
Firstly, the implementation of the variants of the proposed approach to morphol-
ogy on irregularly sampled signals could probably be sped up by incorporating
some additional structure to aid computation (e.g. by creating a graph span-
ning the samples of the input signal or some subset thereof in order to enable
quicker search for neighbors, which is a necessary part of the proposed algo-
rithms). Another speed improvement may be possible by performing openings
and closings in one step, not as a composition of erosions and dilations.
It should be possible to extend the work to higher dimensions. In the case of
path openings and USPO presented in paper VI, there is not much hindering a
generalization to nD. The main obstacle is the skeletonization step. However,
assuming a suitable grayscale nD-skeletonization algorithm, the rest of the
algorithm is readily modified to deal with higher dimensionalities by using
different adjacency graphs and adjusting the weights appropriately. In the 3D
case, a suitable skeletonization algorithm might be developed by combining the
ideas of Verwer et al. [72] with some 3D-skeletonziation algorithm [18, 60].
For the gradient estimation presented in paper V, the same approach, essentially,
should be applicable in higher dimensions as well, simply by exchanging the
ellipses by n-dimensional ellipsoids oriented along relevant directions. The
remaining papers dealing with the general case of morphology on irregularly
sampled signals should also be possible to extend, although there may be issues
to consider regarding what is the inside/outside of objects as the morphological
operators are applied.
There are many questions of interest regarding how to sample the trans-
formed, irregularly sampled signals. This has been treated, to some extent,
in the papers introducing the variants of the approaches to MM on irregularly
sampled signals (specifically papers I, II, III, and IV), for example by experi-
menting with adapting sampling density based on properties of the input signal
(e.g. the gradient or the Laplacian), but more extensive experiments would be
desirable. A related issue is how to best sample the structuring elements. In-
tuitively it would seem beneficial to increase the sampling density near sharp
50
features of the border of the support of the SE. Making sure to sample the exact
positions of extreme values of non-flat SEs, as well as increasing the sampling
density in areas where the height of the SE does not vary smoothly should also
be helpful.
Although the proposed approach allows for dealing with irregularly sampled
signals directly, there are sometimes cases where interpolation of the resulting
transformed, irregular signal is desired. In these cases it is not suitable to apply
some smooth interpolation everywhere, since the morphological operators
generally create cusps in the output signal, meaning it is not smooth. On
the other hand, there are generally parts of the transformed signal that behave
smoothly. Thus a hybrid approach may be suitable, in cases where interpolation
is desired, applying a spline-based interpolation at smooth parts of the signal
for example, but using linear interpolation near sharply varying parts of the
transformed signal.
Another avenue of potential interest is to use the proposed approach together
with the pixel coverage model [59, 36], which converts an image into a fuzzy
set where boundary pixels belong to the object by some fraction that indicates
how much of the pixel is covered by the object, in order to further improve
image-based measurements.
Finally, when dealing with irregularly sampled signals, commonly there are
holes in the data, in the case of range cameras due to the surface geometry.
In the different variants of the proposed approach such cases are never dealt
with explicitly, instead treating every part of the signal the same, regardless
of sampling density. However, this may not be the best approach, since the
information content near these regions is sparse. Thus, it could be useful to
explicitly handle holes by marking such parts of the signal and taking into
account these regions when applying different operations.
51
5. Brief Summaries of the Included Papers
Paper I:This paper introduces a new approach to one-dimensional morphology on
irregularly sampled signals. The paper gives three main reasons as to why
allowing for irregularly sampled signals as output is helpful if one is interested
in approximating continuous morphology in the discrete domain. Empirically,
this is also shown by applying the approach to continuous, synthetic 1D-signals
as well as their regularly sampled counterparts. The irregularly sampled output
as well as the result of applying regular, discrete morphology is compared
against the transformed continuous signal and the proposed approach is shown
to yield a better approximation.
Additionally, allowing for irregularly sampled input makes it possible to
apply morphology directly, without resampling onto a regular grid. The ben-
efit of this is illustrated on an example where paths are preselected through
an image containing a long, thin structure using the same approach as for
parsimonious path opening. These preselected paths are one dimensional sig-
nals and can be interpreted as irregularly sampled, since the distance between
horizontal/vertical neighbors is shorter than that between diagonal neighbors.
The proposed approach is applied to these irregularly sampled signals to yield
an approximate path opening and shown to give better results than regular,
discrete morphology.
Paper II:In this paper the approach to morphology on irregularly sampled signals is
generalized to two dimensions. The paper shows that the proposed approach
yields better approximations of continuous morphology in two dimensions
than regular, discrete morphology. The paper also deals with issues of how
to sample the transformed signal and shows that the sampling density can be
drastically reduced in parts of the transformed signal (namely on plateaus)
without adversely affecting the result, thus significantly decreasing the num-
ber of samples in the transformed signal, which leads to faster subsequent
computations.
Paper III:We improve the approach presented in paper II by sampling the structuring
element differently and modifying the proposed approach appropriately. This
deals with some problems with fuzzy edges that sometimes arose when ap-
plying morphological operators using the previous approach. The paper also
52
applies this improved variant to 3D point clouds (which is a natural target for
the approach, since it deals with irregularly sampled data) of urban scenes in
order to segment the scene into objects of interest.
Paper IV:This paper generalizes the results of paper III by allowing for a much wider
range of structuring elements, both flat and non-flat. The generalization also
allows for adaptive morphology. We make use of the approach presented in
paper V to estimate the gradient for irregularly sampled signals and use this
to compute the local structure tensor in order to apply adaptive morphology,
shaping the structuring element based on directional information in the un-
derlying signal, as is done by Landström and Thurley in the case of regularly
sampled images [32].
This paper also discusses, in some detail, how to sample the output signal
based on the input signal as well as the shape of the structuring element.
Paper V:Regularly sampled images with missing samples can be seen as a special
case of irregularly sampled signals (where all samples fall on grid points, but
not at regular intervals). In this paper we propose a way of using rotated
half-ellipses as structuring elements in order to estimate components of the
gradient. The proposed approach is compared against normalized differential
convolutions and Gaussian derivatives (where the input is preprocessed to
deal with missing samples) and performs favorably. An additional advantage
of the proposed approach is that it generalizes readily to the general case of
irregularly sampled input, by replacing the regular, discrete morphology with
the previously proposed irregular variants.
Paper VI:This paper proposes an approximate variant (called USPO) that is much faster
(approximately an order of magnitude for large images) than implementa-
tions of the traditional path opening. The proposed variant also makes use
of weighted adjacency graphs to achieve less biased measurements of length.
USPO takes some inspiration from another proposed variant, called the par-
simonious path opening, which preselects one-dimensional paths through the
image. However, the preselection has problems where bright structures closer
to the edge of the image may occlude structures of interest toward the center of
the image. USPO also preselects paths, but the proposed approach of USPO
does not have the same problem of occlusions. The paper demonstrates benefits
over several path opening variants and also shows experiments where the pro-
posed path opening variant is applied to the problem of detecting blood vessels
in retinal fundus images following the approach of Sigurðsson et al. [58] but
using the USPO instead of the regular path opening. In summary, the proposed
variant has two advantages: speed and less biased length measurements.
53
6. Summary in Swedish
Denna avhandling “Precise Image-Based Measurements through Irregular
Sampling,” eller “Noggranna bildbaserade mätningar via irreguljär sampling”
behandlar frågor gällande så kallad matematisk morfologi för både kontin-
uerliga signaler och deras motsvarande samplade (diskreta) signaler.
Matematisk morfologi är ett brett använt verktyg för signalbehandling. Den
grundläggande idén är att transformera en signal genom att systematiskt föra
en “sond” över signalen, ett s.k. strukturelement, och undersöka hur väl den
passar in i signalen, i någon bemärkelse. Detta strukturelement kan ses som
ett geometriskt objekt, t.ex. en cirkelskiva. Ibland associeras en funktion till
strukturelementet som anger en viss höjd för varje punkt. Om strukturele-
mentet saknar en sådan funktion (alternativt om funktionen är 0 överallt i
strukturelementet) kallas det platt.
I denna avhandling fokuseras speciellt på bilddata. Ett annat välanvänt
verktyg för (bland annat) bildbehandling är s.k. faltning med olika faltnings-
matriser. Detta angreppssätt ger upphov till linjära filter, som inte påverkar
represenationsegenskaperna för signaler samplade på ett samplingsgitter, enligt
Nyquist-Shannon samplingsteori [47, 57]. Morfologiska filter, å andra sidan,
är icke-linjära och därmed gäller ej detsamma för dessa filter. Som följd av
detta kan olika resultat nås t.ex. beroende endast på ett objekts exakta position
i en kameras synfält. Detta är normalt inte önskvärt, eftersom det vanligtvis är
det avbildade objektet som är av intresse, inte själva bilddatan i sig. Detta är en
av anledningarna till att irreguljär morfologi är av intresse. Genom att ta fram
ett tillvägagångssätt för morfologi på irreguljärt samplad data kan vi undvika,
eller åtminstone minska påverkan av problem som uppstår p.g.a. samplings-
gittret då vi är intresserade av att approximera den kontinuerliga signalen som
bakomligger den digitala signalen, d.v.s. det faktiska objektet, inte bilddatan i
sig.
Utöver de fördelar som finns med att gå ifrån reguljär sampling (såsom
pixlar i ett reguljärt gitter) gällande approximering av kontinuerlig morfologi,
så finns också en stor mängd av data som till sin natur är irreguljärt samplad
från början, t.ex. är data från olika typer av avståndskameror ofta represen-
terad som irreguljärt samplade datapunkter. Traditionellt skulle sådan data
behandlas morfologiskt genom att först interpolera sampel på ett reguljärt git-
ter och sedan applicera reguljär, diskret morfologi. Interpolationen kan leda
54
till icke önskvärda bildartefakter som påverkar resultatet negativt, speciellt om
större hål, d.v.s. områden där sampelpunkter fattas, förekommer i datat. Det
tillvägagångssätt som tagits fram i denna avhandling kan appliceras direkt på
irreguljärt samplad data (såsom i artikel III t.ex.) och kräver alltså inte detta
interpolationssteg.
Denna avhandlig har fem huvudmål, nämligen:
• Utveckling av ett ramverk för matematisk morfologi på irreguljärt sam-
plade signaler.
• Förbättrad approximation av kontinuerlig morfologi i den diskreta domä-
nen via irreguljär sampling eller viktade mått.
• Tillämpning av ramverket för morfologi på irreguljärt samplad data så-
som 3D-punktmoln.
• Särskild behandling av specialfallet av irreguljärt samplade signaler som
kan tolkas som reguljärt samplade signaler där vissa sampel fattas.
• Utveckling av adaptiv och icke-platt morfologi i fallet med irreguljärt
samplad data.
I avhandlingen ingår sex artiklar som sammanbinds av en röd tråd, nämligen
irreguljär sampling. Artikel VI behandlar s.k. path opening som är en sorts
morfologisk operation på reguljärt samplade bilder. I denna artikel föreslås
en variant som approximerar vanlig path opening genom att välja ut “vägar”
((eng.) paths) av intresse och sedan behandla endast dessa. Detta utval ger
en samling pixlar som kan tolkas som en irreguljärt samplad signal. Artikeln
innehåller bl.a. experiment som visar att den föreslagna varianten är snabb att
beräkna och ger ett mer korrekt resultat vad gäller uppskattning av längder.
Artikel V behandlar också reguljärt samplade bilder. Kopplingen till temat
här är att vi undersöker fallet då en del sampel (pixlar) saknas, vilket kan ses
som ett specialfall av irreguljärt samplade bilder. I artikeln föreslås ett sätt att
använda morfologi för att uppskatta gradienten i denna typ av bilder.
Kvarvarande artiklar behandlar det generella fallet där signaler är samplade
irreguljärt från början. Dessa artiklar utvecklar successivt mer generella verk-
tyg för behandling av irreguljärt samplade signaler från endimensionell data till
två och (i viss mån) tre dimensioner och från platta strukturelement med strikta
begränsningar på deras form till strukturelement som kan anta en stor mängd
olika former, inte nödvändigtvis platta. Slutligen generaliseras även resultaten
så att adaptiv morfologi är möjlig, d.v.s. morfologi med strukturelement som
ändrar form.
Sammanfattningsvis innehåller denna avhandling en samling av sex artiklar
gällande matematisk morfologi och irreguljärt samplade signaler och en intro-
duktion till dessa. Matematisk morfologi används ofta på diskreta, reguljärt
samplade bilder, men är också definierad i det kontinuerliga fallet. Av flera skäl
kan en bättre approximation av kontinuerlig morfologi uppnås i den diskreta
domänen, om irreguljär sampling tillåts. Detta är en motiverande faktor till
arbetet. En annan fördel med att kunna applicera morfologi i det irreguljära
55
fallet är att man då enkelt kan behandla data som är irreguljärt samplad från
början. Denna typ av data är vanligt förekommande. Ett exempel på detta är
data från avståndskameror.
56
7. Acknowledgments
My time at the Centre for Image Analysis at the Division of Visual Information
and Interaction has been a pleasure, not least because of the great company. I
would like to express my thanks to the following people:
• Robin Strand, my main supervisor for most of my PhD. Thank you for
the support and for giving me a lot of freedom to choose my own research
directions.
• Cris L. Luengo Hendriks, my main supervisor during the first part of
my studies. Thank you for the stimulating discussions. I hope you are
having a great time at your new job!
• Gunilla Borgefors, for being a supportive co-supervisor, especially dur-
ing the beginning of my PhD.
• Matthew J. Thurley, my other co-supervisor, for helpful comments
during paper writing and research.
• Elisabeth Wetzer, for helpful comments on a draft of this thesis, for your
efforts on the SSBA newsletter, and for being such a nice office mate!
• Johan Öfverstedt, for helpful comments on a draft of this thesis and for
the fun discussions on research and more.
• Eva Breznik, for being a good friend and research collaborator. Hope-
fully we can continue working together on the Graph CNNs. Also, thanks
for the vegan cakes!
• Damian Matuszewski, for the nice lunch discussions on games, TV,
swords, research, etc., and thanks for organizing the board game evenings!
• Fredrik Nysjö. I had fun teaching the graphics course together with you
(and learned a lot)!
• The MIDA research group, for being such a nice bunch of people. The
Monday group meetings always go by so quickly.
• My office mates. Thanks to everyone (Axel Andersson, Ankit Gupta,Raphaela Heil, Gabriele Partel, Nicolas Pielawski, Leslie Solorzano,Elisabeth Wetzer, Håkan Wieslander, and Johan Öfverstedt) for mak-
ing the office such a good working environment.
• All my past and present colleagues at CBA, and Vi2 in general. As I
said, it has been a pleasure.
• The group at the Centre for Mathematical Morphology in Fontaine-bleau, for welcoming me during my visit, and for the fruitful and fun
collaboration.
• My parents, Elof Asplund and Tua Borgmästars, for being supportive
and for motivating me to work harder.
57
• My brother, Sam Asplund, for being a great older brother (even if you
sometimes hogged the family computer, or jumped out from behind a
corner to scare me, when we were kids).
• My younger brother, Björn Asplund. It is always interesting to talk with
you. I hope, and think, you have had a lot of fun at Gotland’s the past
year and I am looking forward to seeing what you will do in the future.
• My nephews, Elof Birger Brøvig Asplund, and Osvald JohannesBrøvig Asplund for being welcome distractions when I see you dur-
ing the holidays, especially when needing a break from thinking about
work.
58
References
[1] J. Angulo. Morphological scale-space operators for images supported on point
clouds. In International Conference on Scale Space and Variational Methods inComputer Vision, pages 78–89. Springer, 2015.
[2] T. Asplund, C. L. Luengo Hendriks, M. J. Thurley, and R. Strand. A new
approach to mathematical morphology on one dimensional sampled signals. In
2016 23rd International Conference on Pattern Recognition (ICPR), pages
3904–3909. IEEE, 2016.
[3] S. Beucher. Segmentation d’images et morphologie mathématique. PhD thesis,
1990.
[4] S. Beucher, J. M. Blosseville, and F. Lenoir. Traffic spatial measurements using
video image processing. In Intelligent Robots and Computer Vision VI, volume
848, pages 648–655. International Society for Optics and Photonics, 1988.
[5] G. Birkhoff. Lattice theory, volume 25. American Mathematical Soc., 1940.
[6] H. Blum. A transformation for extracting new descriptors of shape. Models forthe perception of speech and visual form, 19(5):362–380, 1967.
[7] M. Breuß, B. Burgeth, and J. Weickert. Anisotropic continuous-scale
morphology. In Iberian Conference on Pattern Recognition and Image Analysis,pages 515–522. Springer, 2007.
[8] R. W. Brockett and P. Maragos. Evolution equations for continuous-scale
morphology. In Acoustics, Speech, and Signal Processing, 1992. ICASSP-92.,1992 IEEE International Conference on, volume 3, pages 125–128. IEEE, 1992.
[9] R. W. Brockett and P. Maragos. Evolution equations for continuous-scale
morphological filtering. Signal Processing, IEEE Transactions on, 42(12):
3377–3386, 1994.
[10] P. Brodatz. Textures: A Photographic Album for Artists and Designers. Dover
Publications, 1966.
[11] M. Buckley and H. Talbot. Flexible linear openings and closings. In
Mathematical Morphology and its Applications to Image and Signal Processing,
volume 18, pages 109–118. Springer, Proceedings of the International
Symposium on Mathematical Morphology 2000.
[12] B. Burgeth and A. Kleefeld. An approach to color-morphology based on Einstein
addition and Loewner order. Pattern Recognition Letters, 47:29–39, 2014.
[13] S. Calderon and T. Boubekeur. Point morphology. ACM Transactions onGraphics (TOG), 33(4):45, 2014.
[14] L. Cammoun, C. A. Castaño-Moraga, E. Muñoz-Moreno, D. Sosa-Cabrera,
B. Acar, M. Rodriguez-Florido, A. Brun, H. Knutsson, and J. Thiran. A review
of tensors and tensor signal processing. In Tensors in Image Processing andComputer Vision, pages 1–32. Springer, 2009.
[15] C.-S. Chen, J.-L. Wu, and Y.-P. Hung. Theoretical aspects of vertically invariant
gray-level morphological operators and their application on adaptive signal and
59
image filtering. IEEE transactions on signal processing, 47(4):1049–1060,
1999.
[16] F. Cheng and A. N. Venetsanopoulos. An adaptive morphological filter for
image processing. IEEE Transactions on Image Processing, 1(4):533–539,
1992.
[17] F. Cheng and A. N. Venetsanopoulos. Adaptive morphological operators, fast
algorithms and their applications. Pattern recognition, 33(6):917–933, 2000.
[18] M. Couprie and G. Bertrand. A 3D sequential thinning scheme based on critical
kernels. In International Symposium on Mathematical Morphology and ItsApplications to Signal and Image Processing, pages 549–560. Springer, 2015.
[19] J. Cousty, L. Najman, F. Dias, and J. Serra. Morphological filtering on graphs.
Computer Vision and Image Understanding, 117(4):370–385, 2013.
[20] V. Ćurić, A. Landström, M. J. Thurley, and C. L. Luengo Hendriks. Adaptive
mathematical morphology–a survey of the field. Pattern Recognition Letters,47:18–28, 2014.
[21] M. J. E. Golay. Hexagonal parallel pattern transformations. IEEE Transactionson computers, 100(8):733–740, 1969.
[22] H. Hadwiger. Minkowskische addition und subtraktion beliebiger punktmengen
und die theoreme von Erhard Schmidt. Mathematische Zeitschrift, 53(3):
210–218, 1950.
[23] M. S. Hassouna and A. A. Farag. Multistencils fast marching methods: A highly
accurate solution to the eikonal equation on cartesian domains. IEEEtransactions on pattern analysis and machine intelligence, 29(9), 2007.
[24] H. Heijmans, M. Buckley, and H. Talbot. Path openings and closings. Journal ofMathematical Imaging and Vision, 22(2-3):107–119, 2005.
[25] H. J. A. M. Heijmans and C. Ronse. The algebraic basis of mathematical
morphology I. Dilations and erosions. Computer Vision, Graphics, and ImageProcessing, 50(3):245–295, 1990.
[26] S. Klemenjak, B. Waske, S. Valero, and J. Chanussot. Automatic detection of
rivers in high-resolution SAR data. IEEE Journal of Selected Topics in AppliedEarth Observations and Remote Sensing, 5(5):1364–1372, 2012.
[27] D. E. Knuth. The art of computer programming, vol. 3: Sorting and searching
(2nd ed.). 1998.
[28] H. Knutsson. Representing local structure using tensors. In 6th ScandinavianConference on Image Analysis, Oulu, Finland, pages 244–251. Linköping
University Electronic Press, 1989.
[29] U. Köthe. What can we learn from discrete images about the continuous world?
In International Conference on Discrete Geometry for Computer Imagery, pages
4–19. Springer, 2008.
[30] A. Landström. Elliptical Adaptive Structuring Elements for MathematicalMorphology. PhD thesis, Luleå tekniska universitet, 2014.
[31] A. Landström. An approach to adaptive quadratic structuring functions based on
the local structure tensor. In International Symposium on MathematicalMorphology and Its Applications to Signal and Image Processing, pages
729–740. Springer, 2015.
60
[32] A. Landström and M. J. Thurley. Adaptive morphology using tensor-based
elliptical structuring elements. Pattern Recognition Letters, 34(12):1416–1422,
2013.
[33] C. Lantuéjoul and S. Beucher. On the use of the geodesic metric in image
analysis. Journal of Microscopy, 121(1):39–49, 1981.
[34] C. Lantuéjoul and F. Maisonneuve. Geodesic methods in quantitative image
analysis. Pattern recognition, 17(2):177–187, 1984.
[35] R. Lerallut, E. Decenciere, and F. Meyer. Image filtering using morphological
amoebas. Image and Vision Computing, 25(4):395–404, 2007.
[36] J. Lindblad and N. Sladoje. Coverage segmentation based on linear unmixing
and minimization of perimeter and boundary thickness. Pattern RecognitionLetters, 33(6):728–738, 2012.
[37] F. Lozes, A. Elmoataz, and O. Lézoray. Partial difference operators on weighted
graphs for image processing on surfaces and point clouds. IEEE Transactions onImage Processing, 23(9):3896–3909, 2014.
[38] C. L. Luengo Hendriks. Constrained and dimensionality-independent path
openings. IEEE Transactions on Image Processing, 19(6):1587–1595, 2010.
doi: 10.1109/TIP.2010.2044959.
[39] C. L. Luengo Hendriks, G. M. van Kempen, and L. J. van Vliet. Improving the
accuracy of isotropic granulometries. Pattern Recognition Letters, 28(7):
865–872, 2007.
[40] P. Maragos and C. Vachier. Overview of adaptive morphology: Trends and
perspectives. In 2009 16th IEEE International Conference on Image Processing(ICIP), pages 2241–2244. IEEE, 2009.
[41] G. Matheron. Random sets and integral geometry. 1975.
[42] H. Minkowski. Volumen und oberfläche. In Ausgewählte Arbeiten zurZahlentheorie und zur Geometrie, pages 146–192. Springer, 1989.
[43] V. Morard, P. Dokládal, and E. Decenciere. One-dimensional openings,
granulometries and component trees inO(1) per pixel. Selected Topics in SignalProcessing, IEEE journal of, 6(7):840–848, 2012.
[44] V. Morard, P. Dokládal, and E. Decencière. Parsimonious path openings and
closings. IEEE Transactions on Image Processing, 23(4):1543–1555, 2014. doi:
10.1109/TIP.2014.2303647.
[45] L. Najman and J. Cousty. A graph-based mathematical morphology reader.
Pattern Recognition Letters, 47:3–17, 2014.
[46] L. Najman and H. Talbot, editors. Mathematical morphology: from theory toapplications. John Wiley & Sons, 2013.
[47] H. Nyquist. Certain topics in telegraph transmission theory. Transactions of theAIEE, pages 617–644, 1928. [reprinted in: Proceedings of the IEEE, vol. 90, no.
2, pp. 280-305, February 2002].
[48] I. Nyström. On quantitative shape analysis of digital volume images. PhD
thesis, 1997.
[49] M. Peternell and T. Steiner. Minkowski sum boundary surfaces of 3D-objects.
Graphical Models, 69(3-4):180–190, 2007.
[50] D. Proffitt and D. Rosen. Metrication errors and coding efficiency of
chain-encoding schemes for the representation of lines and edges. Computergraphics and image processing, 10(4):318–332, 1979.
61
[51] J.-F. Rivest, P. Soille, and S. Beucher. Morphological gradients. Journal ofElectronic Imaging, 2(4):326–337, 1993.
[52] C. Ronse. Why mathematical morphology needs complete lattices. Signalprocessing, 21(2):129–154, 1990.
[53] C. Ronse and H. J. A. M. Heijmans. The algebraic basis of mathematical
morphology: II. Openings and closings. CVGIP: Image Understanding, 54(1):
74–97, 1991.
[54] P. K. Saha, G. Borgefors, and G. S. di Baja. A survey on skeletonization
algorithms and their applications. Pattern Recognition Letters, 76:3–12, 2016.
[55] G. Sapiro, R. Kimmel, D. Shaked, B. B. Kimia, and A. M. Bruckstein.
Implementing continuous-scale morphology via curve evolution. Patternrecognition, 26(9):1363–1372, 1993.
[56] J. Serra. Image analysis and mathematical morphology. Academic Press, Inc.,
1983.
[57] C. E. Shannon. Communication in the presence of noise. Proceedings of theIRE, 37(1):10–21, 1949. [reprinted in: Proceedings of the IEEE, vol. 86, no. 2,
pp. 447–457, February 1998].
[58] E. M. Sigurðsson, S. Valero, J. A. Benediktsson, J. Chanussot, H. Talbot, and
E. Stefánsson. Automatic retinal vessel extraction based on directional
mathematical morphology and fuzzy classification. Pattern Recognition Letters,47:164–171, 2014.
[59] N. Sladoje and J. Lindblad. High-precision boundary length estimation by
utilizing gray-level information. IEEE Transactions on Pattern Analysis andMachine Intelligence, 31(2):357–363, 2008.
[60] A. Sobiecki, A. Jalba, and A. Telea. Comparison of curve and surface
skeletonization methods for voxel shapes. Pattern Recognition Letters, 47:
147–156, 2014.
[61] P. Soille. Morphological Image Analysis: Principles and Applications. Springer
Science & Business Media, 2013.
[62] P. Soille, E. J. Breen, and R. Jones. Recursive implementation of erosions and
dilations along discrete lines at arbitrary angles. IEEE Transactions on PatternAnalysis & Machine Intelligence, (5):562–567, 1996.
[63] T. Statella, P. Pina, and E. A. da Silva. Image processing algorithm for the
identification of martian dust devil tracks in MOC and HiRISE images.
Planetary and Space Science, 70(1):46–58, 2012.
[64] S. R. Sternberg. Grayscale morphology. Computer vision, graphics, and imageprocessing, 35(3):333–355, 1986.
[65] M. J. Thurley. Three dimensional data analysis for the separation and sizing ofrock piles in mining (PhD thesis). Monash University, 2002.
[66] S. Valero, J. Chanussot, J. A. Benediktsson, H. Talbot, and B. Waske. Advanced
directional mathematical morphology for the detection of the road network in
very high resolution remote sensing images. Pattern Recognition Letters, 31
(10):1120–1127, 2010.
[67] J. J. van de Gronde and J. B. T. M. Roerdink. Group-invariant colour
morphology based on frames. IEEE Transactions on Image Processing, 23(3):
1276–1288, 2014.
62
[68] R. van Den Boomgaard and A. Smeulders. The morphological structure of
images: The differential equations of morphological scale-space. IEEEtransactions on pattern analysis and machine intelligence, 16(11):1101–1113,
1994.
[69] R. van den Boomgaard and A. W. M. Smeulders. The morphological structure
of images. In Proceedings., 11th IAPR International Conference on PatternRecognition. Vol. III., pages 268–271. IEEE, 1992.
[70] L. J. Van Vliet, I. T. Young, and G. L. Beckers. A nonlinear Laplace operator as
edge detector in noisy images. Computer vision, graphics, and imageprocessing, 45(2):167–195, 1989.
[71] J. G. Verly and R. L. Delanoy. Adaptive mathematical morphology for range
imagery. IEEE Transactions on Image Processing, 2(2):272–275, 1993.
[72] B. J. H. Verwer, L. J. van Vliet, and P. W. Verbeek. Binary and grey-value
skeletons: Metrics and algorithms. International Journal of Pattern Recognitionand Artificial Intelligence, 7(05):1287–1308, 1993.
[73] L. Vincent. Graphs and mathematical morphology. Signal Processing, 16(4):
365–388, 1989.
[74] L. Vincent. Morphological grayscale reconstruction in image analysis:
Applications and efficient algorithms. Transactions on Image Processing, 2(2):
176–201, 1993.
[75] J. Weickert. Anisotropic diffusion in image processing. Teubner Stuttgart, 1998.
[76] M. Welk, M. Breuß, and V. Sridhar. Matrix morphology with extremum
principle. In International Symposium on Mathematical Morphology and ItsApplications to Signal and Image Processing, pages 177–188. Springer, 2019.
[77] B. Yin, H. Li, B. Sheng, X. Hou, Y. Chen, W. Wu, P. Li, R. Shen, Y. Bao, and
W. Jia. Vessel extraction from non-fluorescein fundus images using
orientation-aware detector. Medical image analysis, 26(1):232–242, 2015.
63
Acta Universitatis UpsaliensisDigital Comprehensive Summaries of Uppsala Dissertationsfrom the Faculty of Science and Technology 1869
Editor: The Dean of the Faculty of Science and Technology
A doctoral dissertation from the Faculty of Science andTechnology, Uppsala University, is usually a summary of anumber of papers. A few copies of the complete dissertationare kept at major Swedish research libraries, while thesummary alone is distributed internationally throughthe series Digital Comprehensive Summaries of UppsalaDissertations from the Faculty of Science and Technology.(Prior to January, 2005, the series was published under thetitle “Comprehensive Summaries of Uppsala Dissertationsfrom the Faculty of Science and Technology”.)
Distribution: publications.uu.seurn:nbn:se:uu:diva-395205
ACTAUNIVERSITATIS
UPSALIENSISUPPSALA
2019