Multiple Instance Learning
description
Transcript of Multiple Instance Learning
![Page 1: Multiple Instance Learning](https://reader036.fdocuments.net/reader036/viewer/2022062520/568163f7550346895dd58c97/html5/thumbnails/1.jpg)
Multiple Instance Learning
![Page 2: Multiple Instance Learning](https://reader036.fdocuments.net/reader036/viewer/2022062520/568163f7550346895dd58c97/html5/thumbnails/2.jpg)
Outline Motivation Multiple Instance Learning (MIL) Diverse Density Single Point Concept Disjunctive Point Concept SVM Algorithms for MIL
Single Instance Learner (SIL) Sparse MIL mi-SVM MI-SVM
Results Some Thoughts
![Page 3: Multiple Instance Learning](https://reader036.fdocuments.net/reader036/viewer/2022062520/568163f7550346895dd58c97/html5/thumbnails/3.jpg)
Part I: Multiple Instance Learning (MIL)
![Page 4: Multiple Instance Learning](https://reader036.fdocuments.net/reader036/viewer/2022062520/568163f7550346895dd58c97/html5/thumbnails/4.jpg)
Motivation It is not always possible to provide labeled
data for training Reasons:
Requires substantial human effort Requires expensive tests Disagreement among experts Labeling is not possible at instance level
Objective: present a learning algorithm that can learn from ambiguously labeled training data
![Page 5: Multiple Instance Learning](https://reader036.fdocuments.net/reader036/viewer/2022062520/568163f7550346895dd58c97/html5/thumbnails/5.jpg)
Multiple Instance Learning (MIL) In MIL, instead of giving the learner labels for
the individual examples, the trainer only labels collections of examples, which are called bags.
A bag is labeled positive if there is at least one positive example in it
It is labeled negative if all the examples in it are negativeNegative Bags (Bi
-) Positive Bags (Bi+)
![Page 6: Multiple Instance Learning](https://reader036.fdocuments.net/reader036/viewer/2022062520/568163f7550346895dd58c97/html5/thumbnails/6.jpg)
Multiple Instance Learning (MIL) The key challenge with MIL is coping with the
ambiguity of not knowing which examples in the positive bag are actually positive and which are not
MIL model was first formalized by Dietterich et al. to deal with the drug activity prediction problem
Following that, an algorithm called Diverse Density was developed to provide a solution to MIL
Later, the method was extended to deal real-valued labels instead of binary labels.
![Page 7: Multiple Instance Learning](https://reader036.fdocuments.net/reader036/viewer/2022062520/568163f7550346895dd58c97/html5/thumbnails/7.jpg)
Diverse Density Diversity Density solves MIL problem by
examining the distribution of the instances It looks for a point that is close to instances in
different positive bags and that is far from the instances in the negative bags
Such a point represents the concept that we would like to learn
Diversity Density is the measure of the intersection of the positive bags minus the union of the negative bags.
![Page 8: Multiple Instance Learning](https://reader036.fdocuments.net/reader036/viewer/2022062520/568163f7550346895dd58c97/html5/thumbnails/8.jpg)
Diversity Density – Molecular Example Suppose the shape of candidate molecule can
be described by a feature vector If a molecule is labeled positive, then at least
one place along the manifold it took the right shape to fit into the target protein
![Page 9: Multiple Instance Learning](https://reader036.fdocuments.net/reader036/viewer/2022062520/568163f7550346895dd58c97/html5/thumbnails/9.jpg)
Diversity Density – Molecular Example
![Page 10: Multiple Instance Learning](https://reader036.fdocuments.net/reader036/viewer/2022062520/568163f7550346895dd58c97/html5/thumbnails/10.jpg)
Noisy-Or for Estimating the Density It is assumed that the event can only happen
if at least one of the causations occurred It is also assumed that the probability of any
cause failing to trigger the event is independent of any other cause
![Page 11: Multiple Instance Learning](https://reader036.fdocuments.net/reader036/viewer/2022062520/568163f7550346895dd58c97/html5/thumbnails/11.jpg)
Diverse Density - Formally By maximizing the Diverse Density we can
find the point of intersection (the desired concept)
where
Alternatively, one can use most-likely-cause estimator
![Page 12: Multiple Instance Learning](https://reader036.fdocuments.net/reader036/viewer/2022062520/568163f7550346895dd58c97/html5/thumbnails/12.jpg)
Single Point Concept A concept that corresponds to single point in feature space Every Bi
+ has at least one instance that is equal to the true concept corrupted by some Gaussian noise.
Every Bi- has no instances that are equal to the true concept
corrupted by some Gaussian noise
Where
k = number of dimensions in feature space sk = scaling vector
![Page 13: Multiple Instance Learning](https://reader036.fdocuments.net/reader036/viewer/2022062520/568163f7550346895dd58c97/html5/thumbnails/13.jpg)
Disjunctive Point Concept More complicated concepts are disjunction of
d-single point concepts A bag is positive if at least one of its instances
is in the concept xt1, xt
2 or xtd
![Page 14: Multiple Instance Learning](https://reader036.fdocuments.net/reader036/viewer/2022062520/568163f7550346895dd58c97/html5/thumbnails/14.jpg)
Density Surfaces
![Page 15: Multiple Instance Learning](https://reader036.fdocuments.net/reader036/viewer/2022062520/568163f7550346895dd58c97/html5/thumbnails/15.jpg)
Part II: SVM Algorithms for MIL
![Page 16: Multiple Instance Learning](https://reader036.fdocuments.net/reader036/viewer/2022062520/568163f7550346895dd58c97/html5/thumbnails/16.jpg)
Single Instance Learning MIL SIL-MIL: Single Instance Learning approach
Applies bag’s label to all instances in the bag A normal SVM is trained on the resulting dataset
![Page 17: Multiple Instance Learning](https://reader036.fdocuments.net/reader036/viewer/2022062520/568163f7550346895dd58c97/html5/thumbnails/17.jpg)
Sparse MIL All instances from negative bags are real negative instances Small positive bags are more informative than large positive
bags A bag is represented as the sum of all its instances
normalized by its 1 or 2-norm
![Page 18: Multiple Instance Learning](https://reader036.fdocuments.net/reader036/viewer/2022062520/568163f7550346895dd58c97/html5/thumbnails/18.jpg)
Results Datasets used:
AIMed: sparse dataset created from a corpus of protein-protein interactions. Contains 670 positive and 1,040 negative bags
CBIR: Content Based Image Retrieval domain. The task is to categorize images as to whether they contain an object of interest
MUSK: drug activity dataset. Bags corresponds to molecule, while bag instances correspond to three dimensional conformation of same molecule
TST: text categorization dataset in which MEDLINE articles are represented as bags of overlapping text passages.
![Page 19: Multiple Instance Learning](https://reader036.fdocuments.net/reader036/viewer/2022062520/568163f7550346895dd58c97/html5/thumbnails/19.jpg)
Results
![Page 20: Multiple Instance Learning](https://reader036.fdocuments.net/reader036/viewer/2022062520/568163f7550346895dd58c97/html5/thumbnails/20.jpg)
mi-SVM Instance level classification Treats label instance labels yi as unobserved hidden
variable Goal is to maximize the margin over the unknown instance
labels Suitable for instance classification
![Page 21: Multiple Instance Learning](https://reader036.fdocuments.net/reader036/viewer/2022062520/568163f7550346895dd58c97/html5/thumbnails/21.jpg)
MI-SVM Bag level classification Goal is to maximize the bag margin, which is
The “most positive” instance in case of positive bags
The “least negative” instance in case of negative bags
Suitable for bag classification
![Page 22: Multiple Instance Learning](https://reader036.fdocuments.net/reader036/viewer/2022062520/568163f7550346895dd58c97/html5/thumbnails/22.jpg)
Results: mi-SVM vs. MI-SVM
Corel image data sets
TREC9 document categorization sets
![Page 23: Multiple Instance Learning](https://reader036.fdocuments.net/reader036/viewer/2022062520/568163f7550346895dd58c97/html5/thumbnails/23.jpg)
Some Thoughts Can find multiple positive concepts in a single
bag and learn these concepts? Does varying sizes of negative bags have an
influence on the learning algorithm? Can we re-formulate MIL using Fuzzy Logic?
![Page 24: Multiple Instance Learning](https://reader036.fdocuments.net/reader036/viewer/2022062520/568163f7550346895dd58c97/html5/thumbnails/24.jpg)
References O. Maron and T. Lozano-Pérez, "A framework for multiple-
instance learning," 1998, pp. 570-576.
R. C. Bunescu and R. J. Mooney, "Multiple instance learning for sparse positive bags," 2007, pp. 105-112.
J. Yang, "Review of Multi-Instance Learning and Its applications," 2008.
S. Andrews, et al., "Support vector machines for multiple-instance learning," Advances in neural information processing systems, pp. 577-584, 2003.