Applying Computer Vision to Art History - John Resig · 2015. 2. 10. · Computer Vision •...
Transcript of Applying Computer Vision to Art History - John Resig · 2015. 2. 10. · Computer Vision •...
Applying Computer Vision to Art History
John Resig - http://ejohn.org/research/ Visiting Researcher, Ritsumeikan University
What “Works” TodayReading license plates, zip codes, checks
Optical Character Recognition
• Tesseract
• https://code.google.com/p/tesseract-ocr/
What “Works” TodayFace recognition
What “Works” TodayRecognition of flat, textured, objects
Computer Vision
• Unsupervised (requires no labeling):
• Comparing an entire image
• Categorizing an image
• Supervised (requires labeling):
• Finding parts of an image
• Finding and categorizing parts of an image
Unsupervised Training
• Requires little-to-no prepping of data
• Can just give the tool a set of images and have it produce results
• Extremely easy to get started, results aren’t always as interesting.
Supervised Training
• Need lots of training data
• Needs to be pre-selected/categorized
• Think: Thousands of images.
• If your collection is smaller than this, perhaps it may not benefit.
• Or you may need crowd sourcing.
• Results can be more interesting:
• “Find all the people in this image”
Image Similarity
• imgSeek (Open Source)
• http://www.imgseek.net/
• TinEye’s MatchEngine
• http://services.tineye.com/MatchEngine
• Both are completely unsupervised. No training data is required.
imgSeek
• Compares entire image.
• Finds similar images, not exact.
• Does not find parts of an image.
• Color sensitive.
Ukiyo-e.org (Using MatchEngine)
• Compares portions of images.
• Finds exact matches.
• Finds images inside other images.
• Color insensitive.
Anonymous Italian Art (Frick PhotoArchive) Using MatchEngine
Conservation
Copies
Partial Image vs. Much Larger Image
Image Portion
Frick 420
420
Zeri 1583642090
Frick 417
417
?
8132a 8132
57129
57134
57130
57138
8131a 8131
?
Image Categorization
• Deep neural networks
• Requires minimal categorization
• Very little user-input required.
• Ersatz
• http://www.ersatzlabs.com/
Requires a lot of training data (thousands of images)
Takes a lot of computers(Not cheap)
The less categories you have, the better.
General Computer Vision
• Ideal for some supervised training problems
• CCV
• http://libccv.org/
• https://github.com/liuliu/ccv
• OpenCV
• http://opencv.org/
Object Detection
Training Caveats
• Requires thousands (if not 10s of thousands) of images
• Will take at least a week to run on a very powerful computer
• Does not work with 3D objects
Learn More about Computer Vision
• Learn more:
• http://cs.brown.edu/courses/csci1430/
• Paper on Frick Computer Vision work:
• http://ejohn.org/research/