R2-D2 analysed your collection images and here's what he found
-
Upload
paul-rowe -
Category
Technology
-
view
209 -
download
0
Transcript of R2-D2 analysed your collection images and here's what he found
![Page 1: R2-D2 analysed your collection images and here's what he found](https://reader030.fdocuments.net/reader030/viewer/2022020411/5a6e9ae27f8b9a42298b5047/html5/thumbnails/1.jpg)
R2-D2 analysed your collection images and here’s what he found
Paul Rowe
National Digital Forum, Nov 2017
![Page 2: R2-D2 analysed your collection images and here's what he found](https://reader030.fdocuments.net/reader030/viewer/2022020411/5a6e9ae27f8b9a42298b5047/html5/thumbnails/2.jpg)
Why do it?Provide access to data already there.
![Page 3: R2-D2 analysed your collection images and here's what he found](https://reader030.fdocuments.net/reader030/viewer/2022020411/5a6e9ae27f8b9a42298b5047/html5/thumbnails/3.jpg)
Create text data that we don’t have time to enter manually
![Page 4: R2-D2 analysed your collection images and here's what he found](https://reader030.fdocuments.net/reader030/viewer/2022020411/5a6e9ae27f8b9a42298b5047/html5/thumbnails/4.jpg)
Find patterns: Colours, lines, light, space
![Page 5: R2-D2 analysed your collection images and here's what he found](https://reader030.fdocuments.net/reader030/viewer/2022020411/5a6e9ae27f8b9a42298b5047/html5/thumbnails/5.jpg)
Facial recognition
![Page 6: R2-D2 analysed your collection images and here's what he found](https://reader030.fdocuments.net/reader030/viewer/2022020411/5a6e9ae27f8b9a42298b5047/html5/thumbnails/6.jpg)
Youtube – Closed Captions
![Page 7: R2-D2 analysed your collection images and here's what he found](https://reader030.fdocuments.net/reader030/viewer/2022020411/5a6e9ae27f8b9a42298b5047/html5/thumbnails/7.jpg)
IBM Watson speech to text
![Page 8: R2-D2 analysed your collection images and here's what he found](https://reader030.fdocuments.net/reader030/viewer/2022020411/5a6e9ae27f8b9a42298b5047/html5/thumbnails/8.jpg)
Image analysis isn’t easy
![Page 9: R2-D2 analysed your collection images and here's what he found](https://reader030.fdocuments.net/reader030/viewer/2022020411/5a6e9ae27f8b9a42298b5047/html5/thumbnails/9.jpg)
Cogapp test siteMicrosoft / Google / Clarifai
Nationalmuseum, Sweden: Painting - "Hedvig Sofia, princess of Sweden"
![Page 10: R2-D2 analysed your collection images and here's what he found](https://reader030.fdocuments.net/reader030/viewer/2022020411/5a6e9ae27f8b9a42298b5047/html5/thumbnails/10.jpg)
Captioning is the weakest link
Nationalmuseum, Sweden: Painting
Madonna with child
![Page 11: R2-D2 analysed your collection images and here's what he found](https://reader030.fdocuments.net/reader030/viewer/2022020411/5a6e9ae27f8b9a42298b5047/html5/thumbnails/11.jpg)
Google Vision tags
![Page 12: R2-D2 analysed your collection images and here's what he found](https://reader030.fdocuments.net/reader030/viewer/2022020411/5a6e9ae27f8b9a42298b5047/html5/thumbnails/12.jpg)
Unexpected new connections
![Page 13: R2-D2 analysed your collection images and here's what he found](https://reader030.fdocuments.net/reader030/viewer/2022020411/5a6e9ae27f8b9a42298b5047/html5/thumbnails/13.jpg)
… and the captions can be surprisingly precise
Nationalmuseum, Sweden - Painting, Landskap met kor
![Page 14: R2-D2 analysed your collection images and here's what he found](https://reader030.fdocuments.net/reader030/viewer/2022020411/5a6e9ae27f8b9a42298b5047/html5/thumbnails/14.jpg)
Text analysis
http://ketechristchurch.peoplesnetworknz.info/places_and_streets/images/show/
18306-action-figure-paste-ups-greedo#.WgtonXZpHs0
![Page 15: R2-D2 analysed your collection images and here's what he found](https://reader030.fdocuments.net/reader030/viewer/2022020411/5a6e9ae27f8b9a42298b5047/html5/thumbnails/15.jpg)
Colour Analysis
![Page 16: R2-D2 analysed your collection images and here's what he found](https://reader030.fdocuments.net/reader030/viewer/2022020411/5a6e9ae27f8b9a42298b5047/html5/thumbnails/16.jpg)
Mapping to smaller palette
![Page 17: R2-D2 analysed your collection images and here's what he found](https://reader030.fdocuments.net/reader030/viewer/2022020411/5a6e9ae27f8b9a42298b5047/html5/thumbnails/17.jpg)
Combine it all and find “house AND white AND landscape”
![Page 18: R2-D2 analysed your collection images and here's what he found](https://reader030.fdocuments.net/reader030/viewer/2022020411/5a6e9ae27f8b9a42298b5047/html5/thumbnails/18.jpg)
These are the droids you’re looking forWebsites
Barnes Foundation: https://collection.barnesfoundation.org/
Cogapp image tagging test site: http://labs.cogapp.com/iiif-ml/
Invisible Australians: http://invisibleaustralians.org/faces/
Sarjeant Gallery: https://collection.sarjeant.org.nz
Tools
Artrendex ArtPI: http://www.artrendex.com/
Clarifai: https://www.clarifai.com/demo
Color Thief: http://lokeshdhakar.com/projects/color-thief/
Google Cloud Vision API: https://cloud.google.com/vision/docs/
IBM Watson Speech to Text: https://www.ibm.com/watson/services/speech-to-text/
Microsoft Azure Cognitive Services: https://azure.microsoft.com/en-us/try/cognitive-services/
OpenCV Library: https://sourceforge.net/projects/opencvlibrary/
![Page 19: R2-D2 analysed your collection images and here's what he found](https://reader030.fdocuments.net/reader030/viewer/2022020411/5a6e9ae27f8b9a42298b5047/html5/thumbnails/19.jpg)
“That’s no moon”
Paul Rowe, Vernon Systems
@armchair_caver
Generated Subject Tags:
Moon Planet
Satellite Space