IMPACT Interoperability and Evaluation Framework. Clemens Neudecker
-
Upload
biblioteca-nacional-de-espana -
Category
Technology
-
view
695 -
download
0
description
Transcript of IMPACT Interoperability and Evaluation Framework. Clemens Neudecker
![Page 1: IMPACT Interoperability and Evaluation Framework. Clemens Neudecker](https://reader034.fdocuments.net/reader034/viewer/2022052621/557dcbead8b42a63048b50e8/html5/thumbnails/1.jpg)
IMPACT is supported by the European Community under the FP7 ICT Work Programme. The project is coordinated by the National Library of the Netherlands.
IMPACT Interoperability and Evaluation FrameworkClemens Neudecker, National Library of the NetherlandsIMPACT Demo Day, Biblioteca Nacional de España
![Page 2: IMPACT Interoperability and Evaluation Framework. Clemens Neudecker](https://reader034.fdocuments.net/reader034/viewer/2022052621/557dcbead8b42a63048b50e8/html5/thumbnails/2.jpg)
IMPACT is supported by the European Community under the FP7 ICT Work Programme. The project is coordinated by the National Library of the Netherlands.
OCR: A multitude of challenges…I. OCR challenges (gothic fonts, bleed-through, warping, etc.)
![Page 3: IMPACT Interoperability and Evaluation Framework. Clemens Neudecker](https://reader034.fdocuments.net/reader034/viewer/2022052621/557dcbead8b42a63048b50e8/html5/thumbnails/3.jpg)
IMPACT is supported by the European Community under the FP7 ICT Work Programme. The project is coordinated by the National Library of the Netherlands.
OCR: A multitude of challenges…II. Language challenges (spelling variants, inflection, and many more!)
Example: historical variants of the Dutch word ‘wereld’ (world):werelt weerelt wereld weerelds wereldt werelden weereld werrelts waerelds weerlyt wereldts vveerelts waereld weerelden waerelden weerlt werlt werelds sweerels zwerlys swarels swerelts werelts swerrels weirelts tsweerelds werret vverelt werlts werrelt worreld werlden wareld weirelt weireld waerelt werreld werld vvereld weerelts werlde tswerels werreldts weereldt wereldje waereldje weurlt wald weëled
![Page 4: IMPACT Interoperability and Evaluation Framework. Clemens Neudecker](https://reader034.fdocuments.net/reader034/viewer/2022052621/557dcbead8b42a63048b50e8/html5/thumbnails/4.jpg)
IMPACT is supported by the European Community under the FP7 ICT Work Programme. The project is coordinated by the National Library of the Netherlands.
And a multitude of solutions!22 different ‘tools’ from diverse developers:OCR (C++, C#), Image Processing & Lexica (DLL), Command Line Tools (Win/Linux), Java, Ruby, PHP, Perl, etc. + 3rd party software!
“One ring to rule them all...”
→ IMPACT Interoperability Framework (IIF)
![Page 5: IMPACT Interoperability and Evaluation Framework. Clemens Neudecker](https://reader034.fdocuments.net/reader034/viewer/2022052621/557dcbead8b42a63048b50e8/html5/thumbnails/5.jpg)
IMPACT is supported by the European Community under the FP7 ICT Work Programme. The project is coordinated by the National Library of the Netherlands.
Main requirementsBehavioural:
Minimize integration effortMinimize deployment effortMaximize usabilityMaximize scalability
Functional:ModularTransparentExpandableOpen sourcePlatform independent
![Page 6: IMPACT Interoperability and Evaluation Framework. Clemens Neudecker](https://reader034.fdocuments.net/reader034/viewer/2022052621/557dcbead8b42a63048b50e8/html5/thumbnails/6.jpg)
IMPACT is supported by the European Community under the FP7 ICT Work Programme. The project is coordinated by the National Library of the Netherlands.
ArchitectureIMPACT Interoperability Framework: Technologies- Java 6- Generic Web Service Wrapper- Apache Ant/Maven- Apache Tomcat/httpd- Apache Axis2- Apache Synapse- Taverna Workflow Engine
IMPACT Evaluation Framework: Dataset- approx. 5 TB raw data (images, text files, metadata) and growing- Ground truth transcriptions- Evaluation modules
![Page 7: IMPACT Interoperability and Evaluation Framework. Clemens Neudecker](https://reader034.fdocuments.net/reader034/viewer/2022052621/557dcbead8b42a63048b50e8/html5/thumbnails/7.jpg)
IMPACT is supported by the European Community under the FP7 ICT Work Programme. The project is coordinated by the National Library of the Netherlands.
Components I: IIF
Enterprise Service Busreceives (SOAP) requests from users and distributes the load to the availableworker nodes
Main effect: Process parallelization,Load distribution,Fail over
![Page 8: IMPACT Interoperability and Evaluation Framework. Clemens Neudecker](https://reader034.fdocuments.net/reader034/viewer/2022052621/557dcbead8b42a63048b50e8/html5/thumbnails/8.jpg)
IMPACT is supported by the European Community under the FP7 ICT Work Programme. The project is coordinated by the National Library of the Netherlands.
Framework integrationEasy to use generic command line wrapper (open source)
![Page 9: IMPACT Interoperability and Evaluation Framework. Clemens Neudecker](https://reader034.fdocuments.net/reader034/viewer/2022052621/557dcbead8b42a63048b50e8/html5/thumbnails/9.jpg)
IMPACT is supported by the European Community under the FP7 ICT Work Programme. The project is coordinated by the National Library of the Netherlands.
Workflow development
OCR workflow = data pipeline
Building blocks = processing steps (nodes)
Integration = interaction between nodes(mashup)
![Page 10: IMPACT Interoperability and Evaluation Framework. Clemens Neudecker](https://reader034.fdocuments.net/reader034/viewer/2022052621/557dcbead8b42a63048b50e8/html5/thumbnails/10.jpg)
IMPACT is supported by the European Community under the FP7 ICT Work Programme. The project is coordinated by the National Library of the Netherlands.
Workflow managementWeb 2.0 style registry: myExperimentLocal client: Taverna WorkbenchWeb client: project websiteAPI: SOAP/REST
![Page 11: IMPACT Interoperability and Evaluation Framework. Clemens Neudecker](https://reader034.fdocuments.net/reader034/viewer/2022052621/557dcbead8b42a63048b50e8/html5/thumbnails/11.jpg)
IMPACT is supported by the European Community under the FP7 ICT Work Programme. The project is coordinated by the National Library of the Netherlands.
CommunityWeb2.0 style workflow registry
Community of experts
Sharing of resources
Knowledge exchange
A central meeting point for users and researchers
![Page 12: IMPACT Interoperability and Evaluation Framework. Clemens Neudecker](https://reader034.fdocuments.net/reader034/viewer/2022052621/557dcbead8b42a63048b50e8/html5/thumbnails/12.jpg)
IMPACT is supported by the European Community under the FP7 ICT Work Programme. The project is coordinated by the National Library of the Netherlands.
Components II: DatasetDatabase and front end, hosted at the PRIMA research group at University of Salford, School of Computing, United Kingdom
- more than 500.000 images from Digital Libraries- more than 50.000 ground truth representations- up to 10.000 direct access calls per month- 4 TB of space and growing
![Page 13: IMPACT Interoperability and Evaluation Framework. Clemens Neudecker](https://reader034.fdocuments.net/reader034/viewer/2022052621/557dcbead8b42a63048b50e8/html5/thumbnails/13.jpg)
IMPACT is supported by the European Community under the FP7 ICT Work Programme. The project is coordinated by the National Library of the Netherlands.
DatasetAccess to a representative and annotated dataset of significant size, with metadata, ground truth and search facilities
![Page 14: IMPACT Interoperability and Evaluation Framework. Clemens Neudecker](https://reader034.fdocuments.net/reader034/viewer/2022052621/557dcbead8b42a63048b50e8/html5/thumbnails/14.jpg)
IMPACT is supported by the European Community under the FP7 ICT Work Programme. The project is coordinated by the National Library of the Netherlands.
Evaluation featuresText based comparison of result with ground truth, using Levenshtein distance methodLayout based comparison of result with ground truth,using the Page Analysis And Ground Truth Elements FrameworkExample:
![Page 15: IMPACT Interoperability and Evaluation Framework. Clemens Neudecker](https://reader034.fdocuments.net/reader034/viewer/2022052621/557dcbead8b42a63048b50e8/html5/thumbnails/15.jpg)
IMPACT is supported by the European Community under the FP7 ICT Work Programme. The project is coordinated by the National Library of the Netherlands.
![Page 16: IMPACT Interoperability and Evaluation Framework. Clemens Neudecker](https://reader034.fdocuments.net/reader034/viewer/2022052621/557dcbead8b42a63048b50e8/html5/thumbnails/16.jpg)
IMPACT is supported by the European Community under the FP7 ICT Work Programme. The project is coordinated by the National Library of the Netherlands.
Ground-Truthing Tools
Aletheia
FineReaderPAGE Exporter
GT Validator
GT Normalizer
16
![Page 17: IMPACT Interoperability and Evaluation Framework. Clemens Neudecker](https://reader034.fdocuments.net/reader034/viewer/2022052621/557dcbead8b42a63048b50e8/html5/thumbnails/17.jpg)
IMPACT is supported by the European Community under the FP7 ICT Work Programme. The project is coordinated by the National Library of the Netherlands.
![Page 18: IMPACT Interoperability and Evaluation Framework. Clemens Neudecker](https://reader034.fdocuments.net/reader034/viewer/2022052621/557dcbead8b42a63048b50e8/html5/thumbnails/18.jpg)
IMPACT is supported by the European Community under the FP7 ICT Work Programme. The project is coordinated by the National Library of the Netherlands.
18
Partial MissMiss
Merge
Measures – Segmentation Errors
Split
Ground Truth
Segmentation Result
Mis-classi-fication
Paragraph
Caption
![Page 19: IMPACT Interoperability and Evaluation Framework. Clemens Neudecker](https://reader034.fdocuments.net/reader034/viewer/2022052621/557dcbead8b42a63048b50e8/html5/thumbnails/19.jpg)
IMPACT is supported by the European Community under the FP7 ICT Work Programme. The project is coordinated by the National Library of the Netherlands.
OCR Accuracy
![Page 20: IMPACT Interoperability and Evaluation Framework. Clemens Neudecker](https://reader034.fdocuments.net/reader034/viewer/2022052621/557dcbead8b42a63048b50e8/html5/thumbnails/20.jpg)
IMPACT is supported by the European Community under the FP7 ICT Work Programme. The project is coordinated by the National Library of the Netherlands.
Thank you! Questions?