Goobi user meeting 2014

30.05.2014

Steff

en Hankiew

icz, intranda GmbH

Goobi development overviewGoobi User Mee?ng, 30.05.2014

1

1. Goobi developments in the last two yearsDevelopments for version 1.9.x and 2.0

2

30.05.2014

Steff

en Hankiew

icz, intranda GmbH

1. Goobi developments -‐ overview

Goobi 1.9.x ‣Workflows ‣Dockets ‣ Proper?es ‣METS-‐Editor ‣ Plugins

3

30.05.2014

Steff

en Hankiew

icz, intranda GmbH

Goobi 2.0 ‣ Infrastructure ‣Workflow ‣METS-‐Editor ‣QA ‣UI

1.1. Goobi 1.9.x -‐ Workflows

4

30.05.2014

Steff

en Hankiew

icz, intranda GmbH

Goobi 1.9.x ‣ Workflows ‣ Dockets ‣ Proper?es ‣ METS-‐Editor ‣ Plugins

Goobi 2.0 ‣ Infrastructure ‣ Workflow ‣ METS-‐Editor ‣ QA ‣ UI


5

30.05.2014

Steff

en Hankiew

icz, intranda GmbH



Import from catalogue

Scanning

Quality control

Image processing

OCR

Structure & metadata

Identifier generation

Export into DMS

Archiving


Automatic image import

Image processing

OCR


Export into DMS

Archiving


6

30.05.2014

Steff

en Hankiew

icz, intranda GmbH




7

30.05.2014

Steff

en Hankiew

icz, intranda GmbH




8

30.05.2014

Steff

en Hankiew

icz, intranda GmbH




9

30.05.2014

Steff

en Hankiew

icz, intranda GmbH




10

30.05.2014

Steff

en Hankiew

icz, intranda GmbH




11

30.05.2014

Steff

en Hankiew

icz, intranda GmbH



1.2. Goobi 1.9.x -‐ Dockets

12

30.05.2014

Steff

en Hankiew

icz, intranda GmbH



12teja17g_PPN718954203

Projekt: VD18_IIPPN digital: 12teja17g_PPN718954203Goobi Identifier: 43014Anlegedatum: 2012-07-05 12:32:34.0Regelsatz: vd18.xmlSignatur: DD98 A 209 (9)

12teja17g_PPN718954203

DD98 A 209 (9)

Öffnungswinkel: X 180° 90°

Farbtiefe: Bitonal Graustufen X Farbe

Stück 9 = 2 Bl. *** Ke

Kiste: 12 ÖW: 180

UB Rostock: UBR_MD17_extern

acclfapaq_729461343

acclfapaq_729461343

Signatur:R.U.-phil 1637 Raue, JohannTitelanfang: Acclamatio Fausta panegyrica

Quam in...ProzessID: 473

Ordnername:

acclfapaq_729461343__[473]

Digitalisierungsparameter:- Oeffnungswinkel_max : 180

Bemerkungen08.11.2012 14:38:05: (Adam, Lisa)

KatalogisierungVD17

f vdn 28:720552DGVK

f ppn 729461343

1.3. Goobi 1.9.x -‐ Properties

13

30.05.2014

Steff

en Hankiew

icz, intranda GmbH

Goobi 1.9.x ‣ Workflows ‣ Dockets ‣ Proper8es ‣ METS-‐Editor ‣ Plugins


1.4. Goobi 1.9.x -‐ METS-‐Editor

14

30.05.2014

Steff

en Hankiew

icz, intranda GmbH



1.4. Goobi 1.9.x -‐ METS-‐Editor

15

30.05.2014

Steff

en Hankiew

icz, intranda GmbH



1.5. Goobi 1.9.x -‐ Plugins

16

30.05.2014

Steff

en Hankiew

icz, intranda GmbH



17

30.05.2014

Steff

en Hankiew

icz, intranda GmbH




18

30.05.2014

Steff

en Hankiew

icz, intranda GmbH




19

30.05.2014

Steff

en Hankiew

icz, intranda GmbH




hVp://intranda.com/goobi/wi? command=addToProcessLog &token=test &stepId=1234 &value=my Debug Message &type=debug

hVp://intranda.com/goobi/wi? command=addToProcessLog &token=test &processId=1234 &value=my Error Message &type=error

closeStep exportDms addStep addProcess deleteProcess version help ucc ucc_upload addToProcessLog runScript processFilter stepFilter findWrongStatus reportProblem

1.6. Goobi 2.0 -‐ Infrastructure

20

30.05.2014

Steff

en Hankiew

icz, intranda GmbH



!

‣ Updated Backend API ‣ JavaServer Faces 2.2 ‣ xhtml instead of jsp ‣ Servlet specifica?on 3.0 / Apache Tomcat 7.0 !

‣ Different database communica?on ‣ Local configura?on files

1.6. Goobi 2.0 -‐ Infrastructure

21

30.05.2014

Steff

en Hankiew

icz, intranda GmbH



1.7. Goobi 2.0 -‐ Workflow

22

30.05.2014

Steff

en Hankiew

icz, intranda GmbH




23

30.05.2014

Steff

en Hankiew

icz, intranda GmbH



Plugin with part UI


24

30.05.2014

Steff

en Hankiew

icz, intranda GmbH



Plugin with fullscreen UI

25

30.05.2014

Steff

en Hankiew

icz, intranda GmbH




1.8. Goobi 2.0 -‐ METS-‐Editor

26

30.05.2014

Steff

en Hankiew

icz, intranda GmbH



1.8. Goobi 2.0 -‐ METS-‐Editor

27

30.05.2014

Steff

en Hankiew

icz, intranda GmbH



1.9. Goobi 2.0 -‐ QA

28

30.05.2014

Steff

en Hankiew

icz, intranda GmbH



1.9. Goobi 2.0 -‐ QA

29

30.05.2014

Steff

en Hankiew

icz, intranda GmbH



1.10. Goobi 2.0 -‐ UI

30

30.05.2014

Steff

en Hankiew

icz, intranda GmbH



1.10. Goobi 2.0 -‐ UI

31

30.05.2014

Steff

en Hankiew

icz, intranda GmbH



1.10. Goobi 2.0 -‐ UI

32

30.05.2014

Steff

en Hankiew

icz, intranda GmbH



1.10. Goobi 2.0 -‐ UI

33

30.05.2014

Steff

en Hankiew

icz, intranda GmbH



1.10. Goobi 2.0 -‐ UI

34

30.05.2014

Steff

en Hankiew

icz, intranda GmbH



1.10. Goobi 2.0 -‐ UI

35

30.05.2014

Steff

en Hankiew

icz, intranda GmbH



1.10. Goobi 2.0 -‐ UI

36

30.05.2014

Steff

en Hankiew

icz, intranda GmbH



1.10. Goobi 2.0 -‐ UI

37

30.05.2014

Steff

en Hankiew

icz, intranda GmbH



1.10. Goobi 2.0 -‐ UI

38

30.05.2014

Steff

en Hankiew

icz, intranda GmbH



‣ OCR

‣ JPEG

‣ JPEG 2000

‣ Jpylyzer

‣ Archiving

‣ Download-‐Jobs

1.10. Goobi 2.0 -‐ UI

39

30.05.2014

Steff

en Hankiew

icz, intranda GmbH



2. How is Goobi used across Europe?Using the same sogware -‐ but doing different stuff

40

30.05.2014

Steff

en Hankiew

icz, intranda GmbH

41

30.05.2014

Steff

en Hankiew

icz, intranda GmbHCommon workflows

‣ Typical steps ‣ Legacy import

2. How is Goobi used across Europe?

Very special cases ‣Material ‣Workflow ‣ Sources ‣ Targets

42

30.05.2014

Steff

en Hankiew

icz, intranda GmbH

2.1. Common workflow

43

30.05.2014

Steff

en Hankiew

icz, intranda GmbH

2.2. Legacy import


Scanning

Quality control

Image processing



Export into DMS

Archiving

Standard workflow Legacy import

Import from source

Image processing



Export into DMS

Archiving

44

30.05.2014

Steff

en Hankiew

icz, intranda GmbH

2.3. Greifswald: OAI-‐Harvester

45

30.05.2014

Steff

en Hankiew

icz, intranda GmbH

2.4. HU Berlin: EOD import

46

30.05.2014

Steff

en Hankiew

icz, intranda GmbH

2.5. SUB Hamburg: Newspapers

47

30.05.2014

Steff

en Hankiew

icz, intranda GmbH

2.6. DIPF Berlin: Catalogue enrichment

48

30.05.2014

Steff

en Hankiew

icz, intranda GmbH

2.7. Leiden: Goobi File Delivery

49

30.05.2014

Steff

en Hankiew

icz, intranda GmbH

2.8. GEI Braunschweig: Enrich content

50

30.05.2014

Steff

en Hankiew

icz, intranda GmbH

2.9. Linz & AK Wien: Deposit copies

51

30.05.2014

Steff

en Hankiew

icz, intranda GmbH

2.10. Wellcome Library: FTP and IA

Content Analyser Deamon analyses if upload is finished, valid and no viruses are found. If all conditions are fulfilled the clean and valid content gets moved into secure area on internal storage

Goobi: FTP upload

Version: 1.1Author: Steffen HankiewiczCopyright: intranda GmbH, 2014

The FTP upload consists of two automatic steps:

Step 1: Content Analyser

The Content Analyser analyses the uploaded content and makes sure, that its upload is finished already. The following scenarios may happen:

1a) The content upload of a folder gets never finishes: After a given time (e.g. one month) the Content Analyser should create an error message and move the folder to a separate storage area 'FTP suspicious upload' where problematic folders can be checked by a human person to clarify the status with the customer

1b) The content is uploaded but a virus is found: The content gets moved into a 'FTP suspicious upload' folder. A human person gets involved to clarify with the customer what to do next

1c) The content folder is successfully uploaded and clean: The content is moved into the secure internal 'FTP Migration Area'.

Step 2: Content Migrator

The Content Migrator checks if in the 'FTP Migration Area' are new folders available that shall be imported into Goobi. During the analyzation of this content the following scenarios may happen:

2a) In case that no Goobi process exists where this content can be assigned to, the content stays in the 'FTP Migration Area' for a given time (e.g. one month)

2b) In case of invalid content, errors or if no matching Goobi process can be found in a given time range (e.g. in one month after uploading) the content is moved into a separate 'FTP suspicious content' folder to be manually checked by the Wellcome stuff; the content folder gets a readable error description message to help the user to understand what kind of error has happened

3c) In case of valid content and an existing Goobi process for this content can be found, the content gets moved into the matching Goobi process folder. Afterwards the workflow step 'Receive content from FTP server' gets closed by the Content Migrator to trigger the next Goobi task for this process (e.g. SDB ingest).

FTP upload service

Public available FTP server writes data into insecure storage area

Customer 1

Customer 2

Customer 3

FTP Server

etc.

DMZ storage

FTP suspicious upload

Content from customer 3



etc.

Internal storage for all Wellcome services

FTP Migration area

etc.




Storage area for

Wellcome Service 1

Content Migrator Deamon analyses uploaded and clean content and triggers data import into Goobi.

Storage area for

Wellcome Service 2

Storage area for

Wellcome Service 3

Storage area for Goobi

FTP suspicious

content

3. What is coming next in Goobi?Currently, s?ll this year and poten?ally soon …

52

30.05.2014

Steff

en Hankiew

icz, intranda GmbH

53

30.05.2014

Steff

en Hankiew

icz, intranda GmbH

3. What is coming next in Goobi?

2.0 3.0

‣ Finish new UI ‣ Add some minor features for usability

‣ Metadata & formats ‣ Scalability ‣ Func?onality enrichment with plugins

‣ Efficiency / Usability ‣ More automa?c ‣ Data enrichment

3.1. Efficiency

54

30.05.2014

Steff

en Hankiew

icz, intranda GmbH

-‐ Layout Wizzard -‐

3.1. Goobi 2.0 -‐ Plugins

55

30.05.2014

Steff

en Hankiew

icz, intranda GmbH


56

30.05.2014

Steff

en Hankiew

icz, intranda GmbH


57

30.05.2014

Steff

en Hankiew

icz, intranda GmbH


58

30.05.2014

Steff

en Hankiew

icz, intranda GmbH


59

30.05.2014

Steff

en Hankiew

icz, intranda GmbH


60

30.05.2014

Steff

en Hankiew

icz, intranda GmbH


61

30.05.2014

Steff

en Hankiew

icz, intranda GmbH

3.2. Data enrichment

62

30.05.2014

Steff

en Hankiew

icz, intranda GmbH

-‐ Authority data -‐


63

30.05.2014

Steff

en Hankiew

icz, intranda GmbH


64

30.05.2014

Steff

en Hankiew

icz, intranda GmbH


65

30.05.2014

Steff

en Hankiew

icz, intranda GmbH


66

30.05.2014

Steff

en Hankiew

icz, intranda GmbH


67

30.05.2014

Steff

en Hankiew

icz, intranda GmbH


68

30.05.2014

Steff

en Hankiew

icz, intranda GmbH


69

30.05.2014

Steff

en Hankiew

icz, intranda GmbH

3.3. Perspectives

70

30.05.2014

Steff

en Hankiew

icz, intranda GmbH

‣Newspaper material ‣Alterna?ve Editors ‣ TEI ‣ (addi?onal) content for web presenta?on ‣ Thumbnails in METS-‐Editor ‣ Indexing of structure data, page assignment and metadata in one single step ‣OLR-‐Support for indexing ‣OCR-‐supported pagina?on

3.3. Perspectives

71

30.05.2014

Steff

en Hankiew

icz, intranda GmbH

‣Quality assurance ‣Quality control in the GUI ‣More automa?c valida?on ‣ Cropping / Deskew

!‣ Controlling ‣ BeVer restric?on model ‣ Sta?s?cs for Non-‐Admins

!‣Op?misa?on ‣ SQL-‐Op?misa?on ‣ Search for slowest workflow steps (Snagging) ‣ Clustering

3.3. Perspectives

72

30.05.2014

Steff

en Hankiew

icz, intranda GmbH

‣Workflows ‣ Start Workflows before the digi?sa?on begins ‣ Status ‚skipped‘ ‣Op?misa?on for the coopera?on between mul?ple Goobis (Data import, Sync2Goobi etc.) ‣ BeVer handling for mass manipula?ons ‣ Extension of search func?onality (incl. informa?ons about the file system)

3.3. Perspectives

73

30.05.2014

Steff

en Hankiew

icz, intranda GmbH

‣Other ‣ IPTC-‐Header ‣Archiving / Repository ‣ Correc?on loops ‣ Correc?ons triggered from external persons ‣GoobiScript as plugins ‣More plugins (Sidebar etc.)

4. Questions

74

30.05.2014

Steff

en Hankiew

icz, intranda GmbH

‣ hVp://www.intranda.com

‣ hVp://www.digiverso.com

!

‣ [email protected]

‣ +49 551 29176100

Ques?ons, feedback and contact

Goobi user meeting 2014

Internet

Transcript of Goobi user meeting 2014