Dam It !

Larry Gall

Peabody Museum of Natural HistoryYale University

Dam It !

Dam It !

You crashed my EMu client

again, you %}&$ !

It wasn’t me, you

#@^* loser !

Dam It !

You crashed my EMu client

again, you %}&$ !

It wasn’t me, you

#@^* loser !

Dam It !

Dam It ! noun

Dam It ! noun

Dam It ! noun

amAM

Dam It ! noun

amAM

Dam It ! noun

amAMS^

About DAM Peabody

About DAM Peabody

DAM Brief DAM Review

About DAM Peabody

DAM Brief DAM Review

DAM Yale & DAM EMu

Peabody CollectionsCounts & Functional Cataloguing Unit

• Anthropology 325,000 Lot• Botany 350,000 Individual• Entomology 950,000 Lot / Individual• Invertebrate Paleontology 325,000 Lot• Invertebrate Zoology 325,000 Lot• Mineralogy 35,000 Individual• Paleobotany 150,000 Individual• Scientific Instruments 4,000 Individual• Vertebrate Paleontology 125,000 Individual• Vertebrate Zoology 185,000 Lot / Individual

2.7 million database-able units => ~12 million items

Peabody CollectionsFunctional Units Databased

• Anthropology 325,000 95 % • Botany 350,000 22 %• Entomology 950,000 16 % • Invertebrate Paleontology 325,000 80 % • Invertebrate Zoology 325,000 31 % • Mineralogy 35,000 85 %• Paleobotany 150,000 60 %• Scientific Instruments 4,000 100 %• Vertebrate Paleontology 125,000 60 %• Vertebrate Zoology 185,000 95 %

1,331,000 of 2.7 million => 49 % overall

About DAM Peabody

About DAM Peabody

DAM Brief DAM Review

DAM: management tasks and decisions surrounding the ingestion, annotation, cataloguing, storage, retrieval and distribution of digital assets.

MAM: sub-category of DAM, mainly for audio, video and other media content.

ECM: enterprise content management describes solutions which address similar features but in a wider range of industries

or applications.

DAM: management tasks and decisions surrounding the ingestion, annotation, cataloguing, storage, retrieval and distribution of digital assets.

DAM: management tasks and decisions surrounding the ingestion, annotation, cataloguing, storage, retrieval and distribution of digital assets.

MAM: sub-category of DAM, mainly for digital photographs, audio, video and other media content.

DAM: management tasks and decisions surrounding the ingestion, annotation, cataloguing, storage, retrieval and distribution of digital assets.

MAM: sub-category of DAM, mainly for digital photographs, audio, video and other media content.

DAM: management tasks and decisions surrounding the ingestion, annotation, cataloguing, storage, retrieval and distribution of digital assets.

MAM: sub-category of DAM, mainly for digital photographs, audio, video and other media content.

DAM: management tasks and decisions surrounding the ingestion, annotation, cataloguing, storage, retrieval and distribution of digital assets.

MAM: sub-category of DAM, mainly for digital photographs, audio, video and other media content.

[CIA] assets

DAM: management tasks and decisions surrounding the ingestion, annotation, cataloguing, storage, retrieval and distribution of digital assets.

MAM: sub-category of DAM, mainly for digital photographs, audio, video and other media content.

DAM: management tasks and decisions surrounding the ingestion, annotation, cataloguing, storage, retrieval and distribution of digital assets.

MAM: sub-category of DAM, mainly for digital photographs, audio, video and other media content.

Asset preservation,

[CIA] assets

DAM: management tasks and decisions surrounding the ingestion, annotation, cataloguing, storage, retrieval and distribution of digital assets.

MAM: sub-category of DAM, mainly for digital photographs, audio, video and other media content.

Asset preservation, not deletion

[CIA] assets

DAM: management tasks and decisions surrounding the ingestion, annotation, cataloguing, storage, retrieval and distribution of digital assets.

MAM: sub-category of DAM, mainly for digital photographs, audio, video and other media content.

DAM: management tasks and decisions surrounding the ingestion, annotation, cataloguing, storage, retrieval and distribution of digital assets.

MAM: sub-category of DAM, mainly for digital photographs, audio, video and other media content.

DAM: management tasks and decisions surrounding the ingestion, annotation, cataloguing, storage, retrieval and distribution of digital assets.

MAM: sub-category of DAM, mainly for digital photographs, audio, video and other media content.

ECM: enterprise content management describes solutions which address similar features but in a wider range of industries

or applications.

Large heavyweight DAMs, small lightweight DAMs

Large heavyweight DAMs, small lightweight DAMs

Large heavyweight DAMs, small lightweight DAMs

DAM architectures complex & variable

DAM architectures complex & variable

DAM architectures complex & variable - DAM vendor/integrator relevant

DAM architectures complex & variable - DAM vendor/integrator relevant

DAM architectures complex & variable - DAM vendor/integrator relevant

DAM architectures complex & variable - DAM vendor/integrator relevant

DAM architectures complex & variable - DAM vendor/integrator relevant

About DAM Peabody

DAM Brief DAM Review

About DAM Peabody

DAM Brief DAM Review

DAM Yale & DAM EMu

Motivation - why give a DAM ?

Motivation - why give a DAM ?

Yale actively pursuing a campus-wide strategy for all digital assets whosearchitecture includes DAM and cross-domain discovery/search mechanisms Get YPM assets into new contexts for scholarly engagement; cost share

Place YPM non-biological collections in proximity to their Yale peers Items in e.g., YPM Anthropology and Instruments Departments more closely related to Art Department & Art Gallery than to YPM biological collections

Integrate all of YPM’s digital assets into a common environment Digital collections assets managed in EMu, but no such “curatorial system” exists for non-collections assets (e.g., Events, Exhibits, Public Education) as these are scattered widely on CDs, hard drives, shoe boxes, et al.

Motivation - why give a DAM ?

Yale actively pursuing a campus-wide strategy for all digital assets whosearchitecture includes DAM and cross-domain discovery/search mechanisms Get YPM assets into new contexts for scholarly engagement; cost share

Place YPM non-biological collections in proximity to their Yale peers Items in e.g., YPM Anthropology and Instruments Departments more closely related to Art Department & Art Gallery than to YPM biological collections

Integrate all of YPM’s digital assets into a common environment Digital collections assets managed in EMu, but no such “curatorial system” exists for non-collections assets (e.g., Events, Exhibits, Public Education) as these are scattered widely on CDs, hard drives, shoe boxes, et al.

Motivation - why give a DAM ?

Yale actively pursuing a campus-wide strategy for all digital assets whosearchitecture includes DAM and cross-domain discovery/search mechanisms Get YPM assets into new contexts for scholarly engagement; cost share

Place YPM non-biological collections in proximity to their Yale peers Items in e.g., YPM Anthropology and Instruments Departments more closely related to Art Department & Art Gallery than to YPM biological collections

Integrate all of YPM’s digital assets into a common environment Digital collections assets managed in EMu, but no such “curatorial system” exists for non-collections assets (e.g., Events, Exhibits, Public Education) as these are scattered widely on CDs, hard drives, shoe boxes, et al.

Motivation - why give a DAM ?

Yale actively pursuing a campus-wide strategy for all digital assets whosearchitecture includes DAM and cross-domain discovery/search mechanisms Get YPM assets into new contexts for scholarly engagement; cost share

Place YPM non-biological collections in proximity to their Yale peers Items in e.g., YPM Anthropology and Instruments Departments more closely related to Art Department & Art Gallery than to YPM biological collections

Integrate all of YPM’s digital assets into a common environment Digital collections assets managed in EMu, but no such “curatorial system” exists for non-collections assets (e.g., Events, Exhibits, Public Education) as these are scattered widely on CDs, hard drives, shoe boxes, et al.

Motivation - why give a DAM ?

Yale actively pursuing a campus-wide strategy for all digital assets whosearchitecture includes DAM and cross-domain discovery/search mechanisms Get YPM assets into new contexts for scholarly engagement; cost share

Place YPM non-biological collections in proximity to their Yale peers Items in e.g., YPM Anthropology and Instruments Departments more closely related to Art Department & Art Gallery than to YPM biological collections

Integrate all of YPM’s digital assets into a common environment Digital collections assets managed in EMu, but no such “curatorial system” exists for non-collections assets (e.g., Events, Exhibits, Public Education) as these are scattered widely on CDs, hard drives, shoe boxes, et al.

Motivation - why give a DAM ?

Yale actively pursuing a campus-wide strategy for all digital assets whosearchitecture includes DAM and cross-domain discovery/search mechanisms Get YPM assets into new contexts for scholarly engagement; cost share

Place YPM non-biological collections in proximity to their Yale peers Items in e.g., YPM Anthropology and Instruments Departments more closely related to Art Department & Art Gallery than to YPM biological collections

Integrate all of YPM’s digital assets into a common environment Digital collections assets managed in EMu, but no such “curatorial system” exists for non-collections assets (e.g., Events, Exhibits, Public Education) as these are scattered widely on CDs, hard drives, shoe boxes, et al.

So, which DAM did Yale select ?

YUAG YCBA

YUAG YCBA

xxxxxxxxxxxxxx

YUAG YCBA

Artesia application programming interface (API) was well exposed

xxxxxxxxxxxxxx

YUAG YCBA

Artesia application programming interface (API) was well exposed

About DAM Peabody

DAM Brief DAM Review

DAM Yale & DAM EMu

Synopsis of the path to

YPM

Synopsis of the path to

YPM

DAM Politics^

Only a small percentage of digital projects are using any recognized metadata standards

Long-term maintenance and support is a critical but largely unrecognized need 

No current mechanism exists for disseminating information on existing digital projects

Only a small percentage of digital projects are using any recognized metadata standards

Long-term maintenance and support is a critical but largely unrecognized need 

No current mechanism exists for disseminating information on existing digital projects

Only a small percentage of digital projects are using any recognized metadata standards

Long-term maintenance and support is a critical but largely unrecognized need 

No current mechanism exists for disseminating information on existing digital projects

Only a small percentage of digital projects are using any recognized metadata standards

Long-term maintenance and support is a critical but largely unrecognized need 

No current mechanism exists for disseminating information on existing digital projects

Only a small percentage of digital projects are using any recognized metadata standards

Long-term maintenance and support is a critical but largely unrecognized need 

No current mechanism exists for disseminating information on existing digital projects

Only a small percentage of digital projects are using any recognized metadata standards

Long-term maintenance and support is a critical but largely unrecognized need 

No current mechanism exists for disseminating information on existing digital projects

Only a small percentage of digital projects are using any recognized metadata standards

Long-term maintenance and support is a critical but largely unrecognized need 

No current mechanism exists for disseminating information on existing digital projects

Yale Digital Commons (YDC)

2009-2010

2009-2010

Yale-wide partnership led by ODAI between Museums, Libraries …

YUAG YCBA

YPM YUL

YUAG YCBA

YPM YUL

All partners looking for lots of asset storage and cross-domain asset discovery

YUAG YCBA

YPM YUL

All partners looking for lots of asset storage and cross-domain asset discovery

Essential Beyond Critical

Important

YUAG YCBA

YPM

Phase I partners

Cross Collections Discovery (CCD)

Cross Collections Discovery (CCD)

OAI-PMH

Cross Collections Discovery (CCD)

Harvest /store XML from native schemas (e.g., DwC, CDWA)Crosswalk all schemas to a “Yale CCD Common Discovery Model”Deliver Yale collections items in one-stop-shopping environment

OAI-PMH

Cross Collections Discovery (CCD)

OAI-PMH

Q1 2010BETA

Harvest /store XML from native schemas (e.g., DwC, CDWA)Crosswalk all schemas to a “Yale CCD Common Discovery Model”Deliver Yale collections items in one-stop-shopping environment

Artesia DAM

Artesia DAM

Harvest /store XML into a common Artesia-specific modelThat model mirrors “Yale CCD Common Discovery Model”Use Library of Congress BagIT to deliver assets to DAM

Artesia DAM

Harvest /store XML into a common Artesia-specific modelThat model mirrors “Yale CCD Common Discovery Model”Use Library of Congress BagIT to deliver assets to DAM

Q2 2010

Artesia DAM

Harvest /store XML into a common Artesia-specific modelThat model mirrors “Yale CCD Common Discovery Model”Use Library of Congress BagIT to deliver assets to DAM

Q2 2010

Artesia DAM

Harvest /store XML into a common Artesia-specific modelThat model mirrors “Yale CCD Common Discovery Model”Use Library of Congress BagIT to deliver assets to DAM

Q2 2010

ODAI adopted BagIt to movestuff around the entire YDC

XML

XML

OAI-PMH

MD5

Tag

“hot folder” in the DAM

where an ingestmonitor grabs it

“Holey” bag

“Holey” bag

“Holey” bag

“Holey” bag

“Holey” bag

“Holey” bag

^&*($

Select a multimedia IRNSelect operation (e.g., new, update)Decide contents of bag based on operationBuild the bag using BagItDrop bag into DAM hot folderDAM monitor ingests the bag/assetDAM monitor provides logging/messagesRetrieve and parse the messages for the bagRound-trip DAM info into multimedia IRN

^&*($

Select a multimedia IRNSelect operation (e.g., new, update)Decide contents of bag based on operationBuild the bag using BagItDrop bag into DAM hot folderDAM monitor ingests the bag/assetDAM monitor provides logging/messagesRetrieve and parse the messages for the bagRound-trip DAM info into multimedia IRN

^&*($

Select a multimedia IRNSelect operation (e.g., new, update)Decide contents of bag based on operationBuild the bag using BagItDrop bag into DAM hot folderDAM monitor ingests the bag/assetDAM monitor provides logging/messagesRetrieve and parse the messages for the bagRound-trip DAM info into multimedia IRN

^&*($

Select a multimedia IRNSelect operation (e.g., new, update)Decide contents of bag based on operationBuild the bag using BagItDrop bag into DAM hot folderDAM monitor ingests the bag/assetDAM monitor provides logging/messagesRetrieve and parse the messages for the bagRound-trip DAM info into multimedia IRN

^&*($

Select a multimedia IRNSelect operation (e.g., new, update)Decide contents of bag based on operationBuild the bag using BagItDrop bag into DAM hot folderDAM monitor ingests the bag/assetDAM monitor provides logging/messagesRetrieve and parse the messages for the bagRound-trip DAM info into multimedia IRN

^&*($

Select a multimedia IRNSelect operation (e.g., new, update)Decide contents of bag based on operationBuild the bag using BagItDrop bag into DAM hot folderDAM monitor ingests the bag/assetDAM monitor provides logging/messagesRetrieve and parse the messages for the bagRound-trip DAM info into multimedia IRN

^&*($

Select a multimedia IRNSelect operation (e.g., new, update)Decide contents of bag based on operationBuild the bag using BagItDrop bag into DAM hot folderDAM monitor ingests the bag/assetDAM monitor provides logging/messagesRetrieve and parse the messages for the bagRound-trip DAM info into multimedia IRN

^&*($

Select a multimedia IRNSelect operation (e.g., new, update)Decide contents of bag based on operationBuild the bag using BagItDrop bag into DAM hot folderDAM monitor ingests the bag/assetDAM monitor provides logging/messagesRetrieve and parse the messages for the bagRound-trip DAM info into multimedia IRN

^&*($

Select a multimedia IRNSelect operation (e.g., new, update)Decide contents of bag based on operationBuild the bag using BagItDrop bag into DAM hot folderDAM monitor ingests the bag/assetDAM monitor provides logging/messagesRetrieve and parse the messages for the bagRound-trip DAM info into multimedia IRN

^&*($

Select a multimedia IRNSelect operation (e.g., new, update)Decide contents of bag based on operationBuild the bag using BagItDrop bag into DAM hot folderDAM monitor ingests the bag/assetDAM monitor provides logging/messagesRetrieve and parse the messages for the bagRound-trip DAM info into multimedia IRN

^&*($

Select a multimedia IRNSelect operation (e.g., new, update)Decide contents of bag based on operationBuild the bag using BagItDrop bag into DAM hot folderDAM monitor ingests the bag/assetDAM monitor provides logging/messagesRetrieve and parse the messages for the bagRound-trip DAM info into multimedia IRN

^&*($ Resize the EMu copy ofasset to small derivative

YUAG YCBA

YPM

YUAG YCBA

YPM

Since early mid summer,>250,000 assets ingested,many exceeding 100 mB

YUAG YCBA

YPM

2008-2010 DAM timeframewas not without anxiety…

YUAG YCBA

YPM

YUAG YCBA

YPM

YUAG YCBA

YPM

YUAG YCBA

YPM

YUAG YCBA

YPM

YUAG YCBA

YPM

YUAG YCBA

YPM

Hey, I’ve seen her before …

YUAG YCBA

YPM

YUAG YCBA

YPM

YaleDigitalSilos

YaleDigitalSilosX

Dam It !

Dam It !

^&*($

Dam It !

^&*($

Dam It !

^&*($@%!+