1 ArrayExpress Ugis Sarkans, EBI. 2 Overview Underlying standards –MIAME –MAGE* Data submission...
-
Upload
nelson-shaw -
Category
Documents
-
view
219 -
download
0
Transcript of 1 ArrayExpress Ugis Sarkans, EBI. 2 Overview Underlying standards –MIAME –MAGE* Data submission...
![Page 1: 1 ArrayExpress Ugis Sarkans, EBI. 2 Overview Underlying standards –MIAME –MAGE* Data submission Data access –annotations –actual data –array design descriptions.](https://reader036.fdocuments.net/reader036/viewer/2022062410/5697bfa51a28abf838c97ab7/html5/thumbnails/1.jpg)
1
ArrayExpress
Ugis Sarkans, EBI
![Page 2: 1 ArrayExpress Ugis Sarkans, EBI. 2 Overview Underlying standards –MIAME –MAGE* Data submission Data access –annotations –actual data –array design descriptions.](https://reader036.fdocuments.net/reader036/viewer/2022062410/5697bfa51a28abf838c97ab7/html5/thumbnails/2.jpg)
2
Overview• Underlying standards
– MIAME– MAGE*
• Data submission• Data access
– annotations– actual data– array design descriptions
• Some technical details• Future developments
![Page 3: 1 ArrayExpress Ugis Sarkans, EBI. 2 Overview Underlying standards –MIAME –MAGE* Data submission Data access –annotations –actual data –array design descriptions.](https://reader036.fdocuments.net/reader036/viewer/2022062410/5697bfa51a28abf838c97ab7/html5/thumbnails/3.jpg)
3
What information should be exchanged?
• MIAME - Minimum Information About a Microarray Experiment– informal specification– paper published in Nature Genetics– goal - to initiate discussion:
• which details are important and which may not be
– ArrayExpress can store MIAME data (and more)
![Page 4: 1 ArrayExpress Ugis Sarkans, EBI. 2 Overview Underlying standards –MIAME –MAGE* Data submission Data access –annotations –actual data –array design descriptions.](https://reader036.fdocuments.net/reader036/viewer/2022062410/5697bfa51a28abf838c97ab7/html5/thumbnails/4.jpg)
4
MAGE-OM
• MAGE-OM: MicroArray Gene Expression Object Model– in January 2002 became an “adopted” OMG
specification– January to August 2002 - finalization process– in September became an “available” specification– should be set in stone for the next 2 years– thinking about MAGE v2 started
• user feedback• support for other types of functional genomics data• more precise handling of data manipulation
![Page 5: 1 ArrayExpress Ugis Sarkans, EBI. 2 Overview Underlying standards –MIAME –MAGE* Data submission Data access –annotations –actual data –array design descriptions.](https://reader036.fdocuments.net/reader036/viewer/2022062410/5697bfa51a28abf838c97ab7/html5/thumbnails/5.jpg)
5BioEvent
Experiment
ArrayDesign
BioMaterialBioAssayData
BioAssay
DesignElement
UML Packages of MAGE
HigherLevelAnalysis
BioSequence
Array QuantitationType
DescriptionProtocol
MeasurementAuditAndSecurity
BQS
what was used what was done results
miscellaneous
![Page 6: 1 ArrayExpress Ugis Sarkans, EBI. 2 Overview Underlying standards –MIAME –MAGE* Data submission Data access –annotations –actual data –array design descriptions.](https://reader036.fdocuments.net/reader036/viewer/2022062410/5697bfa51a28abf838c97ab7/html5/thumbnails/6.jpg)
6
MAGE-ML
• MAGE-ML: MicroArray Gene Expression Markup Language– generated from MAGE-OM, therefore
evolved automatically– translation from Jan 2002 to Sep 2002
DTD quite easy
![Page 7: 1 ArrayExpress Ugis Sarkans, EBI. 2 Overview Underlying standards –MIAME –MAGE* Data submission Data access –annotations –actual data –array design descriptions.](https://reader036.fdocuments.net/reader036/viewer/2022062410/5697bfa51a28abf838c97ab7/html5/thumbnails/7.jpg)
7
ArrayExpress: data• currently - 9 experiments, 4 array designs:
– from EMBL - human, yeast– from Sanger - pombe
• coming:– array descriptions: Affymetrix, Agilent– labs: TIGR, Utrecht, more from Sanger, ...– export from existing DBs: SMD, RAD– tools - MAGE-ML export: Jexpress, BASE, ...– ILSI project
• journal requirements: Nature, Lancet, ...
![Page 8: 1 ArrayExpress Ugis Sarkans, EBI. 2 Overview Underlying standards –MIAME –MAGE* Data submission Data access –annotations –actual data –array design descriptions.](https://reader036.fdocuments.net/reader036/viewer/2022062410/5697bfa51a28abf838c97ab7/html5/thumbnails/8.jpg)
8
Help with MAGE-ML: MAGEstk
• MAGE-ML - the only way of getting data into ArrayExpress
• MAGEstk: MicroArray Gene Expression Software ToolKit– Jamboree IV in Stanford, beginning of
December– used in MIAMExpress (MAGE-ML export)
![Page 9: 1 ArrayExpress Ugis Sarkans, EBI. 2 Overview Underlying standards –MIAME –MAGE* Data submission Data access –annotations –actual data –array design descriptions.](https://reader036.fdocuments.net/reader036/viewer/2022062410/5697bfa51a28abf838c97ab7/html5/thumbnails/9.jpg)
9
MAGEstk
• Programming APIs• Mapping of MAGE-OM to language-
specific OMs• API’s are automatically generated from
the OM specifications– get/set methods for associations– get/set methods for attributes
• XML <=> language-specific OM marshallers/unmarshallers - also automatically generated
![Page 10: 1 ArrayExpress Ugis Sarkans, EBI. 2 Overview Underlying standards –MIAME –MAGE* Data submission Data access –annotations –actual data –array design descriptions.](https://reader036.fdocuments.net/reader036/viewer/2022062410/5697bfa51a28abf838c97ab7/html5/thumbnails/10.jpg)
10
MAGEstk (cont.)
• Use opensource/standard modules/packages– Xerces, JDBC, etc.
• Implementation in Java, C++, Perl, Python
• database access modules on top of these APIs– Postgres schema– DB access layer
• annotation tools - planned
![Page 11: 1 ArrayExpress Ugis Sarkans, EBI. 2 Overview Underlying standards –MIAME –MAGE* Data submission Data access –annotations –actual data –array design descriptions.](https://reader036.fdocuments.net/reader036/viewer/2022062410/5697bfa51a28abf838c97ab7/html5/thumbnails/11.jpg)
11
ArrayExpress data retrieval
• main objective - help in finding and initial exploration of data; download for detailed analysis
• data repository (now) + data warehouse (in development)
![Page 12: 1 ArrayExpress Ugis Sarkans, EBI. 2 Overview Underlying standards –MIAME –MAGE* Data submission Data access –annotations –actual data –array design descriptions.](https://reader036.fdocuments.net/reader036/viewer/2022062410/5697bfa51a28abf838c97ab7/html5/thumbnails/12.jpg)
12
Array Design- accession
- name
Protocol- accession
Experiment- accession
Organisation- name
Array
Species Sample
Hybridisation
ExperimentDesign
ExperimentType
ExperimentalFactor
Person- last name
Protocol Type
Queries - logical structure
![Page 13: 1 ArrayExpress Ugis Sarkans, EBI. 2 Overview Underlying standards –MIAME –MAGE* Data submission Data access –annotations –actual data –array design descriptions.](https://reader036.fdocuments.net/reader036/viewer/2022062410/5697bfa51a28abf838c97ab7/html5/thumbnails/13.jpg)
13
Query form
![Page 14: 1 ArrayExpress Ugis Sarkans, EBI. 2 Overview Underlying standards –MIAME –MAGE* Data submission Data access –annotations –actual data –array design descriptions.](https://reader036.fdocuments.net/reader036/viewer/2022062410/5697bfa51a28abf838c97ab7/html5/thumbnails/14.jpg)
14
Annotation browsing
![Page 15: 1 ArrayExpress Ugis Sarkans, EBI. 2 Overview Underlying standards –MIAME –MAGE* Data submission Data access –annotations –actual data –array design descriptions.](https://reader036.fdocuments.net/reader036/viewer/2022062410/5697bfa51a28abf838c97ab7/html5/thumbnails/15.jpg)
15
Data representation
spots
measurements
BioAssays (hybridizations, data transformations)
QuantitationTypes (signal intensity, ratio etc.)
DesignElements (spots, genes)
in MAGE/ArrayExpress
in Expression Profiler
![Page 16: 1 ArrayExpress Ugis Sarkans, EBI. 2 Overview Underlying standards –MIAME –MAGE* Data submission Data access –annotations –actual data –array design descriptions.](https://reader036.fdocuments.net/reader036/viewer/2022062410/5697bfa51a28abf838c97ab7/html5/thumbnails/16.jpg)
16
Exporting data to Expression Profiler
BioAssays (hybridizations, data transformations)
QuantitationTypes (signal intensity, ratio etc.)
DesignElements (spots)
BioAssayData1
BioAssayData2
select BioAssayData cubes
select QuantitationTypes
select BioAssaysDesignElements
(QT,BA) pairs
![Page 17: 1 ArrayExpress Ugis Sarkans, EBI. 2 Overview Underlying standards –MIAME –MAGE* Data submission Data access –annotations –actual data –array design descriptions.](https://reader036.fdocuments.net/reader036/viewer/2022062410/5697bfa51a28abf838c97ab7/html5/thumbnails/17.jpg)
17
Data export form
![Page 18: 1 ArrayExpress Ugis Sarkans, EBI. 2 Overview Underlying standards –MIAME –MAGE* Data submission Data access –annotations –actual data –array design descriptions.](https://reader036.fdocuments.net/reader036/viewer/2022062410/5697bfa51a28abf838c97ab7/html5/thumbnails/18.jpg)
18
Array representation - ADF format
![Page 19: 1 ArrayExpress Ugis Sarkans, EBI. 2 Overview Underlying standards –MIAME –MAGE* Data submission Data access –annotations –actual data –array design descriptions.](https://reader036.fdocuments.net/reader036/viewer/2022062410/5697bfa51a28abf838c97ab7/html5/thumbnails/19.jpg)
19
Experiment plan display
![Page 20: 1 ArrayExpress Ugis Sarkans, EBI. 2 Overview Underlying standards –MIAME –MAGE* Data submission Data access –annotations –actual data –array design descriptions.](https://reader036.fdocuments.net/reader036/viewer/2022062410/5697bfa51a28abf838c97ab7/html5/thumbnails/20.jpg)
20
ArrayExpress(Oracle + Tomcat)
OtherMicroarraydatabases
www
EBI
ExpressionProfiler
ExternalBioinformatics
databases
Data analysis
www
Queries
www
MIAMExpress(MySQL)
MAGE-ML
Submissions
Array Manufacturers
LIMS
Microarray
software
Data Analysissoftware
ArrayExpress Infrastructure
MAGE-ML import,
export
Local MIAMExpressInstallations
Data
pipelines
MAGE-ML
![Page 21: 1 ArrayExpress Ugis Sarkans, EBI. 2 Overview Underlying standards –MIAME –MAGE* Data submission Data access –annotations –actual data –array design descriptions.](https://reader036.fdocuments.net/reader036/viewer/2022062410/5697bfa51a28abf838c97ab7/html5/thumbnails/21.jpg)
21
Tomcat
ArrayExpress architecture
ArrayExpress(Oracle)
MAGE-ML(DTD)
MAGE-OMMAGE-ML (doc)MAGE-ML (doc)MAGE-ML (doc)
MAGEloader
Velocitytemplateengine
Castor
object/relationalmapping
Web pagetemplateWeb pagetemplate
Java servlets
MAGEvalidator
MAGEunloader
error.log
![Page 22: 1 ArrayExpress Ugis Sarkans, EBI. 2 Overview Underlying standards –MIAME –MAGE* Data submission Data access –annotations –actual data –array design descriptions.](https://reader036.fdocuments.net/reader036/viewer/2022062410/5697bfa51a28abf838c97ab7/html5/thumbnails/22.jpg)
22
ArrayExpress: other technical details
• Data matrices - stored in NetCDF format:– binary format for efficient storage of
multidimensional array
• Arrays - stored as ADF spreadsheets (in addition to normal MAGE structures)
![Page 23: 1 ArrayExpress Ugis Sarkans, EBI. 2 Overview Underlying standards –MIAME –MAGE* Data submission Data access –annotations –actual data –array design descriptions.](https://reader036.fdocuments.net/reader036/viewer/2022062410/5697bfa51a28abf838c97ab7/html5/thumbnails/23.jpg)
23
In development
• Immediate:– interface efficiency improvements– BioAssays - graphical display– better integration with Expression Profiler
• Medium-term:– user management
• non-public data (e.g., for reviewers)
– MAGE-ML export
• Curation tool
![Page 24: 1 ArrayExpress Ugis Sarkans, EBI. 2 Overview Underlying standards –MIAME –MAGE* Data submission Data access –annotations –actual data –array design descriptions.](https://reader036.fdocuments.net/reader036/viewer/2022062410/5697bfa51a28abf838c97ab7/html5/thumbnails/24.jpg)
24
ratio absolute change
confidence measure
namedesign element type
speciessample type
bioassay type
performer labexper. type
array design name
platform type
provider
Properties
Properties
Properties
Properties Properties
Data warehouse - for gene- and data-driven queries
namebiological entity type
![Page 25: 1 ArrayExpress Ugis Sarkans, EBI. 2 Overview Underlying standards –MIAME –MAGE* Data submission Data access –annotations –actual data –array design descriptions.](https://reader036.fdocuments.net/reader036/viewer/2022062410/5697bfa51a28abf838c97ab7/html5/thumbnails/25.jpg)
25
Microarray Informatics team at EBIAlvis Brazma - group leader
ArrayExpress Curation MIAMExpress
•Ugis Sarkans
•Gonzalo Garcia •Helen Parkinson •Mohammadreza Shojatalab
Expression Profiler
•Jaak Vilo
Research, students•Thomas Schlitt•Katja Kivinen•Johan Rung•Patrick Kemmeren
•Misha Kapushesky•Lev Soinov
•Koichi Tazaki
•Anastasia Samsonova
•Susanna Sansone
•Philippe Rocca-Serra
•Ele Holloway
•Niran Abeyguna- wardena
•Ahmet Oezcimen
•Gaurab Mukherjee •Sergio Contrino
•Anjan Sharma
•Aurora Torrente