Post on 31-Jan-2016
description
Scaling distributed search for diagnostics and prognostics
applications
Prof. Jim AustinComputer Science, University of York UK
CEO Cybula Ltd.
OGF 20072
Outline Challenge to be addressed Background to Signal Data Explorer The industry problem: Aerospace The science problem: Neuroscience Scaling to other industries
The challenge
OGF 20074
Challenge
Asset orSystem
Data
Data Store
Massive data collection happeningeverywhere
Need to get knowledge from the data
Knowledge
OGF 20075
Challenge
Europe
China
US
Japan
OneData
Warehouse
OGF 20076
Challenge Data too large for a single data store Data changes too fast to keep up to date Networks too slow to transfer the data The system has a central point of failure Large processing needed on centralised
store
OGF 20077
Signal Data Explorer
PME PMC
PME PMC
PME PMC
PME PMC
SDE data architecture
Local nodes
OGF 20078
Challenge Do not move the data - move the processing
That’s much lower effort More reliability Scalable Networks need not be high bandwidth Leverages local computation for free (the
Grid vision)
The Signal Data Explorer
OGF 200710
SDE
!Event detector
Data silosAsset
Data feed
Basic SDE data process
OGF 200711
SDE Allows a user
Set up triggers on multiple, complex, real time data feeds.
Find examples of events that are seen but unknown.
Manage distributed data.
OGF 200712
SDE Signal Search
and analysis Finds
examples like previous
OGF 200713
Enterprise SDE system
Distributed Data Repositories
DataDownload
Novel
Data
En
gin
e
Da
ta
Se
rvic
eD
ata
SignalProcessing
AURA-GCBR
DecisionSupport
EngineModel
DataVisualisation
PortalSecurity management Workflow Manager
Collaborative Working
En
gin
e
Da
ta
En
gin
e
Da
ta
AU
RA
DA
TA
Middleware Services & Data Management
Resource Broker PMC & SRBGrid Service Manager Quality of Service/SLA’s
Grid Infrastructure
GT3
IndustryAerospace: DAME, BROADEN
OGF 200715
Industry Industry demonstrator for the SDE
technology Demonstrates the impact SDE can have
on the business process Allows asset management – full life
support processes
OGF 200716
Rolls-Royce
Engine flight data
Airline office
MaintenanceCentre
London AirportNew York Airport
GRID DiagnosticsCentre
Engine flight data
European data centre
Engine flight data
Airline office
MaintenanceCentre
London AirportNew York Airport
US data centre
GRID DiagnosticsCentre
Engine flight data
OGF 200717
DAME and BROADEN DAME developed basic idea – developed
in lab (2000-2004) BROADEN now has demonstrated this in
Rolls-Royce (2004-2007) Engineers can now use SDE to visualise and
analyse data from Trent engines and test rigs
OGF 200718
DAME and BROADEN The example has shown how this can be
applied in many industries.
ScienceNeuroscience: CARMEN
OGF 200720
Science Data is often not shared from experiments Individuals duplicate many expensive
experiments Should share both data and methods Distributed data and service repositories
needed to support this
OGF 200721
CARMEN Tackles the scientific application of SDE 4 year project to build a neuroscience
repository and experimental platform Project started Oct 2006 £4.5M over 4 years
OGF 200722
CARMEN
resolving the ‘neural code’ from the timing of action potential activity
determining ion channel contribution to the timing of action potentials
examining integration within networks of differing dimensions
Understanding the brain may be the greatest
informatics challenge of the 21st century
OGF 200723
NeuroscienceNeuroscience Gain
Recording from brain tissue removed from epileptic patients (scarce tissue and data rates up to 20 Gb/h)
On line analysis by distributed collaborators will enable experiment to be defined
Repository will enable integration of rare case types from different laboratories
New knowledge will lead to advances in treatment
Scaling to other sectors
OGF 200725
Other sectors Concept is to provide full asset monitoring
Provision of the complete maintenance package, not just the asset
Better value add Manufacturers of the asset are the best
people to diagnose faults, manage the maintenance.
The SDE system allows this
OGF 200726
Other sectors Rail industry
Track and carriage monitoring Oil and Gas
Pipeline monitoring – for leaks Power Generation
Monitoring of generation equipment Transport
Road traffic management
OGF 200727
Thanks DAME, BROADEN and CARMEN teams Support of EPSRC, DTI and industrial
collaborators