OpenStack in Action 4! Susheel Varma - VPH-Share: Patient-Centred Multi-scale Cloud-enabled...

53
Reproducibility a principal driver of the scientific method

description

Paris, 5th December 2013 : OpenStack in Action 4! organized by eNovance, brings together members of the OpenStack community.

Transcript of OpenStack in Action 4! Susheel Varma - VPH-Share: Patient-Centred Multi-scale Cloud-enabled...

Page 1: OpenStack in Action 4! Susheel Varma - VPH-Share: Patient-Centred Multi-scale Cloud-enabled Scientific Workflows

Reproducibility a principal driver of the

scientific method

Page 2: OpenStack in Action 4! Susheel Varma - VPH-Share: Patient-Centred Multi-scale Cloud-enabled Scientific Workflows

Patient Centred Multi-Scale Cloud-Enabled Computational Workflows

Dr Susheel Varma University of Sheffield

Page 3: OpenStack in Action 4! Susheel Varma - VPH-Share: Patient-Centred Multi-scale Cloud-enabled Scientific Workflows

3  #OIA4        Paris,  France        04-­‐Dec-­‐13  

in silico Medicine

“…is the direct use of computer simulation in the diagnosis, treatment, or prevention of a disease.”

Predict disease Personalise treatment

Page 4: OpenStack in Action 4! Susheel Varma - VPH-Share: Patient-Centred Multi-scale Cloud-enabled Scientific Workflows

4  #SummerSchool    20-­‐Jun-­‐13  Cloud Platform

(Public / Private)

Patient Data Workflow Inputs

Workflow Outputs

Semantic Services

Patient Centred Computational Workflows

Patie

nt A

vata

r

App

licat

ion

Info

stru

ctur

e

HPC Infrastructure (DEISA / PRACE)

Pers

onal

ised

M

odel

Knowledge Discovery Data Inference

Compute Services

Storage Services

Knowledge Management

Data Services: Patient/Population

euHeart @neurIST

VPH OP ViroLab

Project No: 269978 VPH-Share

Partners: CYFRONET, PL Sheffield Teaching Hospitals, UK ATOS Origin, ES Kings College London, UK Empirica, DE CiNECA, IT Nine Health CIC, UK INRIA, FR IOR, IT Open Univ., UK Philips Elec., NL TU Eindhoven, NL Univ. Auckland, NZ Uv Amsterdam, NL UCL, UK Univ. Vienna, AT AATRM, ES FCRB, ES

Co-ordinator: University of Sheffield, UK

Select  Workflow   Retrieve  ExisGng  Data  

Transform    or  Infer  Data  

Run  Workflow  

Return  Results  

Page 5: OpenStack in Action 4! Susheel Varma - VPH-Share: Patient-Centred Multi-scale Cloud-enabled Scientific Workflows

VPH-Share A0 Master Plan

5  #OIA4        Paris,  France        04-­‐Dec-­‐13  

ATM

Atmosphere Cloud Platform

Atomic Service Deployment Wizard

MAFEventBus

Authentication Services Workflow

Execution Service

Workflow Registry

Atomic Service Registry

Atomic Service Manager Data Browser Atomic Service

Generic Invoker

Master Interface

Cloud Facade

Visualisation Tools Workflow Composer

VPH-

Share

Client

Generic Workflow Documen

t

Atomic Service

Description

Cloud Clients

libcloud

provider

libcloud

provider

Monitoring Controller

High PerformanceExecution Engine (AHE)

Extension Points

SPRUCE

HARC

Steering

AHE Services API

AHE Runtime

AHE Engine

App Regis

try

JBPM

Workflow &

Main Logic

AHE Database Hibernate ORM

App State Objec

t

Storage Module

Connector Module

External Data

Storage

External HPC

Platform

Security Module

Allocation Management Service

AMS Manag

er (Java) OSGi

bundles

Apache

Karaf

Scheduler / Optimizer

… Algo n Algo 1

REST API & HTML Service

(Ruby) Sinatra & Passenger

Domain Model (Ruby)

Atmosphere Internal Registry

MongoDB

Virtual Machine Template Registry

Data Buckets (C-DISC, CSV,

…)

Databases (SQLServer, …)

External Structured

Data Providers

Data Publishing

Suite (GUI)

Schema Crawler

SPARQL Discovery

Browser Search

RDB2RDF Service

LOD Databases

Silk, LinQuer Service

LD Databases

Multi-Ontology/Archetype

Search Services

Taverna Server

Service Registry

Load Balancer

Proxy Controller

Data Reliability & Integrity Services

PSLoader

External Cloud Data

Storage

Semantic Services

VoID Document Database

Database 1 Query Services

(SPARQL & SQL)

Database Services

Integration Points

Database 2 Query Services

(SPARQL & SQL)

Database n Query Services

(SPARQL & SQL)

Individual Relational Databases

VoID Services

……

……

VoID Document

Atmosphere Cloud Platform

Monitoring System

Atomic Service Instance Contents

Raw Operating System (Linux)

LOBCDER Federated Storage Access

Root Volume

VPH-Share Tool / App

Web Service Wrapper Soaplab2, CXF, soap4r Remote

Access Service

Web Service Security Agent

Monitoring Agent (Munin)

Hypervisor

Driver

Manager

Compute Worker

Network Worker

Object Storage (Swift)

Dashboard

Queue Scheduler

Proxy

Account

Container

Object

ASIProxy

Private Compute & Storage Cloud (OpenStack Example)

Data Volume

Data Resource Catalog

LOBCDER Data Federation Middleware

Data Stores

Connection Module

Request Manager

Access & Control Frontend

Virtual Resource System

… Driver 1 Driver n Cloud Storage Driver

Data Infrastructure Services

Images

NOVA API

Page 6: OpenStack in Action 4! Susheel Varma - VPH-Share: Patient-Centred Multi-scale Cloud-enabled Scientific Workflows

6  

Multi-Scale Scientific Workflows

#OIA4        Paris,  France        04-­‐Dec-­‐13  

Page 7: OpenStack in Action 4! Susheel Varma - VPH-Share: Patient-Centred Multi-scale Cloud-enabled Scientific Workflows
Page 8: OpenStack in Action 4! Susheel Varma - VPH-Share: Patient-Centred Multi-scale Cloud-enabled Scientific Workflows
Page 9: OpenStack in Action 4! Susheel Varma - VPH-Share: Patient-Centred Multi-scale Cloud-enabled Scientific Workflows
Page 10: OpenStack in Action 4! Susheel Varma - VPH-Share: Patient-Centred Multi-scale Cloud-enabled Scientific Workflows

Multi-Scale Challenges

•  State Space Explosion

•  Inverse Parameter Identification

•  Parameter Sensitivity

•  Incomplete Inputs

•  Strong Spatio-Temporal Coupling

•  Chaos and Unstablility

•  Uncertainty Cascade

10  #OIA4        Paris,  France        04-­‐Dec-­‐13  

Page 11: OpenStack in Action 4! Susheel Varma - VPH-Share: Patient-Centred Multi-scale Cloud-enabled Scientific Workflows

11  #OIA4        Paris,  France        04-­‐Dec-­‐13  

VPH-Share Flagship Workflows

Page 12: OpenStack in Action 4! Susheel Varma - VPH-Share: Patient-Centred Multi-scale Cloud-enabled Scientific Workflows

euHeart 16 Partners € 19.05 million Jun 08 – Nov 12

SoMware  VisualizaGon    

MulG-­‐scale  Models  Imaging  MulGmodal  AcquisiGon    

RegistraGon   Anatomical    

PaGent  Therapy  

Geometry  

FuncGonal    

Microstructure  

Vasculature   Coronary  Flow   Perfusion  

Mechanics   AcGvaGon  

EC  Cell   Ventricular  Flow  

Ontologies  

CellML  

FieldML  

X-­‐Ray/MR   MoGon  

Electrical/MR   ExcitaGon   GIMIAS  

CMGUI  

Modelling  tools  and    technologies    

PaGent  

ComputaGonal    OpenCMISS  SOFA  OPENFEM  LIFEV      Numerical      basis  reducGon,  POD    FEM,  FD,  ALE    Parallel  CompuGng    PETsc,    MUMPS    Data  assimilaGon    unscented  Kalmann  filtering  variaGonal  approaches        

Current  PracGce                                                                          Conges(ve  Heart  Failure  

Page 13: OpenStack in Action 4! Susheel Varma - VPH-Share: Patient-Centred Multi-scale Cloud-enabled Scientific Workflows

13  #OIA4        Paris,  France        04-­‐Dec-­‐13  

euHeart Simulation Workflow

Page 14: OpenStack in Action 4! Susheel Varma - VPH-Share: Patient-Centred Multi-scale Cloud-enabled Scientific Workflows

14  #OIA4        Paris,  France        04-­‐Dec-­‐13  

euHeart Simulation Workflow

Page 15: OpenStack in Action 4! Susheel Varma - VPH-Share: Patient-Centred Multi-scale Cloud-enabled Scientific Workflows

15  #OIA4        Paris,  France        04-­‐Dec-­‐13  

@neurIST Simulation Workflow

15  

   

Medical  image

Input:  DICOM Output:  3D  image DescripGon:  Converts  a  DICOM  image  to  VTK  image

   

Segmenta(on

Input:  Image,ROI Output:  surface  mesh DescripGon:  vessels  and  aneurysm  extracGon

   

Mesh  Edi(ng

Input:  surface  mesh Output:    surface  mesh DescripGon:  clipping  vessels,  cleaning  surface  (cell  removal,  closing  holes,  smoothing…)

   

Skeletoniza(on

Input:  surface  mesh. Output:  skeleton. DescripGon:  necessary  to  set  the  boundary  condiGons

   

Aneurysm  isola(on

Input:  surface  mesh Output:  surface  mesh DescripGon:  aneurysm  isolaGon

   

Morphology  Descriptors

Input:  surface  mesh Output:  xml,  vtk   DescripGon:  surface,  depth…  and  ZMI  calculaGon

   

Volumetric  Mesh

Input:  surface  mesh Output:  volumetric  mesh DescripGon:  creates  a  volumetric  mesh  of  the  selected  geometry

   

Flow  Simula(on

Input:  volumetric  mesh,  ccl Output:  wall  shear  stress    map DescripGon:  solves  flow  equaGons

   

Flow  Simula(on  post-­‐processing

Input:  wall  shear  stress Output:  .csv  file DescripGon:  computes  hemodynamic  descriptors

   

CFD  preprocessor

Input:  xml,  surface  mesh Output:  surface  mesh,  ccl DescripGon:  Defines  hemodynamic  model

   

Input:  surface,  1D  model Output:  xml,  vtk DescripGon:  boundary  condiGons  for  CFD

Selec(ng  Boundary  Condi(ons

   

Neck  Selec(on

Input:  surface  mesh Output:  surface  mesh DescripGon:  aneurysm  neck  surface  and  dome  selecGon

GIMIAS  

@neuFuse  

ANSYS  (ICEM)  

ANSYS  (CFX)  

Manual  interacGon  Common  opera(ons  

Morphological  analysis  

Hemodynamic  analysis  

Page 16: OpenStack in Action 4! Susheel Varma - VPH-Share: Patient-Centred Multi-scale Cloud-enabled Scientific Workflows

16  #OIA4        Paris,  France        04-­‐Dec-­‐13  

•  Gathered: scattered across different repositories/catalogues

•  All necessary elements available and accessible maybe open

•  Documented sufficiently well – Explicitly Transparent: How, Why, What,

Where, Who, When – Comprehensive: Just Enough – Comprehensible: Independent

understanding •  Skills and resources to repeat

–  Crowd sourced? Supercomputer?

PublicaGon  

Models,  Techniques,  Algorithms  

Data  

Provenance  AgribuGon  Credit  

Context  InvesGgaGon  

Study  Assay  

Page 17: OpenStack in Action 4! Susheel Varma - VPH-Share: Patient-Centred Multi-scale Cloud-enabled Scientific Workflows

17  #OIA4        Paris,  France        04-­‐Dec-­‐13  

PublicaGon  

Models,  Techniques,  Algorithms  

Data  

Provenance  AgribuGon  Credit  

Context  InvesGgaGon  

Study  Assay  

Accessible   Capable  Reusable  

INSTRUMENTs  Samples,Specimens  Strains  

74%  /  26%  

31%  /  8%  

Page 18: OpenStack in Action 4! Susheel Varma - VPH-Share: Patient-Centred Multi-scale Cloud-enabled Scientific Workflows
Page 19: OpenStack in Action 4! Susheel Varma - VPH-Share: Patient-Centred Multi-scale Cloud-enabled Scientific Workflows

19  

Embodying a Patient Avatar

#OIA4        Paris,  France    04-­‐Dec-­‐13  

Page 20: OpenStack in Action 4! Susheel Varma - VPH-Share: Patient-Centred Multi-scale Cloud-enabled Scientific Workflows

20  #OIA4        Paris,  France        04-­‐Dec-­‐13  

•  A large virtual catalogue of every item of data, information & knowledge of •  Patient; or •  Collection of patients

(Avatars)

•  It also needs to be shared securely, to be able •  To be Searched, Browsed &

Analysed •  For Healthcare, Research &

Education

Vital Signs

Medical Devices

Results

Patient History

Procedures

Medical Images

Histology

Genomics

Proteomics

Patient Avatar

Page 21: OpenStack in Action 4! Susheel Varma - VPH-Share: Patient-Centred Multi-scale Cloud-enabled Scientific Workflows

Patient Avatar – Goals •  Integrating fragmented data and knowledge into a single

cohesive unit of data

•  Creating a centralised repository with reliable population and patient data around which VPH tools and applications could be built

•  Providing diagnostic or prognostic decision and treatment planning support using predictive models built around a patient-specific avatar

•  Providing a comprehensive overview of a patient with missing data based on averages from population phenotype to explore and test ideas virtually

21  #OIA4        Paris,  France        04-­‐Dec-­‐13  

Page 22: OpenStack in Action 4! Susheel Varma - VPH-Share: Patient-Centred Multi-scale Cloud-enabled Scientific Workflows

Current Access to Clinical Data

22  #OIA4        Paris,  France        04-­‐Dec-­‐13  

Owner or Facilitator Description Number (records) Accessibility

Sheffield Teaching Hospitals

ACS data Clinical data, outcomes ~750 Philips

Others on request

Nine Health HES extract Data catalogues ~ 5 millon On demand

FRCB Clinical data Ethics approved for entire record On demand

AATRM Images ~ 6 million On demand

@neurIST Image data CRIM data

Derived data

~100 (600) ~1400 ~300

On demand

euHeart 3D Models 216 cardiac ~90 other Registered users

Virolab Rule database Publication corpus

4 Rule DBs ~1.2 GB Registered users

VPH-OP Extensive clinical baseline data Images

281 Mixed On demand

Page 23: OpenStack in Action 4! Susheel Varma - VPH-Share: Patient-Centred Multi-scale Cloud-enabled Scientific Workflows

23  

Knowledge Management, Discovery & Semantic Services

#OIA4        Paris,  France  04-­‐Dec-­‐13  

Page 24: OpenStack in Action 4! Susheel Varma - VPH-Share: Patient-Centred Multi-scale Cloud-enabled Scientific Workflows

Health Language Terminology

24  #OIA4        Paris,  France        04-­‐Dec-­‐13  

Health Language Terminology Terminology Sets •  SNOMED CT / CA extensions •  ICD-9P&CM •  ICD-10 •  ICD-10-CM / PCS •  CPT-4 •  HL7 •  HCPCS •  APC, DRG, MS-DRG •  LOINC •  ICPC1&2 •  DSM IV •  MeSH •  Pharmacy (FDB, Multum) – NDC •  RxNorm •  Nursing (NIC, NOC, NANDA) •  LCD/NCD/NCCI •  Consumer Friendly Terminology (CFT)

•  CDT •  Multiple Languages •  Local Codes •  Nomenclature •  ICD-10 (GM/AM/CA) •  ICD-O •  UK Admin Extension •  UK Gap Extension •  HRG •  OPCS-4 •  CCI •  Read 2 •  Read 4-byte •  SNOMED Facets •  Clinical Specialty Subsets

Mappings •  SNOMED CT to ICD-9-CM •  SNOMED CT to ICD-10 •  SNOMED CT to OPCS-4 •  ICD-9-CM to SNOMED CT •  SNOMED CT to CPT •  CPT to SNOMED CT •  ICD-9-CM to ICD-10-CM/

PCS •  ICD-10-CM/PCS to ICD-9-

CM •  SNOMED to MeSH •  DSM IV to SNOMED •  ICD-9-CM Procedures to

SNOMED •  HL7 to CHI •  Language to Language

(e.g., English to Spanish)

Page 25: OpenStack in Action 4! Susheel Varma - VPH-Share: Patient-Centred Multi-scale Cloud-enabled Scientific Workflows

Web of Semantically Linked Data

25  #OIA4        Paris,  France        04-­‐Dec-­‐13  

Page 26: OpenStack in Action 4! Susheel Varma - VPH-Share: Patient-Centred Multi-scale Cloud-enabled Scientific Workflows

26  #OIA4        Paris,  France        04-­‐Dec-­‐13  

Tabular Data

Non-Tabular Data

Clinical Information Systems

Data Publishing Suite

Semantic Services

Computational Workflows and Services 1

2

3 4 8 9 10 6 7

Medications

Vital Signs

Lab Reports

Demographic

Vital Signs

Images

Demographic

Risk Factors

Genomic Data

Parameter Estimation

Uncertainty Propagation

Patient Avatar

RDF Graphs

Reference Data

Pseudo Identifier (UID)Demographics

HeightWeight

Vital SignsHeart RateBlood Pressure

Systolic Diastolic

Aneurysm Related Health EventRisk FactorsAneurysm Imaging StudyMedications

Pseudo Identifier (UID)Demographics

HeightWeight

Vital SignsHeart RateBlood Pressure

Systolic Diastolic

Pulmonary FunctionRisk FactorsCardiac Imaging StudyMedications

Pseudo Identifier (UID)Demographics

HeightWeight

Vital SignsHeart RateBlood Pressure

Systolic Diastolic

Orthopaedic Health EventGynaecological InformationBone Phenotype Imaging StudyMedications Physiological

Envelope

Page 27: OpenStack in Action 4! Susheel Varma - VPH-Share: Patient-Centred Multi-scale Cloud-enabled Scientific Workflows

27  #OIA4        Paris,  France        04-­‐Dec-­‐13  

Patient Avatar & Workflow Archetypes

Baseline HistoryCardiac Health EventLab TestsMedicationsCardiac Imaging StudyImaging Details

(c) euHeart Workflow Archetype

Sexual HealthHIV Subtype InformationSpecimens

(d) ViroLab Workflow Archetype

Aneurysm Related Health EventSupporting Risk FactorsSpecimens (Blood, Tissue)MedicationsAneurysm Imaging StudyImaging Details

(b) @neurIST Workflow Archetype

Hand Grip StrengthOrthopeadic Health EventMedicationsSpecimens (Blood, Tissue)Bone Phenotype Imaging Study

(e) VPHOP Workflow ArchetypePatient Pseudoidentifier (PID)DemographicsPersonal & Social HistoryFitness & LifestyleEmployment DetailsVital SignsSubstance Use

(a) Common Shared Archetype

Baseline HistoryCardiac Health EventLab TestsMedicationsCardiac Imaging StudyImaging Details

(c) euHeart Workflow Archetype

Sexual HealthHIV Subtype InformationSpecimens

(d) ViroLab Workflow Archetype

Aneurysm Related Health EventSupporting Risk FactorsSpecimens (Blood, Tissue)MedicationsAneurysm Imaging StudyImaging Details

(b) @neurIST Workflow Archetype

Hand Grip StrengthOrthopeadic Health EventMedicationsSpecimens (Blood, Tissue)Bone Phenotype Imaging Study

(e) VPHOP Workflow ArchetypePatient Pseudoidentifier (PID)DemographicsPersonal & Social HistoryFitness & LifestyleEmployment DetailsVital SignsSubstance Use

(a) Common Shared Archetype

Baseline HistoryCardiac Health EventLab TestsMedicationsCardiac Imaging StudyImaging Details

(c) euHeart Workflow Archetype

Sexual HealthHIV Subtype InformationSpecimens

(d) ViroLab Workflow Archetype

Aneurysm Related Health EventSupporting Risk FactorsSpecimens (Blood, Tissue)MedicationsAneurysm Imaging StudyImaging Details

(b) @neurIST Workflow Archetype

Hand Grip StrengthOrthopeadic Health EventMedicationsSpecimens (Blood, Tissue)Bone Phenotype Imaging Study

(e) VPHOP Workflow ArchetypePatient Pseudoidentifier (PID)DemographicsPersonal & Social HistoryFitness & LifestyleEmployment DetailsVital SignsSubstance Use

(a) Common Shared Archetype

Baseline HistoryCardiac Health EventLab TestsMedicationsCardiac Imaging StudyImaging Details

(c) euHeart Workflow Archetype

Sexual HealthHIV Subtype InformationSpecimens

(d) ViroLab Workflow Archetype

Aneurysm Related Health EventSupporting Risk FactorsSpecimens (Blood, Tissue)MedicationsAneurysm Imaging StudyImaging Details

(b) @neurIST Workflow Archetype

Hand Grip StrengthOrthopeadic Health EventMedicationsSpecimens (Blood, Tissue)Bone Phenotype Imaging Study

(e) VPHOP Workflow ArchetypePatient Pseudoidentifier (PID)DemographicsPersonal & Social HistoryFitness & LifestyleEmployment DetailsVital SignsSubstance Use

(a) Common Shared Archetype

Baseline HistoryCardiac Health EventLab TestsMedicationsCardiac Imaging StudyImaging Details

(c) euHeart Workflow Archetype

Sexual HealthHIV Subtype InformationSpecimens

(d) ViroLab Workflow Archetype

Aneurysm Related Health EventSupporting Risk FactorsSpecimens (Blood, Tissue)MedicationsAneurysm Imaging StudyImaging Details

(b) @neurIST Workflow Archetype

Hand Grip StrengthOrthopeadic Health EventMedicationsSpecimens (Blood, Tissue)Bone Phenotype Imaging Study

(e) VPHOP Workflow ArchetypePatient Pseudoidentifier (PID)DemographicsPersonal & Social HistoryFitness & LifestyleEmployment DetailsVital SignsSubstance Use

(a) Common Shared Archetype

Page 28: OpenStack in Action 4! Susheel Varma - VPH-Share: Patient-Centred Multi-scale Cloud-enabled Scientific Workflows

Knowledge Management

28  #OIA4        Paris,  France        04-­‐Dec-­‐13  

Page 29: OpenStack in Action 4! Susheel Varma - VPH-Share: Patient-Centred Multi-scale Cloud-enabled Scientific Workflows

29  

Cloud-Enabled Computational Workflows

#OIA4        Paris,  France        04-­‐Dec-­‐13  

Page 30: OpenStack in Action 4! Susheel Varma - VPH-Share: Patient-Centred Multi-scale Cloud-enabled Scientific Workflows

Cloud-Enabled Computation Workflows •  portal.vph-share.eu – Provides access to Clinical Data and Scientific applications

and workflows

•  Taverna Workbench – Provides Desktop tool for composition of Scientific Workflows with VPH-Share applications and data

•  OnlineHPC.com – Allows the creation of Scientific Workflows composed of External and VPH-Share Applications and VPH-Share data

•  Meta-Workflow Manager – Executes multiple Scientific Workflow Engines on the VPH-Share Platform

•  Workflow Cloud Plugin – Allows execution of Scientific Web-Services and Application on multiple cloud platforms

•  Command-Line Wrapper – Allows developers to wrap command-line applications into a (REST/SOAP) webservice via wsme and GIMIAS

•  NoMachine RDP – Provides Remote Desktop services for applications that require user interaction

30  #OIA4        Paris,  France        04-­‐Dec-­‐13  

Page 31: OpenStack in Action 4! Susheel Varma - VPH-Share: Patient-Centred Multi-scale Cloud-enabled Scientific Workflows

Cloud-Enabled Computation Workflows

31  #OIA4        Paris,  France        04-­‐Dec-­‐13  

Page 32: OpenStack in Action 4! Susheel Varma - VPH-Share: Patient-Centred Multi-scale Cloud-enabled Scientific Workflows

32  #OIA4        Paris,  France        04-­‐Dec-­‐13  

Clinical  Researcher  

Workflow  Manager  API  

VPH-­‐Share  plugin  

Taverna  Server  

VPH-­‐Share  Workflow  

Cloud  Façade  

Web-­‐based  Remote    Desktop  

AS  without    interacGon

AS  with    interacGon

CLIENT-­‐SIDE   SERVER-­‐SIDE  

AS  AS  

AS  

AS  AS  AS  

External  ApplicaGon  

STORAGE

VPH-­‐Share  plugin  

Taverna  Workbench  

Web  services  

GIMIAS  CLPs  

…  

VPH-­‐Share  plugin  

Taverna  On-­‐line  

Web  services  

…  

Page 33: OpenStack in Action 4! Susheel Varma - VPH-Share: Patient-Centred Multi-scale Cloud-enabled Scientific Workflows

33  

VPH-Share HPC & Cloud Platform

#OIA4        Paris,  France        04-­‐Dec-­‐13  

Page 34: OpenStack in Action 4! Susheel Varma - VPH-Share: Patient-Centred Multi-scale Cloud-enabled Scientific Workflows

Scientific Cloud Platform

34  #OIA4        Paris,  France        04-­‐Dec-­‐13  

ATM

Atmosphere Cloud Platform

Atomic Service Deployment Wizard

MAFEventBus

Authentication Services Workflow

Execution Service

Workflow Registry

Atomic Service Registry

Atomic Service Manager Data Browser Atomic Service

Generic Invoker

Master Interface

Cloud Facade

Visualisation Tools Workflow Composer

VPH-

Share

Client

Generic Workflow Documen

t

Atomic Service

Description

Cloud Clients

libcloud

provider

libcloud

provider

Monitoring Controller

High PerformanceExecution Engine (AHE)

Extension Points

SPRUCE

HARC

Steering

AHE Services API

AHE Runtime

AHE Engine

App Regis

try

JBPM

Workflow &

Main Logic

AHE Database Hibernate ORM

App State Objec

t

Storage Module

Connector Module

External Data

Storage

External HPC

Platform

Security Module

Allocation Management Service

AMS Manag

er (Java) OSGi

bundles

Apache

Karaf

Scheduler / Optimizer

… Algo n Algo 1

REST API & HTML Service

(Ruby) Sinatra & Passenger

Domain Model (Ruby)

Atmosphere Internal Registry

MongoDB

Virtual Machine Template Registry

Data Buckets (C-DISC, CSV,

…)

Databases (SQLServer, …)

External Structured

Data Providers

Data Publishing

Suite (GUI)

Schema Crawler

SPARQL Discovery

Browser Search

RDB2RDF Service

LOD Databases

Silk, LinQuer Service

LD Databases

Multi-Ontology/Archetype

Search Services

Taverna Server

Service Registry

Load Balancer

Proxy Controller

Data Reliability & Integrity Services

PSLoader

External Cloud Data

Storage

Semantic Services

VoID Document Database

Database 1 Query Services

(SPARQL & SQL)

Database Services

Integration Points

Database 2 Query Services

(SPARQL & SQL)

Database n Query Services

(SPARQL & SQL)

Individual Relational Databases

VoID Services

……

……

VoID Document

Atmosphere Cloud Platform

Monitoring System

Atomic Service Instance Contents

Raw Operating System (Linux)

LOBCDER Federated Storage Access

Root Volume

VPH-Share Tool / App

Web Service Wrapper Soaplab2, CXF, soap4r Remote

Access Service

Web Service Security Agent

Monitoring Agent (Munin)

Hypervisor

Driver

Manager

Compute Worker

Network Worker

Object Storage (Swift)

Dashboard

Queue Scheduler

Proxy

Account

Container

Object

ASIProxy

Private Compute & Storage Cloud (OpenStack Example)

Data Volume

Data Resource Catalog

LOBCDER Data Federation Middleware

Data Stores

Connection Module

Request Manager

Access & Control Frontend

Virtual Resource System

… Driver 1 Driver n Cloud Storage Driver

Data Infrastructure Services

Images

NOVA API

Page 35: OpenStack in Action 4! Susheel Varma - VPH-Share: Patient-Centred Multi-scale Cloud-enabled Scientific Workflows

Platform Architecture

35  #OIA4        Paris,  France        04-­‐Dec-­‐13  

Physical  resources  

Atomic  Service  Instances  Deployed  by  AMS  on  available  resources  as  required  by  WF  mgmt  or  generic  AS  

invoker  

Raw OS (Linux variant)

LOB Federated storage access

Web Service cmd. wrapper

Generic VNC server

VPH-Share Tool / App.

DRI  Service  

Atmosphere  persistence  layer  (internal  registry)  

VM  templates  

AS  images  

Available  cloud  

infrastructure  

Managed  datasets  

101101  011010  111011  

101101  011010  111011  

101101  011010  111011  

AM  Service  

LOB  federated  storage  access  

Cloud  stack  clients  

HPC  resource  client/backend  

Data  and  Compute  Cloud  Planorm  

VPH-­‐Share  Master  UI  

AS  mgmt.  interface  

Generic  AS  invoker  

ComputaGon  UI  extensions  

Data  mgmt.  interface  

Generic  data  retrieval  

Data  mgmt.  UI  extensions  

Remote  access  to  Atomic  Svc.  UIs  

Custom  AS  client  

Workflow  descripGon  and  execuGon  

Developer   ScienGst  

Admin  

Security  mgmt.  interface  

Security  framework  

Web  Service  security  agent  

Modules  available  in  first  prototype  

Page 36: OpenStack in Action 4! Susheel Varma - VPH-Share: Patient-Centred Multi-scale Cloud-enabled Scientific Workflows

Cloud Platform Architecture

36  #OIA4        Paris,  France        04-­‐Dec-­‐13  

VPH-­‐Share  Master  Int.  

Admin  Developer   ScienGst  

Development  Mode  

VPH-­‐Share  Core  Services  Host  

                             ComputaGonal  Cloud  Site  

Worker  Node  

Worker  Node  

Worker  Node  

Worker  Node  

Worker  Node  

Worker  Node  

Worker  Node  

Worker  Node  

Head  Node  

Image  store  (Glance)  

Cloud  Facade  (secure  

RESTful  API  )  

Atmosphere  Management  Service  (AMS)  

Cloud  stack  plugins  (JClouds)  

Atmosphere  Internal  

Registry  (AIR)  

Cloud  Manager  

Generic  Invoker  

Workflow  management  

External  applicaGon  

Cloud  Facade  client  

•  The platform provides a set of APIs for the VPH-Share Master Interface and other applications, enabling Atomic Services to be developed.

Customized  applicaGons  may  directly  interface  the  Cloud  Facade  via  its  RESTful  APIs  

Page 37: OpenStack in Action 4! Susheel Varma - VPH-Share: Patient-Centred Multi-scale Cloud-enabled Scientific Workflows

Accessible HPC Execution Platform

37  #OIA4        Paris,  France        04-­‐Dec-­‐13  

�  Provides  virtualized  access  to  high  performance  execuGon  environments  �  Seamlessly  provides  access  to  high  performance  compuGng  to  workflows  that  

require  more  computaGonal  power  than  clouds  can  provide  �  Deploys  and  extends  the  ApplicaGon  HosGng  Environment  –  provides  a  set  of  web  

services  to  start  and  control  applicaGons  on  HPC  resources  

GridFTP  AHE  Web  Services  

(RESTlets)  

Grid  resources  running  Local  Resource  Manager  (PBS,  SGE,  Loadleveler  etc.)  

ApplicaGon  HosGng  Environment  Auxiliary  component  of  the  cloud  planorm,  responsible  for  managing  access  to  tradiGonal  (grid-­‐based)  high  performance  compuGng  environments.  Provides  a  Web  Service  interface  for  clients.  

Invoke  the  Web  Service  API  of  AHE  to  delegate  computaGon  to  the  grid  

ApplicaGon  -­‐-­‐  or  -­‐-­‐  

Workflow  environment  

-­‐-­‐  or  -­‐-­‐  

End  user  

Present  security  token  (obtained  from  authenGcaGon  service)  

Tomcat  container  WebDAV  

User  access  layer  

QCG  CompuGng  

Job  Submission  Service  (OGSA  BES  /  Globus  

GRAM)  RealityGrid  SWS  

Resource  client  layer  

Delegate  credenGals,  instanGate  compuGng  tasks,  poll  for  execuGon  status  and  retrieve  results  on  behalf  of  the  client  

Page 38: OpenStack in Action 4! Susheel Varma - VPH-Share: Patient-Centred Multi-scale Cloud-enabled Scientific Workflows

Unstructured Data Storage

38  #OIA4        Paris,  France        04-­‐Dec-­‐13  

LOBCDER  host  (149.156.10.143)  

LOBCDER  service  backend  

Resource  catalogue  

WebDAV  servlet  

Resource  factory  

Storage  driver  

Storage  driver  (SWIFT)  

SWIFT  storage  backend  

Core  component  host  (vph.cyfronet.pl)   Data  Manager  

Portlet  (VPH-­‐Share  

Master  Interface  component)  

Atomic  Service  Instance  (10.100.x.x)   Service  payload  

(VPH-­‐Share  applicaGon  component)  

External  host  Generic  WebDAV  client  

GUI-­‐based  access  

Mounted  on  local  FS  (e.g.  via  davfs2)  

•  VPH-­‐Share  federated  data  storage  module  (LOBCDER)  enables  data  sharing  in  the  context  of  VPH-­‐Share  applicaGons  

•  The  module  is  capable  of  interfacing  various  types  of  storage  resources  and  supports  SWIFT  cloud  storage  (support  for  Amazon  S3  is  under  development)  

•  LOBCDER  exposes  a  WebDAV  interface  and  can  be  accessed  by  any  DAV-­‐compliant  client.  It  can  also  be  mounted  as  a  component  of  the  local  client  filesystem  using  any  DAV-­‐to-­‐FS  driver  (such  as  davfs2).    

EncrypGon  keys  

REST-­‐interface  

Master  Interface  component  Ticket  validaGon  service  

Auth    service  

Amazon  S3  Storage  backend  

Page 39: OpenStack in Action 4! Susheel Varma - VPH-Share: Patient-Centred Multi-scale Cloud-enabled Scientific Workflows

Platform by Numbers •  4 Data centers

– CYFRONET, Krakow – UoS, Sheffield – STH, Sheffield – UNV, Vienna

•  80+ Cloud Hosts •  100+ VMs baseline, 331 VMs Peak •  50TB+ Data Storage •  75+ Scientific Applications •  25+ Scientific Workflows •  €70k Public Cloud Burst Funds

39  #OIA4        Paris,  France        04-­‐Dec-­‐13  

Page 40: OpenStack in Action 4! Susheel Varma - VPH-Share: Patient-Centred Multi-scale Cloud-enabled Scientific Workflows

Elephant in the Room

•  Security & Privacy •  Legislation & Ethics •  Training

•  Long Tail of Physicians and Care-takers

•  P4 Medicine Journey •  Predictive •  Preventative •  Personalised •  Participatory

40  #OIA4        Paris,  France        04-­‐Dec-­‐13  

Page 41: OpenStack in Action 4! Susheel Varma - VPH-Share: Patient-Centred Multi-scale Cloud-enabled Scientific Workflows

41  #OIA4        Paris,  France        04-­‐Dec-­‐13  

Page 42: OpenStack in Action 4! Susheel Varma - VPH-Share: Patient-Centred Multi-scale Cloud-enabled Scientific Workflows

<Thank  You!>  

42  

Dr Susheel Varma <[email protected]> VPH-Share - Scientific Workflows Coordinator Department of Cardiovascular Science The Medical School, The University of Sheffield Beech Hill Road, Sheffield S10 2RX UK T: +44 (0)114 271 2863

#OIA4        04-­‐Dec-­‐13  

Page 43: OpenStack in Action 4! Susheel Varma - VPH-Share: Patient-Centred Multi-scale Cloud-enabled Scientific Workflows

•  @neurIST- Integrated Biomedical Informatics for the Management of Cerebral Aneurysms (http://www.aneurist.org)

•  IST project funded within the EU FP6 •  Duration: 2006-2010 •  Budget: 17M€ •  Participants: 28 institutions

–  Public and private, –  Industry, hospitals, academia, –  12 European countries

•  External collaborators: from USA, New Zealand, Japan)

•  @neurIST main objective: @neurIST will transform the management of cerebral aneurysms by providing new insights, personalized risk assessment, and methods for the design of improved medical devices and treatment protocols.

@neurIST Simulation Workflow

Page 44: OpenStack in Action 4! Susheel Varma - VPH-Share: Patient-Centred Multi-scale Cloud-enabled Scientific Workflows

@neurIST Framework

VPH-­‐Share  

Page 45: OpenStack in Action 4! Susheel Varma - VPH-Share: Patient-Centred Multi-scale Cloud-enabled Scientific Workflows

   

DICOM

Input:  DICOM Output:  3D  image DescripGon:  Converts  a  DICOM  image  to  VTK  image

   

Volume  Rendering

Input:  3D  image Output:  3D  image DescripGon:  aneurysm  and  vessels  VisualisaGon

   

Bounding  Box

Input:  3D  image Output:  ROI DescripGon:  volume  selecGon

   

GAR  Segmenta(on

Input:  Image,ROI Output:  surface  mesh DescripGon:  vessels  and  aneurysm  extracGon

   

Mesh  Edi(ng

Input:  surface  mesh Output:    surface  mesh DescripGon:  clipping  vessels,  cleaning  surface  (cell  removal,  closing  holes,  smoothing…)

   

Skeletoniza(on

Input:  surface  mesh. Output:  skeleton. DescripGon:  necessary  to  set  the  boundary  condiGons

   

Aneurysm  isola(on

Input:  surface  mesh Output:  surface  mesh DescripGon:  aneurysm  isolaGon

   

Morphology  Descriptors

Input:  surface  mesh Output:  xml,  vtk   DescripGon:  surface,  depth…  and  ZMI  calculaGon

   

Volumetric  Mesh

Input:  surface  mesh Output:  volumetric  mesh DescripGon:  creates  a  volumetric  mesh  of  the  selected  geometry

   

Flow  Simula(on

Input:  volumetric  mesh,  ccl Output:  wall  shear  stress    map DescripGon:  solves  flow  equaGons

   

Flow  Simula(on  post-­‐processing

Input:  wall  shear  stress Output:  .csv  file DescripGon:  computes  hemodynamic  descriptors

   

CFD  preprocessor

Input:  xml,  surface  mesh Output:  surface  mesh,  ccl DescripGon:  Defines  hemodynamic  model

   

Input:  surface,  1D  model Output:  xml,  vtk DescripGon:  boundary  condiGons  for  CFD

Selec(ng  Boundary  Condi(ons

   

Neck  Selec(on

Input:  surface  mesh Output:  surface  mesh DescripGon:  aneurysm  neck  surface  and  dome  selecGon

GIMIAS  

@neuFuse  

ANSYS  (ICEM)  

ANSYS  (CFX)  

Manual  interacGon  Common  opera(ons  

Morphological  analysis  

Hemodynamic  analysis  

Two Workflows from @neurIST

Page 46: OpenStack in Action 4! Susheel Varma - VPH-Share: Patient-Centred Multi-scale Cloud-enabled Scientific Workflows

•  Morphological, hemodynamic and structural analyses have been linked to aneurysm genesis, growth and rupture.

•  Evidence indicating differences in morphology and flow between ruptured and unruptured aneurysms have been shown for reduced patient cohorts.

•  Structural wall mechanics has been used to justify the growth and remodelling happening at the aneurysm level.

Confidence  in    physical  measures  

+  

images  

+  BC,    material  

+  BC,    material  

Morphological analysis

Direct    diagnos6c  power  

+  

Morphological  descriptors  

Structural  descriptors  

Hemodynamic  descriptors  

Haemodynamic analysis

Structural analysis

PracGcally,  morphological  characterizaGons    might  currently  have  the  highest  predic(ve  capabili(es  with  respect  to  the  other  analyses.  

Morphological Workflow

Page 47: OpenStack in Action 4! Susheel Varma - VPH-Share: Patient-Centred Multi-scale Cloud-enabled Scientific Workflows

The @neurIST morphological workflow specification in Taverna:

Implementation in VPH-Share

Page 48: OpenStack in Action 4! Susheel Varma - VPH-Share: Patient-Centred Multi-scale Cloud-enabled Scientific Workflows

•  An automatic segmentation method based on Geodesic Active Regions (GAR) and an image standardization technique is used

•  The method: –  eliminates most of the dependency on the operator, and on

the specific imaging protocols and equipment used. –  is able to segment (extract the surface mesh) a region of

interest with a size of 2563 voxels in 17+4 min (avg+std dev) on a PC (Intel quad-core, 2.4 GHz, 4GB memory).

Hernandez,  M.  et  al.  2007  Non-­‐parametric  geodesic  acGve  regions:  method  and  evaluaGon  for  cerebral  aneurysms  segmentaGon  in  3DRA  and  CTA.  Med.  Image  Anal.  11,  224–241.    Bogunovic,  H.  et  al.  2011  Automated  segmentaGon  of  cerebral  vasculature  with  aneurysms  in  3DRA  and  TOF  MRA  using  geodesic  acGve  regions:  an  evaluaGon  study.  Med.  Phys.  38,  210–222.  

A  surface  mesh  represenGng  the  vascular  geometry  is  required  to  perform  the  @neurIST  morphological  and  hemodynamic  analyses  

Medical  image  from  imaging  equipment  

Surface  mesh  a<er  segmenta6on  

GAR Segmentation

Page 49: OpenStack in Action 4! Susheel Varma - VPH-Share: Patient-Centred Multi-scale Cloud-enabled Scientific Workflows

•  The surface mesh obtained after the GAR segmentation needs to be manually manipulated by an operator to either remove or correct:

–  some of the artifacts not belonging to the cerebral vasculature –  those parts of the geometry not relevant for the subsequent

analyses (morphological or hemodynamic).

Surface  mesh  a<er  segmenta6on  

A  surface  mesh  represenGng  the  vascular  geometry  is  required  to  perform  the  @neurIST  morphological  and  hemodynamic  analyses  

Kissing  vessels   Remove  cells   Close  holes  

O   P  

Surface  mesh  a<er  correc6on    and  aneurysm  isola6on  

Mesh Editing

Page 50: OpenStack in Action 4! Susheel Varma - VPH-Share: Patient-Centred Multi-scale Cloud-enabled Scientific Workflows

•  User is asked to manually delineate the neck •  Unfortunately, automatic methods are not an option

because there are: –  unacceptable differences in a large number of cases among

methods and manual selection of experts –  too complex vascular topologies where there is not even an

agreement among experts about where the aneurysm neck is

Several  morphological  measurements  are  based  on  the  aneurysm  sac,  to  idenGfy  it,  the  aneurysm  neck  is  required  

Surface  mesh  a<er  correc6on    and  aneurysm  isola6on  

Manual  delinea6on  

Manual  aneurysm    neck  selec6on  

vs  Automat  ic  

Too  large  differences  in  performance  and  lack  of  consensus  

Too  complex  vascular  topologies  

Neck  DelineaGon  

Page 51: OpenStack in Action 4! Susheel Varma - VPH-Share: Patient-Centred Multi-scale Cloud-enabled Scientific Workflows

•  Among the wide variety of existing morphological descriptors, @neurIST chose to compute:

–  Basic size indices describing the aneurysm sac: aspect ratio, non-sphericity index, aneurysm volume and surface area.

–  Complex indices describing the sac and a portion of the surrounding vasculature: Zernike moment invariants (volume and surface-based).

Morphological  descriptors  of  various  complexity  are  automaGcally  extracted  and  stored  for  their  subsequent  analysis  

Ujiie,  H.  et  al.  1999  Effects  of  size  and  shape  (aspect  raGo).  Neurosurgery  45,  119–130.  /  Ma,  B.,  Harbaugh,  R.  E.  &  Raghavan,  M.  L.  2004  Three-­‐dimensional  geometrical  characterizaGon  of  cerebral  aneurysms.  Ann.  Biomed.  Eng.  32,  264–273.  /  Raghavan,  M.  L.,  Ma,  B.  &  Harbaugh,  R.  E.  2005  QuanGfied  aneurysm  shape  and  rupture  risk.  J.  Neurosurg.  102,  355–362.  Pozo,  J.  M.  et  al.  2011  Efficient  3D  Geometric  and  Zernike  moments  computaGon  from  unstructured  surface  meshes.  IEEE  Trans.  Pagern  Anal.  Machine  Intell.  33,  471–484.    

Manual  aneurysm    neck  selec6on  

Complex  indices  (Zernike  moment  invariants)  

Basic  size  indices  describing  aneurysm  sac  

depth  

neck  

@neurIST    morphological    descriptors  

Morphological Descriptors

Page 52: OpenStack in Action 4! Susheel Varma - VPH-Share: Patient-Centred Multi-scale Cloud-enabled Scientific Workflows

Medical  image  from  imaging  equipment  

@neurIST    morphological    descriptors  

Complex  indices  (Zernike  moment  invariants)  

Basic  size  indices  describing  aneurysm  sac  

depth  

neck  

Morphological Analysis Workflow

Page 53: OpenStack in Action 4! Susheel Varma - VPH-Share: Patient-Centred Multi-scale Cloud-enabled Scientific Workflows

@neurIST: Morphology Results