Architectural Styles of HEP Experiments Presented by RD Schaffer CERN/EP and LAL Orsay LCB Workshop...
Transcript of Architectural Styles of HEP Experiments Presented by RD Schaffer CERN/EP and LAL Orsay LCB Workshop...
Architectural Styles Architectural Styles of HEP Experimentsof HEP Experiments
Presented by RD Schaffer
CERN/EP and LAL Orsay
LCB Workshop
Marseille, September 29 1999
29 September 1999 Architecture Session – Rapporteur talk
2
Architectural Styles of HEP Architectural Styles of HEP ExperimentsExperiments
HEP experiments have been moving their software to Object-oriented systems for a few years
Each LHC experiment needs to develop a framework to be used in their event data processing applications physics/detector simulation, high level triggers,
reconstruction, analysis, visualization, etc.
These experiment frameworks will incorporate external frameworks/toolkits: e.g. persistency, simulation, GUI
29 September 1999 Architecture Session – Rapporteur talk
3
Architectural Styles of HEP Architectural Styles of HEP ExperimentsExperiments (2) (2)
In this talk we would like to begin to
Distill abstract views of the architecture of our software systems High level view is not completely accurate
Any real architecture is a hybrid of styles
This should help us reason and to discuss about the different architectural designs and choices
Goal: help us to define simple and clear architectures so that the software systems we build are easy to understand and to interact with
29 September 1999 Architecture Session – Rapporteur talk
4
Software ArchitectureSoftware Architecture
Outline of talk Architecture: what is it?
Definitions of the terminology/vocabulary
Architecture styles: examples from the literature
Architectural styles in HEP An example architecture: LHCb’s GAUDI Variations of algorithm/event data styles Architectural issues of persistency Application Meta Information User interface (UI) and visualization
Implementation and physical design
29 September 1999 Architecture Session – Rapporteur talk
5
BibliographyBibliography
Notation: […] is used as cross-reference in slides.
Architecture
[G&S]: D. Garlan and M. Shaw. "An Introduction to Software Architecture," 1994
http://www.cs.cmu.edu/afs/cs/project/vit/ftp/pdf/intro_softarch.pdf
[S&C]: Shaw and Clements, “Preliminary classification of architectural styles for software systems”, 1996
http://www.cs.cmu.edu/afs/cs.cmu.edu/project/vit/www/paper_abstracts/Boxology.html
[USDP]: I. Jacobson, et al. “The Unified Software Development Process”, Addison Wesley 1999
[Booch]: G. Booch, “Object Solutions”, Addison Wesley 1996
29 September 1999 Architecture Session – Rapporteur talk
6
BibliographyBibliography (2) (2)
Frameworks
[Gamma]: E. Gamma, et al., “Design Patterns”, Addison-Wesley 1995
[IBM1]: Building Object Oriented Frameworks (html) http://www.ibm.com/java/education/oobuilding/index.html
[IBM2]: Leveraging Object Oriented Frameworks (html) http://www.ibm.com/java/education/ooleveraging/index.html
Physical design
[Martin]: R. Martin, “Designing O-O C++ Applications using the Booch Method”, Prentice Hall 1995
[Lakos]: J. Lakos, “Large Scale C++ Software Design”, Addison-Wesley 1998
29 September 1999 Architecture Session – Rapporteur talk
7
Architecture: What Is It?Architecture: What Is It?
Definitions architecture [USDP]
Software architecture is also concerned functionality, performance, resilience, reuse, comprehensibility, economic and technology constraints and trade-offs, and aesthetic concerns.
Set of significant decisions about the organization of the software system
Selection of the structural elements and their interfaces which compose the system
Their behavior — collaboration among the structural elements
Composition of these structural and behavioral elements into progressively larger subsystems
The architectural style that guides this organization
29 September 1999 Architecture Session – Rapporteur talk
8
Architecture: DefinitionsArchitecture: Definitions (2) (2)
toolkit [Gamma]
Examples C++ I/O stream library, containers/iterators/algorithms library, CLHEP, GEANT4 (can be toolkit or framework continuous spectrum)
Comments They do not impose a particular design on one’s application They provide functionality to help one’s application do it’s job
set of related and reusable classes designed to provide useful, general-purpose functionality
29 September 1999 Architecture Session – Rapporteur talk
9
Architecture: DefinitionsArchitecture: Definitions (3) (3)
framework [Booch] + [Gamma]
A kind of micro-architecture that codifies a particular domain
Provides the suitable knobs, slots and tabs that permit clients to use and adapt to specific applications within a given range of behavior
A framework is generally composed of two elements a set of classes that capture the vocabulary of a particular domain a control policy that orchestrates the instances of those classes
A framework realizes an architecture
A large O-O system is constructed from severalcooperating frameworks
29 September 1999 Architecture Session – Rapporteur talk
10
Architecture: DefinitionsArchitecture: Definitions (4) (4)
interface [USDP]
A physical or replaceable part of a system that conforms to and provides the realization of one or more interfaces
component [USDP]
A collection of operations that are used to specify a service of a class or component
29 September 1999 Architecture Session – Rapporteur talk
11
Architecture: DefinitionsArchitecture: Definitions (5) (5)
design pattern [Gamma]
Description of communicating objects and classes that are customized to solve a general design problem in a particular context
Is more abstract than a framework a framework itself can be embodied in code a pattern can only have examples embodied in code
Is a smaller architectural unit than a framework Is less specialized than a framework
can apply to >1 application domains
Architectural abstractions can be described as patterns
29 September 1999 Architecture Session – Rapporteur talk
12
Architecture Styles: OutlineArchitecture Styles: Outline
Part I General categorization of systems [Booch]
user-centric data-centric computation-centric
Part II Further classification of architectural styles [C&S]
Constituent parts: components and connectors Examples of architectural patterns
data abstraction (object-oriented) pipe-and-filter systems data-centered repository implicit invocation
29 September 1999 Architecture Session – Rapporteur talk
13
Architecture Styles: Part IArchitecture Styles: Part I
General categorization of systems [Booch]
user-centric focus on the direct visualizationand manipulation of the
objectsthat define a certain domain
data-centric focus upon preserving theintegrity of the persistentobjects in a system
computation-centric focus is on the transformationof objects that are interestingto the system
Our applications have elements with all three.
The interesting question is which one dominates?
29 September 1999 Architecture Session – Rapporteur talk
14
Architecture Styles: Part I (2)Architecture Styles: Part I (2)
Booch describes three dominant layers for each system type user-centric
classes that provide the system’s look-and-feel classes that map the GUI layer to the domain model classes that denote the domain model
data-centric classes that access and manipulate the domain model classes that denote the domain model classes that provide persistence for the domain model
29 September 1999 Architecture Session – Rapporteur talk
15
Architecture Styles: Part I (3)Architecture Styles: Part I (3)
computation-centric
classes whose objects act as agents responsible for carrying out algorithms that involve the collaboration of several other objects
classes that model the objects transformed by the system
classes to which higher level objects delegate certain more primitive responsibilities, so that common behaviors can be localized and thus reused
In my opinion it is the computation-centric architectural style that is at the heart of the reconstruction, analysis, and high-level triggers
29 September 1999 Architecture Session – Rapporteur talk
16
Architecture Styles: Part IIArchitecture Styles: Part II
Further classification of architectural styles [C&S]
Constituent parts – the building blocks of architecture components – a functional unit of software
e.g. objects, processes, filters
connectors – mechanisms that mediate communication, coordination, or cooperation of components
e.g. shared representations, data streams, or data format converters
To uniquely identify a style, one must also specify the control discipline, the data organization, and the interaction of control and data
29 September 1999 Architecture Session – Rapporteur talk
17
Architecture Styles: Part IIArchitecture Styles: Part II (2) (2)
Control issues How control passes among components
control topology linear or acyclic, hierarchical/tree-like, star/hub-and-spoke, or
arbitrary
How components work together in time synchronicity lockstep (sequential or parallel),
synchronous, or asynchronous
Data issues How data moves around a system
Data topology (as for control) Continuity continuous/sporadic, high/low volume Mode passed, shared, or broadcast
29 September 1999 Architecture Session – Rapporteur talk
18
Architecture Styles: Part IIArchitecture Styles: Part II (3) (3)
Control/data interaction issues Are control-flow and data-flow topologies isomorphic?
If isomorphic, is the direction the same or opposite?
Useful examples of architectural patterns Call-and-return styles data abstraction
(object-oriented)
Data flow styles pipe-and-filter systems
Data-centered repository styles blackboard
Interacting process styles implicit invocation
Note: few systems are purely any one of these!
29 September 1999 Architecture Session – Rapporteur talk
19
Architecture Styles: Part IIArchitecture Styles: Part II (4) (4)
Call-and-return styles: data abstraction Localized state maintenance (encapsulation) components: managers connectors: method invocation control: decentralized, usually single thread data: passed disadvantage: interface coupling Same topology
and directiontopology: arbitrary
method calls
managers
29 September 1999 Architecture Session – Rapporteur talk
20
Architecture Styles: Part IIArchitecture Styles: Part II (5) (5)
Data flow styles: pipe-and-filter systemsFilters transform input into output components: filters - computational i.e. retain minimal
state connectors: data streams control: asynchronous data: passed
pipefilter
Same topology and directiontopologies: arbitrary, acyclic (no feedback), fanout, pipeline (linear)
29 September 1999 Architecture Session – Rapporteur talk
21
Architecture Styles: Part IIArchitecture Styles: Part II (6) (6)
Data-centered repository styles: blackboard or DBCentralized data, usually structured components: central data store, many computational objects connectors: computational objects interact with central
store directly or via method invocation control: may be external, predetermined or
internal data: shared or passed topology: star
computational objects
memory
Blackboard (shared data)
29 September 1999 Architecture Session – Rapporteur talk
22
Architecture Styles: Part IIArchitecture Styles: Part II (7) (7)
Interacting process styles: implicit invocationIndependent reactive objects (or processes) components: objects that register interest in “events” and
objects that “signal events” connectors: automatic method invocation control: decentralized, de-coupling of sender and
receiver data: passed with event, may also require a shared
repository topology: arbitrary
“event” signals?!
?!
!
?
method invocation
29 September 1999 Architecture Session – Rapporteur talk
23
Architectural Styles in HEP: OutlineArchitectural Styles in HEP: Outline
Global structure
Foundation libraries
An example architecture: LHCb’s GAUDI
Variations of algorithm/event data styles
Application Meta Information
Architectural issues of persistency
User interface (UI) and visualization
29 September 1999 Architecture Session – Rapporteur talk
24
Architectural Styles of HEP Architectural Styles of HEP ExperimentsExperiments
I believe that most people would agree with a global software structure as expressed by
FrameworksToolkits
Reco
nst
ruct
ion
Sim
ula
tion
Analy
sis
Foundation LibrariesH
igh level tr
iggers
One main framework
Various specialized frameworks: visualization, persistency, interactivity, simulation, etc.
A series of basic libraries widely used: STL, CLHEP, etc.
Applications built on top of frameworks Implements the required physics algorithms.
29 September 1999 Architecture Session – Rapporteur talk
25
Foundation LibrariesFoundation Libraries
These form a basic vocabulary which is used throughout the code
Most experiments today propose to use C++ Standard library (STD)
collections, iterators, algorithms, stings, streams, numerics
For missing or HEP-specific pieces (e.g. via LHC++) CLHEP - random number generators, physics vectors,
geometry and linear algebra
ROOT libraries: containers, vectors, etc.
open questions: more linear algebra (Blitz++ or SL++), units - G4 or Fermilab’s SIunits package, pieces missing - error logger, exception handling
29 September 1999 Architecture Session – Rapporteur talk
26
An Example Architecture: LHCb’s An Example Architecture: LHCb’s GAUDIGAUDI
LHCb has specified the high level view of their software system’s architecture
The specification consists of Scenarios and requirements
Overall system design set of major design criteria identification of the major components and their interactions physical design of their system (i.e. packaging)
More detailed specification of the individual components e.g. purpose, interface, dependencies
Currently GAUDI is being implemented and deployed
29 September 1999 Architecture Session – Rapporteur talk
27
GAUDI ArchitectureGAUDI Architecture (2) (2)
Major design criteria Clear separation between data and algorithms
Three basic types of data: event, detector, statistics
Clear separation between persistent and transient data
Data store -centered architectural style
User code encapsulated in few specific places algorithms and converters
All components with well defined interfaces and as generic as possible
29 September 1999 Architecture Session – Rapporteur talk
28
GAUDI ArchitectureGAUDI Architecture (3) (3)
TObjTObj
Obj2
AppManager
PersistencySvc
Algorithm1Algorithm1
Algorithm1
JobOptionsSvc
TObj
TObjContainerTObjContainer
ObjContainer
Obj3
MessageSvc
TObj1Obj1
DetDataSrv
TDetElem1TDetElem1
TDetElem1
PObjectPObjectPDetElem
EventDataSvc
AlgFactory
AnotherPercySvc
Transient Event Store
PObjPObj
PObj
PObjPObj
PObj
DetPerstySvc
Alg Properties
T Detector Store
T Histogram Store
HistogramSvc
Hist1Hist1
Hist1
HistPerstySvc PHistPHist
ConverterConverter
Converter ConverterConverter
Converter
EventSelector
Converter
TObjObj1
uses
creates
navigability
User code
User code
User code
29 September 1999 Architecture Session – Rapporteur talk
29
Architectural Styles: Use of Architectural Styles: Use of InterfacesInterfaces
GAUDI’s top level components are interacting
objects Application Manager, Algorithms and
Services
To limit the coupling between objects, they make use of Java-like interfaces
any component implements one or more interfaces
clients of a component hold references to interfaces, not to the concrete componentIAlgorithm
IProperty
IDataProvider
IDataProvider
IHistogramSvc
IMessageSvc
Client
EventDataService
DetectorDataService
HistogramService
MessageService
ConcreteAlgorithm
29 September 1999 Architecture Session – Rapporteur talk
30
Architecture Styles: Part IIArchitecture Styles: Part II (5) (5)
Call-and-return styles: data abstraction Localized state maintenance (encapsulation) components: managers connectors: method invocation control: decentralized, usually single thread data: passed
method calls
managers Same topology and directiontopology: arbitrary
29 September 1999 Architecture Session – Rapporteur talk
31
GAUDI AlgorithmsGAUDI Algorithms
AlgorithmA
AlgorithmB
AlgorithmC
Transient EventData Store
Data T1
Data T2, T3
Data T2
Data T3, T4
Data T4
Data T5
T1
T5
Real data flow Apparent data flow
• Each algorithm only knows what data (type and name) is expecting as input and creating as output
• The only coupling is through the data
• Scheduling of sub-algorithms is responsibility of the parent algorithm
A
C
B
Parent
T2
T4
T3
Control flow
29 September 1999 Architecture Session – Rapporteur talk
32
GAUDI’s Algorithm/Event Data GAUDI’s Algorithm/Event Data Style Style
Algorithm and event data logical view – follows pipe-and-filter style
algorithms – filters event data – data stream
data flow is implemented as a data-centered repository
this implies that data is effectively shared between all filters issues: policy on shared data needed, e.g. who can modify what?
control flow follows the logical data flow no general mechanism for control (who calls whom) control is left up to the application manager and each “parent”
algorithm
data and control flow: in same direction, but different paths
29 September 1999 Architecture Session – Rapporteur talk
33
Architecture Styles: Part IIArchitecture Styles: Part II (5) (5)
Data flow styles: pipe-and-filter systemsFilters transform input into output components: filters - computational i.e. retain minimal
state connectors: data streams control: asynchronous data: passed
pipefilter
Same topology and directiontopologies: arbitrary, acyclic (no feedback), fanout, pipeline (linear)
29 September 1999 Architecture Session – Rapporteur talk
34
Architecture Styles: Part IIArchitecture Styles: Part II (6) (6)
Data-centered repository styles: blackboard or DBCentralized data, usually structured components: central data store, many computational objects connectors: computational objects interact with central
store directly or via method invocation control: may be external, predetermined or
internal data: shared or passed topology: star
computational objects
memory
Blackboard (shared data)
29 September 1999 Architecture Session – Rapporteur talk
35
Variations of Algorithm/Event Data Variations of Algorithm/Event Data StylesStyles
Many of today’s HEP experiments use a similar style BaBar and CDF share AC++ framework
framework is a state machine: loops over modules at transitions
module interface: Init, BeginRun, Event, EndRun, EndJob, TalkTo
follows pipe-and-filter style allow parallel data flow paths, filters can stop flow data flow is the event: central repository control flow (done by framework) is separate but parallel
input modul
e
output modul
eeven
t
control framework
app modul
e
app modul
e
app modul
e
app modul
e
control
input/output modules do
event selection
29 September 1999 Architecture Session – Rapporteur talk
36
CMS CARF: Reconstruction via notification andaction-on-demand
follows implicit invocation and central repository styles
No central ordering of actions, no explicit control of data flow: only implicit dependencies
External dependencies managed through an Event Driven Notification to “subscribers”
Internal dependencies through an Action on Demand mechanism
how does it work? reconstruction algorithms register themselves saying what
they produce and their name a client asks the event for something, e.g. a list of tracks, and
may specify which algorithm is to be used iterating over the list induces the algorithms to be run
Variations of Algorithm/Event Data Variations of Algorithm/Event Data StylesStyles (2) (2)
29 September 1999 Architecture Session – Rapporteur talk
37
CMS CARF: example
EventEvent
Rec T2Rec T2
Rec T1Rec T1
AnalysisAnalysis
HitsHits
T1T1
T2T2
DetectorDetectorElementElementRec HitsRec Hits
Variations of Algorithm/Event Data Variations of Algorithm/Event Data StylesStyles (3) (3)
29 September 1999 Architecture Session – Rapporteur talk
38
Architecture Styles: Part IIArchitecture Styles: Part II (7) (7)
Interacting process styles: implicit invocationIndependent reactive objects (or processes) components: objects that register interest in “events” and
objects that “signal events” connectors: automatic method invocation control: decentralized, de-coupling of sender and
receiver data: passed with event, may also require a shared
repository topology: arbitrary
“event” signals?!
?!
!
?
method invocation
29 September 1999 Architecture Session – Rapporteur talk
39
Object Networks under consideration by ATLAS follows both pipe-and-filter and implicit invocation
styles: has general mechanism to pass by reference any object: event,
tracks, clusters - no central repository control flows in same direction as data flow (done by framework) module execution is triggered by an input which has changed
(uses observer pattern - implicit invocation) filters can stop flow, as for CDF/BaBar
input modul
e
output modul
eeven
t
module
module
module
module
module
module
module
event
intermediate data objects
Variations of Algorithm/Event Data Variations of Algorithm/Event Data StylesStyles (4) (4)
29 September 1999 Architecture Session – Rapporteur talk
40
Variations of Algorithm/Event Data Variations of Algorithm/Event Data StylesStyles (5) (5)
Clustermanager
Data
AliRoot base classes
Raw datamanager
Data
Trackmanager
Data
TPC
Hits Pars
MUON
Hits Pars
Run Manager
ITS
Hits Pars
GlobalReconstruction
control
dataDetector modules
Global data managersblackboards
Global algorithms
ROOT IO
IOIO
ALICE’s reconstruction: data abstraction and data-centered repository
29 September 1999 Architecture Session – Rapporteur talk
41
Architectural Issues of PersistencyArchitectural Issues of Persistency
The questions of persistency or I/O are primarily treated in another session
Which issues are pertinent to software architecture? What is the desired coupling of the system to the
persistency mechanism?
How does one design an architecture which allows access to the “advantages” of a desired I/O system, e.g.
ODBMS? access to different types of I/O systems? migration to new technologies?
Are there performance implications?
What are the implications of wide-area distribution of the data and the users?
29 September 1999 Architecture Session – Rapporteur talk
42
Architectural Issues of PersistencyArchitectural Issues of Persistency (2)(2)
From the application’s point of view, clients (e.g. algorithms) want to see objects as if they were transient. They
do not want to “know” about persistency details
just want dereferencing semantics: ptr->object
want to be able to traverse relationships
Track HitsOnTrack* Hit*
Where is the knowledge about what is in memory or on disk, tape or WAN?
the I/O implications when one follows an association or dereferences a pointer?
TrackSet
29 September 1999 Architecture Session – Rapporteur talk
43
Architectural Issues of PersistencyArchitectural Issues of Persistency (3)(3)
Example architecture 1)
algo 2
algo 1converted shapepersistent
shapetransient
shapedisk
to memory
ODBMS knowledge
Forced conversion when bringing objects in - do not let application see “persistent object”
Advantages: application insulation, gain control over the persistency interfaces, future migration
Disadvantages: potential loss of ODBMS benefits - may need to implement on
transient side number of classes or converters
optimized for a specific purpose
29 September 1999 Architecture Session – Rapporteur talk
44
Architectural Issues of PersistencyArchitectural Issues of Persistency (4)(4)
Example architecture 2)ODBMS
knowledge
Allow application to access persistent objects directly
Reduces to architecture 1) if Conversions are used everywhere Paged-in from disk with non-trivial conversion/streamer
Advantages: ODBMS benefits – avoids reimplementing object manager, avoids extra classes where unnecessary
Disadvantages: must take care not to contaminate application with ODBMS knowledge, future migration
optimized for a specific purpose
algo 2
algo 1converted shapepersistent
shapedisk
to memory no transient
shape
29 September 1999 Architecture Session – Rapporteur talk
45
Architectural Issues of PersistencyArchitectural Issues of Persistency (5)(5)
ROOT and Objectivity can be used with either architecture
RDBMS & legacy systems require architecture 1)– needs transient shape
Who chose what? Arch 1 Arch 2 BaBar CDF (non-trivial streamers) D0 (done by D0om layer) ALICE ATLAS ? CMS LHCb (use converters)
29 September 1999 Architecture Session – Rapporteur talk
46
Architectural Issues of PersistencyArchitectural Issues of Persistency (6)(6)
What was the motivation?
What price was paid? (see the persistency session)
29 September 1999 Architecture Session – Rapporteur talk
47
Application Meta InformationApplication Meta Information
This is run-time knowledge about classes
What is it? Can take a number of forms
ROOT dictionary – CINT & RTTI Database schema & indexes CORBA interface repositories Java reflection Automation or scripting by SWIG
Degree & ease of access to this information varies
What is the role of scripting? Complete scriptability requires rich meta-information
29 September 1999 Architecture Session – Rapporteur talk
48
Application Meta InformationApplication Meta Information (2) (2)
Architects need to answer some questions What do you want to do with it?
What role does it play in the architecture?
How many separate repositories?
How complete should the information be?
Example from ROOT: the meta-information core is the key component of the architecture, used for I/O,
interpreter,
context-sensitive GUI, browsers,
class documentation
29 September 1999 Architecture Session – Rapporteur talk
49
Application Application Meta Meta InformationInformation (3) (3)
29 September 1999 Architecture Session – Rapporteur talk
50
User Interface (UI) and User Interface (UI) and VisualizationVisualization
It is important to be able to visualize and interact easily with objects in the application
One also must be able to run in batch mode
Are visualization and UI important? Yes!
Should they dominate the architecture? No
Questions Where should the knowledge about the UI be?
Can the architecture be designed with a detachable UI?
If the UI is detachable, what should be the back-bone communication channel? Decoupled in design? Scripting?
29 September 1999 Architecture Session – Rapporteur talk
51
Implementation and Physical Implementation and Physical DesignDesign
Large software systems need to be decomposed into small and manageable components
A decomposition has significant implementation-related consequences Compile-time coupling, link-time dependencies, size of
executables, etc.
Physical design focuses on grouping classes together into packages Minimizing the number of package dependencies is a key
issue
Because of the consequences, system architects should be involved in the physical design
29 September 1999 Architecture Session – Rapporteur talk
52
Implementation and Physical Implementation and Physical DesignDesign (2) (2)
Many HEP experiments find “guidance” in[Lakos] and [Martin]
Examples that help packaging succeed Abstract interfaces
Factory and strategy patterns [Gamma]
Dynamic loading
Component technology, e.g. CORBA or DCOM
Splitting into packages according to abstractness vs. instability
Should fall on line from abstract and stable/independent to concrete and unstable/dependent