MACS Architecture MACS Week June 24th, 2010 Johannes Gutleber PR-100616-b-JGU, June 24th, 2010 J....
-
Upload
asher-rose -
Category
Documents
-
view
215 -
download
0
Transcript of MACS Architecture MACS Week June 24th, 2010 Johannes Gutleber PR-100616-b-JGU, June 24th, 2010 J....
J. Gutleber1
MACS ArchitectureMACS Week
June 24th, 2010Johannes Gutleber
PR-100616-b-JGU, June 24th, 2010
R. Gutleber
J. Gutleber2
Functional Modeling Concept (FMC)
• Notation used to describe architecture
PR-100616-b-JGU, June 24th, 2010
J. Gutleber3
Scope
PR-100616-b-JGU, June 24th, 2010
MedicalSoftwareSystems
Access ControlSystem
BuildingAutomation
System
AcceleratorFrontendSystems
Beam deliveryFrontendSystems
BeamInterlockSystem
Authentication&
AuthorizationSupervisory Control System
AcceleratorControl System
Beam DeliveryControlSystem
Scope of Work Package
SafetyManagement
System
RadiationMonitoring
System
Scope of MACS
J. Gutleber4
3 + 1 Tier Architecture
PR-100616-b-JGU, June 24th, 2010
Control System Work Package
Presentation tier (1)
Processing tier (2)
Equipment tier (3)
Subsystem Work Package
Frontend tier (4)
Common work
Protocols and data format adapters, fan-out and data concentration, timing, SADS
Configuration for devicesSupervisory state machinesData logging, SCADA
Graphical user interfacesRemote operation
Control logic and closedReal-time control loops
Front-end adaptation and lean framework layer
J. Gutleber5
Repository Management System
PR-100616-b-JGU, June 24th, 2010
Graphical User Interfaces & Data Import/Export Interfaces
Data Model Views
Data Model Access Controller
Data Model
ComponentSetpoints
Waveforms
Alarm Thresholds
Alarm Conditions
AcceleratorComponentDefinitions
ElectronicsComponentDefinitions
SoftwareComponentDefinitions
ComponentConnectionProtocols
Relations amongComponents
ManufacturingData
Virtual AcceleratorDefinitions
Data ArchivingSpecifications
Associations Components and User Interfaces
Geographical Component
Layouts
Accelerator Component
Optics
Main TimingSequences
Accelerator Cycle Definitions
Rooms
Working Set Definitions
Asset TrackingReports
Definition of Device
Permissions
DBMS
ComponentCommands
ComponentEvents
ComponentMonitorables
J. Gutleber6
Component Model
• Physical accelerator element or assemblies of physical elements are represented in software, called component
• Each component has• Unique identifier• Name• Properties• Ownership at runtime• State machine• May react to timing events (cycle dependent operation)
PR-100616-b-JGU, June 24th, 2010
J. Gutleber7
Uniform State Machine
• Every device in 1 of 4 modes• Service, Physics, QA, Clinical
• Every device implements all states and transitions
• Additional, device specific commands possible in “Op” state
• State machine implemented in “software representation” of device• Front End Controller (FEC)
PR-100616-b-JGU, June 24th, 2010
Init
Reset
Hold Ready Op
FaultFail
Reset Reset
EnableConfigure
Clear or Modify QuiesceClear or Modify
Clear
Reset
Initialize
J. Gutleber8
Working Sets• Working Set is a collection of defined components• Can be controlled as a “single virtual device”• Can be assigned a “single set of configuration parameters”
PR-100616-b-JGU, June 24th, 2010
Working Set
DeviceSoftware
Component
Device
Limited to 1 layerImplement the sameState-machine
J. Gutleber9
Errors and Alarms• Devices (hardware and software) limit themselves to provide
information that can be processed by SCS to generate alarms• Alarms are defined in PVSS based on numeric quantities and
Boolean expressions• “Failed” and “Fault” states for each device
• Device must stop working (fail-silent behaviour)• Possibility to recover from “Failed” via operator intervention• Alarm raised by SCS in either case that must be acknowledged
• Error during processing a command is notified to SCS• Not a failure! Alarm can be defined by operator at SCS level is needed
• QoS indicator for each “software” service
PR-100616-b-JGU, June 24th, 2010
J. Gutleber10
Logging (Execution)• Process of tracking changes in execution of a program• Rely on standard protocol, data format and tools
• Log4Net tool, Log4J protocol, all logs end up in PVSS
PR-100616-b-JGU, June 24th, 2010
Presentation TierProcessing TierEquipment Tier
Log Files
FECOSComponent
ProShellComponent
Other Component
LogCollector
SCADATool
Visualization
J. Gutleber11
Data Logging• Automated process of continuously collecting and archiving
process measurements on a timely basis. • O(3000) values collected at average frequency of O(1) Hz. • Carried out by PVSS SCADA tool• For each data item conditions can be specified that determine
when the value is collected or archived. • Limited transformations (e.g. unit conversion) may be applied
to the gathered values in order to record a set of derived data values.
• Archiving can be configured for each data item independent of the time when it is collected• tradeoff between storage space efficiency and measurement accuracy
PR-100616-b-JGU, June 24th, 2010
J. Gutleber12
Data Logging Architecture
PR-100616-b-JGU, June 24th, 2010
Processing TierEquipment Tier
FECOSComponent
ProShellComponent
OtherComponent
SCADATool
R
Configuration
RepositoryManagement
System
Archive
Alarms
Field data
Oracle
PVSS
Oracle
OPC
NI LV-RT
C#
PVSS Internal
J. Gutleber13
Beam Generation Process
PR-100616-b-JGU, June 24th, 2010
Generate beams
J. Gutleber14
Operation Principle
• Accelerator is split into logical machine subsystems (LMS)• Sources• LEBT• Linac• MEBT• Main Ring• Extraction Line• Individual irradiation lines
• Virtual Accelerator (Vacc) comprises at least 1 LMS working set
• VAcc works on a “cycle” basis
PR-100616-b-JGU, June 24th, 2010
J. Gutleber15
Cycle Definition
• Accelerator cycle describes beam characteristics to be produced during a period of time by specifying a set of configuration parameters.• What (parameters)• When (timing)
• Generation of cycle causes activation of pre-defined actions at pre-defined, cycle-dependent times in front-end software components that control the physical elements that constitute the accelerator.
• At each point in time, a single cycle is active per VAcc• A defined cycle can be played over a “VAcc”.
PR-100616-b-JGU, June 24th, 2010
J. Gutleber16
Cycle Definition Contents
• Ion type• Source type (optional, allows binding of cycle to source)• Target room of beam (optional, for clinical mode)• Energy• Beam dimension• Intensity• Spill length
PR-100616-b-JGU, June 24th, 2010
J. Gutleber17
Cycle Code64 bit wide identifier to which configuration parameters can be associated for each front end controller
PR-100616-b-JGU, June 24th, 2010
MSB LSBWord
31 20 29 28 27 26 25 24 23 22 21 20 19 18 17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 0
Beam line Spill length Degrader Source Particle Energy 0
Major Version Minor Ver. Fmt Mode Y size X size 4
Bit Size Name Description0-15 16 Energy n x 0,1 MeV from 0 to 800 MeV (8000 steps)
16-18 3 Particle species
0 unused, p, C and space for other light ions. 7 in total
19-21 3 Source number 0 means unbound, 7 sources if indication needed22-24 3 Degrader 0: 0%, 1: 10%, 2: 20%, 3: 50% (3 values)25-28 4 Spill length 0,1; 0,2; 0,5; 1, 2, 3, 4, 5, 6, 7, 10 sec (11 values)29-31 3 Beam line 0 means unbound, up to 7 if indication needed32-35 4 Beam x size Size in mm from 1 to 1036-39 4 Beam y size Size in mm from 1 to 1040-41 2 Mode 0: service, 1: physics, 2: QA, 3: clinical42-43 2 Code format 0: current version, others for future format changes44-47 4 Minor version Intermediary versions for MD, modes 0 (S) and 1 (P).48-63 16 Major version Version indicates which settings to use for this cycle
J. Gutleber18
Beam Stamp• A record of information updated throughout the entire
lifetime of the facility• Unambiguously associates in time any piece of information
generated during operation to a cycle • e.g. occurrence of a command, acquired data, alarms and errors
• Consists of a run number and a cycle number
PR-100616-b-JGU, June 24th, 2010
Start Cycle Stop CycleStart Cycle Stop CycleStart Run
action action action action
time
UTC timestamp
Run number,Cycle number,UTC timestamp
UTC timestamp
Run number,Cycle number,UTC timestamp
Cycle n Cycle n+1
J. Gutleber19
Virtual Accelerators
• Virtual accelerator (VAcc) is a collection of working sets (LMS)• Used for operation purposes (beam generation)• Freely definable using RMS• Non-overlapping VAccs can be active concurrently• If VAccs overlap, only 1 VAcc active at that time
PR-100616-b-JGU, June 24th, 2010
VAcc 1
VAcc 2
J. Gutleber20
VAcc State Machine
• BDCS allocates a Vacc from Virtual Accelerator Allocator (VAA)• If VAcc overlaps becomes “pending”• If “allocated”, resources are owned by requestor• If “active” VAcc is ready for beam generation
• Beam interception devices and safety elements are “armed”
• If VAcc is released/deactivated, “pending” gets allocated
PR-100616-b-JGU, June 24th, 2010
allocate activate
deactivate
Inactive Pending Allocated Active
Finished
release
J. Gutleber21
Beam Allocation
PR-100616-b-JGU, June 24th, 2010
BDCS
BeamGeneratorProcedure
MACSR
PRVSR
R
BeamGeneratorProcedure
SoftBDCS
R
VAA
R
BDCSBDCS VAA
SCADA Tool
MTS
RBDCS
R
R
J. Gutleber22
Blueprint ofSupervisory andAcceleratorControl SystemComponents
PR-100616-b-JGU, June 24th, 2010
J. Gutleber23
Network Organization
• SCS/ACS devices in physically separate technical network (TN) • Applies also to control room equipment
• General purpose network for office IT and private computers
PR-100616-b-JGU, June 24th, 2010
GPN
Switch
TN
Appliance
J. Gutleber24
Status Displays
• Current status of operation and near real-time update of cycle on various displays in the facility
• Available in techn. network and general purpose network
PR-100616-b-JGU, June 24th, 2010
SCADATool
StatusApplication withUser interface
PublisherSubscriber
B-TrainApplication
StatusApplication withUser interface
Push statusand field data
Push statusand
field data Push status
Push field data
StatusMessageGenerator
Server
Server
Client
General purpose network (GPN) Technical Network (TN)
J. Gutleber25
SCADA Tool
• PVSS to be configured from RMS database• Keeps state of entire system• Single point of entry to interact with all subsystems
PR-100616-b-JGU, June 24th, 2010
SCADA Tool
User InterfaceManager
ControlManager
ProShellManager
EventManager
DatabaseManager
ConfigurationManager
ArchiveManager
Driver
ArchiveDatabase
OnlineDatabase
RepositoryManagement
System
RedundancyManager
J. Gutleber26
Datapoint Concept
• Data structures that abstract from physical devices (DP)• PVSS interacts with devices by reading and writing to
elements (DPE) of data structures that represent those devices.
• Data point type (DPT) defines the structure of DP and contains a set of hierarchically ordered DPEs. • Can be of a simple type (e.g. string or integer) or a complex type (e.g.
static or dynamic lists of integers).
• Every subsystem or component is represented as a datapoint in the system
• All interaction with components/subsystems goes through the datapoints to maintain consistency and operation safety
PR-100616-b-JGU, June 24th, 2010
J. Gutleber27
Accessing Device Components• Supervisory panels
to control entire subsystems
• Virtual Instrument panels• Communication via
PVSS datapoints• Communication via
NI SV
PR-100616-b-JGU, June 24th, 2010
28
SADS
• Connect analogue signal sources to devices acting as sampling scopes display it on operator screens.
• Select video sources and tell on which screen to show them.• Acts as MD, analysis tool not as primary operation display• Requirements to be captured by October 2010
PR-100616-b-JGU, June 24th, 2010 J. Gutleber
SADS
Signalsource
Videosource
Signal anddestination
selector
Oscilloscopedisplay
Video sourceand destination
selector
Videodisplay
R
R
R
R
Signalsource
J. Gutleber29
SADS Architecture
PR-100616-b-JGU, June 24th, 2010
SADSRF
SwitchingMatrix
SignalSource 1
SADSProcessing
Element
SignalSource 100
MTS
Timing event
SwitchController
GUI 2
GUI 1
R
Disk
J. Gutleber30
Procedure Framework (ProShell)
• Functionality to automate repetitive supervisory tasks• Implement beam diagnostics procedures• Framework written in-house in C#• Procedures to be written by “experts” in C#
• Training and documentation to be provided by WP CO.
PR-100616-b-JGU, June 24th, 2010
J. Gutleber31
ProShell Architecture
PR-100616-b-JGU, June 24th, 2010
ProShell
ProshellManagement
GUI
ProcedureSpecific
GUIProcedure
R
Load, run, stopprocedure
Procedure Storage
Load procedures
SCADAEvent
Manager
Publisher/Subscriber
ConfigurationRepository
ProcedureLoader and
Registry
SCADAProxy
Dispatcher
ComponentFacade
R
Commands
Publisher/Subscriber
Proxy
R
Commands
Events
RepositoryProxy
R
Events
J. Gutleber32
T3 and T4 Architecture
• WP CO provides LV-RT programming framework (FECOS)• Uniform state management, configuration, communication• FEC software components provided by WP CO for individual
work packages on case-by-case basis (by default in WP)• FEC can be operated standalone or via Working Set/VAcc
PR-100616-b-JGU, June 24th, 2010
FE
CO
S
FEC FED Device
Analogueand
DigitalIO
Industrialfieldbus
TechnicalNetwork
Slow Control
Technical Network
Fast Control
Tier 2 Tier 3 Tier 4
J. Gutleber33
FEC Communication• FEC is building
block at T3• Defined
interfaces to T2• State machine
and slow control via OPC
• Direct data access via NI Shared Variable engine and DIM
• BLOBs via HTTP
PR-100616-b-JGU, June 24th, 2010
FEC
ProShell
SCADA Tool
NIMeasurement
Studio
SharedVariableEngine
OPCServer
SV
OPC
SV
DIM
HTTPWeb
Server
Tier 2 Tier 3
J. Gutleber34
Main Timing System• Coordinates devices in each subsystem (ES-090512-a-JGU)
• Precision of ordinary timing events O (1 μsec)• Synchronization of 2 receivers O(50 nsec), local solution• Non real-time events distributed via DIM (Ethernet, TN)• Reference clock O(10 MHz) foreseen• Provision O(10 Hz) stability pulse that can be picked on demand
• Broadcasts events• Occurrences in time that are assigned a logical name• E.g. start injection, start acceleration, start extraction, etc.• Sequences are statically defined for individual cycles
• Cycle definition comprises• Record of settings (WHAT)• Sequence of timing events (WHEN)
PR-100616-b-JGU, June 24th, 2010
J. Gutleber35
Typical Timing Events
PR-100616-b-JGU, June 24th, 2010
J. Gutleber36
Definition of Timing Sequence
PR-100616-b-JGU, June 24th, 2010
Generation of events is dynamicsince the BDCS decides, whichcycle to generate next.
J. Gutleber37
Main Timing System Concept
PR-100616-b-JGU, June 24th, 2010
MTG
Fanout
MTR
SRC RFQ/IH RFC Kickers Dipoles Fast Deflector BDIQuad.-
polesBetatron
CoreSext.-poles
OrbitCorrectors
MTR MTR MTR MTR MTR MTR MTR
Reference clock
J. Gutleber38
MTS Architecture
• Event broadcast sequencer (Main Timing Generator)• Real-time and non real-time event broadcast• Timing Box for scope trigger signal reception
PR-100616-b-JGU, June 24th, 2010
Main Timing System
Subsystem (RT) or TiBx
Beam DeliveryControl System
ProShellFramework
TechnicalNetwork
Main TimingGenerator
R
R
ReferenceClock
Main TimingReceiver
Physicaldeviceor TiBx
software
Timing Broadcast Network
Main TimingTransmitters
Gateway toTechnicalNetwork
Subsystem (Non-RT)
Gateway toTechnicalNetwork
TechnicalNetwork
Technical Network
Physicaldevice
R
J. Gutleber39
High-Precision Operation
• Precision of MTS events 1 µsec• For higher-precision, local solution based on programming the
timing receiver board (MTR) possible• Dynamic reconfiguration not foreseen• Output can be simple optical or electrical signals
• Precision of O(nsecs) can be achieved
PR-100616-b-JGU, June 24th, 2010
Main Timing Receiver
Delay 1
Delay 2
EventReceiver
TimingBroadcastNetwork
InjectionBumper
PCO
FastDeflector
PCO
Delay in nsec range
Event
Optical or electrical trigger link
J. Gutleber40
Power Converter Controller Architecture
PR-100616-b-JGU, June 24th, 2010
PCO FEC
PCO FEC
PCCComponent
PCOFED
PCCComponent
PCOFED
PCOFED
PCOFED
PCO
PCO
PCO
PCO
PCCComponent
PCOFED
PCO
CPU
PCCComponent
CPU
100 MBit/s (optical)1-to-1
MTS
Setpointsequence
driven(MR, BDS)
SetpointDriven(LEBT, MEBT, HEBT)
Ethernet (non-realtime)and
Optical (realtime)
Responsibility of WP:CO Responsibility of WP:PCO
MTR
Slow control
Fast control
PX
I ba
ckp
lan
e
MTR
PX
I ba
ckp
lan
e
Slow control
Optical trigger LEMO trigger
Optical trigger LEMO trigger
J. Gutleber41
PCC-FED-PCO Chain
PR-100616-b-JGU, June 24th, 2010
FED
AnalogueRegulation
BoardMemorybus
PCOV
V
I
Slow control, serial
FECCustomOptical
link
FED PCODigital values
Digital values
Slow control, serial
FECCustomOptical
link
Responsibility of WP:PCOResponsibility of WP:CO
J. Gutleber42
Conventional Magnets Slow Control
• Only Boolean values• Will be implemented reading out Beam Interlock System (BIS)• Saving on cabling via Profinet instead of point-to-point
PR-100616-b-JGU, June 24th, 2010
Magnet
TemperatureSwitch
FlowIndicator
PLCSCADAR
TechnicalNetwork
Tier 2 Tier 3 Tier 4
PLCIO
ProcessorFieldbus
Magnet
TemperatureSwitch
FlowIndicator
PLCIO
ProcessorFieldbus
J. Gutleber43
Special Magnet Slow Control
• Flow and Temperature via BIS• Cooling and positionning subject to analysis (summer student)
PR-100616-b-JGU, June 24th, 2010
Magnet
TemperatureSensor
FlowMeter
PLC
SCADA
R
TechnicalNetwork
PLCIO
ProcessorFieldbus
FrontendPositioning
System
PositionDetector
MotorR
TechnicalNetwork
MagnetTemperatureSensor
FlowMeter
MagnetCoolingSystem
Tier 4
Valve
R
TechnicalNetwork
Tier 2 Tier 3 Tier 4
J. Gutleber44
Vacuum Control Architecture
PR-100616-b-JGU, June 24th, 2010
VAC FEC
VACController
VACFED
Gauge
VACController
Gauge
CPU
PX
I ba
ckpl
ane
PumpVACFED
Valve
Pump
Fieldbus
Digital IO
Tier 3 Tier 4
FECOS and CrateProvided by WP CO,Mounted in same rackAs FEDs
Component programmedBy WP VAC
FED provided by WP VAC,serial RS422/RS485 interfaces,digital IO interfaces
J. Gutleber45
Sources Slow Control• Existing LV software
from supplier to be re-engineered
• CWO to Cosylab for provision of FECOS based component
• Beam diagnostics integrated into overall BDI system by BDI programmer
• Provision of MTS events foreseen• BDI, pulsed operation
PR-100616-b-JGU, June 24th, 2010
SRC BDI FEC
SRC FEC
Gas Injection
FEDValves
SRC BDIComponent
Gauge
CPU
PX
I ba
ckp
lane
BDIDevices
Fieldbus
Analogue and Digital IO
Tier 3 Tier 4
MTR
Pump
FED Microwaveamplifier
Microwavetuner
Microwavegenerator
PlasmaGeneration
HV
Chiller
FEDValves
Gauge
Fieldbus
Pump
VacuumController
CoolingController
ExtractionController
PX
I ba
ckp
lane CPU
MTR
J. Gutleber46
Beam Diagnostics & Instrumentation
• WP BDI programmer works with FECOS provided by WP CO• Standalone operation possible for each FEC• GUI: ProShell and PVSS with Measurement Studio plugins
PR-100616-b-JGU, June 24th, 2010
SCADAEvent
Manager
Publisher/Subscriber
TN
BDI FEC
BDIMonitor
Controller
BDIMonitor
CPU
PX
I bac
kpla
ne
MTR
FE
CO
S
LocalStorage
R
Local debug GUI
BDI FEC
TN
J. Gutleber47
Technologies Used• Primary means of communication with front-end:
• Transport: Ethernet + TCP/IP, RS 232/422/485, digital I/O for interlocks• Protocols: OPC (+ NI SV) + DIM (+ HTTP)• Data contents: Definition starts now based on CNAO XML schema• PLCs: Siemens, S7 protocol over Ethernet preferred
• SCADA + GUIs: PVSS II + Measurement Studio• Platform for Frontend:
• PXI crates and crate controllers, FlexRIO FPGAs• LV-RT operating system
• Platform for T2,T3:• Rack-mounted server PCs• Virtualized MS Windows XP (upgrade to 7 for deployment)
PR-100616-b-JGU, June 24th, 2010
J. Gutleber48
Technology Overview
PR-100616-b-JGU, June 24th, 2010
Presentation tier
Processing tier
Equipment tier
Frontend tier
Ethernet, TCP/IP
Ethernet, TCP/IP
Ethernet, TCP/IP
Recommendations forRS 232/422/485Digital IO
OPC, DIM, MRF for timing8B10B custom for setpoint sequences
PXI
Virtual Machines
Rack mountedPC (PCIe)
PLC
OPC, DIM, HTTP
PVSS, SRDP
MeasurementStudio
Inhouse C#
Ethernet,NI SV