07 th April, 2006IT Controls Group1 C5 Presentation: IT-CO 7 th April, 2006 Group set up in ECP...

07th April, 2006 IT Controls Group 1

C5 Presentation: IT-COC5 Presentation: IT-CO77thth April, 2006 April, 2006

Group set up in ECP 1993 General Goals:

Pool research sector resources in Controls Avoid duplication of effort

Promote standard solutions Industrial where applicable

Collaborate CERN-wide via Controls Board

From 1998 Support Joint COntrols Project (JCOP) activities

Long-Term Assume responsibility for all JCOP developments


OrganigramOrganigram

GL: D.R. Myers BE: W. Salter (also Deputy GL)

P. Burkimsher, F. Calheiros, D. Davids, L. Fernandez, P. Golonka, M. Gonzalez Berges, O. Holme,S. Schmeling, F. Varela

FE: R. Barillère A. Burmyakov , M. Beharrell, N. Boroukhoff , S. Cabaret,

N. Kulman, J. Ortola, J. Rochez, G. Thomas,

SI: B. Flockhart D. Filippov, A. Karlov, S. Lüders, G. Morpurgo, J. Pires,

R. Stampfli, S. Wolff(Fellows and Students in Italics: 11 out of 28)


Controls Technologies

Supervision

ProcessManagement

FieldManagement

Sensors/devices

Field buses & Nodes

PLC/UNICOS

OPC

Communication Protocols

SCADA

Technologies

VME/PC/SLiC

MODBUS…

FSM

Commercial Custom

Stolen from Wayne and based on an original slide from LHCb

Layer Structure

Experimental equipment

LAN

WAN

Storage

Oth

er

sy

ste

ms

(LH

C,

Sa

fety

, ..

.)

Configuration DB,Archives,Log files, etc.

Controller/PLC VME

Field Bus

LAN

Node Node

D.I.P. DIM


ServicesServices

Front-End Systems Research: Field Buses, PLCs, SLiC, DIM

CERN-wide: OPC, DIP, CAN Bus

PVSS (SCADA) Support Lab/World-wide (~400 licenses)

Experiment Support “Hands-On” for LHC + remaining Fixed Target

National Instruments Support Lab-wide: LabVIEW + N.I. Hardware (~600 users)


ProjectsProjects

DCS Framework Customize controls solutions for H.E.P.

GCS: Gas Control Systems In collaboration with PH-DT1/GS, provide

controls for all (~23) LHC experiments’ gas systems

DSS: Detector Safety Systems Highly reliable systems to ensure safety of

equipment for all four LHC experiments


External ActivitiesExternal Activities

JCOP Project Leader (WS) Controls Board membership (DRM, WS) Data Interchange W.G. (Chair WS) CNIC (Chair SL) LECC (LHC Electronics Coordination) Representative to OPC foundation (RB) Outsourcing, ICALEPCS, …


Front-End SystemsFront-End Systems

Supervisory System

Front End

Devices

DCS


F.E. Systems … Field Bus’F.E. Systems … Field Bus’

What are they? Serial bus, deterministic, cheap, easy to implement

Types at CERN (selected by the Field Bus WG) CAN/CanOpen, ProfiBus WorldFip, Modbus/TCP

Users Research (ProfiBus) CERN wide (CAN)

Service Interface selection, Integration, Trouble shooting

Supervisory System

Front End

Devices


F.E. Systems…PLCF.E. Systems…PLC

What are they? Micro Controller based, minimal OS, small footprint, no disk

Why use them? Reliable, Deterministic, Stable platform

Which Schneider, Siemens

Users Research sector Cooling, Magnet, CRYO, etc.

Service Selection and acquisition Developing and integration Trouble shooting

Supervisory System

Front End

Devices


F.E. Systems…SLiCF.E. Systems…SLiC

What are they? PC/VME based HW, Framework based SW on XP/Linux

Why use them? Specialized HW.

Users Experiments (fixed target)

Service Develop and support Framework solution. Assist users in development using FW.

Supervisory System

Front End

Devices


F.E. Systems… OPCF.E. Systems… OPC

What is it OLE for process control Designed specifically for device access.

Why Vastly reduced integration time Low maintenance overheads

Users CERN wide

Services Custom server development COTS requirements and acceptance testing Product selection. Training, trouble shooting.

Supervisory System

Front End

Devices


F.E. Systems…DIM/DIPF.E. Systems…DIM/DIP

What is it Publish/subscribe. Similar to OPC in functionality.

Advantages Multi-platform. Simple to learn/use.

Users DIM - Experiments. DIP – CERN wide

Gas, DSS, CSAM , accelerator, etc. Services

Custom server development DIM - 1st level support

General questions Problem and fix verification

DIP – Design, develop and maintain DIP libraries. Administer the DIP infra-structure.

Supervisory System

Front End

Devices


PVSSPVSS

Prozessvisualisierungs- und Steuerungssystem Produced by ETM (Austria) Extensive market survey

Evaluation 1997-99 ( >10 man-year effort) Selection in 2000 after CERN tender Partnership with company

CERN wide standard tool for control systems LHC experiments Fixed target experiments Accelerator applications

LHC Cryogenics, LHC and SPS Vacuum, LHC Quench Protection System (QPS), etc


SCADASCADA

Common toolkit to build Controls

Applications

Communications

Hardware

Middleware

User Interfaces

Trending

Alarm handling

Long term archiving

...

OPC

DIMDIP

MODBUS

S7

Database

File system

SCADA

System


PVSS Main FeaturesPVSS Main Features

Openness Easy customization for HEP environment Scripting language C++ API

Device Oriented (~Object Oriented) Large number of similar devices

Scalable ~1 million devices per experiment

Distributed systems ~100 computers per LHC experiments

Runs on Windows and Linux


PVSS Support (I)PVSS Support (I)

Management of users/licenses > 2000 licenses generated since 2001 CERN and around 100 institutes in 26 countries

Software Distribution of new versions and patches Certification on CERN environment: OS, databases, etc

User support 5 persons first line, 9 (whole section) second line Remedy ~2000 calls in 2005 Regular on site interventions

Link with ETM Regular meetings Gathering of CERN requirements Discussion of new developments Specific developments for CERN


PVSS Support (II)PVSS Support (II)

Web material Tutorials, FAQs, Twiki

Courses Preparation and delivery Three levels

Basic PVSS and JCOP Framework Finite States Machines Course JCOP Framework Advanced Course

Very popular (~300 participants since 2003)

Consultancy Advice on best usage of the tools Joint development with subdetectors


PVSS OPC, DIM, DIP

FSM Databases Web

LHC Experiment Controls Application

Framework Core

Framework Devices

JCOP FrameworkJCOP Framework

Framework Tools

LHC Experiments Applicattions

…

…

…

Fixed Target Experiments Applicattion

s

UNICOS Framework

Accelerator

Applications

GCS Framework

LHC Exp Gas Applics LHC

Exp Magnet

s


JCOP Fw: IntegrationJCOP Fw: IntegrationFull experiment

CaenSY1527SpeedAlarmClearPrimPSClockFqSoftRel

Subdetector 1 Subdetector 2

CaenCrateclearAlarmclkFreqprimaryPSfanSpeedswRelease

Subdetector n

SY1527CrateFanSpeedPrimaryPowerSupplyClockFrequencySoftwareRelease

SY1527CrateCommandsAlarmClear





DeliverablesDeliverables

Guidelines E.g. Look and Feel, Naming conventions, Alarm classes, code

organization, etc Devices

Support for common hardware in the experiments Easy extension to specific devices

Tools Automatic generation Hierarchical organization Finite States Machines Configuration from a DB Access Control

Development driven by users Iterative process

Contributions from the experiments


Example of User InterfaceExample of User Interface


Diversity Optional gas modules Optional devices

Commonalities Standard devices:

Valves, flow meters, mass flow controllers, etc… Modular architecture

Gas Modules Mixer, Pump, Distribution, Purifier, etc

4 experiments ~23 gas systems to build

Gas Control Systems Gas Control Systems Problem descriptionProblem description

Gas System Racks

profiBUS-daisy-chain of4 Mass Flow Controllers


GCS- Scope/objectivesGCS- Scope/objectives

graphical object

Supervision Layer

Middleware

Process ControlLayer

Gas module operational states (FSM)

Trees RecipesHMI ApplicationSCADA - PVSS

Communication

Logic Control Application IEC 61131 Languages

Interlock logic

PLC objects Library

I/O objectsField objects

Control objects

objects Library

PLC object proxies

Views


GCS- Strategy/principlesGCS- Strategy/principles

Principles Model-oriented design for both

Supervision Process control

Tools for the automatic generation of PVSS and PLC code

Strategy Gas systems’ architecture based on a generic model Use industrial technologies

SCADA, PLC, fieldbuses Use of UNICOS FW

PVSS and PLC object libraries Data-driven code generator tools

A complete automatic generation process to produce the PVSS and PLC code of any GCS instance.

Application developer

meta-model

tools

Turn-key control system

Alice TPC(2 gas lines)

PVSS code

Supervision files

Logic files

PLC code

Objects options Objects options

Turn-key control system

Alice (PMD)

PVSS code

PLC code

1. Generate any GCS instance

2. Generate same instance but with different options

3. Add a new gas module type

Alice TPC(3 gas lines)


GCS- Development GCS- Development status/planningstatus/planning

GCS Framework and model-driven tools Tested/validated/in use Open architecture –extensible Flexibility Time saving

GCS instances- commissioning Alice TPC delivered and in operation Atlas RPC in preparation (delivery end of April) Mid 2006: 6 small-size systems (5 Alice and 1 Atlas) End 2006: 4 CMS systems Before LHC start: Remaining ~13


Detector Safety SystemDetector Safety System Why : to cover a “safety gap” left between

CSAM and DCS, and protect the LHC Detectors against dangerous situations.

What does it do : the DSS “protects” the LHC Experiment’s equipment by autonomously taking User-defined ActionsActions as a consequence of the detection of User-defined Alarm Conditions

CSAM == CERN Safety Alarm Monitoring system (mainly deals with personnel safety)

DCS == Detector Control System


DSS Main FeaturesDSS Main Features

Clear split between Safety part (PLC) versusConfiguration & Monitoring part (PVSS)

Data-Driven approach 4 (+1) Experiments, 1 Identical Software Simplicity in Safety part, sophistication in User Interface Flexibility (User can add or modify Safety Rules at any

time, without any software change)


DSS Front End (PLC)DSS Front End (PLC)

Redundant PLC System to insure very high availability

Passive Safety to improve reliability PLC Code implementing the Safety

kept as simple as possible. Cyclical repetition of basic operations on the contents of the PLC datablocks.

Capable of stand-alone operation


DSS Back End (PVSS)DSS Back End (PVSS) Configuration : validates the User-defined

Safety Rules, and decomposes them into basic operations in the PLC data-blocks

Monitoring : displays the Alarms detected by the Front End, and the Actions that have been taken

Interactive User-configurable Synoptic System

Interface to PLC through OPC


DSS Current StatusDSS Current Status

Five DSS systems installed, and slowly coming into real operation

System capacity : ~1000/2000 Sensors, ~1000 Alarm Conditions, ~1000 Actions

PLC cycle time <= 1 sec.

Thanks to the Data-Driven approach, the DSS is a “General Purpose” Safety System, reusable in other domains


National InstrumentsNational Instruments

Support for NI products at CERN Community of ~600 active users (50%

CERN, 50% visitors) Software Hardware Training Assistance with selected projects


NI SoftwareNI Software

LabVIEW, DSC, FPGA and several other packages available

Supported platforms: Windows, Linux, Mac, (Sun) Unlimited site license for LabVIEW and DSC through

license server Limited floating licenses for other products Distribution for Windows through NICE/CMF Version management

Evaluation of new products & releases


NI HardwareNI Hardware

Lab available for product evaluation and testing

Advice and support for hardware selection Modules Interfacing with real world

Support for ordering when requested Support for repairs (“lend & send”)


NI TrainingNI Training

NI training available on site for all NI products

Organized through Technical Training for major packages

Organized by IT/CO/SI for specialized subjects

NI Seminars (~6 per year)


NI Selected ProjectsNI Selected Projects

Direct involvement with selected end-user projects (help, consultancy and development)

Good opportunity to maintain a level of expertise within the section

Feedback to NI (bugs, user requirements, etc.)


Controls Goes IT…Controls Goes IT…

Controls networks mate business networks Proprietary field buses replaced by Ethernet & TCP/IP Field devices connect to Ethernet & TCP/IP VPN connections onto the Controls Network

Use of IT protocols & gadgets: SNMP, SMTP, FTP, Telnet, HTTP (WWW), … Wireless LAN, Notebooks, USB sticks, …

Migration to Microsoft Windows O/S platform


Controls Under Attack !?Controls Under Attack !?

2000: Ex-Employee hacks “wirelessly”46 times into sewage plant and spills basement of Hyatt Regency hotel.

2003: The “Slammer” worm disables safety monitoring system of the David-Besse nuclear power plant for 5h.

2003/08/11: W32.Blaster.Worm

220-<<<<<<>==< Haxed by A¦0n3 >==<>>>>>>220- ¸,ø¤º°^°º¤ø,¸¸,ø¤º°^°º¤ø,¸¸,ø¤º°^°º¤ø,¸¸,ø¤º°^°º¤ø220-/220-| Welcome to this fine str0220-| Today is: Thursday 12 January, 2006220-|220-| Current througput: 0.000 Kb/sec220-| Space For Rent: 5858.57 Mb220-|220-| Running: 0 days, 10 hours, 31 min. and 31 sec.220-| Users Connected : 1 Total : 15220-|220^°º¤ø,¸¸,ø¤º°^°º¤ø,¸¸,ø¤º°^°º¤ø,¸¸,ø¤º°^°º¤ø,¸¸,ø¤º

2006: Hacked “LeCroy” oscilloscope at CERN (running Win XP SP2)


Computing & Network Computing & Network Infrastructure for ControlsInfrastructure for Controls

“…with a mandate to propose and enforce that the computing and network supportprovided for controls applications is

appropriate” … “to deal with security issues.”


Protection of Controls Protection of Controls NetworksNetworks

Further protection of vulnerable devices like PLCs (II/2006)

Remote access schemes via WTS(ongoing in 2006)

CERNGeneral Purpose Network

Experiment Network

“trusted”

Fat pipe

Dedicated networks forALICE, ATLAS, Cast?, CMS, Compass,LHCb, and accelerators / safety (“TN”)

Rules for new connections, wireless access & laptops (2005)

Restriction ofcross-communication (since I/2006)


Centralized Installation Centralized Installation of Control PCsof Control PCs

Benefit of a central installation scheme butallow the user to take responsibility HE decides WHEN to install WHAT on WHICH Control PCs HE has to ensure security (with threat of being blocked)

CMF used now forAB/CO, ALICE, ALPHA, ATLAS, CAD, CCC, DSS, MCS, TS/CSE, TS/CV, TS/EL, TS/FM, …

LinuxFC needs more pushing…


Forum for CNIC UsersForum for CNIC Users

Frequent presentations on “User Awareness”

Bi-weekly user meetings ─ the CNIC UX Regular presentations on tools and solutions Exchange on how to use CMF, LinuxFC, Network, WTS, …

Dedicated discussions on Operator Accounts (I/2006) Intrusion detection systems on the TN (I/2006) Separate domains for fixed-target experiments (II/2006)

20.01.2004 42

The EndThe End

07 th April, 2006IT Controls Group1 C5 Presentation: IT-CO 7 th April, 2006 Group set up in ECP...

Documents

Transcript of 07 th April, 2006IT Controls Group1 C5 Presentation: IT-CO 7 th April, 2006 Group set up in ECP...