New Control Systems at KSTAR Compatible with ITER ... IAEA Technical Meeting on Control, Data...
Transcript of New Control Systems at KSTAR Compatible with ITER ... IAEA Technical Meeting on Control, Data...
11th IAEA Technical Meeting on Control, Data Acquisition, and Remote Participation for Fusion Research,
08~12 May, 2017, Greifswald, Germany Page 1
New Control Systems at KSTAR
Compatible with ITER Standard Technologies
8th
May, 2017
Woongryol.Lee ([email protected]), Taegu.Lee, Giil.Kwon, Taehyun.Tak,
Myungkyu.Kim, Jinseop.Park, Yeon-jung.Kim, Jaesic.Hong, and KSTAR team
National Fusion Research Institute, Daejeon, Korea
11th IAEA Technical Meeting on Control, Data Acquisition, and Remote Participation for Fusion Research,
08~12 May, 2017, Greifswald, Germany Page 2
Outline
Overview the control system implemented with
CODAC standard technologies
Expansion of KSTAR control system by using
CODAC technologies
Improve network infrastructure
New software components for system I&C
Device standardization
Summary
11th IAEA Technical Meeting on Control, Data Acquisition, and Remote Participation for Fusion Research,
08~12 May, 2017, Greifswald, Germany Page 3
Evaluation and Demonstration of ITER CODAC Technologies at KSTAR
ITER CODAC technologies successfully demonstrated their capability for
Tokamak control at KSTAR through the discharge experiment
Main missions:
1. Develop the plant system I&C having real-time feedback control function with the CODAC technologies
2. Perform plasma experiments at KSTAR using CODAC compliant plant systems
Evaluate the new features of CODAC technologies according to new releases
HPN interfaces (DAN, SDN, TCN)
New features of device drivers/supports (NI PXI-6259, NI X-series I/O board, NI FlexRIO board)
Investigate and collect technical requirements for ITER operational applications to evaluate the initial release.
Plant Configuration Editor
Supervision System (SUP)
Data Archiving and Visualization tools
11th IAEA Technical Meeting on Control, Data Acquisition, and Remote Participation for Fusion Research,
08~12 May, 2017, Greifswald, Germany Page 4
ITER standard density feedback control systems in KSTAR
MTCA.4
Milimeter-wave Interferometer
FUELFast
Controller
AIO
GPS PASS
new PCS
miniCODAC (DEV2)(ITER operation
application)
PO
N
DAN
Fast Controller
SDN
TCN
PO
N
SDN
Data archiver
DAN
PO
N
TCN
TCN
PO
N
TCN
PO
N
DAN
KSTAR Timing ITER Timing
CCS
HMI
PO
N
H-alpha
FIRFast
Controller
TCN
PO
N
DAN
SMBIFast
Controller
AIO
SDN
TCN
PO
N
DAN
TCN
AI
AI
AI
IRIG-B
IEEE 1588
SD
N
SDN
SDN
SDN
KO-DA VUV spectrometer
Operation Sequence Inerlock
• Schedule preparation & validation
• Operation Request Gateway(ORG)
• Supervision & Automation
Additional system for fast detection of L-H
transition
Additional function for Unified
Data Access (UDA)
ITER VUV spectrometer
from KO-DA
11th IAEA Technical Meeting on Control, Data Acquisition, and Remote Participation for Fusion Research,
08~12 May, 2017, Greifswald, Germany Page 5
Experiment result
#14102 • The last shot of CODAC at 2015
CODAC PCS command [x 1e19]
Plasma density on KSTAR PCS [x 1e19]
Plasma current [x 100kA]
Vacuum pressure [x 1e-5 mbar]
NBI-A injected [about 1MW, 0.5~9.0]
NBI Shot: energy 70keV,
power 0.95MW, on time:
0.5s, width: 9.5s
IP min fault: 9.34s
Density value on D1-J4-MMWI
11th IAEA Technical Meeting on Control, Data Acquisition, and Remote Participation for Fusion Research,
08~12 May, 2017, Greifswald, Germany Page 6
Experiment result
#14102 • Zoom in the PID enabled section
CODAC PCS command [x 1e19]
Plasma density on KSTAR PCS [x 1e19] Target Reference density
NBI-A injected [about 1MW, 0.5~9.0]
Controlled density
converged to target density.
target reference wave form.
PID enable period
11th IAEA Technical Meeting on Control, Data Acquisition, and Remote Participation for Fusion Research,
08~12 May, 2017, Greifswald, Germany Page 7
Expansion of KSTAR control system by using CODAC technologies
Identified the reliability of CODAC technologies by performing multiple
contractual tasks
Development efforts are naturally concentrated on the system automation and
standardization
Continue the system renovation by adopting new technologies including the ITER
CODAC features
Main applied target:
1. Network Infrastructure
2. Software enhancement
3. Device standardization
Supervisory &
Automation
systems
Integrated
Information System
Database / Server
Data Acquisition Systems
Diagnostics
Local
System I&C
3rd Party
procurements
Plasma Control
System Network
infrastructure
Machine
Protection
11th IAEA Technical Meeting on Control, Data Acquisition, and Remote Participation for Fusion Research,
08~12 May, 2017, Greifswald, Germany Page 8
Real-time network reconfiguration.
Native real time network
• Using a hardware-driven communication devices, i.e. RFM card. It has benefited from
low communication latency on dissimilar hardware platforms running different
operating systems, and lack of software overhead.
• Higher data rate and fast response time are required. Lack of diagnostic tools drive
us to find an alternate solution.
ITER Synchronous Data Bus Network
• Confirmed a stable operation during an evaluation task with IO. The popularity of
Ethernet product give lots of diagnostic tools.
• Decided to adopt an Ethernet-based high-speed control network, i.e. SDN
• Develop RT-GATE for gradually migrate from RFM to SDN
• Ready to use fast SDN to RFM
gateway system
• The real-time network data archiving
system has been developed, refer to the
Post P2/7/S7_3
11th IAEA Technical Meeting on Control, Data Acquisition, and Remote Participation for Fusion Research,
08~12 May, 2017, Greifswald, Germany Page 9
10GbE SDN S/W
(Main control room)
Server room
Fueling facility (D2)Real time diagnostics
RFM network
Control device room (5th
)
Local SDN for Heating
NBI ICS
SDN-Gateway
Integrated
Interferometer
FUEL SMBI
RTP#1 RTP#2 RTP#3
Pellet
Injector
LCS#1 LCS#2 LCS#3 H-Alpha
Native KSTAR
Real time control
systems
Current design and progress of real-time network infra
SDN adoption is actively pursued in various control systems.
11th IAEA Technical Meeting on Control, Data Acquisition, and Remote Participation for Fusion Research,
08~12 May, 2017, Greifswald, Germany Page 10
Complement of time synchronization system
KSTAR native time synchronization system
• Aims to synchronize clocks / triggers with a single clock source and supply them
directly to the controller. It has a star topology connection and provides a high-
precision time protocol.
Up to 5ns resolution, over 50 programmable multi trigger chain, 1~100MHz output clock
Embedded EPICS IOC in ARM® processor
Adopt ITER TCN for system clock synchronization to increase
application design flexibility
Custom protocol
L1, 1.575GHz
GPS Antenna
KSTAR NTPGPS receiver
(Grand Master Clock)
Central Timing Unit(CTU)
Multi-protocolphysical layer switch
Local Timing Unit(LTU)
Local Timing Unit(LTU)
Local Timing Unit(LTU)
IRIG-B DCLS 10MHz Ref. Clock
Local Timing Unit(LTU)
PTP
Layer 3 Precision Time Protocol(PTPv2) switch
ITER PTP
Clock/Trigger supplier
Host time synchronize
Custom
protocol via
Fiber
11th IAEA Technical Meeting on Control, Data Acquisition, and Remote Participation for Fusion Research,
08~12 May, 2017, Greifswald, Germany Page 11
New software components for system I&C
11th IAEA Technical Meeting on Control, Data Acquisition, and Remote Participation for Fusion Research,
08~12 May, 2017, Greifswald, Germany Page 12
Basic software architecture of control system I&C
We do not use an integrated solution (eg: sdd-editor) for control system development,
but we use standard libraries, templates, and example code.
KSTAR widely use a fundamental core library. The sfwLib has a device support structure
that provides non-blocking behavior through message queues.
Additional new features are implemented based on the CODAC standards.
• S/W components supporting absolute time based operation
• SDN support
• NI RIO libraries for Fast Interlock System
• Typical IOC based on “sfwLib”
• More than 20 control systems have been implemented using sfwLib
EPICS Standard LayerSystem environments
Template file
Predefined records
KSTAR Network Infrastructure
MDS+ Lib
Thread Manager
Infromation translator
Sequence handler
Custom libraries
State indicator
User function mapper
Operating System
Hardware
11th IAEA Technical Meeting on Control, Data Acquisition, and Remote Participation for Fusion Research,
08~12 May, 2017, Greifswald, Germany Page 13
Real time core engine for advanced control
Demanding more flexible and stable methodology for implementation of
a real time application
• Inspired by a design philosophy of the ITER RTF, we adopt the function block based
high abstraction layer for implementation of a real time application
• To evaluate, we agilely prototyped an EPICS IOC which is configured with several
libraries include RT-core engine (TAC-engine) from IO
Software design concept, libraries, internal functions are shared with CCS basis
• By developing intuitive HMI system, we will provide a convenient communication and
configuration logging system
• Intel® Xeon® based high end server
• MRG Real-time Linux
• Base s/w package CCS v5.4
• KSTAR standard libraries
• tacLib for RT core
Fundamental of real time core node
Intel® Xeon® based HPC
MRG Real time based on Scientific Linux 6.x
MDS+ lib.
SDN lib. TCN lib. PON lib. LOG lib.
Real time core engine (TAC-engine)
KSTAR customized EPICS libraries
EPICS standard layer
11th IAEA Technical Meeting on Control, Data Acquisition, and Remote Participation for Fusion Research,
08~12 May, 2017, Greifswald, Germany Page 14
Feasibility check on web based configurator and application
Thread A: 1KHz Thread B: 100Hz
kClose
kPusleChain
SDNPub
kPID1
SDNSub
SDNPub
kLinearWave
kOpen
kChopper
kLevelShift
Offset
Trigger
chain
Fuel commander
PID function
Real time
density
Prefill gas
Target density
Offset
neV command
PCSPCS
FUELFUEL
MMWIMMWI
V command
Rate
param.
level
Conceptual connection
diagram
Web-based dynamic configurator provides drag-
and-drop interface
(will continue to update)
• Measure the processing
time for each block
• Seen the peak noise up to
35us
Organized a single node executor based on current CCS v5.4 environment at KSTAR
MRG-R kernel 3.10.33-rt32.52 (FC based on CCS v5.4)
Intel® Xeon® E5-2620 v3 2.4GHz (Single Socket)
Chelsio - S320E-CT-CR (standard SDN card)
TCN driver with i350 chipset
Representative
measurement value
11th IAEA Technical Meeting on Control, Data Acquisition, and Remote Participation for Fusion Research,
08~12 May, 2017, Greifswald, Germany Page 15
First step toward advanced control based on TAC
Second NBI is designed to provide off-axis CD by 3 beam injection with
vertical steering on the equatorial plane
Parameters Designed
Beam energy 100 keV
Beam power 6 ㎿ ( 2.0 ㎿ × 3 ea)
No. of Ion source 3 EA
Beam divergence 1°<
Beam target position,
Tangent radius (Rt) 1.6 m
Vertical slant angle 5.5°
Beam size (@outlet) 450 ㎜ × 130 ㎜
Beam input port H port
Max Operation time 300 sec
Focal position (m) 10.2 m
Beam modulation frequency 100 ㎐
Beam line
Ion source Power Supply
Ion source in vertical plane
11th IAEA Technical Meeting on Control, Data Acquisition, and Remote Participation for Fusion Research,
08~12 May, 2017, Greifswald, Germany Page 16
Infrastructure of Integrated Control System for NBI2
Manipulate each P/S Local Control Systems (LCS) and Beam line LCS
Sequential logic control according to the beam initiation time
Generate predefined/arbitrary reference waveform synchronize to the PCS command
Power Supply Local Control
System
Machine Network(TCP/IP)Real Time Network(RFM)Timming Network(TCN)Interlock Network(Slow)Interlock Network(Fast)
KSTAR MDSPlus
KSTAR PCS KSTAR CCS
RT-Module Server
PCIe
CPU(Xeon)
RFM( PMC5565)
10GbENetwork
Card
1GbE Network
Card
Synchronous Data Network(Fast)
Beam Line Local Control System(PC)
Integrated Control System(PC)ICS-Module
PCIe
CPU(Xeon)
1GbE Network
Card
KSTAR LTU KSTAR LTU KSTAR LTU KSTAR LTU
Local RFM Network
3 * MTCA.4 based LCS
• NBICS : Communicate and
process epics data
Control period : 1KHz
Network : 1Gb based channel
access
• NBRT : Communicate and
process data in real time
Control period : 10KHz
Network : RFM(or SDN),TCN
Time Synchronization : TCN and
KSTAR TSS
• RT SIM : Generate simulated
reference waveform
Control period : 10KHz
11th IAEA Technical Meeting on Control, Data Acquisition, and Remote Participation for Fusion Research,
08~12 May, 2017, Greifswald, Germany Page 17
Device standardization for system I&C
11th IAEA Technical Meeting on Control, Data Acquisition, and Remote Participation for Fusion Research,
08~12 May, 2017, Greifswald, Germany Page 18
Shift to the next generation control device
An extension of MTCA (Micro Telecommunications Computing Architecture) initiated by the
Physics community, MTCA.4, provides high precision clock and trigger through the
backplane, intelligent system management interface, modular structure of high-speed
links, and allows flexible reconfiguration of system functionality.
AMC RTM
Front Back
Analog Digital
Zone 3
AMC backplane
Distribution of clocks and triggers within a shelf
12 slot backplane topology
6 slot chassis for
laboratory or diagnostics
12-Slot fully redundant
11th IAEA Technical Meeting on Control, Data Acquisition, and Remote Participation for Fusion Research,
08~12 May, 2017, Greifswald, Germany Page 19
Device standardization
CODAC recommend several platforms for a Fast I/O system depend on performance grades.
Refer to “ITER Catalog of I&C products - Fast Controllers”(IDM: 345X28)
For the systematic standardization of a fast
controller at KSTAR, we adopt the MTCA.4 standard. Host side adaptor
Target side uplink
module
MTCA.4 Optical PCI Express 4x
Link to an external CPU dedicated
for μTCA.4 systems (Max. 20GT/s)
11th IAEA Technical Meeting on Control, Data Acquisition, and Remote Participation for Fusion Research,
08~12 May, 2017, Greifswald, Germany Page 20
Develop a general-purpose FPGA based control device
Diverse local system require different ADC rates and can be categorized several
performance grade. The most demand is 10KSPS ~ 2MSPS sampling speed.
Through practical experience and strategic factors, we are considering
manufacturing a common device.
We developed bespoke KSTAR Multifunction Control Unit (KMCU) with benefit of
modular design
Item KMCU-Z30 (K-Z30) KMCU-Z35 (KZ35)
FPGA XC7Z30T XC7Z35T
Memory 1GB DDR3
Front Interface - 1xLPC FMC site
2x SFP+ cadge 1x SFP+ cadge
Ethernet RJ45
microSD for Linux
LEMO CLK/TRIG
Backplane
interface
PCIe x1(port4) Gen 2 PCIe x4(port 4-7)
Ethernet port-0
TCLK, FCLK
Zone 3 Class D1.0.
11th IAEA Technical Meeting on Control, Data Acquisition, and Remote Participation for Fusion Research,
08~12 May, 2017, Greifswald, Germany Page 21
MicroTCA.4: DAQ use case
Several data acquisition systems were renovated or newly developed using
KMCU-Z30
Renovate magnetic diagnostics (unlimited acquisition time), Mirnov coil diagnostics,
and H-alpha monitoring system (higher ADC rate)
New DAQ for a multi chord photo-elastic modulator based MSE system
System Specification Remark
DDS1 (MD &Probe) Max 1MSPS, 576ch, 9*kz30, 16bit PCIe uplink to external host
MSE Max 2MSPS, 16bit, 32ch KSTAR Multichord MSE
Max 2MSPS, 16bit, 32ch PPPL/MIT MSE-MSLP
MC 2MSPS, 16bit, 64ch Front aurora interface
H-Alpha 1MSPS, 16bit, 32ch Front aurora interface
Interferometer 125MSPS, 16bit, 12ch (COTS) Real time density calculation
DDS1
KSTAR MSE
MC
H-Alpha
MSE-MSLP
11th IAEA Technical Meeting on Control, Data Acquisition, and Remote Participation for Fusion Research,
08~12 May, 2017, Greifswald, Germany Page 22
MicroTCA.4: High end system use case
Use commercial products for a high performance application demanding
lots of FPGA slices
Integrated interferometer system
Resource Available MMWI
Utilization MMWI + FIR Utilization
LUT 254200 63991 (25.17%) 69663 (27.40%)
LUTRAM 90600 13283 (14.66%) 13433 (14.83%)
FF 508400 99256 (19.52%) 109029 (21.45%)
BRAM 795 44.50 (5.60%) 44.50 (5.60%)
DSP 1540 6 (0.39%) 14 (0.91%)
Device : xc7k410tffg900-2
Details available at the
Poster P2/3/05
AMC523
μTCA.4 AMC, Dual DAC 16-Bit @
250 MSPS
MRT523
μTCA.4 RTM For AMC523, 12 CH
ADC 16-Bit @ 125 MSPS
Packet Gen & Control
Period Flag Gen
PCI ExpressDMA
S/W
LPF
Zero CrossPoint Detect
Period Lock Check
Phase Compare
&FrequencyEstimate
Phase Average
Over Sample
Data Gen
LPF
Zero CrossPoint Detect
Period Lock Check
Over Sample
Data Gen
Ref Wave
Prob Wave
DegreeConvert
&Remove
Channel 1
Channel 2
Zero CrossPoint Detect
Period Lock Check
Phase Compare
&FrequencyEstimate
Phase Average
Over Sample
Data Gen
Zero CrossPoint Detect
Period Lock Check
Over Sample
Data Gen
DegreeConvert
&Remove
11th IAEA Technical Meeting on Control, Data Acquisition, and Remote Participation for Fusion Research,
08~12 May, 2017, Greifswald, Germany Page 23
MicroTCA.4: High end system use case
Compatibility of 3rd party IP logic inside the FlexRIO : provides design flexibility
Measured by KSTAR system Measured by FlexRIO (DAN plot)
• Out put data has 0 ~ -360
phase information
• Makes 1280 data counts per
1msec
3rd-party IP
MMWI Interferometer
PXI Express
MTCA.4 based MMWI
D1-J4-MMWISDN
DAN
Bus ext.
PON
Bus ext.
TCN
FlexRIOPXIe-7966R / 5734
11th IAEA Technical Meeting on Control, Data Acquisition, and Remote Participation for Fusion Research,
08~12 May, 2017, Greifswald, Germany Page 24
Integrated information system
11th IAEA Technical Meeting on Control, Data Acquisition, and Remote Participation for Fusion Research,
08~12 May, 2017, Greifswald, Germany Page 25
KSTAR Data Integration System (KDIS)
The data integration system of KSTAR supporting a wide range of KSTAR
pulse operation automation with off-line data process.
• Scheduled process of stream and batch data according to KSTAR events with user
interface service, including H/W and S/W infrastructure, applications, and libraries.
KSTAR
Systems
KSTAR
Pulse Automation
System
On-line Operation Process
KSTAR
Main Storage
(MDSPlus,
Channel Archiver
, ETC)
CCS
ICS
PCS
NBI
ECH
MPS
PASS
Data Stream
(EPICS, SDN,
Logs, RestAPI,
ETC)
User Interface (OPI, Web Service, ETC)
SIS
KDIS
System Infrastructure
Distributed Clustering Platform
Clustered Servers
(Resource Manager)
KSTAR Integrated Stream
Processing Framework and
Applications
Schedule Algorithm
and Stream
Processing Logic
KSTAR Integrated Batch
Processing Frameworks and
Applications
KSTAR Integrated User Interface
Web Application Service
Optional Back-End Storage
(HDFS, NoSQL, RDB, ETC…)
Data Processing and
Analysis Algorithm
Integrated web view
Supporting Data
interface
11th IAEA Technical Meeting on Control, Data Acquisition, and Remote Participation for Fusion Research,
08~12 May, 2017, Greifswald, Germany Page 26
KDIS Data Cycle Orchestration
Design of data cycle orchestration with developed applications
• KSTAR operation related data cycle ecosystem synchronized with KSTAR event
KSTAR
Stream Processing
Framework
Applications for
KSTAR Event
Stream
KSTAR
Data Processing
Framework
Applications for
KSTAR Data
Processing
Clients
KDIS Main
Web Service
Data Source and Storage
MDSPlus
MemoryDB
MessageQ HDFS
ETC Retrieving
Task Launch
Stream Processing,
Data Archiving
EPICS
RDB
Data
Exchange
NoSQL
• KSTAR Shot Summary
• Shot Fault Analysis Application
• EPICS PV Snapshot for
Operation Parameter
• TSS Parameter Snapshot
• TSS Template Automation
• Processing KSTAR
Pulse Automation
System Event
• Supports PV snapshot View
• Supports Summary View
• Statistical Information Summary view
• Supports Logbook and Integrated view
Details available at the Poster
P1/2/S2_1
11th IAEA Technical Meeting on Control, Data Acquisition, and Remote Participation for Fusion Research,
08~12 May, 2017, Greifswald, Germany Page 27
Summary and Future work
ITER CODAC technologies were successfully evaluated in the real-life
environment of an operational tokamak.
KSTAR has been investigating the next generation control platform and
new standards both hardware and software infrastructure.
By adopting TCN and SDN, we decided to increase the performance of
control infrastructure and increase the flexibility of system design.
We will use the core engine to reflect the real-time framework design
concept and use it for integrated control of the second NBI system.
KSTAR adopt the MTCA.4 standard for the systematic standardization of
a fast controller and real time diagnostics.
A KSTAR data integration and information system based on the big-data
platform is under development and will be serviced in 2017.
11th IAEA Technical Meeting on Control, Data Acquisition, and Remote Participation for Fusion Research,
08~12 May, 2017, Greifswald, Germany Page 28
Thank you for your attention. Q & A