PAL EPICS Meeting Oct. 2012
description
Transcript of PAL EPICS Meeting Oct. 2012
![Page 1: PAL EPICS Meeting Oct. 2012](https://reader035.fdocuments.net/reader035/viewer/2022062309/56813d44550346895da702db/html5/thumbnails/1.jpg)
Archive Systems
What you always wanted to know but were afraid to ask:
What’s available?
Who’s doing what?
PAL EPICS MeetingOct. 2012
![Page 2: PAL EPICS Meeting Oct. 2012](https://reader035.fdocuments.net/reader035/viewer/2022062309/56813d44550346895da702db/html5/thumbnails/2.jpg)
1. Performance?
How many samples/second write & read?
2
![Page 3: PAL EPICS Meeting Oct. 2012](https://reader035.fdocuments.net/reader035/viewer/2022062309/56813d44550346895da702db/html5/thumbnails/3.jpg)
2. How to get at the data?• CSS archive plugin?
3
• Easy web reports?
![Page 4: PAL EPICS Meeting Oct. 2012](https://reader035.fdocuments.net/reader035/viewer/2022062309/56813d44550346895da702db/html5/thumbnails/4.jpg)
3. All the Rest?• How to configure?• How to
– start/stop sample engines?– Monitor if they’re
• running?• connected to all channels?• write without errors?
• Impact of disk failure, engine crash?• Limits when data grows?• How to spread data across disks, servers?
– How to delete certain channels or time ranges?– Data backup?
• Info:– Which channel has the most samples in last 24 hours?– Which subsystem generates the most samples?
4
![Page 5: PAL EPICS Meeting Oct. 2012](https://reader035.fdocuments.net/reader035/viewer/2022062309/56813d44550346895da702db/html5/thumbnails/5.jpg)
![Page 6: PAL EPICS Meeting Oct. 2012](https://reader035.fdocuments.net/reader035/viewer/2022062309/56813d44550346895da702db/html5/thumbnails/6.jpg)
System
• Language, Platform: Java, C++, ??• Data StoreAdvantageGreat Stuff- Disadvantage- ProblemUser, user, user
6
![Page 7: PAL EPICS Meeting Oct. 2012](https://reader035.fdocuments.net/reader035/viewer/2022062309/56813d44550346895da702db/html5/thumbnails/7.jpg)
Channel Archiver• C++• Custom Data/Index FilesFast: Write 60k, 70k samples/sec- Need library to read data- Trouble when engine crashed- File size limits- Maintenance nightmareSNS until 2009, KEK, J-PARC,
SLAC, KSTAR, ..
7
![Page 8: PAL EPICS Meeting Oct. 2012](https://reader035.fdocuments.net/reader035/viewer/2022062309/56813d44550346895da702db/html5/thumbnails/8.jpg)
Channel Archiver, 64 bit
.. As before, butExtended file size limit-Even Bigger Trouble when engine crashed?-Even Bigger Data maintenance nightmare?Diamond
8
![Page 9: PAL EPICS Meeting Oct. 2012](https://reader035.fdocuments.net/reader035/viewer/2022062309/56813d44550346895da702db/html5/thumbnails/9.jpg)
RDB-Based Channel Archiver (BEAUTY)
• Java (Eclipse, CSS)• RDB: MySQL, PostgreSQL, Oracle, … Any data maintenance possible
No size limits; partitions, cluster, …Any type of reporting on dataAnybody can reach data from any programming language
Engines don’t crash, and if they do: Just no new samples for that engine
- Slow: 8000 samples/sec write- ~15k for special MySQL setup
SNS since 2009, KEK, ITER?
9
![Page 10: PAL EPICS Meeting Oct. 2012](https://reader035.fdocuments.net/reader035/viewer/2022062309/56813d44550346895da702db/html5/thumbnails/10.jpg)
DESY MySQL Archiver• Java (started with BEAUTY code)• RDB: MySQL with special setup– Writing original sample plus a few averages
Faster writing because of some MySQL optimization
Faster retrieval of long time rangesMost of the usual RDB benefits- Only MySQLDESY?
10
![Page 11: PAL EPICS Meeting Oct. 2012](https://reader035.fdocuments.net/reader035/viewer/2022062309/56813d44550346895da702db/html5/thumbnails/11.jpg)
Fermilab (not EPICS)• Java• MySQL ‘BLOBs’ used as ring bufferFasterAutomatically replace oldest data, not running
out of space- Need library to write and read, encode and
decode content of BLOBs- None of the usual RDB advantagesFermilab
11
![Page 12: PAL EPICS Meeting Oct. 2012](https://reader035.fdocuments.net/reader035/viewer/2022062309/56813d44550346895da702db/html5/thumbnails/12.jpg)
MyA (Jlab)
• C++?• Homebrew MySQL ‘cluster’– Several MySQL Servers. One table per channel.
FasterMost of the usual RDB benefits- Need library to read from correct MySQL
server and tableJLab
12
![Page 13: PAL EPICS Meeting Oct. 2012](https://reader035.fdocuments.net/reader035/viewer/2022062309/56813d44550346895da702db/html5/thumbnails/13.jpg)
Archive Appliance
• Java• Google Protocol BuffersFaster“Appliance” Idea
Submit channel name. Rest Auto-configured.- Need library to access data?SLAC, BNL, IHEP, PSI, ITER?
13
![Page 14: PAL EPICS Meeting Oct. 2012](https://reader035.fdocuments.net/reader035/viewer/2022062309/56813d44550346895da702db/html5/thumbnails/14.jpg)
CERN
• Language, Platform: Java, C++, ??• OracleFast because they tuned it well- not EPICSCERN
14
![Page 15: PAL EPICS Meeting Oct. 2012](https://reader035.fdocuments.net/reader035/viewer/2022062309/56813d44550346895da702db/html5/thumbnails/15.jpg)
SDDS
• C• SDDS FilesIntegrate4d w/ SDDS tools- Needs library to access- Data managementAPS, …
15
![Page 16: PAL EPICS Meeting Oct. 2012](https://reader035.fdocuments.net/reader035/viewer/2022062309/56813d44550346895da702db/html5/thumbnails/16.jpg)
Fast Acqu. Archive
• C• Raw diskFast, for specific purpose- Need lib.- Data managementDiamond
16
![Page 17: PAL EPICS Meeting Oct. 2012](https://reader035.fdocuments.net/reader035/viewer/2022062309/56813d44550346895da702db/html5/thumbnails/17.jpg)
SLC History Buffers
• Fortran• Binary filesWorked for decades- SlowSLAC
17