Design and Development of STCF Offline Software

Xingtao HuangFor the STCF Software Team

Shandong University

Dec.4-7, 2018Joint Workshop of future tau-charm factory

Orsay, France

Outline

u Requirements u Architecture Designu SNiPER Frameworku Detector Geometry Descriptionu Event Data Model u Current Statusu Summary

Dec.06,2018 2

Super Tau-Charm Facility in Chinal e+ e- Colliderl Ecm = 2-7 GeVl L = 0.5-1´1035 cm-2s-1 @4 GeVl Event Rates: ~100 X BESIII

Dec.06,2018 3

l General purpose Detectorl PXDl MDCl PIDl EMCl MUD

l Event size: larger than BESIII

Requirements of STCF offline software

Dec.06,2018 4

u Procedure and requirements of STCF are similar with BESIIIu Many of BESIII software could be reused in STCFu But event rates and event size much larger than BESIIIu Need faster and more powerful offline software system than

BOSS, The BESIII Offline Software System.ð Underlying Framework: Gaudià SNiPERð Detector Simulationð Event Data Modelð Event Data Management Systemð IO systemð Parallelismð ……

OSCAR: Offline Software of Super Tau-Charm Facility

Architecture Design

Dec.06,2018 5

SNiPER

Generator Analysis

Simulation

Calibration

Reconstruction

OfflineExLibs

DD4HEP ROOT Geant4 HepMC CLHEP Boost Python

……

l ExLibs: include frequently used third-party software and toolsl SNiPER: a new framework to provide core functionalities and common services.l Offline: all software specific to STCF, including Generator, Simulation,

Calibration, Reconstruction and Analysis.

Software Environmentsu Operation System and main Libraries:

ð Scientific Linux 6.10 ð Gcc 4.9.4ð ROOT 6.10.08ð Geant4.10.04.p01

u Programming language: Hybrid programming of C++ and Pythonð C++ : main part implementationð Python : job configuration interface

u Packages management tool: CMT (Configuration Management Tool)ð Help developers to compile packages easilyð Help users to setup the environment for running the application easily

u Codes Management: SVN repositoryð Keep the history of code evolutionð Synchronization and sharing between developersð Tag and release

Dec.06,2018 6

SNiPER Framework u Developed by JUNO Collaboration since 2013u Main goals:

ð lightweight framework, less dependence on third-party softwareð Flexible and fast execution ð Easy to learn and convenient to useð Support multithreading

u Design and developmentð Learn a lot from other software frameworks, such as Gaudið Based on the valuable experiences of DayaBay and BESIIIð Coding from scratch

u Current Statusð Successfully used for JUNO and LHAASO Experimentsð Tested with BESIII Real Data Analysis ð Under investigation by several other experiments, sucha as nXEO, CEPC…ð We Started for STCF since Aug. 2018

Dec.06,2018 7

Key Functionality of SNiPER

u Dynamically Loadable ElementðAlgorithmðServiceðTask

u Data Storeu Parallelismu Propertyu Logging

Dec.06,2018 8

Dec.06,2018 9

TaskaddAlg(name, scope)addSvc(name, scope)addTask(name, scope)

DlElementDlElement(string)

name()type()

AlgBaseinitialize()execute()finalize()

SvcBase

initialize()finalize()

DleManager

create(string)get<type>(string)

IListenerregister( MsgType)register(string)handle( IMessage)

u Algorithmð An unit of codes for Data Processingð Perform calculations during event loopð Users usually write their own algorithms

• Inherit from the AlgBase

u Serviceð Similar with Algorithmð A piece of code for common use

• GeometrySvc• DatabaseSvc• ……

u Task

Dynamically Loadable Element

Task: A lightweight Application Manageru Task manages its algorithms, services and sub-tasks

10

Dec.06,2018

u One job can has more than one Tasksð SubTask provides nested event loopð Natral interface for multithread

Computing (run each task in an individual thread)

u All DLEs are organized in a tree structure

u Task controls the execution of algorithms

Data Store for Event Data Management

u Data Store is the dynamically allocated memory place to hold events data which are being processed

u Applications (in terms of algorithms) get events data from the Data Store and add/update objects after processing

u Data type of DataStore is flexible

11

DataStore

Dec.06,2018

Parallelism

u Developed based on Intel TBB to Support event level parallelismð Muster: Multiple SNiPER Task Schedulerð SniperTbbTask: Binding of a SNiPER Task to a TBB task

u Global DataStore to provide events for multi-tasks (or multi-threads)u A dedicated task(thread) is used to read/write event data from/to files

Dec.06,2018 12

u Configurable variable at run time

Dec.06,2018

Property

13

u Declare a property in DLElement(Alg, Svc, Tool and Task)

u Configure a property in Python script

u Types can be declared as properties:ð scalar: C++ build in types and std::stringð std::vector with scalar element typeð std::map with scalar key type and scalar value type

u SniperLog: a simple log mechanism supports different output levels

0: LogTest2: LogDebug3: LogInfo4: LogWarn5: LogError6: LogFatal

u Each DLElement(Alg,Svc,Tool, Task) has its own LogLevel and can be set at run timeð very helpful for debugging

u The output message includes more information , such asð where it happensð message levelð message contents

Dec.06,2018

Logging

14

More specific to Experimentsu SNiPER provides the core functionalities and services

ðUser interfaces: Algorithm and ServiceðTask for management of event loopðDataStore for management of events in memoryðParallelisam for multithreadingðPropertyðLogging

u But without coveringðDetector SimulationðEvent Data ModelðFile Input/OutputðOthers more specific to the experiments

Dec.06,2018 15

DD4hep

u Key functionalitiesðFull Detector Description

• Includes geometry, materials, visualization, readout, alignment, calibration, etc.

ðFull Experiment life cycle• From concept development, detector optimization, construction to operation.

ðConsistent Description with single source• for simulation, reconstruction, analysis, etc.

ðEase of Use

Dec.06,2018 16

u A general detector (geometry) description toolkit for HEP u Developed in AIDA and AIDA2020

u Used by ILC, CLIC, FCC u Under Evaluation by EIC, CMS

and LHCb for upgrade

DD4hep

u DD4hep Uses Root TGeo for the underlying geometry descriptionð Access to ROOT Open GL viewer for geometryð Debugging of geometry

u Default geometry descriptionð Compact xml-files and C++ drivers

u Ouput formats/interfaceð GDMLð Geant4 geometry

Dec.06,2018 17

Visualization of STCF Detector with DD4hep

Dec.06,2018

<detectors><comment>Trackers</comment>

<detector name="AirTube" type="AirTube" vis="VXDVis" id="42"insideTrackingVolume="true">

<dimensions rmin="10.*mm" rmax="11.*mm" zhalf="6.250000000e+01*mm"/></detector>

...

</detectors>

From Dong Liu

18

Event Data Model

u Defines the data unit to be processed in offline data processingu In form of C++ classesu Can be converted into persistent type and saved into disk

Dec.06,2018 19

Data Processing

Generator Simulation Reconstruction

Analysis

DataStore (Memomery)

GenEvent SimEvent RecEvent PhyEvent File

Design of STCF EDM

u Single definition for Events in DataStore and Root Filesu Event Objects will be based on ROOT Tobjec to take

advantages of ROOT powerful functionalitiesð Objects streamersð Class schema evolutionð Run time type identificationð Inspection, drawing etc.

u For each data processing stage, the event data is divided into two layers:ð Header Objectð Event Object

u IO system based on ROOT streamers supports data loading on demand to reduce I/O burden

Dec.06,2018 20

Design Schema of STCF EDM

Dec.06,2018 21

XML Object Description (XOD) Toolu Traditionally writing C++ Code manually

ð Many repeatable work such as Getters and Setters ð Difficult to be maintained

u Use XML file to define EDM ð Strong syntax (DTD, XML Schema)ð More readable, easier to maintainð Automatically generate the Get-, Set-functions, ROOT I/O Streamers

u A tool, XmlObjDesc (XOD), is developed to automatically generate class codes

Dec.06,2018 22

*.xml*.h

*.cc

*LinkDef.h

*Dict.h*Dict.cc lib*.so

XODCINT/CLing G++

Installation,Documentation and SVN

u The first working version of OSCAR has been installed in USTC computing nodes ð stcf01.ustc.edu.cnð stcf01.ustc.edu.cn

u Installationð Automatic installation of the whole offline software with a shell script ð svn export http://202.141.163.202/svn/oscar/installation/trunk/setup-trunkj.sh

u Documentationð OSCAR User Guide

• http://cicpi.ustc.edu.cn/indico/getFile.py/access?contribId=1&resId=0&materialId=slides&confId=1610

ð DD4HEP User guide• http://cicpi.ustc.edu.cn/indico/getFile.py/access?contribId=0&resId=2&materialId=slides&confId=1610

u SVN repositoryð http://202.141.163.202/svn/oscar/ð Account request, please send mail to Wenhao Huang (whyellow@mail.sdu.edu.cn)

Dec.06,2018 23

Summaryu OSCAR, the STCF offline software, is being designed and

developed ð Based on SNiPER and DD4hep

u First working version has been installed and several tutorials have been organized to train more people

u Lots of works ahead, more people are welcomeð Optimization of Detector descriptionð Integration DD4hep with OSCARð Adding frequently used generators ð Detector response simulationð Finalize Event Data Modelð Design of File Input/Output Systemð Developing Reconstruction methods

u The goal: setup a full chain from generator to reconstruction for optimization of Detector design and performance study.

Dec.06,2018 24

Thanks for your attention!