Python -based Physics Analysis Environment for LHCb G. Barrand, M. Frank, P. Mato, E. de Oliveira,...

PythonPython-based Physics -based Physics Analysis Environment for Analysis Environment for

LHCbLHCb

G. Barrand, M. Frank, P. Mato, E. de Oliveira, G. Barrand, M. Frank, P. Mato, E. de Oliveira, A.Tsaregorodtsev, I. BelyaevA.Tsaregorodtsev, I. Belyaev

CHEP 2004, Interlaken, SwitzerlandCHEP 2004, Interlaken, Switzerland

30 Sep'2k+4 Computing30 Sep'2k+4 Computing Python based physics analysis for LHCbPython based physics analysis for LHCb 22

Goals User friendly environment for development of physics analysis User friendly environment for development of physics analysis

codecode•input: event data files (DST); output: AOD, Ntuples, Histos, …input: event data files (DST); output: AOD, Ntuples, Histos, …

““Easy-to-understand”, “readable” Easy-to-understand”, “readable” •Physics-driven semantics (using physics concepts)Physics-driven semantics (using physics concepts)•1-1 matching with physical description 1-1 matching with physical description •all technical details hidden from the userall technical details hidden from the user

CompactCompact• ~1 page of code per “typical” analysis algorithm ~1 page of code per “typical” analysis algorithm

Interactive Interactive •(re)define cuts/algorithms on-flight (re)define cuts/algorithms on-flight •visualization (event and statistics display) visualization (event and statistics display)

Complete functionalityComplete functionality•to be used by “real” analysisto be used by “real” analysis

Rapid Application Development (RAD) Rapid Application Development (RAD) •prototyping, no compilationprototyping, no compilation


Why PythonPython-based ? PythonPython is a language with the special emphasize for fast is a language with the special emphasize for fast

prototyping and developmentprototyping and development Scripting and interactivity combined in a natural way Scripting and interactivity combined in a natural way Easy integration with third party software (Python as the glue)Easy integration with third party software (Python as the glue) Availability of external packagesAvailability of external packages

• Visualization, statistical analysis Visualization, statistical analysis •ROOTROOT, , HippoDrawHippoDraw, , PanoramixPanoramix,, PyX PyX, , GNUplotGNUplot

• Event displayEvent display•Panoramix (LHCb)Panoramix (LHCb)

• Bookkeeping data base Bookkeeping data base • Interface to Interface to GRIDGRID

•GANGAGANGA

• and many othersand many others


LHCb Analysis Applications DaVinci

C++ Analysis Application

LoKiPhysics Analysis ToolKit

LHCbEvent Model

GaudiPython

Bender

LCG

Dict

iona

ries

BenderBenderPython Analysis ApplicationPython Analysis Application

Exte

rnal

s: V

isua

lizat

ion,

Eve

nt D

ispl

ay, e

tc

PyLCGDict

DaVinci Tools

Gaudi FrameworkServices, Algorithms &

Application Control


GaudiPythonGaudiPython

The generic package for The generic package for Gaudi & PythonGaudi & Python bindings bindings Access to major Access to major GaudiGaudi Components Components

• Services, Algorithms and Tools Services, Algorithms and Tools Application ConfigurationApplication Configuration

• Algorithm scheduleAlgorithm schedule• Configuration of all components Configuration of all components • “ “Dynamic” reconfiguration is possibleDynamic” reconfiguration is possible

The technique The technique Using the LCG Dictionary (only for the interfaces) Using the LCG Dictionary (only for the interfaces)

and the PyLCGDict package from SEAL to produce and the PyLCGDict package from SEAL to produce the the C++/PythonC++/Python bindings. bindings. (see Wim Lavrijsen’s talk)(see Wim Lavrijsen’s talk)


LoKiLoKi

High Level High Level C++C++ Physics Analysis ToolKit Physics Analysis ToolKit Set of high level analysis utilities built on top of Set of high level analysis utilities built on top of

DaVinci toolsDaVinci tools Physics oriented semanticsPhysics oriented semantics

•Inspired by Inspired by KALKAL from genius H. Albrecht and from genius H. Albrecht and GPATTERNGPATTERN by by T. Glebe T. Glebe

Compact code Compact code Concept of locality (entities defined at the place they Concept of locality (entities defined at the place they

are needed)are needed) Technicalities are “hidden” from end-userTechnicalities are “hidden” from end-user The kernel has low coupling to event model The kernel has low coupling to event model Templated/Inline/EfficientTemplated/Inline/Efficient


LoKiLoKiLOKI_ALGORITHM( MyAlg ) {

select( “K-” , “K-” == ID && PT > 1 * GeV ); select( “pi+” , “pi+”== ID && P > 3 * GeV );

Cut dmass = ABSDM(“D0”) < 30 * MeV ; for( Loop D0 = loop(“K- pi+”,”D0”) ; D0 ; ++D0 ) { if ( VCHI2(D0) > 4 && dmass( D0 ) ) { D0->save(“D0”) ; } }

for ( Loop Dst = loop( “D0 pi+”); Dst ; ++Dst ) { double dm = M(Dst) – M1(Dst) ;

plot ( dm , “DM for D*+” , 130 , 180 ) ;

}

return StatusCode::SUCCESS ;};

Plenty of useful Plenty of useful “functors”“functors”• P, PT, IP,VCHI2,Q,….P, PT, IP,VCHI2,Q,….

Selection/filtering of Selection/filtering of particles particles

Multi-particle looping Multi-particle looping Kinematical/Topology Kinematical/Topology

fitsfits Intuitive interface to Intuitive interface to

histograms and histograms and N-N-TuplesTuples• Book-and-fill on demandBook-and-fill on demand

Easy matching for MC-Easy matching for MC-truth informationtruth information


Bender = LoKi + Python +Bender = LoKi + Python + … LCG dictionariesLCG dictionaries for for C++/PythonC++/Python binding binding

•A bit of raw A bit of raw Boost.PythonBoost.Python also also >95%>95% of of LoKi’sLoKi’s C++C++ functionality is available in functionality is available in PythonPython•Non-trivial due to heavy templated nature ofNon-trivial due to heavy templated nature of LoKiLoKi •Situation improving with Situation improving with PyLCGDictPyLCGDict evolution evolution

The mixture of The mixture of C++C++ and and PythonPython is possible is possible •C++C++ algorithm with algorithm with PythonPython cuts (“ cuts (“LoKiHybridLoKiHybrid”) ”)

The bulk of actual computations in The bulk of actual computations in C++C++•Minimal Minimal PythonPython-related penalty-related penalty

The conversion between existing The conversion between existing PythonPython BenderBender’s and ’s and C++ C++ LoKiLoKi’ s algorithms is simple in both directions: ’ s algorithms is simple in both directions: • Semantics is very similar Semantics is very similar


Configuration: CMT + GaudiPythonCMT + GaudiPython

Job (self)-configuration/environment: Job (self)-configuration/environment: cmt.pycmt.py

Application configuration: Application configuration: GaudiPythonGaudiPython

import cmt cmt.project( ‘Bender’ , ‘v4r0’ )cmt.use ( package = ‘Ex/BenderExample’)cmt.setup()

g = gaudimodule.AppMgr()g.TopAlg += [ “MyAlgorithm” ] g.ExtSvc += [ “MyService” ]g.OutputLevel = Infog.run( 100 )


Physics analysis Functions and cutsFunctions and cuts

Selection of particles Selection of particles

Looping over combinationsLooping over combinations

Saving/retrieve of interesting combinations, Saving/retrieve of interesting combinations, Vertex/Mass-Vertex/Direction/Lifetime fitsVertex/Mass-Vertex/Direction/Lifetime fits

cut = (ID ==‘D+’) & ( P > 5*GeV ) & ( PT > 2*GeV )

k = self.select( tag = ‘K+’ , cuts = ( ‘K+’ == ID ) & ( PT > 0.5 * GeV )

for phi in self.loop(formula=‘K+ K-’, pid=‘phi(1020)’):m = M( phi ) / GeV p = P( phi ) / GeV


Histos & N-TuplesHistos & N-Tuples

HistogramsHistograms

N-TuplesN-Tuples

h1 = self.plot ( title = “ phi mass ” , value = M( phi ) , low=1000, high=1050 )

tup = self.nTuple( title = “Phi NTuple” )tup.column ( name= “ID”, value= ID(phi) )tup.column ( name= “p” , value= P (phi) )tup.column ( name= “pt”, value= PT(phi) ) tup.write()


HistoHisto visualization The histogram visualization can be done throughThe histogram visualization can be done through•ROOTROOT

• native native ROOTROOT through through PythonPython prompt ( prompt (pyROOTpyROOT))• rootPlotterrootPlotter from LCG-PI through from LCG-PI through AIDAAIDA pointer pointer

•PanoramixPanoramix•Directly through Directly through AIDAAIDA pointer pointer

•HippoDrawHippoDraw• hippoPlotterhippoPlotter from from LCG-PI LCG-PI through through AIDAAIDA pointer pointer

Few lines “common interface” for Few lines “common interface” for trivialtrivial plotting exist plotting exist The interactive analysis of The interactive analysis of GaudiGaudi N-Tuples is possible in N-Tuples is possible in BenderBender with with ROOTROOT persistency and persistency and ROOTROOT module directly module directly• Prototype for Prototype for HippoDrawHippoDraw


Everything can be combined

Bender/Python promptBender/Python prompt

PanoramixPanoramix

PI/ROOTPI/ROOT

ROOTROOT

HippoDrawHippoDraw


Event Display: PanoramixPanoramix

The integrated analysis and visualization of statistical The integrated analysis and visualization of statistical and event data is possibleand event data is possible


Complete Examplefrom bendermodule import *class Dstar(Algo):

def analyse( self ) :self.select ( tag=‘K-’, cuts=(‘K-’ ==ID)&(PT>1*GeV) )self.select ( tag=‘pi+’,cuts=(‘pi+’==ID)&(P >3*GeV) )dmass = ABSDM(“D0”) < 30 * MeV

for D0 in self.loop ( formula=‘K- pi+’ , pid=‘D0’ ) :If ( VCHI2(D0) < 4 ) & dmass( D0 ) : D0.save(‘D0’)

tup = self.nTuple ( title = “D*+ N-Tuple ” )

for Dst in self.loop ( formula=‘D0 pi+’ , pid=‘D*(2010)+’ ) :

dm = M(Dst)-M1(Dst)

h1 = self.plot( title = “Delta mass for D*+” , value = dm , low=130 , high=170 ) tup.column( name = ‘M’ , value = M(Dst) / GeV ) tup.column( name = ‘DM’ , value = dm / GeV ) tup.column( name = ‘p’ , value = P (Dst) / GeV )tup.column( name = ‘pt’ , value = PT(Dst) / GeV ) tup.write ()

return SUCCESS


Conclusions GaudiPython, Loki, Bender are regularly released as GaudiPython, Loki, Bender are regularly released as

part of the official LHCb softwarepart of the official LHCb software• In both LHCb supported platforms (Linux, Windows) In both LHCb supported platforms (Linux, Windows)

Sufficient functionality for Physics Analysis is Sufficient functionality for Physics Analysis is availableavailable• Real analysis can be doneReal analysis can be done

Performance is adequatePerformance is adequate•No noticeable difference between pure C++ (DaVinci) No noticeable difference between pure C++ (DaVinci)

analysis and Python (Bender) analysisanalysis and Python (Bender) analysis About compact and user-friendliness…About compact and user-friendliness…

•Users will have to judge after some time of usageUsers will have to judge after some time of usage

Essential for successful and good physics analysisEssential for successful and good physics analysis


LoKi BenderLoKi Bender

LokiLoki is a god of wit and is a god of wit and mischief in Norse mythologymischief in Norse mythology

LoLoops & ops & KiKinematicsnematics

Ostap Suleiman Berta Maria Ostap Suleiman Berta Maria BenderBender-bei-bei The cult-hero of books by The cult-hero of books by I.Il’fI.Il’f &&

E.Petrov: E.Petrov: “The 12 chairs”“The 12 chairs” ,,“The “The golden calf”golden calf”

The title: The title: ““The great schemer”The great schemer” Attractive & brilliant cheaterAttractive & brilliant cheater

Python -based Physics Analysis Environment for LHCb G. Barrand, M. Frank, P. Mato, E. de Oliveira,...

Documents

Transcript of Python -based Physics Analysis Environment for LHCb G. Barrand, M. Frank, P. Mato, E. de Oliveira,...