A Deep Learning tool for fast detector simulation · A Deep Learning tool for fast detector...
1
Federico Carminati 2 , Gulrukh Khattak 2 , Sofia Vallecorsa 1 , Jean-Roch Vlimant 3 A Deep Learning tool for fast detector simulation 1 Gangneung Wonju U. & CERN, 2 CERN, 3 Caltech [email protected] Simulation of particle transport through matter is fundamental for understanding the physics of High Energy Physics (HEP) experiments, as the ones at the Large Hadron Collider (LHC) at CERN. Currently, most of LHC worldwide distributed CPU budget – in the range of half a million CPU-years equivalent – is dedicated to simulation. A faster approach is to treat Monte Carlo simulation as a black-box and replace it by a deep learning algorithm trained on different particle quantities. Our project intends to test several DL techniques to achieve a speedup of at least x100 with respect to Monte Carlo techniques. Generative models, such as Generative Adversarial Networks (GAN) are particularly suited to replace Monte Carlo: they generate realistic samples modelling complicated probability distributions. They allow multi-modal output, they can do interpolation and they are robust against missing data. We can use Monte Carlo simulation to train GANs to reproduce realistic detector output • Motivation Generative Adversarial Networks for fast simulation One of the first 3D GAN implementations ! Distributed training for HPC Training: 3D GAN are not “out-of-the-box” networks • Complex training process First physics results look very promising Perform detailed validation against standard Monte Carlo comparing high level quantities (energy shower shapes) and detailed calorimeter response (single cell response) Agreement is remarkable (a few percent)! Discriminator auxiliary energy regression task has 5% accuracy over the whole energy range Start with the most time consuming simulations : high granularity calorimeters (CLIC detector studies)*. Single particles deposit energy in an array of calorimeter cells and generate a “energy shower”, interpreted as a 3D image. The idea Achievements Development 200 Computing centers in 20 countries: > 600k cores @CERN (20% WLCG): 65k processor cores ; 30PB Generative Adversarial Networks simultaneously train two models: a Generator G and a Discriminator D ´G reproduces the data distribution starting from random noise ´D estimates the probability that a sample came from the training data rather than G ´Training procedure for G is to maximize the probability of D making a mistake References Data is essentially a 3D image π shower in CLIC calorimeter simulated with Monte Carlo approach 25 25 25 Generator Discriminator ´ Goodfellow et al. 2014 ´ Conditional GAN, arXiv: 1411.1744 ´ Auxiliary Classifier GAN, arXiv:1610.0958 Image from: https://medium.com/@devnag/ge nerative-adversarial-networks- gans-in-50-lines-of-code-pytorch- e81b79659e3f *http://cds.cern.ch/record/2254048# Computing resources All tests run with Intel optimised Tensorflow 1.4.1. + keras 2.1.2 Systems: Intel Xeon Platinum 8180 @2.50 GHz (28 physical cores) NVIDIA GeForce GTX 1080 Time to create an electron shower Method Machine Time/Shower (msec) Full Simulation (geant4) Intel Xeon Platinum 8180 17000 3d GAN (batch size 128) Intel Xeon Platinum 8180 7 3d GAN (batchsize 128) GeForce GTX 1080 0.04 3d GAN (batchsize 128) Intel i7 @2.8GHz (MacBookPro) 66 Time to train for 30 epochs Method Machine Training time (days) 3d GAN (batchsize 128) Intel Xeon Platinum 8180 (Intel optimised TF) 30 3d GAN (batchsize 128) GeForce GTX 1080 1 Inference: Using a trained model is very fast ! • Orders of magnitude faster than standard Monte Carlo • Test inference on FPGA and integrated accelerators Implement data parallelism and study scaling on clusters Modify mpi based library (mpi-learn (#) ) to parallelise adversarial training process Preliminary scaling measured at CSCS Swiss National Super Computing Center Picture from: https://github.com/duanders/mpi_learn # https://github.com/duanders/mpi_learn How generic our network can be? • Our baseline is an example of next generation calorimeter detector • Extend to other calorimeters • Explore optimal network topology according to the problem to solve • Hyper-parameters scans and meta- optimization • Fast training Generalisation Longitudinal shower shape for 400 GeV electron Prototype of the SemiDigital Hadronic Calorimeter during tests at the CERN SPS facility Generator and Discriminator are based on 3D convolutions. The shower shape depends on particle type and energy so we condition training on particle energy Auxiliary energy regression task for discriminator Elastic Average SGD Calorimeter energy response: GAN prediction stays stable through 20 nodes! Monte Carlo 1 GPU GAN 5 GPU GAN 10 GPU GAN 20 GPU Part of this work was conducted at "iBanks", the AI GPU cluster at Caltech. We acknowledge NVIDIA, SuperMicro and the Kavli Foundation for their support of “iBanks” This work is partially funded by Intel Parallel Computing Center program Calorimeter response as a function of primary electron energy 3D GAN for calorimeter simulatio n 3D GAN adversarial training process Start Generate Fake Image Batch Train Discriminator on real images Train Discriminator on fake images Discriminator Loss is average of both losses Train generator twice Epoch< num_epochs Longitudinal shower shape for 100 GeV electron Discriminator energy regression accuracy as a function of primary electron energy Transverse shower shape for 100-500 GeV pions (E p -E discr )/E p JHEP 11 (2017) 047
Transcript of A Deep Learning tool for fast detector simulation · A Deep Learning tool for fast detector...
FedericoCarminati2,Gulrukh Khattak2,SofiaVallecorsa1,Jean-Roch Vlimant3
ADeepLearningtoolforfastdetectorsimulation
1GangneungWonju U.&CERN,2CERN,[email protected]
Simulation of particle transport through matter is fundamental for understanding the physics of High Energy Physics(HEP) experiments, as the ones at the Large Hadron Collider (LHC) at CERN. Currently, most of LHC worldwidedistributed CPU budget – in the range of half a million CPU-years equivalent – is dedicated to simulation. A fasterapproach is to treat Monte Carlo simulation as a black-box and replace it by a deep learning algorithm trained ondifferent particle quantities. Our project intends to test several DL techniques to achieve a speedup of at least x100with respect to Monte Carlo techniques.
Generative models, such as Generative Adversarial Networks (GAN) are particularly suited to replace Monte Carlo: theygenerate realistic samples modelling complicated probability distributions. They allow multi-modal output, they can dointerpolation and they are robust againstmissing data.
We can use Monte Carlo simulation to train GANs to reproduce realistic detector output
• Motivation
GenerativeAdversarialNetworksforfastsimulation
Oneofthefirst3DGANimplementations!
DistributedtrainingforHPC
Training:3DGANarenot“out-of-the-box”networks• Complextrainingprocess
FirstphysicsresultslookverypromisingPerformdetailedvalidationagainststandardMonteCarlocomparinghighlevelquantities(energyshowershapes)anddetailedcalorimeterresponse(singlecellresponse)Agreementisremarkable (afewpercent)!Discriminatorauxiliaryenergyregressiontaskhas5%accuracyoverthewholeenergyrange
Startwiththemosttimeconsumingsimulations:highgranularitycalorimeters(CLICdetectorstudies)*.Singleparticlesdepositenergyinanarrayofcalorimetercellsandgeneratea“energyshower”,interpretedasa3Dimage.
TheideaAchievem
entsDevelopm
ent
200Computingcentersin20countries:>600kcores
@CERN(20%WLCG):65kprocessorcores;30PB
GenerativeAdversarialNetworkssimultaneouslytraintwomodels:aGeneratorG andaDiscriminatorD´G reproducesthedatadistributionstartingfromrandom noise´DestimatestheprobabilitythatasamplecamefromthetrainingdataratherthanG
´TrainingprocedureforGistomaximizetheprobabilityofDmakingamistake
References
Dataisessentiallya3D
image
πshowerinCLICcalorimetersimulatedwithMonteCarloapproach25
2525
Generator
Discriminator
´ Goodfellow etal.2014´ ConditionalGAN,arXiv:1411.1744´ AuxiliaryClassifierGAN,arXiv:1610.0958
Image from:https://medium.com/@devnag/generative-adversarial-networks-gans-in-50-lines-of-code-pytorch-e81b79659e3f
*http://cds.cern.ch/record/2254048#
ComputingresourcesAlltestsrunwithInteloptimised Tensorflow 1.4.1.+keras 2.1.2
Systems:[email protected](28physicalcores)NVIDIAGeForceGTX1080
Timetocreateanelectronshower
Method Machine Time/Shower (msec)
FullSimulation(geant4) IntelXeonPlatinum8180 17000
3dGAN(batchsize128) IntelXeonPlatinum8180 7
3dGAN(batchsize 128) GeForceGTX1080 0.04
3dGAN(batchsize 128)
Intel [email protected](MacBookPro) 66
Timetotrainfor30epochs
Method Machine Trainingtime(days)
3dGAN(batchsize 128)
Intel XeonPlatinum8180
(Inteloptimised TF)30
3dGAN(batchsize 128) GeForceGTX1080 1
Inference: Usingatrainedmodelisveryfast!• OrdersofmagnitudefasterthanstandardMonteCarlo
• TestinferenceonFPGAandintegratedaccelerators
ImplementdataparallelismandstudyscalingonclustersModifympi basedlibrary(mpi-learn(#))toparallelise adversarialtrainingprocessPreliminaryscalingmeasuredatCSCSSwissNationalSuperComputingCenter
Picturefrom:https://github.com/duanders/mpi_learn #https://github.com/duanders/mpi_learn
Howgenericournetworkcanbe?• Ourbaselineisanexampleofnextgeneration
calorimeterdetector• Extendtoothercalorimeters• Exploreoptimalnetworktopologyaccordingto
theproblemtosolve• Hyper-parametersscansandmeta-
optimization• Fasttraining
Generalisation
Longitudinalshowershapefor400GeVelectron
PrototypeoftheSemiDigital HadronicCalorimeterduringtestsattheCERNSPSfacility
GeneratorandDiscriminatorarebasedon3Dconvolutions.Theshower shape dependsonparticletypeandenergysoweconditiontrainingonparticleenergyAuxiliaryenergyregressiontaskfordiscriminator
ElasticAverageSGD
Calorimeterenergyresponse:GANpredictionstaysstablethrough20nodes!
MonteCarlo1GPUGAN5GPUGAN10GPUGAN20GPU
Partofthisworkwasconductedat"iBanks",theAIGPUclusteratCaltech.WeacknowledgeNVIDIA,SuperMicro andtheKavli Foundationfortheirsupportof“iBanks”
ThisworkispartiallyfundedbyIntelParallelComputingCenterprogram
Calorimeterresponseasafunctionofprimaryelectronenergy
3DGANforcalorimetersimulation
3DGANadversarialtrainingprocess Start
GenerateFakeImageBatch
TrainDiscriminatoronrealimages
TrainDiscriminatoronfakeimages
DiscriminatorLossisaverageofbothlosses
Traingeneratortwice
Epoch<num_epochs
Longitudinalshowershapefor100GeVelectron
Discriminatorenergyregressionaccuracyasafunctionofprimaryelectronenergy
Transverseshowershapefor100-500GeVpions
(Ep-E d
iscr)/E p
JHEP11(2017)047
