Speeding up the Monte Carlo simulations is of interest because such simulations take vast computational resources to run. What if we could dramatically speed things up but still have results that did as well as today’s classical Monte Carlo simulations?
This is on the minds of a number of research scientists out there. Their thinking? Harvest the flexibility of a modern computer system with a novel application of machine learning. The idea boils down to replacing a computationally driven behavior with a learned behavior. Other researchers are exploring variations that leverage the performance of a trained neural net but still connect with the original simulator to offer interpretability not available from the neural net alone.
Why displace Monte Carlo?
Simulations—critical to data analytics—consume vast amounts of computing doing Monte Carlo simulations. Sofia Vallecorsa of CERN explained her motivation in exploring the possibility of displacing classical Monte Carlo simulations, by explaining that half of the computing in the Worldwide LHC Computing Grid (WLCG-a global collaboration of more than 170 computing centers in 42 countries) is used for simulation.
Future plans to upgrade CERN’s LHC (the Large Hadron Collider— the world’s largest particle accelerator) will dramatically increase particle collision rates, meaning it is important to explore new frameworks likethis and assess their potential in helping to ensure data rates remain manageable.
Could a trained neural network displace Monte Carlo simulations?
Building on previous work, a team of researchers at CERN, SURFsara, and Intel, have focused on the possibility of using deep-learning engines for fast simulation. These are trained using the original Monte Carlo simulators, but then do the actual simulation without employing classical Monte Carlo simulators at all. This work is being carried out through a project in what is known as CERN openlab, a unique public-private partnership between CERN and leading ICT companies.
The team worked on a new way to do detector simulation in the hope that it would prove faster than existing simulation methods. Simulations of detectors allow scientists to understand how designs affect measurements and physics. They can use the information to understand inefficiencies, and inaccuracies in their equipment. Most importantly, simulation enables comparisons to be made between actual data and theory, with conclusions then reached about the physics being studied.
The team is investigating the use of deep-learning for fast simulation. The approach effectively encapsulates expensive computations into a neural net via training, and then is able to extract — through inferencing (using the net) — a result much faster than the standard algorithmic approaches of a classical Monte Carlo simulation. The team is also exploring how much accuracy can be maintained while exhibiting much faster performance. Early results reported by the team in simulating a calorimeter for the proposed CLIC (Compact LInear Collider) accelerator — using a conditional generative adversarial network (GAN) — have compared very well with Monte Carlo simulation output, while using only a fraction of the compute resources.
The detector being simulated has a 3D output they refer to as a particle “shower.” The classical Monte Carlo (geant4 – a toolkit for the simulation of the passage of the resulting particles through detectors) is able to simulate a shower in just over 17 seconds, whereas the trained model (a 3D-GAN) produces output at a rate of about 7 msecs/shower — running on the same Intel Xeon Platinum 8180 processor-based hardware. It is, of course, important to note that this comparison does not include the time required for training the GAN.
Furthermore, the team was able to take their work to a distributed system. Their runs on the TACC Stampede2 cluster (dual socket Intel Xeon processors 8160 with Intel Omni-Path architecture) exhibited 94% scaling efficiency on 128 nodes while maintaining accuracy. The results are cautiously called “very promising from the physics perspective” by the researchers as they continue to examine their error bars and look to understand how to reduce them. They also plan to consider how the model will generalize to other uses.[i]
Why does it work?
Recently, upon conclusion of a talk where they presented their work, a fellow scientist in the audience pointed out that Monte Carlo simulation involves complex calculations, requires recursion, projection sampling, and other things which are not representable in a neural net. So, referring to the 3D-GANs output, he asked “Why does it work?” After a little laughter died down, CERN’s Sofia Vallecorsa said “honestly, we don’t know (yet).” She went on to explain that the trained network obviously learned the outputs that it was exposed to in training and effectively found a faster way to approximate their output without doing all the same calculations. Vallecorsa described the 3D outputs as “much more blurry” than the outputs of the actual simulations. It is “just learning the results”, not the Monte Carlo algorithm. I can’t help but think that if this was done by a human we would just say they were good at “faking it.” Well, faking it “very, very, very well” given the error bars that were presented!
Can you add interpretability?
The comment that “we don’t know (yet)” how it works, is a common observation about any trained neural net. The concept of “interpretability” is one that is debated and explored a great deal.
Concerned about this exact problem, a group of researchers and scientists at Oxford, the University of British Columbia, NYU, LBNL, University of Liège, and Intel, have been exploring probabilistic compilation, and a way of connecting an original simulator with the inferencing engine, to obtain degrees of interpretability. They share that physical sciences can benefit very much from interpretability — for instance it can give the ability to inspect aspects of processes encoded in the simulation, such as the chain of particle decays and interactions within the detector that led to predictions. They assert that such a capability is not possible in inference techniques that do not have access to the simulator, such as those based on neural nets.
In published results, they highlighted using probabilistic programming to control a state-of-the-art Monte Carlo event generator of high-energy reactions of particles (SHERPA1), which is commonly used with geant4. The “inference efficiency” is achieved via inference compilation. Inference compilation is a technique for performing fast repeated inference using deep neural nets to parameterize proposal distributions, essentially displacing Monte Carlo! The extra twist, in the quest for interpretability, is the use of the neural nets to control the original simulator to achieve useful results at a fraction of the computational cost of Markov chain Monte Carlo sampling. The authors assert that their approach is readily scalable to more complex events and full detector simulators, paving the way to its use in the discovery of new fundamental physics.[ii] To learn more, read the paper “Efficient Probabilistic Inference in the Quest for Physics Beyond the Standard Model.”
One tool they share is pyprob, a PyTorch-based library for probabilistic programming and inference compilation, described by the creators as being a research prototype in alpha testing stage. The focus for the library is on coupling existing simulation codebases with probabilistic inference with minimal intervention. As mentioned in their work, one of their inference engines is for Markov chain Monte Carlo.
Is a post-modern Monte Carlo world possible?
The extent to which classical Monte Carlo simulations can be replaced will depend a great deal on how well the new methods are trusted. Like much of machine learning, the trained behavior seems to work, but exactly how it works is more difficult to interpret. Without a solid understanding of how it works, it is difficult to have a confident understanding of its meaning, potential shortcomings, or failure modes. This brings up the special interest in interpretability that researchers are exploring. Given the potential gold mine in terms of performance gain, it is reasonable to assume that this avenue of thinking will see some adoption in production usage. Whether that proves a competitive edge, or an Achilles’ heel, will determine the future fate of this novel approach to have a learned behavior replace a computationally driven behavior. If this is productive, what other complex computations might be displaced by a learned behavior? That is an interesting question indeed.
Sofia Vallecorsa and Federico Carminati of CERN, Damian Podareanu and Valeriu Codreanu of SURFsara B.V., and Vikram Saletore and Hans Pabst of Intel, presentation “Distributed Training of Generative Adversarial Networks for Fast Detector Simulation on Intel Xeon HPC Cluster,” NERSC Big Data Summit, held July 18, 2018, presentations available at http://www.nersc.gov/research-and-development/data-analytics/big-data-center/big-data-summit/.
Frank Wood of the University of British Columbia, for his presentation titled “Working Towards Distributed Inference Compilation at Scale – Efficient Probabilistic Inference in the Quest for Physics Beyond the Standard Model,” NERSC Big Data Summit, held July 18, 2018, presentations available at http://www.nersc.gov/research-and-development/data-analytics/big-data-center/big-data-summit/.
# # #
This article was produced as part of Intel’s HPC editorial program, with the goal of highlighting cutting-edge science, research and innovation driven by the HPC community through advanced technology. The publisher of the content has final editing rights and determines what articles are published.