With the challenges posed by the architecture and software environments of today’s most powerful supercomputers, and even greater complexity on the horizon from next-generation and exascale systems, there is a critical need for specialized, in-depth training for the computational scientists poised to facilitate breakthrough science and engineering using these amazing resources. The Argonne Training Program for Extreme-Scale Computing (ATPESC) program is designed to provide intensive hands-on training on the key skills, approaches and tools to design, implement and execute computational science and engineering applications on current supercomputers and the HPC systems of the future.
As a bridge to that future, the two-week ATPESC program was held from August 2-14, 2015, at the Pheasant Run Resort in suburban Chicago. Wrapping up its third year with a class of 65 researchers, the intensive two-week curriculum filled many gaps that exist in the training computational scientists typically receive through formal education or shorter courses.
Below, six participants reflect on their experience and how it will impact their future.
Jason Bender
Q: What is your current position (or path of study) and how does it tie in with high-performance computing?
I am currently a Ph.D. student at the Department of Aerospace Engineering and Mechanics at the University of Minnesota in Minneapolis. Also, I am a fourth-year fellow in the DOE Computational Science Graduate Fellowship program, which has funded most of my graduate research and education. I am planning to finish graduate school early in 2016.
My thesis work is on high-temperature chemical reactions that occur in the hot gases around aerospace vehicles traveling at hypersonic speeds (roughly five times the speed of sound or faster). Hypersonic flow experiments are difficult and expensive, so high-performance computing plays a crucial role in this field. My research is inherently multidisciplinary, involving computational fluid dynamics, computational chemistry, and molecular dynamics.
Q: Why did you decide to apply for ATPESC? What did you hope to get out of the program?
Over the course of graduate school, I’ve learned just how exciting the field of computational science is. High-performance computing enables us to simulate and analyze physical phenomena in ways that are impossible with experiments or theory alone. Computing is having a revolutionary impact on science, and DOE is largely leading that charge.
In my future career, I want to continue working as a computational scientist, so applying to ATPESC was a no-brainer. By bringing together a diverse array of experts from the DOE national laboratories, academia, and industry, ATPESC promised a survey of important topics in computational science that I couldn’t find anywhere else. I was interested both in learning new skills for my own research and in getting deeper insight into the computational science ecosystem at DOE.
Q: What was your biggest takeaway(s) from ATPESC?
The survey of hardware trends, algorithms, mathematical and scientific libraries, software engineering practices, performance toolkits, visualization software, and tools for data-intensive I/O have all been fantastic. Some of these I had already used extensively, some I had just heard about or used only sparingly, while others were totally new to me. In all cases, it has been tremendously insightful to hear perspectives and overviews from leading experts.
I am certain that what I’ve learned at ATPESC will impact my current and future research. The training has expanded the set of tools in my toolbox and grown the range of scientific problems that I am confident tackling. The hands-on exercises have been especially useful for trying out new tools, making it easier to hit the ground running when I return to my university.
Q: How will this experience impact your career path/studies moving forward?
ATPESC has further reinforced my interest in and excitement for computational science. Before coming to ATPESC, I was already pursuing job prospects at the DOE National Laboratories. I am now even more excited by the opportunity to do computational science at DOE after graduate school.
Q: If you had a colleague or peer considering applying for ATPESC next year, what would you tell them about the program?
ATPESC is a remarkable resource for early-career computational scientists. I know of no other program that offers such a comprehensive and in-depth training in high-performance computing. The program is definitely intense, but it is its total immersion approach that makes it so effective. If you are serious about computational science, you should absolutely take advantage of the opportunity to attend ATPESC.
Britni Crocker
Q: What is your current position (or path of study) and how does it tie in with high-performance computing?
I’m a fourth-year graduate student in Medical Engineering and Medical Physics at the joint Harvard Medical School/MIT department of Health Sciences and Technology. My research is in the neurology department at Massachusetts General Hospital, where we collect 10s of terabytes of time-series data from patients. Essentially, our datasets are so large that they can’t be analyzed on a desktop computer, and even with the research computing cluster at the hospital, we are limited by computational resources.
Q: Why did you decide to apply for ATPESC? What did you hope to get out of the program?
As a scientist, programming is often just something you pick up rather than something you ever formally learn. When you really need performance, though, there’s so much you need to learn – from computer architecture to file I/O. I applied to ATPESC because I’m in the process of scaling up my code, and I wanted to make sure I was taking full advantage of the capabilities of our machines. I was hoping to understand the basics behind getting your applications optimized for performance and to come away with some experience and skills in HPC.
Q: What was your the biggest takeaway(s) from ATPESC?
One thing I have learned is that the field of HPC is constantly moving. You really need to be continually improving and learning new techniques, because the available architectures and software are always changing.
Q: How will this experience impact your career path/studies moving forward?
In the short term, many of the skills and tools I’ve learned in the past two weeks will help me scale up my application faster and much less painfully. In the long term, there are very few people in neuroscience doing HPC right now, but I think we will soon see large increases in that area due to new technologies that are enabling high-throughput data collection. Having these skills in HPC will help me to take full advantage of these advances in my field.
Q: If you had a colleague or peer considering applying for ATPESC next year, what would you tell them about the program?
Honestly, I’d just tell them that they’ll have a chance to run their code on a supercomputer! And at the same time, they’ll have experts from all different aspects of HPC coming in to talk to them and give them advice. I was very happily surprised with how eager the lecturers were to help us get everything running smoothly.
Lisa Lowe
Q: What is your current position (or path of study) and how does it tie in with high-performance computing?
I am a high performance scientific programmer for Lockheed Martin, who supports the U.S. Environmental Protection Agency (EPA) with the Environmental Modeling and Visualization Laboratory. One of my tasks is to help EPA scientists run their codes on our local cluster. I have parallelized two ocean modeling codes using MPI and OpenMP in support of EPA studies of hypoxia in the Gulf of Mexico.
Q: Why did you decide to apply for ATPESC? What did you hope to get out of the program?
I know the fundamentals of high-performance computing (HPC), but not the very newest technologies. I had not had any experience in GPU or accelerator programming. I hoped to move beyond fundamentals and on to an expert level of HPC knowledge.
Q: What was your the biggest takeaway(s) from ATPESC?
ATPESC reinforced the ideas of portability and scalability, and reminded me that I need to spend more time profiling my codes and capturing provenance (reproducibility).
Q: How will this experience impact your career path/studies moving forward?
ATPESC will help me be ready for any computational problem an EPA scientist might come up with. I will be able to help the agency, and in turn, help the greater community by writing and optimizing large-scale, realistic simulations that can deal with real-world problems like climate change.
Q: If you had a colleague or peer considering applying for ATPESC next year, what would you tell them about the program?
One of the best things about ATPESC is meeting the other participants and interacting personally with the speakers. I met the authors of books that I had used to study numerical analysis and HPC. I met participants who exchanged research ideas from their projects that were similar to mine. Also, being able to sit down one-on-one with a main author of a visualization tool saved me several dozens of hours of trying to do the same thing myself.
Jonathan Madsen
Q: What is your current position (or path of study) and how does it tie in with high-performance computing?
I am a PhD student at Texas A&M University in the Department of Nuclear Engineering, work as a part-time Graduate Research Assistant at Los Alamos National Lab (XCP-3, Monte Carlo codes group), and for the past 4 years I have been a member of the Geant4 collaboration. I primarily work on Monte Carlo particle transport codes – FinMcool (TAMU), MCATK (LANL), and Geant4 (CERN) – and on a deterministic transport code, PDT (TAMU).
Q: Why did you decide to apply for ATPESC? What did you hope to get out of the program?
I want to be an exceptional computational scientist. I want the codes that I help develop to run faster, scale better, and be more flexible than any of the alternatives. I also want to have the utmost confidence that the codes that I work on are doing exactly what they are supposed to do in a safe and efficient manner. I applied to ATPESC as part of my pursuit of achieving these goals. More specifically, I applied with the hope of learning paradigms for implementing mixed-mode parallelism and how to increase parallel efficiency. Although, the former wasn’t discussed at length, the latter was certainly covered and, in addition to this, I gained perspective on the parallelism requirements for colleagues in different fields – which aids in gaining a broader mental picture for my pursuit of flexibility. Plus, two weeks in the Chicago area appealed to me because I’ve always wanted to go see a game at Wrigley (Cubs won 2-0 after the closer loaded the bases and then proceeded to strike out the side to end the game).
Q: What was your biggest takeaway(s) from ATPESC?
Parallel programming is an art. There is no cookie-cutter solution to scale on supercomputers, but instead your approach (shared memory, distributed memory, or both) has to viewed through the lens of the problem you are trying to solve and the most time-efficient and typically best option is to take advantage of libraries that experts have spent a painstaking amount of time developing to ensure performance.
Q: How will this experience impact your career path/studies moving forward?
This experience will impact my career path in many ways, but in more subtle ways than profound changes. I do not foresee a drastic change in my coding style or a change in programming languages because prior to this, I was experienced in methods of avoiding parallel bottlenecks and I work on several established codes that will not be changing languages any time soon, respectively. However, this experience has made me, and will continue to make me, a better parallel programmer in many ways. For example, I was exposed to many tools that I am now utilizing to debug and improve the performance of my code(s), such as the Intel Cluster Studio and HPCToolkit. It has opened me up to considering using Python to control the flow of my programs and executing C/C++ behind the scenes – which would greatly benefit the usability of my programs. It has reiterated the importance of using good programming practices, documenting everything, and encouraged me to share some collections of routines and frameworks I have developed over the years for quickly and safely implementing shared memory parallelism. Additionally, in contrast to the previous statement, this program has encouraged me to utilize more libraries provided by others, as I now have insight into the aptitude and thought process of the core developers. And finally, this experience has encouraged me – through the hands-on experiences – to pursue the benefits of utilizing GPUs, where possible.
Q: If you had a colleague or peer considering applying for ATPESC next year, what would you tell them about the program?
I would tell them to absolutely apply for ATPESC. I would tell them that, yes, the training program will cut into your summer work or detract from time that could be well spent working on research; however, the long-term benefit of being exposed to the myriad of architectures, languages, libraries, performance, visualization, and debugging tools, is almost secondary to the understanding and problem-framing conceptualization you gain from listening to the tremendous amount of experts this training program makes available to you. This training program exposes you to not only an understanding of how the parallel libraries and tools are used – which can be found in the documentation – but how the parallel libraries are meant to be used and the thought process and ideas behind them – straight from the mouths of the core developers. All throughout the various walks of life, there are two types of people that are successful: those that are extremely diverse in their skills set and those that are uniquely exceptional at specific things. This training program exposes you to everything you need as a computer scientist to become uniquely diverse and the connections and insight one would need to become uniquely exceptional at something specific. Both or either of these qualities will set you apart and it is an investment that will far exceed any two-week setback in pay or research progress.
Ivy Bo Peng
Q: What is your current position (or path of study) and how does it tie in with high-performance computing?
I am currently a second-year PhD student in computer science at KTH Royal Institute of Technology, Sweden. I completed my bachelor’s and master’s degrees in computer science in Singapore. My dissertation topic is the hybrid programming models for extreme-scale computing. I am working on the Message Passing and PGAS model to investigate their limitations at extreme scale and to implement innovative algorithms that could hide the process imbalance in collective operations. I am also working on preparing an exascale-ready application called iPIC3D that simulates plasma physics.
Q: Why did you decide to apply for ATPESC? What did you hope to get out of the program?
I got to know ATPESC from the lecture videos uploaded to YouTube last year. At first, I was only looking for lectures in network architecture and programming models in HPC. After following the program, I found ATPESC covers much broader topics that are highly relevant to my research and that the lecturers are world-class experts in their fields.
I decided to apply for ATPESC to get an overview of the pillars of the HPC ecosystem: architectures, programming models, algorithms, libraries, applications, tools, etc. I wanted to know how these aspects interact and impact each other. I also hoped to hear about the state-of-art in each field, so that I have a starting point if I encounter a problem that requires knowledge beyond my research field.
Q: What was your the biggest takeaway(s) from ATPESC?
My main takeaways from ATPESC were learning about the philosophy behind different programming systems and the state-of-the-art research in extreme-scale algorithms. I also gained a basic understanding of network architectures and their future trends, found some numerical libraries that can accelerate our application development, and got to know some profiling tools that may provide performance information on real machines to feed into our theoretical simulation model.
In addition, I was able to use Mira to perform a recent weak scaling test with iPIC3d, which ran at 81% efficiency on a half million processors. I plan to apply for a Director’s Discretionary project at the ALCF to continue my code development work in the future.
Q: How will this experience impact your career path/studies moving forward?
I will be able to tackle the challenges from different perspectives in hopes of finding more innovative solutions. ATPESC has given me a bigger picture of HPC and extreme-scale computing. My research may only focus on one small area, but I will now be paying more attention to the interplay with other aspects of HPC. I had the opportunity to meet leading experts working on the research topics similar to mine. Besides sharing ideas on research, I also got to know what possible career paths may be available after graduation and what I need to do to prepare for them.
Q: If you had a colleague or peer considering applying for ATPESC next year, what would you tell them about the program?
This is a very efficient and intensive program. Within two weeks, you will get an overview of the most important aspects of HPC and extreme-scale computing. You can hear from world-class experts about state-of-the-art research and their visions on future HPC trends. You will be amazed by the complexity and variety of your classmates’ applications, and it is fun! You also get a chance to run your code on one of the top-ranking supercomputers in the world. And last but not least, the organizers and staff are always ready to help if and when you need it. ATPESC made my summer meaningful and it can do the same for you.
Kan Wang
Q: What is your current position (or path of study) and how does it tie in with high-performance computing?
I’m a postdoctoral research associate at University of Notre Dame performing research on high-fidelity turbulence simulations, with applications to aero-optics and aeroacoustics. Since my PhD study, I have been working in this field for more than eight years. High-fidelity numerical simulation techniques, including direct numerical simulation and large-eddy simulation, are important investigation tools to tackle the intricate turbulent flows in nature and engineering. These methods are also in great demand of computer power. We write, modify, and optimize codes, and run large simulations on supercomputers. Therefore, high-performance computing is part of my research, and I am very interested in it.
Q: Why did you decide to apply for ATPESC? What do you hope to get out of the program?
I knew ATPESC from its website. After I looked through the 2014 agenda, I knew it was the training program I was looking for. I immediately subscribed to the mailing list and decided to apply for ATPESC 2015. The contents of the program perfectly fit my need. From architecture, programming models, profiling and debugging, numerical algorithms, software engineering, visualization to I/O and data management, the program covers almost every important aspect of high-performance computing. I viewed the training program as an opportunity to take my knowledge of HPC and computational skills to a new level.
Q: What was your the biggest takeaway(s) from ATPESC?
I got what I expected from ATPESC. We received an introduction to the trends in next-generation supercomputer architectures. Around these trends, we learned the leading-edge technologies that are needed to achieve good performance and performance portability on future supercomputers. It is a very intense program and thus it’s almost impossible to digest all information in these two weeks, but still, I feel that I’m well prepared for the new generation of extreme-scale computing technologies.
Q: How will this experience impact your career path/studies moving forward?
ATPESC not only taught me the technologies for extreme-scaling computing, but also provided me good opportunities to communicate with world-class experts in HPC. The information I got from ATPESC makes me rethink the approaches I’ve used in my research, and will guide me in making decisions in future projects. The knowledge of the evolution of future architectures will help me look ahead and choose the right strategy when I design my research tools.
Q: If you had a colleague or peer considering applying for ATPESC next year, what would you tell them about the program?
I would tell them ATPESC is a very unique training program. It is highly likely the only program that can gather so many world-leading lecturers in HPC and provide supercomputer resources like Mira for doing exercises and testing codes. ATPESC is very well organized. From lectures and hands-on exercises to transportation and accommodations, everything is well supported by Argonne staff.
