For 2016, I have a resolution to deepen my ability to “think parallel.” Despite years of parallel programming, I know there is always more to learn. Whether you are new to parallel programming, or do it in your sleep, 2016 is a good year to invest more in yourself.
In regards to learning parallel programming, I have no greater advice than to “think parallel.” Programming is dependent on our ability to reason logically and break a problem down step-by-step. Parallel programming challenges us to use concurrency in order to achieve higher performance. How well we think about parallelism affects how effective we will be in parallel programming.
The parallel programming landscape today covers virtually all devices, ranging from processors for cellphones to those for supercomputers. This is a change from when modern multicore processor designs appeared in 2004 and first began displacing single core processor designs. Since that time, we have seen virtually a 100 percent shift to multicore processors.
Parallel computing is everywhere and is, therefore, both an opportunity and a challenge for every software developer. The most important aspect of being a parallel programmer is having the ability to think about where parallelism opportunities lie within your application, and to exploit them in such a way that is to a parallel computer’s advantage. We just need to “think parallel.”
I like to call the past decade in the parallel computing world the dawn of ubiquitous parallel computing. It was a time when both single core and multicore performance mattered. In this short time, parallel programmers had to develop for both single core and multicore platforms. Luckily, we have been rapidly exiting that era, and software programs that require parallel computing are acceptable everywhere.
In the next decade, which I like to call the coming-of-age for ubiquitous parallel computing, we will see many hardware and software innovations coming together. Innovative hardware is being designed to make parallel computing more affordable and to allow for higher performance. Software efforts ranging from operating systems, to tools, to language standards have all seen a decade of advancements to support parallel programming, on which we can now build upon. With the hardware and software to build parallel systems upon, we are going to see compute power unleashed like never before. This era will be defined by the combination of compute power from parallel computing, with access for everyone via the cloud, combined with an industry that is now equipped and oriented to take advantage of it.
How can software developers take advantage of this? The two greatest lessons to take away are first, to make yourself your greatest asset by investing in learning to “think parallel.” Second, know that parallel computing is coming of age. The tools, techniques and systems available make parallel computing both essential and approachable in a way that is radically different than it was a decade ago. There are limitless opportunities ahead for software programmers to unlock the power of parallel computing to solve needs in all fields.
It all boils down to an investment in ourselves as software developers. Do we “think parallel?” There are many ways to learn to make parallelism an intuitive and everyday part of our thinking. One resource is the Intel Code Modern Web site. There are many books, too, including ones that I’ve worked on — after all, parallel programming is my professional passion. Regardless of where you start, I strongly encourage joining me by putting “think parallel” on your New Year’s resolution list for 2016.
James Reinders is an expert in the area of parallelism, Intel’s leading spokesperson on tools for parallelism, Intel’s parallel programming evangelist, and has books on VTune (2005), TBB (2007), Structured Parallel Programming (2012), Intel Xeon Phi coprocessor programming (2013), Multithreading for Visual Effects (2014), High Performance Parallelism Pearls Volume One (2014) and Volume Two (2015).
