HPC for ML
One of the main reasons why ML (and deep learning in particular) is so successful now is our ability to perform intensive computations on very large amounts of training data. So, I decided to step outside of my comfort zone this semester and take an entire class on supercomputing software, and how it is applied to achieve maximum performance of ML algorithms.
In this class, we studied how to…
- use HPC techniques to find and solve performance bottlenecks,
- do performance measurements and profiling of ML software,
- evaluate the performance of different ML software stacks and hardware systems,
- develop high performance distributed ML algorithms, and
- use fast math libraries, CUDA and C++ to accelerate high-performance ML algorithms.
I definitely learned a lot, and got much more comfortable working with low-level languages.
Til next time,