NOTE: This should be inside a notice box or something
No generative technology has been used in writing of this blog post or the below mentioned code examples, all mistakes are my own. For any comment open a pull request on the blog’s github page or contact me via email.
Introduction
This is a deep overview of openmp1 programming model used in C/C++ and Python, openmp is an implicit programming model used for writing single-node high performance parallel code for cpu as well as accelerators (with the help of openmp offload). OpenMP contains the least amount of friction when it comes to writing parallel programs compared to other programming models and high performance APIs.
In order to follow through with the examples in the blog post, you can clone the OpenMP playground github repo2 on your system, for GPGPU examples you must have a CUDA supported device as well as a openmp offload supported compiler system such as nvc++ or clang with offloading enabled. The guide on how to compile the clang compiler with openmp offload support is available here
Directive based programming model
Directive3 based programming is a way to hint the compiler about what transformation should be applied to which code block. Besides openmp there is also the OpenACC4 programming model for GPGPU applications which works similar to openmp but with some differences.
template <std::size_t row, std::size_t col>
void omp_add(const vector_f32& a, const vector_f32& b, vector_f32& c) {
#pragma omp parallel for
for (int i = 0; i < row; i++) {
#pragma omp simd
for (int j = 0; j < col; j++) {
size_t idx = offset(col, i, j);
c[idx] = a[idx] + b[idx];
}
}
}OpenMP Task
Task based abstraction allows for irregular and dynamic parallel tasks to be executed, besides openmp there are task based abstractions in onetbb, kokkos and taskflow. Task based programs are often structred as a Directed Acyclic Graphs or DAGs with explicit dependencies.
#pragma omp parallel {
#pragma omp single {
#pragma omp task {
printf("hello");
}
#pragma omp task {
printf("world");
}
}
}OpenMP on the Accelerator
#include <stdio.h>
#define NX 102400
int main(void)
{
double vecA[NX], vecB[NX], vecC[NX];
int i;
/* Initialization of the vectors */
for (i = 0; i < NX; i++) {
vecA[i] = 1.0;
vecB[i] = 2.0;
}
#pragma omp target
for (i = 0; i < NX; i++) {
vecC[i] = vecA[i] + vecB[i];
}
return 0;
}OpenMP and the Compiler
OpenMP is relient on the compiler to add hooks to the openmp runtime system.