CSC 5001– High Performance Systems

Introduction to MPI

Hello world with MPI

The aim of this exercise is to discover the compilation and deployment chain of MPI.

Basic parallel hello

Implement a MPI program hello.c where each MPI process you launch prints a message like:

Hello from task <rank> / <nb tasks> on <machine>.

For this purpose:

Initialize MPI,
Retrieve the rank of the current MPI process,
Retrieve the number of running MPI processes,
Retrieve the hostname on which the current MPI process is running on (cf man gethostname),
Close MPI.

On labs machine, OpenMPI is installed by default but in the labs, we use MPICH implementation you can load thanks to the following script (you have to execute it each time you open a new terminal):

Compile your code thanks to MPI compiler mpicc and execute your program on one and several machines. The hosts file indicates the machines on which computation will be deployed by mpirun, transparently through ssh (hence the importance of configuring your SSH keys).

$ mpicc hello.c -o hello
$ srun --account=csc_5001 --partition=starfighter --qos=normal --time=00:10:00 --cpus-per-task=1 -n 4 ./hello

Hello World from task 1 / 4 on starfighter-slurm-node-11-1
Hello World from task 2 / 4 on starfighter-slurm-node-11-1
Hello World from task 3 / 4 on starfighter-slurm-node-11-1
Hello World from task 0 / 4 on starfighter-slurm-node-11-1
	    

You may get a bunch of warning messages such as:

[starfighter-slurm-node-02-1][[28545,0],13][../../../../../../opal/mca/btl/tcp/btl_tcp_proc.c:266:mca_btl_tcp_proc_create_interface_graph] Unable to find reachable pairing between local and remote interfaces

These warning messages are not really important. You can ignore them by running export OMPI_MCA_btl_tcp_disable_family=6. You should add this line to your ~/.bashrc

You should observe that the 4 MPI ranks run on the same machine. By default, Slurm will try to allocate all the MPI ranks on one compute node. You can ask for several nodes using the -N option:

$ srun --account=csc_5001 --partition=starfighter --qos=normal --time=00:10:00 --cpus-per-task=1 -n 4 -N 4 ./hello
Hello World from task 0 / 4 on starfighter-slurm-node-01-1
Hello World from task 1 / 4 on starfighter-slurm-node-05-1
Hello World from task 2 / 4 on starfighter-slurm-node-06-1
Hello World from task 3 / 4 on starfighter-slurm-node-10-1
		    

Ordered parallel hello

Modify your program in order to have ordered printings (i.e. rank 0 first, followed by rank 1, rank 2, etc.). For this purpose, each process N waits that the process of rank N-1 printed its message by waiting a message from it, print its own message and send a message to the process N+1 in order to notify it is its turn. Process 0 initiates the chain by not doing any reception but printing its message directly and sending the token to rank 1.

Mandelbrot with MPI

EasyPAP makes it easy to use MPI by relieving you of the need to call MPI_Init() and MPI_Finalize(), Makefile, and mpirun. To pass parameters to the srun command, use the --slurm --mpirun EasyPAP options followed by a string of characters. You can use the -d M debugging option to display a window for each process:

./run --kernel mandel -v variante_mpi -d M --slurm --mpirun "-n 4 --account=csc_5001 --partition=starfighter --qos=normal --time=00:10:00"	

Baseline MPI version

In this first version, all processes participate in the computation. At the end, the master process receives the data calculated by the other processes to perform the display. The calculation is performed in the procedure mandel_compute_mpi(). Communications are executed by the function mandel_refresh_img_mpi() which is automatically called by EasyPAP before each display.

Integrate the file mandel-mpi-skeleton.c into the file mandel.c and complete the latter. The program can be initially tested without completing the function mandel_refresh_img_mpi(). The processes then calculate their image area independently, without synchronization. It can be visually verified that each process correctly calculates its pixel area thanks to the option -d M.

static int rank, size;

void mandel_init_mpi ()
{
 easypap_check_mpi (); // check if MPI was correctly configured

 // TODO obtenir rank et size ;

 mandel_init ();
}


static int rankTop (int rank)
{
 return 0; // indice de la 1ere ligne du processus rank
}


static int rankSize (int rank)
{
 return 0; // nombre de lignes à traiter par le processus rank
}


void mandel_refresh_img_mpi ()
{
 // le maitre réceptionne les données
 // les autres processus envoient les données
 // &cur_img(ligne,0) correspond à l'adresse d'une ligne du tableau.
}


//////////// MPI basic variant
// Suggested cmdline:
// ./run -k mandel -v mpi -mpi "-np 4" -d M

unsigned mandel_compute_mpi (unsigned nb_iter)
{
 for (unsigned it = 1; it <= nb_iter; it++) {
  do_file (0, rankTop (rank), DIM, rankSize (rank));
  zoom ();
 }
 return 0;
}
#endif

Point-to-point communication

To implement mandel_refresh_img_mpi(), use point-to-point communication. The master receives data contribution from all other ranks while all the others send their data to the master.

Communications collectives

Adapt the function mandel_refresh_img_mpi() by replacing point-to-point communications with collective communications (here gather). Compare execution times obtained with those obtained by the point-to-point version. Refine the measurements by timing the transmission times (distribution and collection phases) with gettimeofday function.

Hybrid MPI + OpenMP version

Using a tiled version parallelized with OpenMP, write a function mandel_compute_mpi_omp(). You will also need to write the functions mandel_init_mpi_omp() and mandel_refresh_img_mpi_omp().