Getting started¶
This 5 minutes tutorial will show you how to execute your first parallel
program on CLEPS cluster.
The acces of the cluster is based on your LDAP ID. Therefor every person from Inria can access CLEPS using:
ssh <ldap_login>@cleps.inria.fr
Warning
If you want to access CLEPS from a different network than that of Inria Paris,
you can consider using Inria’s VPN or adding your public ssh key to the
ssh bastion host. For details on these methods please visit this
webpage.
Build tutorial codes¶
Download and extract cleps-examples.tar.gz:
wget https://paris-cluster-2019.gitlabpages.inria.fr/cleps/cleps-userguide/_static/cleps-examples.tar.gz
tar xzf cleps-examples.tar.gz
cd cleps-examples/
This archive contains a set of examples you can run on CLEPS and modify for
your own usages. Among these examples, you’ll find a minimal Hello World
program using MPI. Let’s try it:
cd mpi
Submit an interactive job to benefit resources of a compute node for the
compilation. Despite the simplicity of this example, it is a good practice
to avoid tasks that require computing resources on the login node. Then create
a separated build folder and activate the gnu8, openmpi3 and cmake
modules necessary for the compilation:
$ salloc
$ mkdir build && cd build
$ module load gnu8 openmpi3 cmake
Build the code and exit interactive job:
$ CC=gcc cmake ..
$ make
$ exit
Execute tutorial code¶
The hello_world_mpi.batch contains commands to execute and instructions for the ressource manager
Slurm:
These instructions begin by #SBATCH and indicate what resources
you need to allocate for your job. The script load the required runtime modules
and executes the srun command, which itself calls mpirun according to the
required resources.
#!/bin/bash
#SBATCH --job-name=hello_world_mpi # Set job name
#SBATCH --time=00:02:00 # How long you think your job will
#SBATCH --nodes=2 # Number of nodes to allocate
#SBATCH --ntasks-per-node=2 # Number of MPI processes per node
#SBATCH --mem=100m # Memory required per node
#SBATCH --output=%x_%j.log # Standard output and error log. %x denotes the job name, %j the jobid.
module purge
module load gnu8 openmpi3
srun ./build/hello_world_mpi
Submit the job:
$ sbatch hello_world_mpi.batch
Standard output and error of the program are written to file:
$ cat hello_world_mpi_<jobID>.log
==========================================
SLURM_JOB_ID = <jobID>
SLURM_JOB_NODELIST = node[053-054]
==========================================
Hello, world (4 procs total)
--> Process 2 of 4 is alive on node054
--> Process 0 of 4 is alive on node053
--> Process 3 of 4 is alive on node054
--> Process 1 of 4 is alive on node053
The node list and the order of the lines may vary from an execution to another.
Simplify login with SSH keys¶
To avoid typing your password each time you log into the cluster, you can add your
public SSH key to CLEPS. This process is similar across all operating systems.
On a linux machine, the default location is in the $HOME/.ssh directory:
$ cd ~/.ssh
$ ls
authorized_keys2 id_dsa known_hosts
config id_dsa.pub
You’re looking for a pair of files named something like id_dsa or id_rsa and a
matching file with a .pub extension. The .pub file is your public key, and the
other file is the corresponding private key. If you don’t have these files (or
if you don’t even have a $HOME/.ssh directory), you can create them by
calling ssh-keygen, which is provided with the SSH package on
Linux/macOS systems (shipped with Git for Windows):
$ ssh-keygen -o
Generating public/private rsa key pair.
Enter file in which to save the key (/home/schacon/.ssh/id_rsa):
Created directory '/home/schacon/.ssh'.
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/schacon/.ssh/id_rsa.
Your public key has been saved in /home/schacon/.ssh/id_rsa.pub.
The key fingerprint is:
d0:82:24:8e:d7:f1:bb:9b:33:53:96:93:49:da:9b:e3 schacon@mylaptop.local
First, it confirms where you want to save the key (.ssh/id_rsa), and then it
asks twice for a passphrase, which you can leave empty if you don’t want to
type it when you log in. However, if you want to use one, make
sure to add the -o option; it saves the private key in a format that is more
resistant to brute-force password cracking than is the default format.
Once created, you have to copy your public key into your
$HOME/.ssh/authorized_keys on the CLEPS login node. A convenient method is
to use the ssh-copy-id command (replace <mykey> and <login> by the name
of your key and your Inria login):
$ ssh-copy-id -i ~/.ssh/<mykey>.pub <login>@cleps
After entering your password (for the last time :) ), you should be able to log into CLEPS without typing it any more.