Running Jupyterlab

In this example we will set up a Python virtual environment, install Jupyter Lab into said virtual environment and run a notebook on a compute node in Orfeo.

Environment setup

Note

This is a one time task. There's no need to replicate these steps every time you need to run a notebook unless you have conflicting packages sets and/or need different packages than those present in your usual environment.

Request an interactive session to perform the install. For example here we request a single core on a THIN node for 20 minutes:

[username@login ~]$ srun -A $ACCOUNT -p THIN --nodes=1 --tasks=1 --mem=2gb --time=0:20:0 --pty bash

Warning

Substitute $ACCOUNT with your SLURM account.

Ensure that you are on a compute node in the requested partition:

[username@thin001 ~]$ hostname
thin001.hpc.rd.areasciencepark.it

Create a virtual environment:

[username@thin001 ~]$ python3 -m virtualenv jupyter

Activate the environment and install the jupyter framework:

[username@thin001 ~]$ source jupyter/bin/activate
(jupyter) [username@thin001 ~]$ pip install jupyterlab

After the installation finishes deactivate the virtual environment and free the node:

(jupyter) [username@thin001 ~]$ deactivate
[username@thin001 ~]$ exit

Note

For now onwards your virtual environment will work on any compute node in the same partition.

A virtual environment created on a compute node in a specific partition might not work on a compute node in a different partition.

Running the notebook

Allocate the desired resources using the queue system and open an interactive shell:

[username@login ~]$ srun -A $ACCOUNT -p THIN --nodes=1 --tasks=24 --mem=512gb --time=1:0:0 --pty bash

Activate the environment and start in headless mode the notebook:

[username@thin002 ~]$ source jupyter/bin/activate
(jupyter) [username@thin002 ~]$ hostname
thin002.hpc.rd.areasciencepark.it
(jupyter) [username@thin002 ~]$ jupyter lab --no-browser --ip "*"

Note

Take note of the URL to connect to Jupyter.

It should look like this:

http://localhost:8888/lab?token=aa84abc...

Now open a terminal on your machine and setup an SSH tunnel targeting the hostname printed above:

username@~$ ssh -L 8888:thin002.hpc.rd.areasciencepark.it:8888 -N username@195.14.102.215

Warning the port number must much the notebook port.

Now open your favourite browser and open the link suggested by the jupyter lab console:

    To access the server, open this file in a browser:
    Or copy and paste one of these URLs:
        http://localhost:8888/lab?token=aa84abc...
     or http://127.0.0.1:8888/lab?token=aa84abc...