Using Jupyter Notebooks on ARC

Interactive Notebook example:

 

The following example shows how to connect to a Jupyter notebook session running on ARC. The example shows the best practice method of running the Jupyter session on an interactive node, and we make the assumption that you are connected to the university network or VPN.

 

1) First we need to log in to an ARC login node, for this example we are using arc-login.arc.ox.ac.uk - and start an interactive session using the standard srun command:

srun --nodes=1 --ntasks-per-node=4 --partition=interactive --pty /bin/bash

 

Once the srun allocation is started on the interactive node (for example: arc-c304) we will get a command prompt and can load the Anaconda module and start Jupyter with the options shown below (Note: it is important to use Anaconda3 version 2021.11 or newer for notebooks to work on ARC):

module load Anaconda3/2021.11

jupyter notebook --no-browser --ip=*

 

The response from Jupyter will be as follows:

jupyter2

Jupyter will provide information on the host name and port that the notebook is running on. In the above case arc-c304 and port 8888 coloured in red and yellow in the above example to show their location - make a note of these. It also provides a URL for accessing the notebook in your local browser (on the last line - highlighted in light green in this example).

You can now minimise (but do not close) your SSH window to the interactive session (in this example arc-c304) - and a new window on your local machine - as we now need to tunnel the Jupyter port to your local desktop.

To do this we run the following command on the local machine:

 

newtunnel

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

The ssh command creates an ssh tunnel for port 8888 from your local machine to the remote machine (arc-c304) - note we have used the same colours to highlight these. Note in the above example you need to replace the username ouit0554 in the connection to arc-login to your own ARC username. For clarity the command is

ssh -L [local port]:[remote host]:[remote port] [ARC username]@arc-login.arc.ox.ac.uk

Once you have entered your password, you will be logged into arc-login and the SSH command will have "tunnelled" port 8888 on your machine to port 8888 on the interactive node. You will then be able to use the URL from the original Jupyter command in your local browser and connect to the remote Jupyter session.

Do NOT disconnect from the remote interactive session OR the arc-login session until you have finished working with your Jupyter browser.

Please note, the ports and node names in the above may (and will) change - so you should always use the values provided by the Jupyter command you have run on the interactive node.

It is possible that you may find that you need to change the [local port] value if the default port (8888) is in use on your local machine (ie. you are running a local Jupyter session already). In this case simply change the port to another value, say 8889 and in the SSH command above (for the local port only) and in your browser URL.