[JupyterHub] Add Your Customized Kernel for Jupyterhub

By default, the kernel that WARA-SW JupyterHub cluster uses is initialized from jupyter/base-notebook. Scientific computing packages and machine learning packages are not preinstalled in this Docker image.

If you would like to customize this running environment, you may need to install some packages by yourself and add the customized environment as a new kernel in your JupyterHub.

Because only the user's home folder is persisted, you cannot directly use conda to install running environments (its default env folder is not located in a user's home folder). In order to customize the running environment, please refer to the following steps.

0. Open a terminal to configure your conda environment Log in to your JupyterHub dashboard, click the New button on the top-right corner of the page and choose Terminal. A new tab should be opened and you should be able to interact with your JupyterHub environment now.

1. Create a .condarc in your home folder using the following commands

echo -e "envs_dirs:\n  - /home/jovyan/.my-conda-envs/" > .condarc
mkdir .my-conda-envs

The commands above configure conda to save your customized running environments in your home folder so that these new environments can be persisted after you restart your server. Please keep the foler's name as .my-conda-envs.

2. Create a new conda environment

# Feel free to change the name testenv
conda create --name testenv

conda init
# Run the following command so that you don't need to reopen a terminal
source .bashrc
conda activate testenv

3. Now you can install the packages you need

# I would recommend installing ipykernel first to avoid some dependency issues
conda install -c anaconda ipykernel

# The following line is an example which installs pytorch and cuda
conda install pytorch torchvision torchaudio cudatoolkit=11.1 -c pytorch -c nvidia

# Install your new running environment so that it can be found on the webpage
python -m ipykernel install --user --name=testenv

Notes

• The self-defined kernels also takes your storage. The maximum amount of storage per user is 10Gi now (was 5Gi before).
• Our current backup strategy does not include the .cache folder and the .my-conda-envs folder. If your running environment is complicated to be set up, it is good to have some documents about the commands as a backup.