Jupyter/ROOT Containers

Users of the Nevis particle-physics Linux cluster have access to a notebook server that lets them execute Python and ROOT C++ scripts on a web browser. This page describes alternative methods for installing and using the Jupyter/ROOT/Python/C++ combination with (hopefully) a minimum of installation overhead. For example, this may be helpful for:

• Nevis researchers who want to be able to run ROOT on their laptops when they don't have internet access (e.g., on an airplane trip);

• Folks who'd like to go through the ROOT tutorial but who don't have computer accounts with a Nevis particle-physics group.

Docker

Docker is probably the method of running Jupyter+pyroot with a minimum of fuss. Its disadvantage is that it requires administrative access to the host computer system (e.g., your laptop), both to install Docker and to run the Docker container.

The first step is to install Docker. For Mac and Windows systems, use Docker Desktop; there's a different procedure needed for Linux systems.

Once Docker is installed and running, you'll be able to download and run a Docker container:

sudo docker run -p 8080:8080 -v $PWD:/work wgseligman/jupyter-pyroot

(Windows users will probably need to use %CD% instead of $PWD.)

The first time you run this command, it will download a ~2.5GB container. Give it time.

Finally you'll see some output. Look at that output carefully, as it will tell you how to access Jupyter via a web browser. For example, assume the output contains something like this:

    To access the notebook, open this file in a browser:
        file:///root/.local/share/jupyter/runtime/nbserver-1-open.html
    Or copy and paste one of these URLs:
        http://649d0c4b4dc1:8080/?token=97d7242fc79734f1429bc425c627ccc4f586675c01ecdd9c
     or http://127.0.0.1:8080/?token=97d7242fc79734f1429bc425c627ccc4f586675c01ecdd9c

Then start up a web browser and visit http://127.0.0.1:8080/?token=97d7242fc79734f1429bc425c627ccc4f586675c01ecdd9c. You'll see the standard Jupyter home page.

Changing the port

Consider the command:

sudo docker run -p 8080:8080 -v $PWD:/work wgseligman/jupyter-pyroot

That first 8080 is the port to use on your local computer. If you want to use a different port on your computer (for example, you're already using port 8080 for something else), change that first 8080 to a different port. Note that if you change the port, you'll also have to change the port in the URL in the output; e.g.,

sudo docker run -p 7000:8080 -v $PWD:/work wgseligman/jupyter-pyroot

means you'll have to change:

http://127.0.0.1:8080/?token=97d7242fc79734f1429bc425c627ccc4f586675c01ecdd9c

to:

http://127.0.0.1:7000/?token=97d7242fc79734f1429bc425c627ccc4f586675c01ecdd9c

Changing the directory

Again, consider:

sudo docker run -p 8080:8080 -v $PWD:/work wgseligman/jupyter-pyroot

That $PWD (%CD% in WIndows) just means "the current directory." The execution environment within the container uses /work for its files; the -v option in the command means "map /work to the current directory in the terminal." If you'd like to use a different directory on your computer as the work directory for the Docker container, just substitute that directory for $PWD. For example:

sudo docker run -p 8080:8080 -v ~jsmith/root-class:/work wgseligman/jupyter-pyroot

Changing the container

You can use New -> Terminal within Jupyter to get a shell. Within that shell, you can modify anything within the container you want; for example, you can use pip3 to install new Python packages or yum to install new Linux packages. (If you install something that might be of general interest, let WilliamSeligman know so he can add it to the main jupyter-pyroot container.)

However, any changes you make to the Docker container won't be automatically saved when you quit the container. When you next start the container, it will start "fresh". If you want to save your changes, you'll have to commit them.

For example, assume that you've made some changes to your copy of the jupyter-pyroot container. Look up the ID of the container as assigned by your local docker process:

sudo docker container ls
CONTAINER ID        IMAGE                       COMMAND                  CREATED             STATUS              PORTS                    NAMES
1105371318e8        wgseligman/jupyter-pyroot   "jupyter notebook ..."   13 minutes ago      Up 13 minutes       0.0.0.0:7000->8080/tcp   cranky_albattani

Your output will be different; you'll have different CONTAINER ID and NAMES. Commit a revised container using your own image name:

sudo docker commit 1105371318e8 $USER/jupyter-pyroot

You'll can see your new image with the docker images command. For example, if $USER is "jsmith":

sudo docker images
REPOSITORY                            TAG                 IMAGE ID            CREATED             SIZE
jsmith/jupyter-pyroot               latest              97ca601cbf9c        7 seconds ago       2.66 GB
docker.io/wgseligman/jupyter-pyroot   latest              16c3bbdc8144        17 hours ago        2.66 GB

From that point forward, you'll probably want to run your new container with your changes:

sudo docker run -p 8080:8080 -v $PWD:/work jsmith/jupyter-pyroot 

Docker container notes

WilliamSeligman prepared the container wgseligman/jupyter-pyroot to be similar to the environment of the notebook server; for example, it runs the same version of the OS and of ROOT (as of Feb-2020, that's CentOS 7 and ROOT 6.18.04).

A little bit web searching will show there are other ROOT containers available. For example:

sudo docker run -p 3000:8080 pedwink/pyroot-notebook

That particular container uses Fedora 28 and ROOT 6.14, and it also offers Python 2 versions of its notebook kernels (wgseligman/jupyter-pyroot only offers Python 3).

So if you can't find the feature you want in wgseligman/jupyter-pyroot, hunt around a bit. It's probably out there.

Singularity

If you don't have admin access to your local computer, or you simply prefer it, you can use Singularity instead. You still need admin access to install Singularity, or a willing sysadmin to do it for you. (Singularity is installed on all the systems in the Nevis Linux cluster.)

To download the container and convert it to Singularity's .sif format:

singularity pull docker://wgseligman/jupyter-pyroot

After some processing, you'll have the image file jupyter-pyroot_latest.sif. Then you can run Singularity on that container:

singularity run --bind=$PWD:/work jupyter-pyroot_latest.sif

Note that while you can change the mapping of the /work directory within the container (see above), you can't change Jupyter's binding to port 8080. This might be a problem if you're running on a shared computer system and more than one user wants to run this container at the same time.

Anaconda

Docker and Singularity are OS-level containers (in contrast to emulators like VMware, which are machine-level containers). Anaconda is an environment-level container; it doesn't change the operating system, but it allows you download and execute packages in your home directory.

If you use the Nevis Linux cluster, then you should consider using environment modules over Anaconda. But if you're on a different system, or the Nevis environment modules don't offer the package or version you're looking for, Anaconda is a better choice. You can install it in your home directory so admin access is not necessary.

To install Jupyter/ROOT, this command is supposed to work:

conda create --name jupyter-pyroot jupyter python root

However, I've found this method to be unreliable. Typically there's no problem with jupyter or python, but installing ROOT via Anaconda is hit-or-miss.

Assuming you succeed, you can run the jupyter command as described near the bottom of the notebook server page; e.g.,:

jupyter notebook

jupyter notebook --no-browser --port=XXXX

The hard way

If all other methods fail, you can embark on the adventure of compiling these packages on your own. You can install Python, ROOT, and Jupyter on your laptop. In fact, Jupyter is meant to be a laptop tool; the container installations described above are to save you time. If you want to try your own installation:

• These are not applications that you can just double-click to install. The process requires some knowledge of the command shell.
• You'll need to read the documentation for the package installations and use some thought and initiative. The links in the previous paragraph point to the installation documentation.
• The Dockerfile used to create wgseligman/jupyter-pyroot may provide a clue to what you can do to create your own installation.