Skip to content

VOBox Container

This guide describes how to create a networked Docker container for VO-Box use.

A guide for Apptainer and Podman may now also be found here. This version can run directly from CVMFS, without the need for any additional packages on the host.

For the previous guide refer to the this page.

Requirements
CentOS 7.0 or later
Docker 1.12 or later (Tested on 17.03.1-CE a.k.a. "1.14")
CVMFS Installed on the host

Setup Networking

Create a new MACVLAN bridge with both IPv4 and IPv6 support, named docknet, using the following command:

# docker network create -d macvlan \
    --subnet=137.138.47.192/26 \
    --gateway=137.138.47.193 \
    --ipv6 \
    --subnet=2001:1458:201:b50e::/64 \
    --gateway=2001:1458:201:b50e::1 \
    -o parent=eth0 docknet
Command details
  • subnet/gateway must be replaced with the settings applicable to your network, while parent must be the network interface.
  • Why MACVLAN? Docker's normal approach to bridging is simply a NAT translation scheme. MACVLAN bridges give the containers direct access to the network.
  • Containers on this bridge will be assigned their own MAC addresses, and appear as conventional computers on the local network.
  • However, due to the way MACVLAN works, the host will not be able to ping the containers (and vice versa).

Create Container

  1. Clone the repository containing the desired preconfigured setup:

    Scheduler Command
    HTCondor # git clone https://gitlab.cern.ch/mstoretv/dockervobox.git
    ARC/Generic # git clone https://gitlab.cern.ch/mstoretv/dockervobox_arc.git

  2. Build container image using the following command:

    # cd dockervobox
# docker build -t voboximg .
    Or pull a pre-built container from our registry: 
    # docker pull gitlab-registry.cern.ch/jalien/dockervobox/voboximg:latest
# docker tag gitlab-registry.cern.ch/jalien/dockervobox/voboximg:latest voboximg:latest

    Warning

    Be sure to change the default root password! This is simply root in the above image.

  3. Create and launch a new CentOS container connected to the MACVLAN bridge:

    # docker run -d -v /cvmfs:/cvmfs \
    -h myalienvo.cern.ch \
    --name=myvocontainer \
    --net=docknet \
    --ip=137.138.47.251 \
    --ip6=2001:1458:201:b50e::100:3e \
    voboximg
    Command details

    -v mounts the directory /cvmfs from the host as /cvmfs within the container
    -h sets the hostname myalienvo.cern.ch
    --name sets the container name (as seen from Docker) to myvocontainer
    --net attaches the container to the previously created MACVLAN bridge docknet
    --ip/ip6 sets the desired network IPs for the container

    Info

    Docker gives containers limited access to the host kernel, which will prevent some tools from working properly. A possible workaround is to give the container additional privileges (see advanced topics for details).

    As opposed to mounting CVMFS from the host, it is also possible to have it installed normally within the container. However, this will only work on a container with full access privileges. Be aware that unless otherwise configured, this will give each container its own CVMFS cache.

Service Management

Almost all services are started automatically alongside the container. If needed, you can manage most of them manually by standard Linux systemctl commands. The exception is the proxy renewal daemon, which is managed as follows:

# /services/alice-box-proxyrenewal {start|stop|restart|status|proxy}

Example Deployment

  1. Install a host certificate.
  2. Start the services using the following commands: 
    # /services/alice-box-proxyrenewal start < /dev/null
# /usr/sbin/edg-mkgridmap --output=/etc/grid-security/grid-mapfile --safe
  3. Adjust ~/.alien/Environment as needed (not for JAliEn).

Useful Commands

The following table lists commands that may be of use when working with the VoBox containers.
Refer to command line Docker documentation for more details.

Command Description
# docker start myvocontainer Start the container
Note: This way, the container will remain active until you type
# docker stop myvocontainer
# docker exec -it myvocontainer /bin/bash Start a bash session within the container
$ ssh root@your.ip.vobox.here Login into the container
# docker ps List active containers
# docker ps -a List all containers
# docker images List all images cached locally
# docker commit myvocontainer myvoimagename Save container state as an image.
With myimagename being the desired image name
# docker export mycontainer > /home/myexportcontainer.tar Save the container as a tar file.
With /home/myexportedcontainer.tar being the directory and name for the exported container
Docker commit vs export

The commit command will save the container as an image in Docker's internal format. It will also preserve the container's history, layers and metadata, and also create a new layer on top. Beware that an image will rapidly grow in size if it is continuously being committed, as a result of all the stacked layers.

The export command will save the container to a tar file, that can be reimported and used to launch new containers on the same machine, or uploaded and/or transferred elsewhere. However, it does not store any history or metadata, and will delete all previous layers except the current. Exporting and reimporting an image can thus be used to flatten a container that has gained a large file size, e.g. as a result of having previously used commit several times.

Advanced Topics

Privileged Mode

For the sake of isolation, access to the kernel is restricted within the containers. To enable some access for networking tools, such as iptables, add --cap-add=NET_ADMIN when creating the container. More access can be given through --cap-add=SYS_ADMIN, and full access can be given with --privileged.

Warning

Be aware that this will create pathways for potentially breaking container isolation.

Open Files Limit

If you plan on running more than one VOBOX container on a single host, the system limit for the maximum number of open file descriptors will need to be increased. The default ulimit is otherwise bound to be reached at some point, causing the containers and their processes to become irresponsive and terminate.

Access from Host

As mentioned earlier, the host will be unable to reach/ping the containers, and vice versa. A workaround is to create an additional ("normal") Docker bridge, and have the host / containers connect to it.

AutoFS Bug

CVMFS may respond with a "too many symbolic links" error when accessed from a new container or after a reboot. This is known as an autofs bug, and can be avoided by disabling autofs on the host and mounting your CVMFS directory manually. If you prefer to leave autofs enabled, you can remove the error by accessing a directory within CVMFS on the host (e.g /cvmfs/alice.cern.ch), and then restarting the container.

Warning

Be aware that this error will likely return on the next reboot if autofs is left enabled.