Building Docker Images for Container Service

Quickstart

For end-to-end container development examples, see the Quickstart Guide.

Docker provides an isolated filesystem that can be configured with all dependencies, configuration, scripts, and binaries necessary to process data in a stable and reproducible operation. Container Service is an XNAT plugin for controlling Docker containers and translating between the XNAT data hierarchy and Docker application runtime. The following documentation is geared towards the development of XNAT-enabled container images.

Before getting started with XNAT Container Service, it may be helpful to review the official Docker documentation, especially the Getting started section.

Docker Docs

Containerizing your Application for XNAT

Building a custom application as a Docker image takes only a few steps, including identification of a base image, defining process I/O and runtime parameters, and integration with the XNAT data hierarchy. This guide will start by developing a container to run using Docker command-line interface (CLI).  Before getting started, check to be sure you have Docker CLI installed by running the command:

docker run hello-world

The response should be a friendly message from Docker.

Hello from Docker!
This message shows that your installation appears to be working correctly.

Build a Docker Base Image

To build a custom Docker image, create a directory to hold your Docker image definition, particularly a text file named Dockerfile. This file contains instructions for building a custom Docker image. See the Dockerfile reference docs for a complete guide. 

For example, if we want our image to have access to python, various packages, and custom files, the following Dockerfile will do that. Note that these are not the processing commands to be executed at runtime, rather, they are commands used to set up the execution environment, i.e. the Docker image.

Here, the FROM command on line 1 indicates the label of a base Docker image of a particular version of Python. You can build your custom image upon any suitable image found on Docker Hub, a private repo, or image an image tagged on your local system. The RUN command on line 2 indicates a shell command that should be run on the base image. You can specify as many RUN commands as desired or concatenate multiple commands on a single line. Finally, COPY on line 3 instructs the Docker builder to copy the ./workspace/ folder from your local filesystem to the Docker image at /workspace/, along with all of its contents. 

Define your process I/O

Under XNAT, containers executed under Docker operate non-interactively.  This means that all process inputs and outputs need to be defined before launching the container. Container Service uses Docker bind mounts to expose parts of the XNAT data archive to a container at run-time. While Container Service orchestrates input and output mounts, the containerized process still needs to be designed around a set of expected input and output directories. For example, your application might be designed to read images from the /input directory and write output to the /output directory.

Define runtime parameters

In addition to specifying input and output directories to mount, you can define various runtime parameters for your container execution. The only required parameter is the command, underlined in orange above.  This is the execution directive followed in the Docker container instance on launch.  It is also good practice to regularly specify other constraining runtime parameters, such as reserve-memory and limit-memory, which will ensure your container process does not consume excessive resources.

• • Docker run reference - Official Docker documentation for the CLI run command.

XNAT Container Integration - Mapping the XNAT Data Hierarchy

To facilitate launching Docker containers via XNAT, process I/O and runtime parameters are saved using a JSON 'Command'. This JSON string encodes the runtime information normally provided to Docker using the Docker run command-line interface.

• • JSON Command Definition – At the core of your container is a JSON command definition
  • Making your Docker Image XNAT Ready – instructions and a Hello World example for building an XNAT-enabled Dockerfile that uses your Command JSON

Additional container behavior options:

• • Setup Commands – How to create a preprocessing step before your main command can run
  • Automatically Uploading Assessors – Optional handling of container outputs as new experiment assessors

Supporting Containers at Scale

• Bulk Launching Containers – What types of batch launching can XNAT support? (See also: Batch Launch Plugin)
• Container Scheduling – Understanding multiple possible mechanisms for managing batches of containers, including Docker Swarm

Understanding the Components of the Container Service

• How the Container Service Launches Docker Containers
• Launching Containers – How your commands are exposed and launched via the UI
• Command Resolution – How XNAT translates JSON command definitions and connects to XNAT data
• Command Secrets – Giving your command the ability to pass secret or sensitive values into your containers without exposing those values to the users launching the containers