Running XNAT in a Vagrant Virtual Machine

Vagrant is not supported on Macs running M1 or later chipsets. For users of those computers, we recommend Running XNAT in a Dockerized Container with Configurable Dependencies

Introduction

In a matter of minutes, you can deploy a running copy of XNAT in a virtual machine, using Vagrant. For advanced users, you can also choose to setup one of multiple configurations for your XNAT VM. 

Prerequisites: 

• Install Vagrant (2.2 or later is recommended)

• Install Git

• Install VirtualBox (6.1 or later is recommended)

One-line XNAT VM Setup

If you have Git, Vagrant, and VirtualBox installed, the fastest way to get a fresh XNAT VM running is to enter the following command from an empty folder at a BASH terminal prompt:

BASH

git clone --branch master https://bitbucket.org/xnatdev/xnat-vagrant.git && cd xnat-vagrant && ./run xnat setup

This will clone the xnat-vagrant repo and run the script to install the latest official release. Just wait a few minutes, and when the text is done flying through your terminal, you should have a new VM running with instructions displayed on how to access the XNAT instance running in it.

Additional Prerequisites for Windows Users

• Bash terminal program - Git Bash is adequate and is included with the Git installer - this is what will be used for these instructions. You can also use  Cygwin, but it isn't as straightforward to use.

• When installing Git, make sure to select to check-out as-is and commit with Unix line endings in your install options.  

• Recommended SSH key management:  PuTTY, PuTTYgen, and Pageant

Vagrant 2.2 or later is the recommended version for building an XNAT VM on Windows. If you find any issues running the XNAT Vagrant project on Windows, please report them to bugs@xnat.org.

Download XNAT Vagrant

There are a couple different ways to get the XNAT Vagrant project.

Option 1: Install the xnat-vagrant Download Bundle

You can download the XNAT Vagrant project directly from our download repository:

Once you've downloaded the archive, unzip it using a standard zip utility. It will expand into a folder named xnat-vagrant. Proceed to Running XNAT in a Vagrant Virtual Machine#run-setup-script.

Option 2: Clone the xnat-vagrant Source Repository

Clone the xnat-vagrant repository to your local computer by logging into the Terminal, navigating to your desired folder location, then running the following command: 

clone

git clone --branch master https://bitbucket.org/xnatdev/xnat-vagrant.git

Follow the instructions at  https://bitbucket.org/xnatdev/xnat-vagrant to clone your repo. 

The 'develop' branch of xnat-vagrant is updated more often, and therefore possibly less stable, but can offer enhancements and improvements sooner. Just change the value for the --branch flag to "develop" when cloning:

clone

git clone --branch develop https://bitbucket.org/xnatdev/xnat-vagrant.git

Configure and Customize XNAT Vagrant

As of VirtualBox version 6.1.30, the available range of IP addresses available to virtual machines has been greatly restricted. The xnat-vagrant project has been updated accordingly.

This means the first two numbers must be 192.68, while the third number has to be between 56 and 63 (inclusive), and the last can be 1 to 255, except:

• With 192.68.63.x, where 255 is reserved for broadcast

• You can use 1, but Vagrant or VirtualBox will issue a warning

The default properties for a particular configuration depend on the settings for that configuration. The available configurations can be found by looking in the configs folder under your xnat-vagrant project. The XNAT Vagrant project currently provides the following configurations, with fully qualified domain names available through the xnat.org DNS:

These configurations are described in more detail in the project README.md and in the README.md for each configuration. The settings for each configuration can be found in the file config.yaml in the configuration folder. For example, the settings for the xnat configuration are in the file configs/xnat/config.yaml. 

 For many purposes, you can simply use the default values for a configuration. If you do need to override values in a configuration's default settings, create a file named local.yaml in the configuration's folder. You can then define your own values for the available properties. There's a file named sample.local.yaml you can refer to as an example. You can also reference the config.yaml file to see all of the properties that are set and available to override.

For example, a common requirement is to specify an IP address compatible with the local network configuration, as well as increase the amount of RAM available to the VM. Looking in the config.yaml for the xnat configuration, these have the following default values:

Default configuration settings

vm_ip:          192.168.56.100
ram:            2048 

Your local.yaml file might look like this:

Custom configuration settings

vm_ip:          192.168.60.184
ram:            4096 

When you run the script that actually creates your XNAT VM, the values in local.yaml override the values in config.yaml.

Create the XNAT Vagrant VM

Once you've downloaded the XNAT Vagrant project and set any custom properties required, you can create the VM one of three ways:

• The master run script

• The setup.sh script for the specific configuration you want to create

• Running the various vagrant commands manually

Master Run Script

The master run script is located in the top-level folder of the xnat-vagrant project. This script requires two arguments:

• The name of the configuration you want to use (see Running XNAT in a Vagrant Virtual Machine#configure-and-customize for information on the various configurations)

• The command you want to execute

The available commands are:

• setup

• ssh

• start

• stop

• destroy

start and stop are aliases for Vagrant's own reload and halt commands respectively and can be used to start and stop VMs that have completed setup.

From the xnat-vagrant folder:

Running master run script

./run xnat setup

This launches the setup script in the xnat configuration folder. The setup script automates downloading the XNAT war and pipeline installer archive, then executes a series of Vagrant commands to get the VM fully provisioned. You'll see a bunch of text fly up the console, and the script should ultimately exit with a success message that displays the URL where your new XNAT instance can be accessed with your web browser.

Configuration Script

Each configuration folder has its own setup.sh script. For the most part, the master run script described in the previous section just invokes the setup.sh script for the specified configuration. You can do this directly as well:

Running configuration setup.sh

$ cd configs/xnat
$ ./setup.sh

The advantage to using a configuration-specific setup.sh script rather than the master run script is that being in the configuration folder makes it easier to work with your local.yaml to test settings and configurations for your VM.

Vagrant Commands

Finally you can run Vagrant commands directly within the configuration folders, with the main difference being you'll run 'vagrant reload' instead of 'vagrant up' after the initial setup. This is a more advanced option that can give you a great deal of control over how your VM is configured and created, but requires more knowledge over the specifics of configuring and running Vagrant. To find the commands used by the XNAT Vagrant project to build VMs, you should look through the setup.sh script that most closely resembles the configuration you want to work with.

Updating the XNAT Stack Base Box

Each Vagrant VM that builds XNAT uses an "xnatstack" base box for its machine environment. This environment is downloaded once and rarely updated. But on those occasions where it does need updating, you should know how to do it.

First, check to see if your installed base box is up to date.

$ vagrant box outdated --global
Loading existing configuration from /repos/xnat-vagrant/configs/xnat-release/vars.yaml...
* 'nrgxnat/xnatstack-ubuntu2004' is outdated! Current: 1.0.0. Latest: 1.0.1

If an outdated box is found, run an update command to update it.

$ vagrant box update --box nrgxnat/xnatstack-ubuntu2004

Work with the XNAT Vagrant VM

Once you've created an XNAT Vagrant VM, you can start to use the XNAT server there. There are a couple of steps you'll need to take to make this work:

• Configure the VM's host and server name

• Configure XNAT

• Log into the VM to manage services

Configure Host and Server Name

You can access your server directly through the configured IP address. The default IP address varies by configuration but can be easily changed in local.yaml before creating the VM. If you're comfortable using the bare IP address, you don't need to worry about the host and server name.

The default address for the xnat-dev configuration is xnatdev.xnat.org. This address is resolved by XNAT's DNS service to 10.1.1.170. That means you don't need to map the address xnatdev.xnat.org in /etc/hosts in order for the IP to resolve properly.

If you'd like something more readable and memorable, you'll probably want to configure the host and server name. The server name, also known as the fully qualified domain name or FQDN, is the standard web address, something like images.xnat.org. The hostname is the first part of the FQDN, in this case images. To work with a custom host and server name, you need to make a couple of changes:

• When customizing your VM configuration, add the name and server settings to your local.yaml configuration. The default value of name is xnat-dev so if you're OK with that as your host name, you don't need to override it. Note that name is also used to name the VM in the VirtualBox inventory, so it must be unique to the VMs running on your host server. You can also specify name and host separately: by default, host is set to the same value as name, but they can be different! The value for server should be set to the value you want for the FQDN:

  Configuring host and server name

  XML

  name:           xnatdev
  server:         xnatdev.xnat.org

  
  

• Creating the VM with the host and server name won't help your host machine find the server. For this, your machine needs to have a way to associate the FQDN with the VM's IP address. The easiest way to do this is to configure the hosts file on your host machine. For most operating systems, this is located in the file /etc/hosts and requires root access to modify:

  Editing /etc/hosts

  BASH

  $ sudo vim /etc/hosts 

  Windows also has a hosts file, but editing it can be trickier.

  Once you've opened the hosts file, you just need to add the line to associate the VM's IP address with the FQDN:

  Adding VM FQDN

  CODE

  10.1.1.17 images.xnat.org 

  Save the hosts file and you're done. No services need to be restarted or reloaded for this change to take effect.

It's possible to configure a DNS server to point to the VM or even modify the VM's network configuration so that it gets its IP from a DHCP server which can then manage the host name resolution to the IP address, but that depends on your local network architecture and configuration and is outside of the scope of this documentation.

Initialize and Configure XNAT

Once you have your VM running and can reach the server through its IP address or FQDN, you can configure your XNAT system. This task is the same regardless of whether you're running a VM, installing on a server, or running on a local development instance. This is described in XNAT Setup - First Time Configuration.

Manage VM Services

If you use a particular VM for more than trivial or trial operations, you'll eventually want to be able to manage the VM itself, not just the XNAT it hosts. To access the VM through the command line, you can use the Vagrant ssh command from the configuration-specific folder. Once on the VM, you can run commands as the default user, which has access to the Tomcat server and XNAT data folders. For more secure operations, you can use the sudo command to get root access. The code below shows a sample set of operations.

$ vagrant ssh
[ ssh ]

Loading existing configuration from /Users/rherrick/Development/XNAT/Tools/Vagrant/xnat-vagrant/configs/xnat-dev/.work/vars.yaml...
Welcome to Ubuntu 20.04.2 LTS (GNU/Linux 5.4.0-73-generic x86_64)

 * Documentation:  https://help.ubuntu.com
 * Management:     https://landscape.canonical.com
 * Support:        https://ubuntu.com/advantage

  System information as of Wed 19 May 2021 04:31:09 PM CDT

  System load:  0.0               Users logged in:          1
  Usage of /:   9.6% of 61.31GB   IPv4 address for docker0: 172.17.0.1
  Memory usage: 21%               IPv4 address for eth0:    10.0.2.15
  Swap usage:   0%                IPv4 address for eth1:    10.1.1.170
  Processes:    148


This system is built by the Bento project by Chef Software
More information can be found at https://github.com/chef/bento
vagrant@xnatdev:~$  

The purpose of this is just to show how you can perform operations on your XNAT VM. The full range of operations you can perform as a Linux and XNAT system administrator is outside the scope of this documentation. For more information, you can find many tutorials and reference guides to working with Linux on the internet.

Navigating the XNAT Vagrant VM File System

Once you have SSH'ed into your Vagrant VM, you'll need to find your way around. The Vagrant install process defines XNAT_HOME at /data/xnat/home, which is the standard practice in the XNAT Installation Guide. Accordingly, there are a few critical folders to be aware of.

Each of these folders is owned by the xnat system user. When you want to install a plugin or otherwise make changes manually in this folder, it is a best practice to be working as that user.

Plugins Folder

Plugins are installed here:

/data/xnat/home/plugins

To copy a plugin jar file from your local file store to this directory, you would do the following, substituting the appropriate IP address of the Vagrant VM you're working with:

$ scp /path/to/plugin.jar xnat@10.1.1.17:/data/xnat/home/plugins

Restart Tomcat to activate the plugin, following the XNAT Plugin Installation guidelines.

Other folders within XNAT_HOME are:

/data/xnat/home/config
/data/xnat/home/logs
/data/xnat/home/themes
/data/xnat/home/work

Archive and Prearchive

XNAT's storage of data files, including the Archive and Prearchive, sit outside XNAT_HOME and can be found here:

/data/xnat/archive
/data/xnat/prearchive

Other, less commonly-used file storage folders at this level are:

/data/xnat/build
/data/xnat/cache
/data/xnat/fileStore
/data/xnat/ftp
/data/xnat/inbox
/data/xnat/src

Troubleshooting your Vagrant VM

See: Troubleshooting a Vagrant XNAT VM