How To Upgrade XNAT

This section describes how to upgrade your installation to the latest version of XNAT:


Backing Up XNAT

A thorough back up of your XNAT installation is a valuable safeguard not only when upgrading XNAT, but also in the event of a system crash, power outage, or other unforeseen event that could cause the loss of valuable data. To back up your XNAT:

  1. If you are backing up a version of XNAT 1.6.x, you should preserve your xnat_builder folder. This is no longer used by XNAT 1.7, but may be necessary if you ever need to restore your previous 1.6.x installation.
  2. Back up the deployed XNAT web application. If you deployed via war file, you can just copy the war file to your back-up location. If you deployed to a folder (as with the -Ddeploy option for XNAT builder's setup or update script), you should back up the folder to your back-up location, although you can create a zip or other archive file instead of copying the folder itself.
  3. Back up your database using pg_dumppgAdmin, or other database tools: pg_dump -d xnat > xnat-backup.sql
  4. Optionally, if you have the space or technical ability, backing up or snapshotting your data archive folder provides an extra level of security for your data assets.

Upgrading XNAT 1.7

Upgrading from an earlier release of XNAT 1.7 to the latest release is a matter of replacing the XNAT war file:

  1. If your Tomcat server is running, shut it down.
  2. Back up your current XNAT installation.
  3. Replace the existing XNAT war file with the upgraded version.
  4. If there is a folder in the webapps folder containing the "exploded" web application, delete that folder.
  5. Start Tomcat.




Upgrading XNAT 1.6.x to 1.7

There are two parts to upgrading an XNAT 1.6.x installation to XNAT 1.7:

  • Migrating XNAT customizations–data types, display templates, code, and so on–to build as XNAT plugins
  • Upgrading an existing deployment from 1.6 to 1.7

You can also migrate your existing customizations but create a new unpopulated XNAT 1.7 deployment rather than upgrading an existing deployment. In this case, refer to XNAT Installation Guide, since you're basically just installing a new XNAT with migrated customizations.

Migrating XNAT customizations

Earlier versions of XNAT provided the ability to extend XNAT's core functionality a couple of ways:

  • Project overlays, where custom files were placed directly in the projects/project folder structure and processed when setup or update scripts were run
  • XNAT modules, which allowed custom files to be packaged in a jar or zip archive

These customizations are no longer supported in XNAT 1.7. To use these customizations with XNAT 1.7, you need to convert the overlay or module structure to build as an XNAT plugin. The XNAT plugin structure and architecture is described in the plugin development documentation.

The XNAT team has developed a tool to assist with the migration of pre-1.7 modules to XNAT 1.7 plugin build structures. If you are trying to convert XNAT modules to plugins, you can go there to download the tool and find information on how to use it.

In most cases, all your customizations to XNAT should be able to be organized into plugins.  XNAT is configured to look first in plugins for content, then to look in the web application.  This should work well for all content that can reside under META-INF/resources, as well as any content you have added to XNAT. 

One other thing to keep in mind is that if the files you customized have changed in XNAT 1.7, XNAT may not work correctly without these changes. If your customization was simple, you may want to start with the version of the files from XNAT 1.7, make your customization to it, and then put it in a plugin. If you don't remember what you changed, you can look up the source code for the version of XNAT you based your customization on in either the 1.6 or 1.7 repo by selecting the right version tag as shown below.


If there are any XNAT files that you customized very extensively, you may want to compare 1.7's version of that file to the 1.6 version (or whatever version you started with when making the customization). You can then make the corresponding changes to your customized version of the file. For example, maybe a method name changed in XNAT and that method was used in the file you customized. You may have very heavily modified the file, but should be able to get your customized version to work with XNAT 1.7 if you simply update the method name when you use it in your customized code.

After you have installed XNAT and put your customizations in plugins, you can simply put these plugins in your plugins directory and restart Tomcat. Your XNAT should then be using all of your customizations.

Plugin upgrade example

As an example, let's say you wanted to upgrade the UserAttributes plugin, the current version of which (as of March 12, 2019) has not been updated in two years. A good place to start would be to identify all the files in the plugin that are modified versions of files in xnat-web (one way to do this is to clone xnat-web and search it for each file name in your plugin). For example, the UserAttributes plugin has custom versions of XDATScreen_report_xnat_projectData.vm, XDATScreen_report_xnat_projectData.java, and minProjectsList.vm. And comparing the version of minProjectsList.vm in the plugin to the one in xnat-web shows that the difference is that it references shortProjectsList.js rather than minProjectsList.js. The shortProjectList.js file in the plugin seems to mainly be different from the minProjectsList.js file in default XNAT because it sets projList.options = undefined; on line 20. But it also does not have the latest changes to the minProjectsList.js file in XNAT. So you would want to not only look for files whose names exactly match, but dig into the code of the files whose names exactly match to see if they reference custom versions of default XNAT files which have been given different names (as is the case for shortProjectsList.js).

Once you have the set of files in the plugin that you want to try to update, you should try to determine either what the plugin version changed, or what changed in XNAT since the plugin version was created. For example, if you can tell that the version of the file in the plugin had only changed a single text string that was displayed to the user but the XNAT version of that file had recently been heavily modified, it might be easiest to take the recent XNAT version of that file, change that one text string, and have that file replace the version of that file in the plugin. However, if it looks like the file had been heavily customized in the plugin, and there has only been one small change to the version of that file in XNAT, it might make more sense to start with the version of the file in the plugin and then make the same change to it that was made on the XNAT side.

So for the the UserAttributes plugin, you would probably want to update shortProjectList.js, XDATScreen_report_xnat_projectData.vm, and XDATScreen_report_xnat_projectData.java. Once you have modified these files, you can make sure the plugin still builds and then make sure the website comes up and seems to work as expected. Often when updating plugins, there will be additional work you need to do at this point because some of the other files in the plugin (files that were not copies of files in XNAT) use methods/functions in XNAT code that have changed since the plugin was written (and so the way they are invoked needs to change). Some of these changes are listed here.

Plugins can be very powerful, however, that power can come with a maintainability trade-off. It is always easier to simply override a file rather than figure out how to use the extension points that exist in XNAT for adding things without overriding files (and unfortunately the documentation of extension points is limited and extension points do not exist for everything people want to do). But the more XNAT files a plugin modifies, the less likely it is to work for any version of XNAT other than the one it was written for. The only way we could ensure compatibility with all plugins that override XNAT files would be to never modify XNAT files. But then we would never be able to release new features or improve existing ones.

Upgrading existing deployment

Earlier versions of XNAT used the "builder" folder, which contained the XNAT source code, libraries, display templates, and so forth. Setup and update scripts generated the web application to be deployed into Tomcat, but also generated scripts to create or update the database based on changes or additions to the data-type schemas. Upgrading an existing deployment for these earlier versions required you to process these update scripts manually in order to update the database properly to manage changes to core XNAT and/or custom data types.

This procedure assumes that you've already migrated any XNAT customizations to XNAT plugin jars and installed those jars into the XNAT plugin folder. See the previous section for more information.

XNAT 1.7 integrates this functionality directly into the application that is deployed in Tomcat. The data-type schemas are now built directly into the plugin archive and accessible at run time. XNAT takes advantage of this to process all of these schemas on application start up, generating and running the database update scripts on the fly. This simplifies the upgrade process significantly.

If you've modified the services.properties file–for example, to add an LDAP authentication repository–you will need to move these changes to other properties files. XNAT 1.7 reduces the number of preferences specified in properties files or XML configurations. Many of the properties that were specified via properties file in XNAT 1.6 are now customizable through the Administer->Site Administration menu command within XNAT. Most of these preference changes take effect instantly without having to restart Tomcat. But if you need XNAT to be initialized with some properties rather than setting them as soon as you start XNAT for the first time, you can still specify them in properties files. See the documentation for custom configuration settings for more information.

To upgrade an existing XNAT deployment from XNAT 1.6.x to 1.7:

  1. If your Tomcat server is running, shut it down.
  2. Back up your current XNAT installation.
  3. Delete the existing web application from Tomcat:
    • If XNAT was deployed into a folder under the Tomcat webapps folder, delete that entire folder
    • If you deployed with a war file, delete the war file, but also check whether the application was "exploded"–that is, extracted into the webapps folder–in which case you should also delete the exploded folder

  4. Make sure that Tomcat is configured as described in XNAT Installation Guide. Most importantly, you must set a value for the xnat.home property. That must indicate a folder accessible by the Tomcat user. That folder should have (at least) four subfolders: configlogsplugins, and work.
  5. In the ${xnat.home}/config folder, create the file xnat-conf.properties (a default version of this can be found in the XNAT Installation Guide documentation). Set the values in this file appropriately for accessing your database.
  6. If you have any plugins, copy them into the ${xnat.home}/plugins folder.
  7. Start Tomcat.

You can watch the server logs, especially the Tomcat catalina.out and XNAT sql.log logs, to monitor the progress of the database update and migration. If anything goes wrong during the upgrade process, you can dump the modified database and new web application, then restore your database and earlier version of XNAT. Please preserve all log and configuration files and contact us with as much information as possible on what happened.

If you've installed the XNAT 1.7 pipeline engine in a different location from the previous version, you must change the location configured in XNAT for the pipeline engine before running any pipelines (this includes before archiving new sessions, since XNAT automatically invokes the auto-run pipeline once a session is archived):

  1. Log into your upgraded XNAT with an administrator account.
  2. Click the Administer->Site Administration menu command.
  3. Select the File System tab.
  4. Enter the new location for your pipeline engine in the Pipeline Path text box.
  5. Click the Save button.

If you don't make this change, you may see all kinds of weird errors, including failing to generate snapshots of your newly archived sessions, verbose messages in error logs, etc.



Troubleshooting

Plugins Not Working

If the code you have in your plugins does not seem to be taking effect, there could be several possible explanations. Here are some of the things you should check:

  • Make sure there is a directory named 'plugins' directly under the xnat.home directory and that xnat.home is defined for Tomcat as explained in the installation instructions.
  • Make sure that the owner of your plugins directory and the owner of each plugin file matches the owner of your XNAT webapp directory and that you have sufficient file permissions
  • Make sure your plugins are correctly structured. It is easy to accidentally create a jar of the directory that contains your plugin's code instead of creating a jar of the contents of that directory. If you open the jar for your plugin, there should be a META-INF directory at the top level of it.
  • Make sure the location of any content that is to be loaded at initialization by Spring (e.g. @Service classes) are indicated in the @ComponentScan annotation definition of your @XnatPlugin configuration class. 
  • Similarly, make sure that the location (package) of any Hibernate entities you have added are included in the "entityPackages" attribute of your @XnatPlugin annotation. 
  • Look at your server.xml and see if you have a Context path section. This may be preventing Tomcat from picking up your plugins. If you have a section like this, remove it:

    <Context path="" docBase="/var/lib/tomcat7/webapps/xnat" >
    	<Resource name="UserTransaction" auth="Container" type="javax.transaction.UserTransaction"                        		
    		factory="org.objectweb.jotm.UserTransactionFactory" jotm.timeout="60"/>
    	<Manager pathname=""/>
    </Context>
  • You can also try checking your logs. It is possible that it is correctly picking up on your plugin code, but there is some coding error that is preventing it from working properly. See if there is an error message in one of the log files that is related to your plugin code.

Database Error

In XNAT, most database updates happen automatically and run upon Tomcat startup.  New tables will be generated based on schema files that you add in your plugins and new columns will be generated for fields you add to your schema.  Additionally, tables will be generated for hibernate entities you define in your plugins and will be updated as you add additional elements to your class.  This process works well for customizations that add database tables and columns, but can fail when the data types of existing fields are changed.

While a number of tables and columns have been added to the database for XNAT 1.7, there have been relatively few changes to existing database columns.  Where there have been changes to existing columns or existing database contents, a number of initializers were written to perform the database updates necessary to move from 1.6.5 to 1.7.  

If you are experiencing a database error after upgrading to XNAT 1.7 and have added one or more plugins to XNAT or you are moving from an older version of XNAT, you could see database errors if the type of a field has changed, either from the older XNAT version, or if you have modified the field type in your plugin during the transition from 1.6 to 1.7.

If the problem is in a database view, you may be able to eliminate the problem simply by deleting the problem views and restarting Tomcat.  Any missing views will then be regenerated. If the problem is in a table, you may need to create and run a SQL query to update your existing table and copy data from the old version of the column to the new version.

$label.name