Migrating XNAT ML data from Beta to Production

The XNAT configurations, datasets, and machine-learning plugins underwent a number of changes as part of the process of preparing for release:

• Dataset collection resources were changed from actual resources on the collection to simple references by ID to the resources. This addressed an issue where deleting a dataset collection resulted in the deletion of the associated resources.
• The data types in the ML plugin were renamed from names based on working code names to the name of the finalized project.
• Unused automation preferences were removed from the database. 

If you are installing and running XNAT 1.8 or the 1.0.0 release version of the XNAT ML plugin for the first time or on a system that didn't have the ML-BETA version of XNAT or the XNAT ML plugin installed, you don't need to do anything. This migration process only applies to users who want to preserve data created using the ML-BETA version.

The pre-release version of the XNAT ML plugin was called the XNAT Clara Plugin. If you have tables in your database with names that start with 'clara_' (e.g. clara_trainsession or clara_model), this migration process converts those to the release naming scheme.

This migration process has a couple of prerequisites:

• Java 8
• Groovy 2.5.0 or later
• Access to the database you want to migrate

It's strongly recommended that you first try this migration process on a copy of your database, or at least have the ability to restore your database to its original state. Once you've completed the migration process, you should only re-start your XNAT system with the release versions of the datasets and ML plugins!

Converting dataset collection resources to references

We provide an SQL script to convert dataset collection resources to references. To perform this conversion:

1. Download the migrate-collection-resources-to-references.sql script.
2. Run this script using psql.

The exact format of the psql command depends on your database configuration. Suppose you have a database named kraken running on a server located at cthulhu-db, and the username and password to access the database are armitage/arcane. You would run the migrate-collection-resources-to-references.sql like this:

$ psql --host=cthulhu-db --dbname=kraken --username=armitage --file=migrate-collection-resources-to-references.sql

You should be prompted to enter the database password for the specified user. Once you've entered the password, psql should execute the script and migrate the collections resources to references. The SQL migration script is wrapped in a database transaction, so that, if an error occurs at any point during script execution, any changes already made to the database are rolled back.

Converting Clara datatypes to ML datatypes

To migrate the ML-BETA Clara datatypes to the release version of ML datatypes:

1. Download the MigrateClaraToMl.groovy script.
2. Run MigrateClaraToMl.groovy. You can get help for the available options with the -H or --help options.
3. The script generates an SQL file with the default name migrate-clara-to-ml.sql (you can specify some other name with the -o/--output option). Run this script using psql.

As with migrating dataset collection resources, the exact format of the commands for MigrateClaraToMl.groovy and psql depends on your database configuration. The options available allow you to specify the specific parameters for your database. With the same database example–a database named kraken running on the cthulhu-db server, username and password of armitage/arcane–you would run the MigrateClaraToMl.groovy like this:

$ groovy MigrateClaraToMl.groovy --server cthulhu-db --database kraken --username armitage --password

You'll be prompted to enter the database password for the specified user. Once the script completes successfully, you can use the generated SQL script to migrate your database:

$ psql --host=cthulhu-db --dbname=kraken --username=armitage --file=migrate-clara-to-ml.sql

Again, you'll be prompted to enter the database password for the specified user.

As with the dataset collection migration script, the Clara-to-ML SQL migration script is wrapped in a database transaction, so that, if an error occurs at any point during script execution, any changes already made to the database are rolled back.

Remove Unused Preference Entries

The XNAT ML-BETA release introduced some preferences that were simplified and subsequently removed in the 1.8 release. These preferences need to be removed from the database before XNAT will start properly using an ML-BETA-generated database.

We provide an SQL script to remove these entries. To perform this step:

1. Download the update-event-service-prefs.sql script.
2. Run this script using psql.

The exact format of the psql command depends on your database configuration. Suppose you have a database named kraken running on a server located at cthulhu-db, and the username and password to access the database are armitage/arcane. You would run the update-event-service-prefs.sql like this:

$ psql --host=cthulhu-db --dbname=kraken --username=armitage --file=update-event-service-prefs.sql