This documentation is for XNAT versions 1.6.0 - 1.6.5. You can find the latest documentation for XNAT 1.7 at https://xnat.org/documentation
Please make sure that the following steps have been carried out before you setup XNAT to invoke pipelines:
A project in a XNAT site can invoke pipelines that are available with the site. The site administrator releases the pipelines and the project owners can choose pipelines which are relavant for their project from the site wide collection of pipelines. Pipelines are typically created for specific data-types in the project. For example, a site may have a freesurfer pipeline which can be invoked on MRSession data or a site can have a PET processing pipeline which can be invoked on a PETSession data. A pipeline can specify the data-type for which the pipeline is relevant. This is done using the appliesTo attribute of the xnatInfo element. (If this element is absent, it is assumed that the pipeline is applicable to All Datatypes)
Most pipelines need input parameters in order to carry out tasks. These input parameters can be described as xpath schema links or discrete values. The pipeline descriptor document has an optional element called documentation where one can define the input parameters for the pipeline. An input parameter has a name and its value can be set either as a schema link or as a constant value.
Once the site administrator has made a pipeline available, the project owner can enable that pipeline for their project. Setting up pipeline for a project involves defining the project specific pipeline input parameters. Any member of the project can invoke the pipeline on a specific datatype.
In this demonstration, you will go through the steps required to make a pipeline available to the site, and set up a pipeline for a project. The pipeline engine comes bundled with a sample pipeline which generates the snapshots of the scans of an imaging session. This pipeline is located in <PIPELINE_HOME>/catalog/images/WebBasedQCImageCreator.xml. For this pipeline, the input-parameters are called sessionLabel, session and xnat_project and all of these are defined as schemalink.
Log on to your XNAT site as an administrator and select ADMINISTER > PIPELINES menu command.
Then, select "ADD PIPELINE TO REPOSITORY".
In the resulting page, insert <PIPELINE_HOME>/catalog/images/WebBasedQCImageCreator.xml in the "Enter Path to Pipeline descriptor xml" box and leave the other box empty and click the ADD button. The pipeline will then be available in your XNAT application.
Make sure you use forward slashes '/' in the path, as using backslashes '\' will lead to errors.
Create a test project using the NEW > PROJECT menu command, and giving it a basic configuration. (Any site user can do this.)
As the project owner, you can now add the pipeline we set up in Step 1. This is done from the project report page, under the PIPELINES tab. Setting up a pipeline for a project involves setting project specific parameters. This is done by clicking on ADD. (Clicking on DETAILS presents the user with documentation which lists details about configuring and using the pipeline.)
If you do not see the pipeline active for your new project, in the PIPELINES tab on the project report, click on ADD MORE PIPELINES.
Select the DicomToNifti pipeline and click ADD. If applicable, you can configure pipelines in this next step. However, this pipeline has been setup to work on MRSession data. Simply scroll down and click SUBMIT.
Let's use a basic MR Session as an example: Upload a zip file containing DICOMs using ACTIONS > UPLOAD IMAGES > OPTION 1: COMPRESSED UPLOAD. After the upload is complete, you will get a message "Your file was successfully uploaded. Click here to review and archive your uploaded sessions." Follow the link to enter the metadata about the session.
Log on as a site administrator and select the ADMINISTER > DATA TYPES menu command. Click on xnat:mrSessionData. In the resulting page, select the ACTIONS > EDIT menu command. Under the Report Actions section of the page, insert the following:
This enables members with Edit access to use ACTIONS > BUILD for an xnat:mrSessionData to launch a pipeline. PipelineScreen_launch_pipeline leads to a page in which the user selects the pipeline to be launched.
Log on as a project owner or a project member. Navigate to the MR Session that you created in Step 5 from the project report page. In the ACTIONS box on the right of this page, you should see the BUILD menu command. Click on this link. Select DicomToNifti.xml and click SUBMIT.
Clicking on BUILD leads the user to a page where the pipeline to be launched is selected.
Enter project-specific parameters for the experiment. The parameter values are populated by resolving the project specific settings and the constant values and the XPATH statements, if any, in /Pipeline/documentation/input-parameters. User can override these values appropriate for the experiment.
Result of clicking on submit
After the pipeline is complete, you should have the following:
The files which enable these steps for adding pipelines to an XNAT site, and to a project, are detailed below:
Velocity File: <XNAT_HOME>/plug-resources/webapp/xnat-templates/screens/XDATScreen_add_pipeline.vm
Pipelines Tab: <XNAT_HOME>/plug-resources/webapp/xnat/scripts/project/pipelineMgmt.js
Velocity File: <XNAT_HOME>/plug-resources/webapp/xnat-templates/screens/PipelineScreen_launch_pipeline.vm
Unless a custom screen class has been associated with the pipeline, the following are used:
ADMINISTER > MORE OPTIONS > SUMMARY can be used to view site activity with regards to pipelines.
ADMINISTER > MORE OPTIONS > VIEW ALL WORKFLOWS can be used to view the entries in the workflow table.
PIPELINE_HOME/logs contains timestamped log files which can be used to troubleshoot.
If a pipeline is launched from XNAT and continues to be in Queued/Running state, the file TOMCAT_HOME/webapp/XNAT_PROJECT/logs/application.log contains the exact command statement which failed to launch. Copy and run this statement from a command prompt as a tomcat user to figure out cause of failure.
Pipeline engine ships with support for Sun Grid Engine (SGE). This is done using the DRMAA API. A site can use its own job scheduling application and integrate it with XNAT/Pipeline Engine by "overloading" the script PIPELINE_HOME/bin/schedule. Out of the box, this script just invokes the command that is passed. PIPELINE_HOME/lib/PipelineClient.jar contains the class org.nrg.pipeline.client.PipelineJobSubmitter which supports scheduling jobs on the SGE.
/Pipeline/resourceRequirements can be used to set resource requirements like Architecture, free memory while submitting the job onto the grid.