Using Direct-to-Archive Uploading

Overview

Added in XNAT 1.8.3, the new "Direct-to-Archive" (DA) importing mechanism allows users improve data intake efficiency by significantly reducing file-IO on the backend file system. The standard mechanism of uploading data involves writing the incoming data to the prearchive and then moving it to the archive afterwards. The new DA functionality bypasses the prearchive and writes the data immediately to the archive, effectively halving the IO load on the file system. DA uploads follow the same workflow for session detection as XNAT applies with a standard C-STORE: the session is considered fully "received" after XNAT has finished waiting the configurable amount of time without receiving a new DICOM instance for the study (see Admin UI - Session Upload and Anonymization Settings). The difference is that while with the default behavior the session would now be ready in the prearchive (triggering autoarchiving, if relevant), DA uploads would build directly into the archive.

Using the feature

Currently, there are two major pathways that make use of the new DA functionality. First, when defining a DICOM SCP receiver in XNAT, the XNAT administrator can choose to enable direct-archive functionality for all imports going to that receiver. This is intended to allow a sort of "fast-track" receiver that can be used by modalities with consistent, well-behaved data. For more information on defining SCP receivers, see Connecting XNAT to DICOM Scanners and PACS . It is important to know therefore that enabling direct-archive on an SCP receiver will override the project's preferences on autoarchiving sessions when data arrives for that receiver.

In this scenario, users can see the status of in-progress DA requests for their projects in the XNAT prearchive (see Using the Prearchive for additional information on the prearchive). Under the main prearchive table, there is now a table for monitoring DA requests:

Second, when a user is uploading via the Import API (see Image Session Import Service API), the user can request that XNAT import the data via direct-archive by including the query parameter Direct-Archive=true . However, this parameter is only handled by the gradual-DICOM  and DICOM-zip import handlers. To track the progress of the session through direct archive, the user will need to poll (GET) the URL returned by the API, as the direct archive table shown above is not present on the prearchive page unless there is a DICOM SCP receiver with DA enabled. (Coming in XNAT 1.8.4: In this scenario, the user may also request immediate archival by POSTing to this URL).

Limitations

If XNAT cannot cleanly archive the session being received, the DA process is designed to move the session data back to the prearchive for standard handling. For efficiency purposes, this not only loses the advantages of direct archival, but also incurs a performance penalty of up to 50% over using the standard import mechanism in terms of file IO. As such, the DA import workflow is ideal for consistent, heavy load inputs, but not for data involving many corner cases. Some scenarios that the DA process will send back to the prearchive include:

1. Splitting of a multimodality PET-MR study into PET Session and MR Session (see Admin UI - Session Upload and Anonymization Settings).
2. Merging additional DICOM series into an already archived XNAT session.