Adding a Custom Importer & Session Builder via a Plugin

A custom importer and session builder can be written in a plugin to handle imports of image session formats not supported by XNAT. Your image sessions can then be imported with XNAT's Import Service API and make use of XNAT's prearchive. Before proceeding, you'll need to create new data types which extend xnat:imageSessionData and xnat:imageScanData.

Creating an Importer

The importer is responsible for reading the incoming imaging data from an InputStream then storing the data in XNAT's prearchive directory.

Creating the importer class starts with creating the class itself. This class needs to extend ImporterHandlerA and be annotated with @ImporterHandler. The code below shows a basic class declaration for an importer:

CODE

@ImporterHandler(handler = "TEMPLATE")
public class TemplateImporter extends ImporterHandlerA {

    private final InputStream in;
    private final Map<String, Object> params;
    private final Format format;

    public TemplateImporter(final Object listenerControl,
                            final UserI u,
                            final FileWriterWrapperI fw,
                            final Map<String, Object> params) throws IOException {
        super(listenerControl, u);

        this.params = params; // Import Service API parameters
        this.in = fw.getInputStream();
        this.format = Format.getFormat(fw.getName());
    }

    @Override
    public List<String> call() throws ClientException, ServerException {         
		return Collections.emptyList();     
	}
}

The call method is where the InputStream is read and the imaging data is written to XNAT's prearchive. The code below demonstrates reading the files from a zip file and storing them into the prearchive directory.

CODE

@Override
public List<String> call() throws ClientException, ServerException {
    // List of paths in the prearchive where incoming image session data has been stored
    final List<String> uris = Lists.newArrayList();

    if (format == Format.ZIP) {
		// Get project, subject, and experiment labels passed to the Import Services API
        final String projectId = (String) params.get(URIManager.PROJECT_ID);
        final String subjectId = (String) params.get(URIManager.SUBJECT_ID);
        final String experimentLabel = (String) params.get(URIManager.EXPT_LABEL);

	    // Image sessions in the prearchive directory are stored as:
        // .../prearchviePath/projectId/timestamp/experimentLabel/SCANS/scanLabel
        // You can have more than one scan.
        final String timestamp = PrearcUtils.makeTimestamp();
        Path prearchivePath = Paths.get(ArcSpecManager.GetInstance().getGlobalPrearchivePath(),
                                        projectId,
                                        timestamp,
                                        experimentLabel,
                                        "SCANS", "1");

        if (!Files.exists(prearchivePath)) {
            try {
                Files.createDirectories(prearchivePath);
            } catch (IOException e) {
                throw new RuntimeException(e);
            }
        }

        try (final ZipInputStream zin = new ZipInputStream(in)) {
            ZipEntry ze;
            while (null != (ze = zin.getNextEntry())) {
                if (!ze.isDirectory()) {
                    String fullFileName = ze.getName();
                    String[] splitFileName = fullFileName.split("/");
                    String fileName = splitFileName[splitFileName.length - 1];

                    Path outputPath = prearchivePath.resolve(fileName);

                    if (!Files.exists(outputPath)) {
                        Files.createFile(outputPath);
                    }

                    ZipEntryFileWriterWrapper zipEntryFileWriterWrapper = new ZipEntryFileWriterWrapper(ze, zin);
                    zipEntryFileWriterWrapper.write(outputPath.toFile());
                }
            }
        } catch (IOException e) {
            throw new RuntimeException(e);
        }

        // The SessionData object is used to represent image sessions stored in the prearchive as the final
        // session.xml has not yet been created by a SessionBuilder
        SessionData session = new SessionData();
        session.setFolderName(experimentLabel);
        session.setTimestamp(timestamp);
        session.setName(experimentLabel);
        session.setProject(projectId);
        session.setSubject(subjectId);
		session.setUploadDate(new Date());
        session.setScan_date(new Date());         
        session.setStatus(PrearcUtils.PrearcStatus.RECEIVING);
        session.setLastBuiltDate(Calendar.getInstance().getTime());
        session.setUrl(prearchivePath.toString());
        session.setAutoArchive(PrearchiveCode.Manual);

        // Add your session to the prearchive
        try {
            PrearcDatabase.addSession(session);
        } catch (Exception e) {
            throw new RuntimeException(e);
        }

        // Add the session directory to our list of uris
        uris.add(Paths.get(ArcSpecManager.GetInstance().getGlobalPrearchivePath(),
                            projectId,
                            timestamp,
                            experimentLabel).toString());
    } else {
        throw new ClientException("Unsupported format " + format);
    }

    return uris;
}

In this example the project, subject, and experiment labels are query parameters passed to the Import Services API. You can pass other parameters in as needed or you may read parameters from a metadata file sent along side the image session. This example only accepts zip files, but of course you can accept any file type.

The last step in creating an importer is to add it to the Spring application context. To do this, in your @XnatPlugin ClassXNAT Plugin Configurations add a @Bean annotated method which returns ImporterHandlerPackages:

CODE

@Bean
public ImporterHandlerPackages templateImporterHandlerPackages() {
    return new ImporterHandlerPackages("org.nrg.xnat.plugins.template.importer");
}

Your importer will now work with the Import Service API. Let's try sending a zip file to our importer:

CODE

curl -u user:password -H 'Content-Type: application/zip' -X POST 'http://yourxnat/data/services/import?inbody=true&import-handler=TEMPLATE&PROJECT_ID=TestProject&SUBJECT_ID=TestSubject&EXPT_LABEL=TestImageSession' --data-binary @'/Path/to/fakeImageSession.zip'

In the XNAT UI, navigate to the prearchive to see the image session.

The image session will first be in a Receiving state then will transition to an Error state. After the image session is received, XNAT will attempt to build the image session XML. Because we have not created a SesisonBuilder for our new image session type, session building fails thus the error state. In this state we will not be able to archive our image session.

Creating a SessionBuilder

The session builder is responsible for building the XML which represents your image session and writing that XML to the prearchive directory alongside the imaging data.

First we need to create our own session building class which extends SessionBuilder:

CODE

public class TemplateSessionBuilder extends SessionBuilder {

    private final File sessionDir;


    public TemplateSessionBuilder(final File sessionDir, final Writer fileWriter) {
        super(sessionDir, sessionDir.getPath(), fileWriter);
        this.sessionDir = sessionDir;
    }

    @Override
    public String getSessionInfo() {
        return null;
    }

    @Override
    public XnatImagesessiondataBean call() throws Exception {
        return null;
    }
}

The getSessionInfo method is only used for logging purpose. We will override that method with something simple, but add more information about your image session if needed.

CODE

@Override
public String getSessionInfo() {
    return "1 scan";
}

The call method is where we build our image session bean and also create the catalog xml files which catalogs all the files in each image scan directory. You'll need to have already created a new image session and image scan data type before implementing this method.

CODE

@Override
public XnatImagesessiondataBean call() throws Exception {
    // Get project, subject and image session labels
    Map<String, String> parameters = getParameters();
    String project = parameters.getOrDefault(PrearcUtils.PARAM_PROJECT, null);
    String subject = parameters.getOrDefault(PrearcUtils.PARAM_SUBJECT_ID, "");
    String label = parameters.getOrDefault(PrearcUtils.PARAM_LABEL, null);

    // Build the image session bean. Set any other fields associated with your image session.
    TemplateImagesessiondataBean templateImageSession = new TemplateImagesessiondataBean();
    templateImageSession.setProject(project);
    templateImageSession.setSubjectId(subject);
    templateImageSession.setLabel(label);

    // Build image scan beans. Your image session can have many scans. Build a scan bean for each one. Scan
    // building can also be handled by another class.
    TemplateImagescandataBean templateImageScan = new TemplateImagescandataBean();
    templateImageScan.setId("1");
    templateImageScan.setType("CUSTOM");
    templateImageScan.setUid(UUID.randomUUID().toString());

    // Create catalog file. This catalogs all the files in a single scan directory. You'll need to build a
    // a catalog for each scan.
    XnatResourcecatalogBean resourceCatalog = new XnatResourcecatalogBean();
    resourceCatalog.setUri(Paths.get("SCANS", "1", "scan_catalog.xml").toString());
    resourceCatalog.setLabel("TEMPLATE");
    resourceCatalog.setFormat("CUSTOM");
    resourceCatalog.setContent("CUSTOM");
    resourceCatalog.setDescription("Template image scan data");

    CatCatalogBean catCatalogBean = new CatCatalogBean();

    Path scanDir = sessionDir.toPath().resolve("SCANS").resolve("1");
    try {
        Files.list(scanDir)
                .map(file -> { CatEntryBean catEntryBean = new CatEntryBean();
                    catEntryBean.setUri(String.valueOf(file.getFileName()));
                    return catEntryBean;
                })
                .forEach(catCatalogBean::addEntries_entry);
    } catch (IOException e) {
        throw new RuntimeException(e);
    }

    templateImageScan.addFile(resourceCatalog);

    // Write the catalog xml to the scan directory
    File resourceCatalogXml = new File(scanDir.toFile(), "scan_catalog.xml");
    try (FileWriter resourceCatalogXmlWriter = new FileWriter(resourceCatalogXml)) {
        catCatalogBean.toXML(resourceCatalogXmlWriter, true);
    } catch (IOException e) {
        throw new RuntimeException(e);
    }

    // Add the scan to the image session
    templateImageSession.addScans_scan(templateImageScan);

    return templateImageSession;
}

You'll need to create and return an image session bean which XNAT will then write to the prearchive directory as an XML file. You also need to create a scan bean for each scan in your image session.

Unlike the importer we created, the session builder is added to XNAT application context via a properties file instead of with Spring. This properties file is stored in META-INF/xnat/**/*-session-builder.properties. We'll create ours at META-INF/xnat/session-builders/custom-session-builder.properties.

CODE

org.nrg.SessionBuilder.impl=TEMP
org.nrg.SessionBuilder.impl.TEMP.className=org.nrg.xnat.plugins.template.sessionBuilder.TemplateSessionBuilder
org.nrg.SessionBuilder.impl.TEMP.sequence=2

The first line is a short code to identify your session builder. The next line identifies the class of your session builder. The last line identifies the sequence or order in which your session builder will run. XNAT's core DICOM and ECAT session builders are set to 0 and 1 respectively. If you'd like your session builder to run before those set the sequence value to -1.

With a SessionBuilder and properties file written we can resubmit our image session to the Import Service API.

The image session should now transition from a Receiving state to a Ready state. You will now be able to archive your image session.

User Interface

The UI for this custom importer is left as an exercise for the reader. You'll need to build a new Velocity template with a <form> that submits the image session to the Import Service API. A link to your form can be added to XNAT's top navigation bar with a second Velocity template:

CODE

<!-- templates/screens/topBar/Upload/CustomUploadTopBar.vm -->
<li><a href="$link.setPage("CustomUpload.vm")">Upload Custom Image Session</a></li>