Page tree
Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 11 Next »

As described in Getting Started / Configuration, the -d data flag is a required parameter to specify a newline-separated text file indicating what projects to import. For convenience, a file called allData.txt is included which will pull all previously prepared sample data. For any project, the script expects two things: a YAML configuration file to define the structure of the project and a folder containing the data for the project. The YAML file must be titled "$projectId_metadata.yaml". The data folder should contain individually zipped sessions at the top level. When the script starts, it will read the text file and begin checking for these files. If the ./data folder does not already contain the YAML file, the script will search for it on the NRG FTP site. Likewise, if the ./data folder does not contain the folder containing the data for the project (or its zip), the script will attempt to locate a zip by the project ID on the NRG FTP site. Below is an example project YAML file which contains most of the currently supported operations (note that the file would necessarily be called study_001_metadata.yaml):

id : study_001
runningTitle : First XNAT Study
title : My First XNAT Study
keywords : 1st XNAT
description : This collection contains data for my new XNAT
anon :
    file : sample_anon.das
subjects :
    Subj_004 :
        gender : Male
        yob : 1950
        education : 16
        share :
            DEST_PROJ :
                label : DEST_SUBJ_LABEL
        complexSessions :
            0000004 :
                share :
                    DEST_PROJ : DEST_LABEL
            empty_session :
                src : empty
                xsiType : "xnat:mrSessionData"
                scans :
                    1 :
                        xsiType : "xnat:mrScanData"
            other_session :
                src : /data/xnat/home/other/
                assessors : [other_session_QC1]
         assessors : [Subj_004_FORM1]
    GSU002 :
        gender : Male
        yob : 1921
        handedness : Right
        sessions : [GSU002_v00_mr, GSU002_v00_pet, GSU002_v01_mr]
    OUA001 :
        gender : Female
        yob : 1937
        handedness : Ambidextrous
        complexSessions :
            OUA001_v00_mr :
                pipelines :
                    DicomToNifti :
                        create_nii : "N"
            OUA001_v01_mr :
                pipelines :
                    DicomToNifti :
                        create_nii : "Y"
users :
    owners : [owner]
    members : [member]
    collaborators : [collaborator]
pi :
    title : Dr
    firstname : Atticus
    lastname : Cromwell
investigators : [{firstname : Jens, lastname : Aasgaard}, {firstname : Arthur, lastname : Compton}]
src : /data/xnat/

Tips on the YAML above:


The anonymization section mirrors the syntax exactly of how it is used in Site Configuration, except now it is added to the project.

sessions versus complexSessions

The sessions element allows you to easily define an array of sessions which will all be uploaded to the XNAT server as is. So, while it is exceedingly easy to specify the sessions as an array like this, you lose power to fine-tune the sessions, as needed. So, if you want to do this, you specify the sessions under complexSessions, instead. The sessions here will also be uploaded through the session importer, but additional modifications will be added. So, the study_001 folder for this project should contain the following files:


Note that there are two sessions left out of the above list: "empty_session" and "other_session". If a complexSession has the src property set to empty, the session will be created as an empty experiment directly through the experiment API. Then, scans can be manually created and modified under the session. For any scan, such as the above scan 1, you can set series_description, type, note in addition to the required xsiType. This is the path "empty_session" will take. If a complexSession has the src property set to something else, the zip for the session to be uploaded will be searched for at this value (interpreted as an absolute path). So, the "other_session" would have its session zip pulled from /data/xnat/home/other/


The format for sharing differs depending on whether the object being shared is a subject or session. For a subject, the child elements of share are one or more destination projects. Each of these destination projects takes zero or more key-value pairs to customize the sharing. In the above example, subject Subj_004 is being shared into a single project, "DEST_PROJ" while being given the label "DEST_SUBJ_LABEL". In addition to specifying the label, also supported is passing shareAllSessions : true, which will share all of the sessions from this subject into the destination project with labels as is (included for convenience). The sharing for a session is easier. The child elements of share on a complexSession element are one or more key-value pairs of destination project to destination label of the session. So, in the above example, session 0000004 would be shared to project DEST_PROJ with the session label DEST_LABEL


If the users already exist, and the XNAT user running the script has access to the site user list, the users will be added to the project in their corresponding roles. If the users do not already exist, insecure users will be created (the password is just the username), which should not be used in production (this requires admin access).


For any project, subject, session, or scan in the YAML, you can add a resources block. For project resources, this would be included at the root level, for subject resources, it would be a property of a subject, and so on for complexSessions and scans.

This block should be formatted as:

resources :
    resource_folder_1 :
        files : [my_file.txt, my_paper.pdf]
    resource_folder_2 :
        files : []
        unzip : true

The above would create 2 resource folders: one called "resource_folder_1" with the files my_file.txt, my_paper.pdf, and one called "resource_folder_2" with the zip extracted. Note that the resources are expected to be in the project's data folder to be picked up for upload.


By default, XNAT Populate will look for the data:

  1. First, it checks in a folder under the "data" folder with a name matching the project ID.
  2. If that doesn't work, it checks for a zip file under the "data" folder named $
  3. If that doesn't work, it checks on the NRG FTP site for the data.

However, there is another option: by specifying the 'src' value at the top level of the project YAML, the script will attempt to load the project's data zip from the file path given here. Technically, this is checked before the FTP site check, so the script will not fail as long as it finds your data via 'src'. This is useful if the data zip is large and mounted in a reliable spot (it will still be extracted into "data" to post the session zips).


What is available already?

See this cheat sheet:

  • No labels