Skip to main content
Skip table of contents

DICOM resource survey user guide

If you are unfamiliar with the DICOM resource survey and mitigation function, please read the technical overview first.

There is no REST API reference for this functionality. Instead you should open the Swagger page for your XNAT installation and find the section labeled resource-survey-api.

This user guide describes how to create and execute resource survey and mitigation requests. This page includes the following sections:


The resource survey and mitigation functionality only targets DICOM resources associated with image session experiments, with data types such as xnat:mrSessionData for MR, xnat:petSessionData for PET, xnat:ctSessionData for CT, and so on. Some other experiment types may include DICOM resources, e.g. icr:roiCollectionData experiments include RTSTRUCT data. However, this type of DICOM is not pushed into XNAT through the C-STORE service and so is not subject to the same file naming issues. 


XNAT does not provide a user interface for the DICOM resource survey functionality. All interactions are through the provided REST API. There are a number of tools that you can use to work with the REST API:

Examples on this page use curl because it's widely available on all major platforms, but any similar tool works as well.

The curl examples are simplified for readability and omit options such as --user, --cookie-jar, and --cookies. For ease of use, you can create a file named .curlrc in your home directory that specifies these options by default. An example might be:

Sample .curlrc file

BASH 
--cookie-jar /Users/username/.curl/cookies.txt
--cookie /Users/username/.curl/cookies.txt

You could then make a call like:

CODE 
curl --user admin http://server/data/projects
Enter host password for user 'admin':

The return value of this call isn't important. What is important is that the cookie jar file specified in your .curlrc file should now contain a session cookie.

Terminology 

This page uses a number of terms that may need clarification or definition.

  • A resource ID is the unique ID for a resource. This is not the same as the label or any other identifier for the resource. Instead it's the unique primary key for each resource, i.e. xnat_abstractresource.xnat_abstractresource_id.
  • A resource survey request is a request to survey a resource for issues and, if necessary, mitigate those issues. Although the name only references surveying, it's used for both operations and, on completion of survey and mitigation operations, includes the resulting survey and mitigation reports.
  • Creating a resource survey request actually implies both creating and queuing that request (queuing means putting the request into a processing queue where it can be acted on when a worker process becomes available). These are really two separate tasks, but have been combined in a single call for ease of use. Note that this does not apply to mitigation: for that the request must be pushed to the queue explicitly.
  • The resource survey request status indicates where a request is in its lifecycle. Request statuses include CREATED, QUEUED_FOR_SURVEY, SURVEYING, DIVERGENT, CONFORMING, QUEUED_FOR_MITIGATION, MITIGATING, RESOURCE_DELETED, CANCELED, and ERROR.
  • An open resource survey request is one that still requires some action to be taken: survey needs to be completed, mitigation should be run, etc., which can be determined by the request status. A request is open if it has any status other than CONFORMING, RESOURCE_DELETED, CANCELED, or ERROR and no value set for closing date.
  • A closed resource survey request indicates that nothing remains to be done for that request. A request is closed if its status is CONFORMING, RESOURCE_DELETED, CANCELED, or ERROR: setting one of those statuses automatically sets the closing date.

Another set of terms to understand pertains to issues found during the resource survey:

  • Mismatched files are files containing valid DICOM and have a unique SOP class and instance UID in the resource, but their names don't match the expected values
  • Duplicates files are files containing valid DICOM that have the same SOP class and instance UID within the resource as one or more other files: in this case at least one of the duplicate files has to have a name that doesn't match the expected value (theoretically all of the files in a set of duplicates might be mismatched, but usually at least one has the correct name)
  • Bad files are files that look like DICOM files and are included in the resource catalog but can't be parsed as valid DICOM for some reason


The DICOM standard requires each DICOM instance to be uniquely identified by SOP Instance UID. XNAT’s resource survey function makes the assumption that two or more files that have the same SOP class and instance UID are therefore duplicates and adds them to the map of duplicates requiring mitigation.

We have found scenarios where DICOM arrives with two files with the same SOP class and instance UID but different instance numbers. In this scenario, generated file names will not match. As of XNAT 1.8.8.2, these "non-actionable" duplicates are logged but no automated mitigation takes place. 

See XNAT-7794 for details. 


The type of file found in a divergent resource dictates what action is required for mitigation. This is discussed in more detail in Reviewing mitigation reports below.

Creating and queuing resource survey requests 

You can create resource survey requests at the project or resource level:

  • POSTing to /xapi /resources/survey/resource/{resourceId} creates and queues a new resource survey request for the resource with the ID specified by resourceId
  • POSTing to /xapi /resources/survey/project/{projectId} creates and queues new resource survey requests for all resources in the specified project that meet the following criteria:
    • Have format DICOM
    • Are not associated with an open resource survey request 

These calls return the ID or IDs of the newly created resource survey request.

Here's an example of creating a resource survey request for a single resource:

Creating resource survey request for a resource

BASH 
# curl -X POST http://localhost:8080/xapi/resources/survey/resource/256
132%

This has created a resource survey request with the ID 132 for resource 256.

Here's an example of creating resource survey requests for all DICOM resources in a particular project:

Creating resource survey request for a resource

BASH 
# curl -X POST http://localhost:8080/xapi/resources/survey/project/PROJECT_01
[133,134,135,136,137]%

This has created resource survey requests with the IDs 133, 134, 135, 136, and 137 for resources in the project PROJECT_01.

Reviewing resource survey requests and reports

You can retrieve resource survey requests individually by resource or request ID or in bulk by project ID or even for your whole XNAT deployment.

Retrieving survey requests

Here's an example of retrieving open resource survey requests for a particular project:

Retrieving resource survey requests for a project

BASH 
# curl http://localhost:8080/xapi/resources/survey/project/PROJECT_01
[
    {
        ...
    },
    {
        ...
    }
 ]

Each of the ... placeholders in the output is a full resource survey request, omitted here for readability.

Retrieving requests by resource or request ID looks very much the same except that each returns only a single request instance.

Retrieving a resource survey request for a resource

BASH 
# curl http://localhost:8080/xapi/resources/survey/resource/256
{
  ...
}
# curl http://localhost:8080/xapi/resources/survey/request/132
{
  ...
}

There is one significant difference between retrieving requests by resource or request ID and by project ID:

  • When retrieving by resource or request ID, the request will always be returned, regardless of state (presuming there is a request with the specified resource or request ID)
  • When retrieving by project ID, only open requests (i.e. those with a status other than CONFORMING, RESOURCE_DELETED, CANCELED, or ERROR) are returned by default

If you need to retrieve closed requests by project ID, you can add a qualifier indicating the status to the REST endpoint:

Retrieving resource survey requests by status for a project

BASH 
# curl http://localhost:8080/xapi/resources/survey/project/PROJECT_01/conforming
[
    {
        "id": 13,
        "rsnStatus": "CONFORMING",
        ...
    },
    {
        ...
    }
]

# curl http://localhost:8080/xapi/resources/survey/project/PROJECT_01/error
[
    {
        "id": 197,
        "rsnStatus": "ERROR",
        ...
    },
    {
        ...
    }
]


Retrieving survey status

In practice, you will usually want to get something less than the full resource survey request. The REST API provides the ability to get just request status or the resource survey report as well. To retrieve just the status, add "/status" to the paths above, so you get:

  • /xapi/resources/survey/project/{projectId}/status
  • /xapi/resources/survey/resource/{resourceId}/status
  • /xapi/resources/survey/request/{requestId}/status

The output is pretty simple:

Retrieving request status

BASH 
# curl http://localhost:8080/xapi/resources/survey/resource/256/status
"DIVERGENT"
# curl http://localhost:8080/xapi/resources/survey/request/132/status
"DIVERGENT"

Retrieving survey reports

You can also retrieve just the survey report by adding "/report" to the paths above:

  • /xapi/resources/survey/project/{projectId}/report
  • /xapi/resources/survey/resource/{resourceId}/report
  • /xapi/resources/survey/request/{requestId}/report

The output here is much more verbose than the status above:

Retrieving request survey report

BASH 
# curl http://localhost:8080/xapi/resources/survey/request/132/report
{"resourceSurveyRequestId":132, ... }

You can also download report summaries as a CSV file with the download endpoint:

  • /xapi/resources/survey/report/download (accepts an optional querystring parameter named projectId)

Note that the session user must be a site administrator if omitting the projectId parameter!

The CSV report includes most of the properties of each resource survey request, including:

  • Project ID
  • Subject ID
  • Experiment ID
  • Scan ID
  • Resource ID
  • Resource label
  • Resource URI (i.e. location of the catalog file)
  • Status

It also includes properties from the survey report if one exists for the request, including:

  • Survey date
  • Total entries
  • Total number of distinct SOP class and instance UID combinations
  • Total number of mismatched files
  • Total number of duplicates
  • Total number of bad files

You may find the CSV output is more compact and easier to manipulate if you have a lot of survey data to review:

Sample CSV download

BASH 
# curl http://localhost:8080/xapi/resources/survey/report/download\?projectId=XNAT_01
projectId,subjectId,experimentId,scanId,resourceId,resourceLabel,resourceUri,requestStatus,surveyDate,totalEntries,totalUids,totalDuplicates,totalBadFiles,totalMismatchedFiles
XNAT_01,XNAT_S00001,XNAT_E00004,22,22,DICOM,/data/xnat/archive/XNAT_01/arc001/XNAT_01_01_MR_02/SCANS/4/DICOM/scan_4_catalog.xml,DIVERGENT,Tue Feb 14 15:08:25 CST 2023,3,3,0,0,3
XNAT_01,XNAT_S00001,XNAT_E00004,23,23,DICOM,/data/xnat/archive/XNAT_01/arc001/XNAT_01_01_MR_02/SCANS/5/DICOM/scan_5_catalog.xml,DIVERGENT,Tue Feb 14 15:08:25 CST 2023,6,3,3,0,0
XNAT_01,XNAT_S00001,XNAT_E00004,24,24,DICOM,/data/xnat/archive/XNAT_01/arc001/XNAT_01_01_MR_02/SCANS/6/DICOM/scan_6_catalog.xml,DIVERGENT,Tue Feb 14 15:08:25 CST 2023,3,2,0,1,0

Since this output is in standard CSV format, you can import it into a spreadsheet or database if you have a lot of resources to review. This CSV output can also be used as an input for launching mitigation operations in bulk, as described in Queuing resource mitigation requests.

Retrieving survey report summaries

You can get a summary of survey reports in a particular project or across the site by calling:

  • /xapi/resources/survey/project/{projectId}/report/summary
  • /xapi/resources/survey/report/summary (accepts an optional querystring parameter named projectId)

Calling for a project summary would look like this:

CODE 
# curl http://localhost:8080/xapi/resources/survey/project/PROJECT_01/report/summary
[{"PROJECT_01":"XNAT_01","totalEntries":738,"totalUids":648,"totalDuplicates":90,"totalBadFiles":0,"totalMismatchedFiles":94,"totalMovedFiles":0,"totalRemovedFiles":0,"totalFileErrors":0,"overallStatus":"IN_PROGRESS"}]%

Note that the value for overallStatus in this example is IN_PROGRESS. This is a "pseudostatus" that indicates that there are open resource survey requests in the project.

Queuing resource mitigation requests 

Queuing resource mitigation requests looks similar to creating new resource survey requests:

  • POST /xapi/resources/mitigate/project/{projectId}
  • POST /xapi/resources/mitigate/resource/{resourceId}
  • POST /xapi/resources/mitigate/request/{requestId}
  • POST /xapi/resources/mitigate/resources/csv (requires uploadFile form parameter with a CSV file containing the IDs of resources to be mitigated: see below)

The primary difference from creating requests is that now you have requests that have IDs so you can queue mitigation based on the request ID in addition to project and resource IDs.

Queuing mitigation by project

BASH 
# curl -X POST http://localhost:8080/xapi/resources/mitigate/project/XNAT_01
[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20]

The list returned here includes the IDs of all of the requests in the project that were divergent and queued for mitigation.

The first three API endpoints are pretty straightforward: run mitigation on open surveyed requests in a project or on a particular resource or or the resource referenced by a particular request. The last one allows you to specify one or more resource IDs in a CSV file, with the CSV file being uploaded in a multi-part form data POST request. This sounds more complicated than it really is:

  • Get the CSV file by calling the bulk survey report summary endpoint /xapi/resources/survey/report/summary, as described in Retrieving survey report summaries
  • Edit the CSV file to include only resources you want to mitigate presently (note that you can even remove all columns other than resourceId from the CSV, although this isn't required)
  • Post the CSV file to /xapi/resources/mitigate/resources/csv with the uploadFile form parameter referencing your CSV file

Using curl, this looks something like this:

CODE 
# curl http://localhost:8080/xapi/resources/mitigate/resources/csv --form uploadFile=@xnat_01.csv
{"queued":[2,3,1]}

Reviewing mitigation reports 

Mitigation reports describe exactly what steps were taken to mitigate divergent resource files. This includes:

  • The cache path used to back up all modified files (unless maintainFileHistory is set to true, in which case the cache path will contain only log files generated during the mitigation process)
  • Which files were moved (renamed) and where they were moved to
  • Which files were removed entirely from the resource
  • Any errors that occurred during the move/delete operations or catalog updates
  • The total number of moved and removed files

Retrieving mitigation reports works much like retrieving survey reports, replacing survey in the path with mitigate:

  • /xapi/resources/mitigate/project/{projectId}/report
  • /xapi/resources/mitigate/resource/{resourceId}/report
  • /xapi/resources/mitigate/request/{requestId}/report

The output here is much more verbose than the status above:

Retrieving request survey report

BASH 
# curl http://localhost:8080/xapi/resources/mitigate/resource/2/report
{"resourceSurveyRequestId":1, ... }

The most useful information you'll find in a mitigation reports are the movedFiles and removedFiles maps:

  • movedFiles tells you which files in the resource were renamed as well as their new names
  • removedFiles tells you which files were backed up and the destination file for the backup
  • retainedFiles tell you which files contained duplicate SOP class and instance UIDs but were not moved or removed because their name matched the name generated by the DICOM file namer

These maps could be used to restore the files in the resource to their original state if that became necessary for some reason.

One thing you won't find when reviewing mitigation reports is any mention of bad files. As mentioned in the Terminology section, bad files look like DICOM files and are included in the resource catalog but can't be parsed as valid DICOM. Since the DICOM isn't parsable, the file namer can't calculate the expected file name nor can it check the SOP class and instance UID against other DICOM files in the resource. Because of this, the mitigation operation has no way of mitigating any issues with bad files. The entries for these files are left in the catalog file. It's left to administrators and project owners of each XNAT deployment to review any bad files in the survey reports and decide on the appropriate resolution.

Sanitizing survey and mitigation reports 

For information on why you might want to sanitize reports on a resource survey request, refer to Sanitizing survey and mitigation reports in the technical overview.

You can sanitize reports at the resource or request level:

  • At the resource level,  all  requests associated with the resource are cleaned, not just the latest one
  • At the request level, only the single request is cleaned

The calls for sanitizing a resource or a request differ only slightly:

  • DELETE /xapi/resources/survey/resource/{resourceId}/report
  • DELETE /xapi/resources/survey/request/{requestId}/report

The example below shows the full versions of the survey and mitigation reports for a resource, then the call to sanitize reports for that resource, then the sanitized versions of the survey and mitigation reports (note that this example may not be consistent in file naming, as it was edited for better readability):

Full vs. sanitized survey and mitigation reports

BASH 
# curl http://localhost:8080/xapi/resources/survey/resource/2/report
{
  "resourceSurveyRequestId": 1,
  "surveyDate": 1675460154441,
  "totalEntries": 12,
  "totalUids": 9,
  "totalBadFiles": 0,
  "totalMismatchedFiles": 2,
  "totalDuplicates": 3,
  "uids": {
    "1.2.840.10008.5.1.4.1.1.4": [
      "1.2.840.113654.2.45.2.108831",
      "1.2.840.113654.2.45.2.111893",
      "1.2.840.113654.2.45.2.113207",
      "1.2.840.113654.2.45.2.113254",
      "1.2.840.113654.2.45.2.115284",
      "1.2.840.113654.2.45.2.119159",
      "1.2.840.113654.2.45.2.120430",
      "1.2.840.113654.2.45.2.124799",
      "1.2.840.113654.2.45.2.126951",
      "1.2.840.113654.2.45.2.130459",
      "1.2.840.113654.2.45.2.132438"
    ]
  },
  "mismatchedFiles": {
    "/data/xnat/archive/XNAT_01/arc001/XNAT_01_01_MR_01/SCANS/1/DICOM/XNAT_01_01.MR.XNAT_01.1.26.20210315.o4uktl.dcm": "1.2.276.0.7230010.3.1.2.0.31909.1624637609.925-1-26-2zncre.dcm",
    "/data/xnat/archive/XNAT_01/arc001/XNAT_01_01_MR_01/SCANS/1/DICOM/XNAT_01_01.MR.XNAT_01.1.28.20210315.npsqlk.dcm": "1.2.276.0.7230010.3.1.2.0.31909.1624637609.925-1-28-2z8pkd.dcm",
  },
  "duplicates": {
    "1.2.840.10008.5.1.4.1.1.4": {
      "1.2.840.113654.2.45.2.113254": {
        "/data/xnat/archive/XNAT_01/arc001/XNAT_01_01_MR_01/SCANS/1/DICOM/1.2.276.0.7230010.3.1.2.0.31909.1624637609.925-1-21-2ylma9.dcm": "1.2.276.0.7230010.3.1.2.0.31909.1624637609.925-1-21-2ylma9.dcm",
        "/data/xnat/archive/XNAT_01/arc001/XNAT_01_01_MR_01/SCANS/1/DICOM/XNAT_01_01.MR.XNAT_01.1.21.20210315.n22d7g.dcm": "1.2.276.0.7230010.3.1.2.0.31909.1624637609.925-1-21-2ylma9.dcm"
      },
      "1.2.840.113654.2.45.2.119159": {
        "/data/xnat/archive/XNAT_01/arc001/XNAT_01_01_MR_01/SCANS/1/DICOM/1.2.276.0.7230010.3.1.2.0.31909.1624637609.925-1-23-2ypfgv.dcm": "1.2.276.0.7230010.3.1.2.0.31909.1624637609.925-1-23-2ypfgv.dcm",
        "/data/xnat/archive/XNAT_01/arc001/XNAT_01_01_MR_01/SCANS/1/DICOM/XNAT_01_01.MR.XNAT_01.1.23.20210315.n5zd02.dcm": "1.2.276.0.7230010.3.1.2.0.31909.1624637609.925-1-23-2ypfgv.dcm"
      },
      "1.2.840.113654.2.45.2.108831": {
        "/data/xnat/archive/XNAT_01/arc001/XNAT_01_01_MR_01/SCANS/1/DICOM/XNAT_01_01.MR.XNAT_01.1.20.20210315.ml4ml3.dcm": "1.2.276.0.7230010.3.1.2.0.31909.1624637609.925-1-20-2y550w.dcm",
        "/data/xnat/archive/XNAT_01/arc001/XNAT_01_01_MR_01/SCANS/1/DICOM/1.2.276.0.7230010.3.1.2.0.31909.1624637609.925-1-20-2y550w.dcm": "1.2.276.0.7230010.3.1.2.0.31909.1624637609.925-1-20-2y550w.dcm"
      }
    }
  }
}
# curl http://localhost:8080/xapi/resources/mitigate/resource/2/report
{
  "resourceSurveyRequestId": 1,
  "cachePath": "file:///data/xnat/cache/XNAT_01/XNAT_E00001/1675460154345/2/",
  "movedFiles": {
    "/data/xnat/archive/XNAT_01/arc001/XNAT_01_01_MR_01/SCANS/1/DICOM/XNAT_01_01.MR.XNAT_01.1.26.20210315.o4uktl.dcm": "/data/xnat/archive/XNAT_01/arc001/XNAT_01_01_MR_01/SCANS/1/DICOM/1.2.276.0.7230010.3.1.2.0.31909.1624637609.925-1-26-2zncre.dcm",
    "/data/xnat/archive/XNAT_01/arc001/XNAT_01_01_MR_01/SCANS/1/DICOM/XNAT_01_01.MR.XNAT_01.1.28.20210315.npsqlk.dcm": "/data/xnat/archive/XNAT_01/arc001/XNAT_01_01_MR_01/SCANS/1/DICOM/1.2.276.0.7230010.3.1.2.0.31909.1624637609.925-1-28-2z8pkd.dcm"
  },
  "removedFiles": {
    "/data/xnat/archive/XNAT_01/arc001/XNAT_01_01_MR_01/SCANS/1/DICOM/XNAT_01_01.MR.XNAT_01.1.26.20210315.o4uktl.dcm": "/data/xnat/cache/XNAT_01/XNAT_E00001/1675460154345/2/XNAT_01_01.MR.XNAT_01.1.26.20210315.o4uktl.dcm",
    "/data/xnat/archive/XNAT_01/arc001/XNAT_01_01_MR_01/SCANS/1/DICOM/1.2.276.0.7230010.3.1.2.0.31909.1624637609.925-1-23-2ypfgv.dcm": "/data/xnat/cache/XNAT_01/XNAT_E00001/1675460154345/2/1.2.276.0.7230010.3.1.2.0.31909.1624637609.925-1-23-2ypfgv.dcm",
    "/data/xnat/archive/XNAT_01/arc001/XNAT_01_01_MR_01/SCANS/1/DICOM/XNAT_01_01.MR.XNAT_01.1.25.20210315.nnpbiv.dcm": "/data/xnat/cache/XNAT_01/XNAT_E00001/1675460154345/2/XNAT_01_01.MR.XNAT_01.1.25.20210315.nnpbiv.dcm",
    "/data/xnat/archive/XNAT_01/arc001/XNAT_01_01_MR_01/SCANS/1/DICOM/XNAT_01_01.MR.XNAT_01.1.24.20210315.nlvlu4.dcm": "/data/xnat/cache/XNAT_01/XNAT_E00001/1675460154345/2/XNAT_01_01.MR.XNAT_01.1.24.20210315.nlvlu4.dcm",
    "/data/xnat/archive/XNAT_01/arc001/XNAT_01_01_MR_01/SCANS/1/DICOM/XNAT_01_01.MR.XNAT_01.1.29.20210315.noiyo6.dcm": "/data/xnat/cache/XNAT_01/XNAT_E00001/1675460154345/2/XNAT_01_01.MR.XNAT_01.1.29.20210315.noiyo6.dcm"
  },
  "totalMovedFiles": 2,
  "totalRemovedFiles": 5,
  "totalFileErrors": 0
}
# curl -X DELETE http://localhost:8080/xapi/resources/survey/resource/2/report
# curl http://localhost:8080/xapi/resources/survey/resource/2/report
{
  "resourceSurveyRequestId": 1,
  "surveyDate": 1675460154441,
  "totalEntries": 12,
  "totalUids": 9,
  "totalBadFiles": 0,
  "totalMismatchedFiles": 2,
  "totalDuplicates": 3
}
# curl http://localhost:8080/xapi/resources/mitigate/resource/2/report
{
  "resourceSurveyRequestId": 1,
  "cachePath": "file:///data/xnat/cache/XNAT_01/XNAT_E00001/1675460154345/2/",
  "totalMovedFiles": 2,
  "totalRemovedFiles": 5,
  "totalFileErrors": 0
}



JavaScript errors detected

Please note, these errors can depend on your browser setup.

If this problem persists, please contact our support.

Contact Support Close