open source imaging informatics
open source imaging informatics
If you want XNAT to execute your docker image, you will need a Command. The Command is a collection of properties that describe your docker image, and which XNAT can read to understand what your image is and how to run it:
{
"name": "",
"label": "",
"description": "",
"version": "",
"schema-version": "1.0",
"type": "docker",
"info-url": "",
"image": "",
"index": "", // only valid for docker images
"hash": "", // only valid for docker images
"working-directory": "",
"command-line": "",
"reserve-memory": "",
"limit-memory": "",
"limit-cpu": "",
"override-entrypoint": false,
"runtime": "nvidia",
"ipc-mode": "host",
"shm-size": "1073741824",
"network": "xnat-services",
"container-name": "",
"container-labels": {},
"generic-resources": {},
"ulimits": {},
"mounts": [
{
"name": "",
"writable": false,
"path": ""
}
],
"environment-variables": {
"envName1": "envVal1",
"envName2": "#inputReplacementKey#"
},
"ports": {
"80": "8080",
"22": "52222"
}, // "ports" only valid for docker images
"inputs": [
{
"name": "",
"description": "",
"type": "",
"required": true,
"matcher": "",
"default-value": "",
"replacement-key": "",
"command-line-flag": "",
"command-line-separator": "",
"true-value": "",
"false-value": ""
}
],
"outputs": [
{
"name": "",
"description": "",
"required": true,
"mount": "",
"glob": ""
}
],
"secrets": [
{
"source": {"type": "", "identifier": "", ...},
"destination": {"type": "", "identifier": "", ...}
}
],
"xnat": [
{
"name": "",
"description": "",
"contexts": [""],
"external-inputs": [
{
"name": "",
"description": "",
"type": "",
"matcher": "",
"default-value": "",
"user-settable": true,
"provides-value-for-command-input": "",
"provides-files-for-command-mount": "",
"via-setup-command": "repo/image:version[:commandname]"
}
],
"derived-inputs": [
{
"name": "",
"description": "",
"type": "",
"matcher": "",
"default-value": "",
"user-settable": true,
"provides-value-for-command-input": "",
"provides-files-for-command-mount": "",
"via-setup-command": "repo/image:version[:commandname]",
"derived-from-wrapper-input": "",
"derived-from-xnat-object-property": ""
}
],
"output-handlers": [
{
"name": "",
"type": "",
"accepts-command-output": "",
"via-wrapup-command": "",
"as-a-child-of-wrapper-input": "",
"label": ""
}
]
}
]
}1.0. All commands should use "schema-version": "1.0".true/false). If false, the entrypoint will not be touched; any entrypoint that the underlying image has will be used. If true, the entrypoint will be overridden with an empty string.67108864 (64mb). The implementation of any specified shm-size request creates a temporary volume mounted to the containers /dev/shm path."container_port": "host_port". Keys and values can be templates.true. Some examples: “true”, “T”, “Y”, “1”, “–a-flag”. Default: “true”.false. Some examples: “false”, “F”, “N”, “0”, “–some-other-flag”. Default: “false”."file") The name of a mount—which must be defined in this command—into which container service will ."glob" is blank, then all files found at relative path "path" within the mount will be uploaded.Session, and this input is a Scan, we can make sure that this input only matches scans with a DICOM resource by setting the matcher to "DICOM" in @.resources[*].label, or only matches scans of a certain type by setting the matcher to @.scan-type == "MPRAGE".repo/image:version[:commandname] where the commandname is optional). See the page on Setup Commands for more.Session, and this input is a Scan, we can make sure that this input only matches scans with a DICOM resource by setting the matcher to "DICOM" in @.resources[*].label, or only matches scans of a certain type by setting the matcher to @.scan-type == "MPRAGE".repo/image:version[:commandname] where the commandname is optional). See the page on Setup Commands for more."Resource" is accepted.repo/image:version[:commandname] where the commandname is optional). See the page on Wrapup Commands for more.Mounts are the way to get files into and back out of your container. If you need the container service to stage files from XNAT into your container, or you want container service to upload any files your container creates back into XNAT, then you need to use mounts.
A mount only has a few properties; we will summarize those here, and go into more detail on each below. When you create a mount, you give it a…
"name" so that you can refer to it in other parts of the command"path", which is the path at which it will be found inside the container. You don’t need to specify the path outside the container; we will manage that for you."writable" boolean, which specifies whether the mount is read-only or writable.A little more detail on the writable flag: Mounts that are referenced by command outputs are always writable. If a mount is not referenced by any output—i.e. it is only used to mount input files—it will typically be read-only. This is so that files can be mounted directly from the XNAT archive, which should never be written to directly. However, this means that if a container does try to write anything to a location that is a read-only mount, the container will fail with a runtime error. If you know that a container will write to a location where you want to place input files, you can explicitly set an input mount to “writable=true”. This means that before the container is launched, any input files will be copied out of the archive into a writable directory which will then be mounted.
A mount can be used for both an input and an output. That means the input files will be copied into the directory before launch, and the same directory will be searched for output files upon container completion. If you aren’t careful, the input files will be re-uploaded along with the output files. The output.path and output.glob properties can be carefully crafted to avoid this effect.
Define a wrapper input for the kind of object that will provide the files you need. Typically this will be a Resource, but that resource input will almost certainly need to be derived from some higher-level input object like a Session or a Scan. On the input that provides the files, set the mount’s name as the value of the "provides-files-for-command-mount" property.
More info to come.
More info to come. # Inputs
What information does your container need?
Command Inputs allow you define what information needs to be provided in order to launch your container: files, command-line arguments, environment variables, etc. If you need some bit of information to be variable and set at launch time, it should be a command input.
How do you get information from XNAT into your Command’s inputs?
At the level of command inputs, it does not necessarily matter where the runtime values come from. They could in principle come from some XNAT object, or from a user’s input, or from some other contextual information.
The XNAT inputs—i.e. the command.xnat.external-inputs and command.xnat.derived-inputs objects—give you a way to tell XNAT how to use XNAT objects—their properties, files, and hierarchical relationships—to provide values to the Command inputs.
If you have a derived wrapper input, it can derive its value from a parent wrapper input.
Info dump:
external input or another derived input. But it must be an input on the same wrapper, not on the command or a different wrapper.type of the parent input must be one of the XNAT object types: Project, Subject, Session, Scan, Assessor, or Resource.type of the child input is also an XNAT type, it must obey the usual XNAT hierarchy.Project -> SubjectSubject -> SessionSession -> ScanSession -> AssessorProject, Subject, Session, Scan, or Assessor -> ResourceSubject parents cannot have children of type Directory. All the other XNAT types can.string or number—you can derive the value by specifying the property on the parent object that you want to pull out using the input’s derived-from-xnat-object-property field. For a list of the properties you can use, see XNAT object properties.A list of all the properties for all the XNAT object types:
ProjectidlabelxsiTypeuridirectorySubjectidlabelxsiTypeuriproject-idSessionidlabelxsiTypeuridirectoryproject-idScanidlabelxsiTypeuridirectoryinteger-idscan-typeAssessoridlabelxsiTypeuridirectoryResourceidlabelxsiTypeuridirectoryinteger-idThis initial list is small. There are many more properties that could be added over time.
Note: The uri property is the REST-style identifier of the object. For instance, a project’s URI will be /projects/{projectId}, subject’s may be /subjects/{subjectId} or /projects/{project}/subjects/{subjectLabel}, etc.
When a user brings up a user interface to launch a container using a Command, they will see some appropriate interface element for each Command input and external XNAT Wrapper input. These interface elements give the user a chance to change the input values before launching the container. If, for some reason, this is not appropriate for a particular input, and users should not be allowed to change the default value, the input can be defined with the property user-settable=false. When the user brings up an interface to launch a container, they will see the input and its value but will not be able to change it.
The more common use-case for this parameter is in XNAT-project-specific settings. Perhaps all the inputs in a Command definition have user-settable=true, but a project owner may choose to configure particular inputs on the Command to have user-settable=false. In this way, they ensure that all containers on their project will be executed with the chosen input value.
Command input types: string, boolean, number
XNAT Wrapper input types: string, boolean, number, Directory, File, File[], Project, Subject, Session, Scan, Assessor, Resource, Config
Note: Some inputs are possible to make but aren’t currently functional.
number inputs should be validated that they are within a range, or that they are in fact numbers and not strings. But currently they are not. They are treated the same as string inputs.Directory, File, and File[] inputs are possible to make, but the files the refer to cannot be mounted.Config inputs are intended to pull their values from the XNAT Config Service. They currently do not.Note A Project input is kind of a bad idea to use on its own. (Probably Subject too.) The performance is really very bad. The entire project, all its subjects, all their sessions, all their scans, and all the resources and files on every one of those things all get loaded into memory and stored in an object. It takes a long time!
If you want to use a Project input, you should set the property "load-children" on the input to false. That will prevent the entire project from being loaded. But, on the other hand, it will prevent you from deriving any child inputs from the Project input. So only use if you need to derive some property from the Project itself.
If you want your container to produce files that get imported into XNAT, you need to define one or more output objects. You need to define where the files can be found (which output mount they are in, and what is the path within that mount) and where the new to-be-created object will live within XNAT. For the latter, you provide the name of an input, which must be an XNAT object type; the output files will be a new child of that parent input.
More info to come.
When you define a Command, you can leave many of the values as “templates”. These templates are placeholder strings, also known as “replacement keys”, which tell the container service “When you launch a container from this Command, you will have values for your inputs; I want you to use one of those values here.”
Lots of properties in the Command can use template strings:
command-line - See a simple example in the Hello world example, but also see the caveats in the complex example below.environment-variables - Both the environment variable name and value can be templates.ports - Both the container port and host port can be templates.output.path - The relative path within a mount at which output files can be found.JSONPath is an expression syntax for searching through a JSON object, similar to the way you can use XPath to search through an XML document. The syntax and operators are documented here at the source repository: https://github.com/jayway/JsonPath.
You can use JSONPath strings as values in several Command fields. When the Command is resolved before it is used to launch a container, those JSONPath strings will be replaced with whatever values they refer to. This is similar to the way you can use a template string in the Command definition, which gets replaced by a value when the Command is resolved. In fact, anywhere in the Command that you can use a template string, you can also use a JSONPath expression.
When you use a JSONPath expression as a value, you must surround it with carets (^...^) to signal to the Container Service that it needs to invoke the JSONPath interpreter.
JSONPath expressions start at the root of the Command, which is referred to as “$”. For instance, you could get the path of a particular mount with name foo using the JSONPath expression
$.mounts[?(@.name = "foo")].pathAgain, remember to surround the JSONPath expression with carets (^) to signal that it should be evaluated.
You can search through the XNAT Command wrappers in a JSONPath expression just like any other part of the Command: ^$.xnat[?(@.name = "wrapper-name")].... To find in the list the particular wrapper that is being evaluated, you would have to hard-code the wrapper name into such an expression, because there is no easy way to find the name at runtime. However, theIre is a shorthand way to search through the particular command wrapper that is being evaulated. Instead of surrounding the JSONPath expression with carets (^$.thing1.thing2...^), surround it with carets and the word "wrapper" as such: ^wrapper:$.thing3.thing4^. Now the root of the JSONPath expression ($) will refer to the Wrapper, not the Command.
The input.matcher property uses a special subset of the JSONPath sytax called a “filter”. In JSONPath expressions that can return a list, you can use one of these expressions to filter out non-matching elements. As a simple example, let’s say I have the JSON
{
"ice-cream": [
{"name": "Strawberry", "flavor": "yummy"},
{"name": "Spaghetti", "flavor": "yucky"}
]
}Say I want just the "name"s. The JSONPath expression “$.ice-cream[*].name” would give back a list:
["Strawberry", "Spaghetti"]But if I filter that list, I can get back just the elements that are "yummy":
$.ice-cream[?(@.flavor == "yummy")].name
["Strawberry"]See the source documentation for the filter syntax. The filter is the part inside the parentheses. In the filter expression, we use the character "@" to refer to the element that we are checking.
We can use these filter expressions as the value of the input.matcher. When the Command is being resolved, a potential input value must not be rejected by the filter, i.e. it must match the filter condition, to be assigned to the input value. In that way, if an input can receive its value from a parent input which has many children of a certain type, we can select just the one child that we want to be the value for our input.
For example, say we have a Scan input, with a Session input as “parent”. That Session may have many child scans, but we can use the "matcher" to select one that has a certain scantype by setting it to a filter expression, something like
@.scan-type in ["T1", "MPRAGE"]Here’s another example from xnat/dcm2niix-scan.
"inputs": [
{
"name": "scan",
"description": "Input scan",
"type": "Scan",
"required": true,
"matcher": "DICOM in @.resources[*].label"
},
{
"name": "scan-dicoms",
"description": "The scans dicom resource",
"type": "Resource",
"parent": "scan",
"matcher": "@.label == DICOM"
}
]This Command expects that it will be given a scan as an input, but it wants to run a matcher anyway just to be sure the scan has a DICOM resource ("matcher": "'DICOM' in @.resources[*].label"). The second input is a child to the first, and it matches the scan’s DICOM resource ("matcher": "@.label == 'DICOM'").
How do you take in one or more XNAT objects and use their properties and files to launch your Command?
More info to come.
Let’s go over a simple use case. You have a string input, and you want whatever value gets put into that input to be printed to the screen. We could define a command like this:
{
"name": "hello-world",
"description": "Prints a string to stdout",
"type": "docker",
"image": "busybox:latest",
"command-line": "echo #my_cool_input#",
"inputs": [
{
"name": "my_cool_input",
"description": "The string that will be printed",
"type": "string",
"default-value": "Hello world"
}
]
}In this example, you have one input, which is a string, and is named "my_cool_input". The value of command-line is what will be executed inside the container when it is launched. However, before container service launches that container, it will look at that command-line value and see that it recognizes the string #my_cool_input# as a replacement key for the input with name my_cool_input. Each input has a replacement key, which is by default just the input’s name surrounded by hash marks (#). But you can customize an input’s replacement key by setting the replacement-key property on the input.
When the container-service sees a replacement key, it will replace it with the input’s value. In this case, that means #my_cool_input# will be replaced by Hello world. (Well, technically, that is only partially correct. See the more complex example below.)
What I said above is that when the container service looks at one of the strings in the list above and sees an input’s replacement key, it fills in the input’s value. That is correct most of the time. However, under certain circumstances tweaks are made to the input’s value before it is inserted in place of the replacement key.
boolean will have their value (true or false) mapped to a string. By default these strings are "true" and "false", but you can customize these values by setting input.true-value and input.false-value respectively. Some example true/false string pairs could be 1/0, Y/N, True/False, etc. Or, if you choose, you could set input.true-value to some flag and set input.false-value to a blank string, or vice versa. In that way, you could set a flag on the command line only if a certain input is true but do not set any flag if the input is false; for example, you could have the input named "recursive" have false-value="" and true-value="-r".input.command-line-flag. So if my input foo has value bar and command-line-flag="--my-input-value", the flag and value will be smashed together and will replace the replacement key as --my-input-value bar. If you want the flag and value to be joined by something other than a space, for instance an equals sign, you can set that as input.command-line-separator.Let’s see an example Command with these properties set.
{
"name": "complex-example",
"description": "An example Command with more complex inputs",
"type": "docker",
"image": "busybox:latest",
"command-line": "/run/my_script.sh [THE_BOOLEAN] a-string",
"environment-variables": {
"STR_VAL": "a-string",
"BOOL_VAL": "[THE_BOOLEAN]"
},
"inputs": [
{
"name": "the_boolean",
"description": "A boolean input",
"type": "boolean",
"default-value": false,
"replacement-key": "[THE_BOOLEAN]",
"true-value": "T",
"false-value": "F",
"command-line-flag": "--bool",
"command-line-separator": "="
},
{
"name": "the_string",
"description": "A string input",
"type": "string",
"replacement-key": "a-string",
"command-line-flag": "--str",
}
]
}Let’s say this Command is used to launch a container, and let’s say that no input values are passed in (i.e. only the defaults will be used). Then the_boolean will have value false and the_string will have no value. The command line would become "/run/my_script.sh --bool=F ". The environment variables would be STR_VAL= and BOOL_VAL=F.
Now let’s say we launch a container from the Command again, and this time we pass in input values the_boolean=true and the_string=Hey. Then the command line would become "/run/my_script.sh --bool=T --str Hey"
XNAT is an open source project produced by NRG at the Washington University School of Medicine | NRG Home
Contributions to the XNAT Documentation site are licensed under a Creative Commons Attribution 3.0 Unported License. 