Test control/specification via annotations

One significant difference between JUnit and TestNG is that JUnit will reinstantiate your test class for each @Test method, while TestNG only does so once per class. Therefore, when designing new tests in the XNAT REST test framework, you can safely include any instance variables which will be shared between the tests without needing to mark them as static . TestNG provides various annotations you can apply to methods to control when they execute. The recommended ones to use are:

  1. @BeforeClass : the method annotated will run once before any of the tests in the class are executed. This is especially useful for:
    • Creating a project shared by all of the tests
    • Setting site config values that are prerequisites for the tests. (Note that the test framework has a feature to automatically restore the previous site config after each class, so undoing your changes made to it should be unnecessary)
  2. @AfterClass : the method annotated will run once at the end of test execution in the class. This is mainly useful for cleaning up after the tests.
  3. @BeforeMethod  : the method annotated will run before each @Test method.
  4. @AfterMethod : the method annotated will run after each @Test method. 


In addition to the standard annotations provided by TestNG, nrg_test provides several additional annotations that may prove useful when designing new tests:

  1. Several annotations provide control over specifying which version(s) of XNAT are compatible with your test. The versions are specified as XnatVersion classes from the org.nrg.xnat.versions package. All of the version annotations can be specified at either the class level or individual @Test level. If the version of XNAT specified when running the tests is not compatible with annotated tests, those tests will be excluded.
    1. @AddedIn : takes a single version class as value. Any version of XNAT released after (or including) the specified version will be considered compatible.
    2. @DeprecatedIn : takes a single version class as value. Any version of XNAT released strictly before the specified version will be considered compatible.
    3. @DisallowXnatVersion : takes an array of version classes. Any version of XNAT other than the versions enumerated in the value will be considered compatible.
    4. @RequireXnatVersion : takes an array of version classes. Only the versions enumerated in the value will be considered compatible.
  2. @ExpectedFailure can be added to an individual test or an entire test class. The jiraIssue property on the annotation should be set to the issue key (e.g. XNAT-6687) of a ticket in the XNAT JIRA. The purpose of the annotation is to allow tests to be written for functionality that may not be functioning correctly. Instead of disabling the tests, the behavior of the test suite when encountering tests it expects to fail is instead configurable.
  3. @HardDependency/@SoftDependency : if your tests contain dependencies on one another, you can specify them with these annotations. For either annotation, the test framework will reorder test execution so that all tests will be executed after their prerequisite test(s). The difference between a "hard" and "soft" dependency is that a "soft" dependency will not actually require the prerequisite test(s) to have all passed successfully, unlike the "hard" variant. Note that test dependencies can only specify other methods within the same class.
  4. @SortLast is essentially a special case of @SoftDependency . It tells the test framework to sort a specific test method to the end of all of the tests in the class.
  5. @TestedApiSpec/@TestedApiSpecs : allow you to specify which REST endpoint(s) are being tested by your test. This information is passed along to the summary email sent upon test completion to make test failures more interpretable. A single @TestedApiSpec  annotation or a @TestedApiSpec annotation containing multiple @TestedApiSpec annotations can be added to a test to identify the endpoints. A complex example could be:

    @TestedApiSpecs({
        @TestedApiSpec(method = Method.POST, url = {
            "/xapi/archive/download",
            "/xapi/archive/downloadwithsize"
        }),
        @TestedApiSpec(method = Method.GET, url = {
            "/xapi/archive/download/{catalogId}",
            "/xapi/archive/download/{catalogId}/test",
            "/xapi/archive/download/{catalogId}/xml",
            "/xapi/archive/download/{catalogId}/zip"
        })
    })
    @Test
    ...
  6. @TestRequires is a multipurpose annotation that allows you to specify various requirements for your test at either the class or test level:
    1. dicomScp can be set to true to enforce a check that the tests have a valid DICOM SCP receiver to which they can send data.
    2. openXnat can be set to true to make the test framework disable the login requirement before executing the test.
    3. closedXnat can be set to true to make the test framework enable the login requirement before executing the test.
    4. users can be set to an integer n > 0  to declare a required number of "generic" accounts required to run the test(s). The framework will automatically create that many non-admin, enabled accounts before running the tests. User objects for these accounts can be obtained by calling getGenericUser() multiple (up to n) times.
    5. plugins can be set to a list of XNAT plugin IDs upon which the tests depend. If the required plugin(s) are not all detected on the tested XNAT instance, the tests will be skipped.
    6. data allows you to specify test data from the TestData enum to use within the tests. Specified data will be downloaded locally and cached for future usage.

$label.name