Introduction and Rationale
A test run status field can be helpful for understanding how far a test run has progressed.
Here are the valid statuses:
Created |
This is the state a test run is in by default, before any results have been submitted for it. Any test run that is in the Created state for 24 hours without test results being submitted is eligible to be automatically deleted by a sweeper process. |
Active |
Once one or more results are submitted to a test run in the Created state, the status should change to submitting. |
Aborted |
A test run was abandoned. This state could be set manually. |
Dormant |
This state should be set automatically after a minimum of one hour has passed without any result submitted for a test run. If a test result is submitted while in this state, the state should revert to Active. |
Complete |
The test run has completed, no more results will be submitted for it. Should an exception be thrown if attempting to submit to a Complete test run? |
API Changes
TestRunStatus Object
A cuanto.api.TestRunStatus object should be created with a single String field named "status". It should have a public getter and a package private setter. The class should have final static fields that are instances of TestRunStatus set to each of the status values specified above.
A public static TestRunStatus valueOf(String) method should also be implemented, which if called should instantiate a new TestRunStatus with the specified status value (if there is a case-insensitive match). If the status value is unknown, a fault should be thrown: "Unknown status value: {status}". See the cuanto.api.TestResult or cuanto.api.AnalysisState classes as an example, but the TestRunStatus shouldn't expose a method for setting arbitrary status values.
TestRun status Field
The cuanto.api.TestRun object should have the following field and methods added:
CuantoConnector.setTestRunStatus()
A method should be provided to allow API users to set the status of a test run:
CuantoAntTask
testrun Attribute
A new attribute named "testrun" will be added to the CuantoAntTask. The value for testrun is expected to be the ID of an existing TestRun on the Cuanto server. When submitting test result files, if the testrun attribute is specified, the results will be submitted to the TestRun with the corresponding ID, otherwise a new TestRun is created (which the current behavior).
keepactive Attribute
The CuantoAntTask should support a new attribute named "keepactive". It's default value is "false". When the value is false (ignoring case), a TestRun should be marked with the status "Complete" after submitting TestRuns. If the value is "true" (again ignoring case), the TestRun should remain in the Active state after submitting TestRuns.
status Action and Attribute
The default action (and currently only action) for the Cuanto ant task is "submit". A new action "status" will be introduced, so that if an attribute of action="status" is specified, the status attribute will be used to set the TestRun status for the TestRun referred to by the testrun attribute. For example:
If nested property or link nodes are specified when action="status", the provided property and/or link values should be set.
If the TestRun referred to by the ID does not exist, a fault is thrown: "Test Run {id} was not found".
If the status attribute does not case-insensitively match one of the known TestRunStatus values, a fault is thrown: "TestRunStatus {attribute value} is not a known status value."
If action="status", but nested fileset nodes or a resultFile attribute are specified, a fault is thrown: "No files may be submitted with a status change."
createTestRun Action
A new action "createTestRun" will be introduced along with a testrunprop attribute. When this action is specified, a TestRun for the specified testProject should be created and the property named in the testrunprop attribute should be set to the ID value of the created TestRun.
In the example above, the property "testRunId" will be set to the ID of the newly-created TestRun.
If nested property or link nodes are specified when action="status", the provided property and/or link values should be set.
If action="createTestRun", and nested fileset nodes or a resultFile attribute are specified, this is functionally equivalent to action="submit" with keepactive="true", with the testrunprop value being set to the new TestRun ID.
submit Action
The existing "submit" action should also be modified to recognize and set the testrunprop attribute if specified.
UI changes
Add a sortable status column to test run history, visible by default, between the TestRun and Tests columns.