Introduction
This specification is for adding test tags (hereafter known as "tags" or "tag") to Cuanto. Tags are intended to correspond with TestNG's groups feature, although Cuanto will present some additional functionality that will not be limited to TestNG. In TestNG, a test can be tagged with one or more groups, and tests can be executed based on the groups to which they belong.
Object Modeling and Basic Tag Functionality
In Cuanto, a test case is represented by the TestCase domain object, and each execution of that TestCase is a TestOutcome object. A collection of TestOutcomes executed together are considered a TestRun. A Project domain object represents a particular set of test cases and their associated TestOutcomes.
Each TestOutcome will have a List of tags associated with it, as will each TestRun.
When a TestOutcome is submitted to Cuanto that has tags associated with it, those tags will be persisted with the TestOutcome. If the TestOutcome is part of a TestRun, the TestRun will have that tag copied to it as well.
User Interface
Test Run Analysis Table
Tag buttons
In the Test Run Analysis view, if any tags are defined for the TestOutcomes of the TestRun being viewed, a row of buttons will be placed underneath the existing filtering options at the top of the analysis table. The button row will have the label "Tags:". The first button will be labeled "All" and adjacent to the All button there will be one button rendered per tag that is associated with the TestRun, with the tag buttons listed in alphabetical order.
(Note the label in the above screenshot is "Groups", an earlier name for Tags. The button will be labelled "Tags" in the final implementation).
The tag buttons can be selected or deselected – in other words, clicking on a button will display it as on and clicking on a button that is already selected will turn it off. When a button is on, the corresponding tags will be displayed in the table. Multiple buttons may be in the on state so that the table can be narrowed to display multiple tags.
The All button will cause all buttons to be selected. If all buttons are already selected, the All button will de-select all buttons. When all buttons are selected, the analysis table will display all TestOutcomes with tags that meet the table's other filter requirements. If any TestOutcomes are not associated with tags, those TestOutcomes will not be displayed. When all buttons are de-selected, all TestOutcomes that meet the table's filter requirements will be displayed, regardless of if they are associated with any tags.
The table will also have an optional Tags column (not displayed by default) which gives a comma-separated list of the tags applied to that TestOutcome. This column will be non-editable and sortable, and the tags will appear in alphabetical order.
Tags column
An optional "Tags" column will be available for the analysis table. It's contents will be a comma-separated list of tags for the current test outcome. It will not be sortable and it will be hidden by default.
Tag Summary Table
The Test Run Summary tab should display a table of tags and their test totals:
Tag |
Passing |
Failed |
Skipped |
Total |
|---|---|---|---|---|
BVT |
60 |
0 |
0 |
60 |
Client |
80 |
6 |
0 |
86 |
Database |
60 |
4 |
0 |
64 |
Graphics |
33 |
8 |
2 |
43 |
Server |
140 |
12 |
10 |
162 |
The table should be sortable by clicking on the column headings.
Manual Tag Support
 | This feature is presently unimplemented. |
This feature can be implemented subsequent to the other functionality.
In this scenario, TestCases also have tags. Project will have an applyTags boolean value which defaults to true.
For each tag that is submitted with a TestOutcome, if the TestOutcome's project applyTags value is true, that tag will be copied to the corresponding TestCase if that TestCase does not already have that tag associated with it.
Tags may be manually added to or removed from a TestCase or TestOutcome via the UI. When a tag is added to a TestCase, the user will be prompted to decide if the tag should be added to all TestOutcomes for this test case or if it should just be added to any future TestOutcomes submitted.
If a TestOutcome is submitted that does not have tags associated with it and the project's applyTags value is true, then any tags associated with that TestOutcome's TestCase will be automatically added to the TestOutcome. This will allow test types that do not have native tag support, like JUnit, to still take advantage of tags. A user can classify each TestCase one time, and then that TestCase's TestOutcomes will automatically get assigned the appropriate tags.