Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ __IMPORTANT NOTICE:__ The contents of this repository currectly reflect a __DRAF
1. EiffelTestCaseStartedEvent
1. EiffelTestCaseFinishedEvent
1. [EiffelTestSuiteStartedEvent](./eiffel-vocabulary/EiffelTestSuiteStartedEvent.md)
1. EiffelTestSuiteFinishedEvent
1. [EiffelTestSuiteFinishedEvent](./eiffel-vocabulary/EiffelTestSuiteFinishedEvent.md)
1. EiffelTestExecutionRecipeCollectionCreated
1. EiffelAnnouncementEvent
1. EiffelConfigurationChangedEvent
Expand Down
53 changes: 53 additions & 0 deletions eiffel-vocabulary/EiffelTestSuiteFinishedEvent.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
# EiffelTestSuiteFinishedEvent
The EiffelTestSuiteFinishedEvent declares that a previously started test suite (declared by [EiffelTestSuiteStartedEvent](./EiffelTestSuiteStartedEvent.md)) has finished and reports the outcome.

Note that while similar, the __data.outcome__ object is different from that of [EiffelActivityFinishedEvent](./EiffelActivityFinishedEvent.md). The outcome of the test suite reports not only the conclusion of the test suite execution - whether the tests were successfully executed - but also passes a verdict on the item or items under test. To highlight this conceptual difference, both __data.outcome.verdict__ and __data.outcome.conclusion__ are included.

## Data Members
### data.outcome
__Type:__ Object
__Required:__ No
__Description:__ The outcome of the test suite.

#### data.outcome.verdict
__Type:__ String
__Required:__ No
__Legal values:__ PASSED, FAILED, INCONCLUSIVE
__Description:__ A terse standardized verdict on the item or items under test. Unlike in [EiffelTestCaseFinishedEvent](./EiffelTestCaseFinishedEvent.md), this property is optional. It offers a method of summarizing the verdict of the test suite as a whole, but may be skipped.
PASSED signifies that the item or items under test successfully passed the test suite.
FAILED signifies that the item or items under test failed to pass the test suite.
INCONCLUSIVE signifies that the verdict of the test suite was inconclusive. This SHOULD be the case if __data.outcome.conclusion__ is not __SUCCESSFUL__, but may in combination with a __SUCCESSFUL__ conclusion be used to represent unreliability or flakiness.

#### data.outcome.conclusion
__Type:__ String
__Required:__ No
__Legal values:__ SUCCESSFUL, FAILED, ABORTED, TIMED_OUT, INCONCLUSIVE
__Description:__ A terse standardized conclusion of the test suite, designed to be machine readable. Unlike in [EiffelTestCaseFinishedEvent](./EiffelTestCaseFinishedEvent.md), this property is optional. It offers a method of summarizing the conclusion of the test suite as a whole, but may be skipped.
SUCCESSFUL signifies that the test suite was successfully concluded. Note that this does not imply that the item under test passed the tests.
FAILED signifies that the test suite could not be successfully executed. To exemplify, one or more tests failed to run due to required environments being unavailable.
ABORTED signifies that the test suite was aborted before it could be concluded.
TIMED_OUT signifies that the test suite did not conclude within the allowed time frame.
INCONCLUSIVE signifies that the outcome of the test suite could not be determined.

#### data.outcome.description
__Type:__ String
__Required:__ No
__Description:__ A verbose description of the test suite outcome, designed to provide human readers with further information.

### data.persistentLogs
__Type:__ Object[]
__Required:__ No
__Description:__ An array of persistent log files generated during execution.

#### data.persistentLogs.name
__Type:__ String
__Required:__ Yes
__Description:__ The name of the log file.

#### data.persistentLogs.uri
__Type:__ String
__Required:__ Yes
__Description:__ The URI at which the log can be retrieved.

## Examples
* [Simple example](../examples/events/EiffelTestSuiteFinishedEvent/simple.json)
34 changes: 34 additions & 0 deletions examples/events/EiffelTestSuiteFinishedEvent/simple.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
{
"meta": {
"type": "EiffelTestSuiteFinishedEvent",
"version": "1.0",
"time": 1234567890,
"id": "aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeee0",
"source": {
"domainId": "example.domain"
}
},
"data": {
"outcome": {
"verdict": "FAILED",
"outcome": "SUCCESSFUL",
"description": "16 test failures."
},
"persistentLogs": [
{
"name": "firstLog",
"uri": "http://myHost.com/firstLog"
},
{
"name": "otherLog",
"uri": "isbn:0-486-27557-4"
}
]
},
"links": [
{
"type": "TEST_SUITE_EXECUTION",
"target": "aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeee1"
}
]
}