Add compatibility tests to rest resources artifact #78534
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Pinging @elastic/es-delivery (Team:Delivery) |
jakelandis
requested changes
Oct 1, 2021
There was a problem hiding this comment.
compatibleApi will also need to be included. We don't transform the API, but we do have 2 different sets of API's on the classpath for compatiblitly testing. One directly from 7.x , and another from the yamlRestCompatTest sourceset (soon to be renamed) . For example, https://github.com/elastic/elasticsearch/tree/master/rest-api-spec/src/yamlRestCompatTest/resources/rest-api-spec/api is only found in master.
|
Good catch, Jake! I've added the compatibility specs into a separate subfolder |
|
@elasticmachine update branch |
jakelandis
added a commit
that referenced
this pull request
Oct 25, 2021
… qa module (#79501) This commit moves where we the _xpack/* REST compatible tests are executed. The _xpack/* prefix was deprecated in 7x and removed in 8.x via #35958. However, when deprecated the paths were also removed from the API spec in 7x. This means that these paths are not available to test with the YAML test framework. As part of the work to support REST API compatibility for 8.x these _xpack prefix paths were re-introduced when using compatibility. These paths were tested via the primary x-pack YAML rest tests using custom specs and transformations. These custom specs are needed to allow the testing framing work to use the _xpack/* paths and the transformations tell the tests to use the _xpack prefix. To avoid re-introducing these prefix paths in the 7.x spec, custom specs for the tests are used. This worked well for testing on the server, but after introducing the compatibility tests via the rest-resources-zip (#78534) the transformed tests in that zip requires those custom specs to execute properly. Since these are re-introduced just for testing we do not want to add these to the zip file of specs. To allow the xpack prefix to continue to be tested AND allow the tests to execute as-is via the specs/tests in the rest-resource-zip BUT do not expose the need for custom specs in the zip file ; these tests have been moved to their own qa module. As an implementation detail, the tests in the new qa module are transformed twice. Once as determined by the main x-pack transforms for things like compatible type support, and then once again solely for the purpose for testing the xpack prefix.
lockewritesdocs
pushed a commit
to lockewritesdocs/elasticsearch
that referenced
this pull request
Oct 28, 2021
… qa module (elastic#79501) This commit moves where we the _xpack/* REST compatible tests are executed. The _xpack/* prefix was deprecated in 7x and removed in 8.x via elastic#35958. However, when deprecated the paths were also removed from the API spec in 7x. This means that these paths are not available to test with the YAML test framework. As part of the work to support REST API compatibility for 8.x these _xpack prefix paths were re-introduced when using compatibility. These paths were tested via the primary x-pack YAML rest tests using custom specs and transformations. These custom specs are needed to allow the testing framing work to use the _xpack/* paths and the transformations tell the tests to use the _xpack prefix. To avoid re-introducing these prefix paths in the 7.x spec, custom specs for the tests are used. This worked well for testing on the server, but after introducing the compatibility tests via the rest-resources-zip (elastic#78534) the transformed tests in that zip requires those custom specs to execute properly. Since these are re-introduced just for testing we do not want to add these to the zip file of specs. To allow the xpack prefix to continue to be tested AND allow the tests to execute as-is via the specs/tests in the rest-resource-zip BUT do not expose the need for custom specs in the zip file ; these tests have been moved to their own qa module. As an implementation detail, the tests in the new qa module are transformed twice. Once as determined by the main x-pack transforms for things like compatible type support, and then once again solely for the purpose for testing the xpack prefix.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Include transformed compatibility tests in our
rest-resources-zipartifact so that client testing can verify REST compatibility as well.Core and x-pack tests will live in
rest-api-spec/compatTest/freeandrest-api-spec/compatTest/platinumrespectively.cc @philkra