Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
test: introduce new StorageITRunner (#1785)
## New StorageITRunner Add two new JUnit runner to provide support for Cross Running test in a class across multiple backends and transports. When a class is annotated `@CrossRun(transports = {HTTP, GRPC}, backends = {PROD, TEST_BENCH})` each test defined in the class will be ran four times, once for each pair from the permutation. Each permutation will add a new entry to the test suite of the form `[Transport][Backend]`. This name form can be thought of as keys to access a location in an adjacency matrix. If only a single backed is needed `@SingleBackend()` can be used to denote the backend. If a class is annotated `@ParallelFriendly` the test suite may attempt to run the tests in parallel. Currently parallelism factor is set to 2 * core count. ### Field `@Inject`ion When defining the test several "globally scoped" items are available for injection to the test class by annotating a public non-final field with `@Inject`. Instances of Storage, along with BucketInfo, TestBench, etc. are registered as injectable values. If a field isn't able to be injected an InitializationError will be raised with details as to why. ### `CrossRun.Ignore` & `CrossRun.Exclude` Not all tests in a class will apply to all intersections, to allow filtering out tests for these cases there are two new annotations which can be used. If a test is both Ignored and Excluded, the Ignore will take precedence. `@Ignore`d functions similar to the standard `@org.junit.Ignore` where the test will be loaded but not ran. `@Exclude`d tests will not be loaded into the suite at all, thereby not being reported at all. ### `@Parameterized` running When a class is annotated `@Parameterized(ParameterProvider)` in addition to and Cross Run previously resolved, each parameter will result in another dimension added to the suite. The name of the suite will be extended with another dimension below any cross name, for example `[HTTP][PROD][myParam]`. ### Runner Registry A new class `Registry` has been added which is responsible for managing "globally scoped" instances of resources associated with running integration tests. All injectable instances are registered with the Registry and will be lazily initialized, reused and cleaned up. Instances like Storage clients, Backend Buckets, standard objects etc are all managed per backend. When a BucketInfo is loaded into the registry it's name is also made available to the provider for Storage instances. If a test attempts to mutate the state of a bucket the operation will be interrupted with a vetoing exception causing the test to fail. ## Refactoring Done 1. TestBench is now a Registry Resource and shouldn't be used outside that context. Since testbench binds ports it is not a multi instance thing anyway. 2. Remove StorageFixture in favor of Registry Field Injection 3. Remove BucketFixture in favor of Registry Field Injection 4. All IT test now use either of the two new runners 5. Remove ParallelParameterized runner, instead annotate your class with `@ParallelFriendly` 6. Move bucket cleanup utility from BucketFixture to new class BucketCleaner * test: add annotations for new Integration tests runners * test: add new Registry subsystem for new Integration Runners * test: add new Integration Runners * chore: add toString to ChecksummedTestContent * test: migrate ITReadMaskTest to use new StorageITParamRunner Split enclosed classes into their own respective classes ITBlobReadMaskTest and ITBucketReadMaskTest * test: migrate ITGrpcTest to use new StorageITRunner * test: migrate ITHmacTest to use new StorageITRunner * test: migrate ITNotificationTest to use new StorageITRunner * test: migrate ITKmsTest to use new StorageITRunner * chore: remove legacy junit fixtures and runner * test: migrate ITStorageLifecycleTest to use new StorageITRunner * test: migrate ITRetryConformanceTest to use new StorageITParamRunner Refactor RetryTestFixture to extend TestWatcher to allow for manual registration * test: migrate ITBatchTest to use new StorageITRunner * test: migrate ITServiceAccountTest to use new StorageITRunner * test: migrate ITSignedUrlTest to use new StorageITRunner * test: migrate ITOptionRegressionTest to use new StorageITRunner * test: migrate ITWriteChannelConnectionPoolTest to use new StorageITRunner * test: migrate ITDownloadBlobWithoutAuth to use new StorageITRunner * test: migrate ITDownloadToTest to use new StorageITRunner * test: migrate ITBlobReadChannelTest to use new StorageITRunner * test: migrate ITBlobWriteChannelTest to use new StorageITRunner * test: migrate ITObjectChecksumSupportTest to use new StorageITParamRunner * test: migrate ITObjectTest to use new StorageITRunner * test: migrate ITBucketTest to use new StorageITRunner * test: migrate ITBucketLifecycleTest to use new StorageITRunner * test: migrate ITBucketLifecycleRulesTest to use new StorageITRunner * test: migrate ITAccessTest to use new StorageITRunner * chore: consolidate StorageITParamRunner and StorageSuite into StorageITRunner
- Loading branch information