Stream Smoke Test

Proton smoke tests structure

Proton smoke tests are located in the folder: proton/tests/stream/test_stream_smoke, overall structure lays in:

proton    
│
└───tests
    │
    └───stream
        │   ci_runner.py
        │   ...
        └───helpers
        │   │   rockets.py 
        │   │   ... 
        │        
        └───test_stream_smoke
            │   test_query_smoke.py
            │   0000_smoke_case.json
            │   0001_view_case.json
            │   ...
            │   0014_versioned_kv_case.json
            │   ...
            └───configs
                │   config.json
                │   docker-compose.yaml
    

Proton stream smoke test leverage Pytest as main test automation framework. The main components includes:

1. ci_runner.py

The main entrance of smoke tests, it will scan all the folders specified in --test_folders for the test suite files (end with .json or .yaml or .yml such as 0000_smoke_case.json, ..., 0099_fixed_issues.json) in the stream folder and run tests based on the command parameters, main parameters are:

• --local: specify if ci_runner runs the tests in local mode, ci_runner.py will assume proton env is ready and Timeplus edition of clickhouse python driver is installed therefore won't use docker-compose to create proton environments or install proton python client. If no --local specified ci_runner will use docker-compose to create several proton dockers and try to install the Timeplus edition of clickhouse python driver and distribute test suite running on proton containers evenly.
• --settings: nativelog or redp, specify which proton stream store settings will be tested. Proton has 2 types of stream store, one is to use own nativelog stream store and another is to use external kafka or redpanda as stream store. Under nativelog setting ci_runner will create nativelog stream store based proton container deployments and under redp setting ci_runner will create redpanda stream store based proton deployments. By specifying --settings=nativelog or --settings=redp or --settings=nativelog,redp, you could let ci_runner to proton container deployments for nativelog stream store setting or redpanda stream store setting or both and run all test suites on proton deployments in parallel.
• --test_folders: specify which folders ci_runner.py should scan for test suite files
• --debug: means open all the debug log, by default only info log is printed if no --debug specified when running ci_runner.py
• --no_retry:
• --test_suites: specify which tests suites will be run, if no test_suties specified, ci_runner.py will run all the test suites
• --id: specify ids of test cases to be run, usually the test_suites need to be specified with id together, otherwise ci_runner.py will try to run all test cases with the id in all test suites that's meaningless
• --help: show this help message and exit.

how to start ci_runner.py

Install python dependencies, get the requirements.txt from tests/stream/helpers and run pip -r requirements.txt

Enter the test/stream folder：

• start ci_runner in local mode： run python3 ./ci_runner.py --local --debug, it will run all the test suites on the nativelog settings. Note that proton-server must be started locally before running ci_runner.py.

2. helpers

The main lib proton smoke tests uses is rockets.py

3. test_stream_smoke

All the proton functional smoke tests.

• configs: config.json contains all the test environment configuration information for the settings including rest_settings which is about URIs of rest endpoints, proton informations (proton_server, proton_server_native_port, proton_admin, proton_server_contain_name) needed by the test scripts. For example there is a json object of "redp" in configs.json, when settings=redp