[Investigation] Benchmark ES Archiver Load Action on Serverless

[wayneseymour]

Used to run against local, ess and serverless to benchmark es archiver.
Currently, it's only focused on the load action.

The serverless and local / ess configs are in the disabled list, so these will no longer be run in Buildkite, for now.

We run the benchmark against the three environs, mostly focused on Serverless, to learn
whether a performance improvement has actually improved performance.

Then, we take the markdown table results from the log or cli and record them, for comparison.

To Run

Bash Shell function to run locally

localFTR() {
  declare -i CNT=${1:-1}
  echo " λsh CNT: ${CNT}"
  local config=${2:-test/api_integration/apis/local_and_ess_is_es_archiver_slow/config.ts}

  set -x

  LOG_FILE_NAME="LOCAL-es_archiver_load_times_log.txt" \
    CI_STATS_DISABLED=true \
    LOOP_LIMIT="${CNT}" \
    node scripts/functional_test_runner \
    --config="${config}" --verbose

  set +x
}

Function to run against ESS

essFTR() {
  declare -i CNT=${1:-50}
  local PASS="SUPER-DUPER-PASSWORD"
  local config=${2:-test/api_integration/apis/local_and_ess_is_es_archiver_slow/config.ts}

  RUNTIME_ENV="ESS" \
    LOG_FILE_NAME="${RUNTIME_ENV}-es_archiver_load_times_log.txt" \
    CI_STATS_DISABLED=true \
    LOOP_LIMIT="${CNT}" \
    TEST_CLOUD=1 \
    ES_SECURITY_ENABLED=1 \
    NODE_TLS_REJECT_UNAUTHORIZED=0 \
    TEST_ES_URL="https://elastic:${PASS}@esarchiver-slow-ess.es.eu-west-3.aws.elastic-cloud.com:443" \
    TEST_KIBANA_URL="https://elastic:${PASS}@esarchiver-slow-ess.kb.eu-west-3.aws.elastic-cloud.com:9243" \
    node --no-warnings scripts/functional_test_runner \
    --config="${config}" \
    --es-version=8.8.1
}

Function to run against serverless

serverlessFTR() {
  declare -i CNT=${1:-50}
  echo "λsh CNT: ${CNT}"
  local PASS="SUPER-DUPER-PASSWORD"
  local config="x-pack/test_serverless/api_integration/test_suites/serverless_is_es_archiver_slow/config.ts"
  local LOG_FILE_NAME="SERVERLESS-es_archiver_load_times_log.txt"
  export LOG_FILE_NAME
  echo "λsh LOG_FILE_NAME: ${LOG_FILE_NAME}"

  CI_STATS_DISABLED=true \
    LOOP_LIMIT="${CNT}" \
    TEST_CLOUD=1 \
    ES_SECURITY_ENABLED=1 \
    NODE_TLS_REJECT_UNAUTHORIZED=0 \
    TEST_ES_URL="https://elastic:${PASS}@d9b5502e181f47cf9dacf93e8a8e5d95.es.eks-eu-west-1-qa-wda7.qa.cld.elstc.co:443" \
    TEST_KIBANA_URL="https://elastic:${PASS}@d9b5502e181f47cf9dacf93e8a8e5d95.kb.eks-eu-west-1-qa-wda7.qa.cld.elstc.co:443" \
    node --no-warnings scripts/functional_test_runner \
    --config="${config}" \
    --es-version=8.10.0-SNAPSHOT
}

The only difference really, is the config is a different file.

Resolves: https://github.com/elastic/appex-qa-team/issues/31

[kibana-ci]

💔 Build Failed

• Buildkite Build
• Commit: 2dd6ab6
• Interpreting CI Failures

Failed CI Steps

• Build API Docs
• Check Types

Metrics [docs]

✅ unchanged

History

• 💚 Build #143174 succeeded 8944cbd28f5cc4557688cf9ddfdcd234124372ea
• 💔 Build #143149 failed 4537baab4a41a895a840f9bf364b13134dfb7741
• 💚 Build #143104 succeeded 35f79cd6d567b66a665439857e6c8aee6ce5b8ca
• 💚 Build #142969 succeeded d8e6d1a92bff58a2e2a73a07e67eb3fc33ec5f7a

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @wayneseymour