Skip to content

Revert "Increase test memory limit"#13314

Merged
martint merged 1 commit intotrinodb:masterfrom
nineinchnick:revert-tests-mem
Jul 22, 2022
Merged

Revert "Increase test memory limit"#13314
martint merged 1 commit intotrinodb:masterfrom
nineinchnick:revert-tests-mem

Conversation

@nineinchnick
Copy link
Copy Markdown
Member

Description

This reverts commit da367a3. I noticed quite a few tests failing with the process being killed. This commit was merged on the 14th in #13014

This is a draft, because I want to see which job would fail so we can figure out how to deal with single tests, without increasing the mem limit for everything.

Some numbers:

    day     | num_runs | num_success | rate_success |                                                                                                                                           top5_failing_jobs                                                                                                                                           | num_all_runs | num_all_success | rate_all_success 
------------+----------+-------------+--------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------+-----------------+------------------
 2022-06-22 |       15 |           6 |          0.4 | [{1, Test Report}, {1, test (plugin/trino-delta-lake)}, {1, test (plugin/trino-redis)}, {2, test (plugin/trino-iceberg)}, {6, hive-tests (config-empty)}]                                                                                                                                             |           68 |              27 |              0.4 
 2022-06-23 |       10 |           2 |          0.2 | [{2, hive-tests (config-hdp3)}, {2, test (plugin/trino-hive)}, {2, test (testing/trino-tests)}, {3, Test Report}, {4, test (plugin/trino-iceberg)}]                                                                                                                                                   |           59 |              20 |             0.34 
 2022-06-24 |        8 |           4 |          0.5 | [{1, test (plugin/trino-pinot)}, {4, hive-tests (config-empty)}]                                                                                                                                                                                                                                      |           78 |              23 |             0.29 
 2022-06-25 |        6 |           0 |          0.0 | [{1, test (testing/trino-tests)}, {2, hive-tests (config-empty)}, {2, test (plugin/trino-kudu)}, {4, Test Report}, {5, test (plugin/trino-iceberg)}]                                                                                                                                                  |           17 |               6 |             0.35 
 2022-06-26 |        0 |           0 |          NaN | NULL                                                                                                                                                                                                                                                                                                  |            4 |               1 |             0.25 
 2022-06-27 |        6 |           2 |         0.33 | [{1, hive-tests (config-hdp3)}, {1, test (plugin/trino-kudu)}, {2, Test Report}, {2, hive-tests (config-empty)}, {4, test (plugin/trino-iceberg)}]                                                                                                                                                    |           44 |              17 |             0.39 
 2022-06-28 |       11 |           7 |         0.64 | [{1, pt (${{ matrix.config }}, ${{ matrix.suite }}, ${{ matrix.jdk }})}, {1, test (plugin/trino-delta-lake)}, {1, test (testing/trino-faulttolerant-tests, test-fault-tolerant-delta)}, {1, test (testing/trino-faulttolerant-tests, test-fault-tolerant-iceberg)}, {2, test (plugin/trino-iceberg)}] |           49 |              25 |             0.51 
 2022-06-29 |       13 |           7 |         0.54 | [{2, hive-tests (config-empty)}, {2, test (plugin/trino-delta-lake)}, {2, test (plugin/trino-kudu)}, {2, test (testing/trino-tests)}, {3, test (plugin/trino-iceberg)}]                                                                                                                               |           48 |              21 |             0.44 
 2022-06-30 |        7 |           3 |         0.43 | [{1, test (plugin/trino-hive)}, {1, test (plugin/trino-pinot)}, {1, test (testing/trino-tests)}, {3, hive-tests (config-empty)}]                                                                                                                                                                      |           54 |              12 |             0.22 
 2022-07-01 |        6 |           2 |         0.33 | [{1, Test Report}, {1, test (plugin/trino-hive)}, {4, hive-tests (config-empty)}]                                                                                                                                                                                                                     |           44 |              19 |             0.43 
 2022-07-02 |        1 |           1 |          1.0 | NULL                                                                                                                                                                                                                                                                                                  |            9 |               4 |             0.44 
 2022-07-03 |        0 |           0 |          NaN | NULL                                                                                                                                                                                                                                                                                                  |           10 |               2 |              0.2 
 2022-07-04 |        6 |           5 |         0.83 | [{1, test (plugin/trino-iceberg)}]                                                                                                                                                                                                                                                                    |           39 |              20 |             0.51 
 2022-07-05 |        3 |           2 |         0.67 | [{1, hive-tests (config-empty)}, {1, pt (default, suite-2, 11)}]                                                                                                                                                                                                                                      |           34 |              14 |             0.41 
 2022-07-06 |        7 |           2 |         0.29 | [{1, pt (default, suite-6-non-generic, 11)}, {1, test}, {2, test (plugin/trino-iceberg)}, {3, Test Report}, {3, hive-tests (config-empty)}]                                                                                                                                                           |           44 |              22 |              0.5 
 2022-07-07 |        6 |           3 |          0.5 | [{1, pt (${{ matrix.config }}, ${{ matrix.suite }}, ${{ matrix.jdk }})}, {1, test (client/trino-jdbc,plugin/trino-base-jdbc,plugin/trino-thrift,plugin/trino-memory)}, {1, test (plugin/trino-bigquery)}, {1, test (plugin/trino-iceberg)}, {2, hive-tests (config-empty)}]                           |           40 |              12 |              0.3 
 2022-07-08 |        2 |           0 |          0.0 | [{1, Test Report}, {1, test (plugin/trino-iceberg)}, {2, hive-tests (config-empty)}]                                                                                                                                                                                                                  |           24 |              10 |             0.42 
 2022-07-09 |        2 |           0 |          0.0 | [{1, hive-tests (config-empty)}, {1, test (plugin/trino-bigquery)}]                                                                                                                                                                                                                                   |            9 |               1 |             0.11 
 2022-07-10 |        1 |           1 |          1.0 | NULL                                                                                                                                                                                                                                                                                                  |            9 |               3 |             0.33 
 2022-07-11 |        7 |           3 |         0.43 | [{1, pt (${{ matrix.config }}, ${{ matrix.suite }}, ${{ matrix.jdk }})}, {2, hive-tests (config-empty)}, {2, test (plugin/trino-hive)}, {2, test (plugin/trino-iceberg)}, {2, test-jdbc-compatibility}]                                                                                               |           32 |              16 |              0.5 
 2022-07-12 |        7 |           1 |         0.14 | [{1, test (plugin/trino-bigquery)}, {1, test (plugin/trino-iceberg)}, {1, test (testing/trino-tests)}, {3, Test Report}, {3, hive-tests (config-empty)}]                                                                                                                                              |           37 |              15 |             0.41 
 2022-07-13 |        9 |           2 |         0.22 | [{3, hive-tests (config-empty)}, {3, hive-tests (config-hdp3)}, {3, test (plugin/trino-hive)}, {3, test (plugin/trino-iceberg)}, {3, test-other-modules}]                                                                                                                                             |           35 |               6 |             0.17 
 2022-07-14 |        6 |           2 |         0.33 | [{1, pt (default, suite-delta-lake-databricks, )}, {1, test-jdbc-compatibility}, {2, test (plugin/trino-delta-lake)}, {2, test (plugin/trino-iceberg)}, {3, hive-tests (config-empty)}]                                                                                                               |           35 |              19 |             0.54 
 2022-07-15 |        7 |           0 |          0.0 | [{1, test (testing/trino-tests)}, {2, test (plugin/trino-delta-lake)}, {2, test (plugin/trino-hive)}, {3, Test Report}, {4, pt (default, suite-delta-lake-databricks, )}]                                                                                                                             |           34 |               9 |             0.26 
 2022-07-16 |        0 |           0 |          NaN | NULL                                                                                                                                                                                                                                                                                                  |           11 |               3 |             0.27 
 2022-07-17 |        0 |           0 |          NaN | NULL                                                                                                                                                                                                                                                                                                  |            2 |               1 |              0.5 
 2022-07-18 |        2 |           0 |          0.0 | [{1, Test Report}, {1, hive-tests (config-empty)}, {1, test (plugin/trino-delta-lake)}, {1, test (plugin/trino-hive)}, {1, test (plugin/trino-iceberg)}]                                                                                                                                              |           39 |              11 |             0.28 
 2022-07-19 |        9 |           2 |         0.22 | [{2, test (plugin/trino-hive)}, {3, Test Report}, {3, pt (default, suite-delta-lake-databricks, )}, {3, test (plugin/trino-iceberg)}, {6, hive-tests (config-empty)}]                                                                                                                                 |           59 |              19 |             0.32 
 2022-07-20 |       19 |           1 |         0.05 | [{6, pt (default, suite-delta-lake-databricks, )}, {7, test (plugin/trino-elasticsearch)}, {9, test (plugin/trino-kudu)}, {16, hive-tests (config-hdp3)}, {18, hive-tests (config-empty)}]                                                                                                            |           83 |              11 |             0.13 
 2022-07-21 |       21 |           1 |         0.05 | [{11, hive-tests (config-empty)}, {13, test (plugin/trino-elasticsearch)}, {13, test (plugin/trino-iceberg)}, {14, test (plugin/trino-hive)}, {14, test (plugin/trino-kudu)}]                                                                                                                         |           96 |               8 |             0.08 
 2022-07-22 |        0 |           0 |          NaN | NULL                                                                                                                                                                                                                                                                                                  |           21 |               2 |              0.1 
(31 rows)

obtained by running:

-- ci workflow failures per day
WITH
runs AS (
  SELECT
      r.id
    , r.created_at
    , r.head_branch
    , r.conclusion
    , j.name AS job_name
  FROM runs r
  LEFT JOIN jobs j ON j.run_id = r.id AND j.conclusion != 'success'
  WHERE r.owner = 'trinodb' AND r.repo = 'trino' AND r.name = 'ci' AND r.created_at >= CURRENT_DATE - INTERVAL '30' DAY
)
SELECT
    date(date_trunc('day', created_at)) AS day
  , count(DISTINCT id) FILTER (WHERE head_branch = 'master') AS num_runs
  , count(DISTINCT id) FILTER (WHERE head_branch = 'master' AND conclusion = 'success') AS num_success
  , round(count(DISTINCT id) FILTER (WHERE head_branch = 'master' AND conclusion = 'success') / CAST(count(DISTINCT id) FILTER (WHERE head_branch = 'master') AS double), 2) AS rate_success
  , array_sort(
     transform(
        transform(
          map_entries(approx_most_frequent(5, job_name, 1000) FILTER (WHERE head_branch = 'master')),
          r -> cast(r AS row(key varchar, value bigint))
        ),
        r -> row(r.value, r.key)
      )
      --(x, y) -> least(greatest(cast(y AS row(key varchar, value double)).value - cast(x AS row(key varchar, value double)).value, -1), 1)
    ) AS top5_failing_jobs
  , count(DISTINCT id) AS num_all_runs
  , count(DISTINCT id) FILTER (WHERE conclusion = 'success') AS num_all_success
  , round(count(DISTINCT id) FILTER (WHERE conclusion = 'success') / CAST(count(DISTINCT id) AS double), 2) AS rate_all_success
FROM runs
GROUP BY 1
ORDER BY 1
;

Is this change a fix, improvement, new feature, refactoring, or other?

Is this a change to the core query engine, a connector, client library, or the SPI interfaces? (be specific)

How would you describe this change to a non-technical end user or system administrator?

Related issues, pull requests, and links

Documentation

( ) No documentation is needed.
( ) Sufficient documentation is included in this PR.
( ) Documentation PR is available with #prnumber.
( ) Documentation issue #issuenumber is filed, and can be handled later.

Release notes

( ) No release notes entries required.
( ) Release notes entries required with the following suggested text:

# Section
* Fix some things. ({issue}`issuenumber`)

@cla-bot cla-bot bot added the cla-signed label Jul 22, 2022
@findepi findepi mentioned this pull request Jul 22, 2022
@nineinchnick nineinchnick marked this pull request as ready for review July 22, 2022 17:20
@nineinchnick
Copy link
Copy Markdown
Member Author

All green, I guess the memory bump wasn't needed.

@martint martint merged commit 1009240 into trinodb:master Jul 22, 2022
@nineinchnick nineinchnick deleted the revert-tests-mem branch July 22, 2022 17:33
@github-actions github-actions bot added this to the 391 milestone Jul 22, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Development

Successfully merging this pull request may close these issues.

2 participants