Skip to content

Fix recoverable execution and enable flaky test in TestHiveRecoverableExecution#22314

Merged
tdcmeehan merged 1 commit intoprestodb:masterfrom
hantangwangd:fix_recoverable_execution
Mar 25, 2024
Merged

Fix recoverable execution and enable flaky test in TestHiveRecoverableExecution#22314
tdcmeehan merged 1 commit intoprestodb:masterfrom
hantangwangd:fix_recoverable_execution

Conversation

@hantangwangd
Copy link
Member

@hantangwangd hantangwangd commented Mar 24, 2024

Description

A committed LIFESPAN_COMMIT task may happens losing contact and being rescheduled. TableFinishOperator should handle this situation and do not try to make statistics for it again. Or we would meet the error described in #20466 and #20272.

This PR fix #20466 and #20272.

Test Plan

  • Existing tests testCreateBucketedTable and testInsertBucketedTable in TestHiveRecoverableExecution
  • Before this fix, this error almost certain to occur when running the tests 300 times continuously. After this fix, it never occur again.

Contributor checklist

  • Please make sure your submission complies with our development, formatting, commit message, and attribution guidelines.
  • PR description addresses the issue accurately and concisely. If the change is non-trivial, a GitHub Issue is referenced.
  • Documented new properties (with its default value), SQL syntax, functions, or other functionality.
  • If release notes are required, they follow the release notes guidelines.
  • Adequate tests were added if applicable.
  • CI passed.

Release Notes


General Changes
* Fix bug with recoverable execution

});
}
else {
LOG.info("The lifeSpanAndStage %s has been committed before", LifespanAndStage.fromTableCommitContext(tableCommitContext));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this isn't actionable and no one needs to read this, this doesn't seem worth an info log. Debug level at most, but better yet nothing at all.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, the log is removed! Thanks for the suggestion.

@hantangwangd hantangwangd force-pushed the fix_recoverable_execution branch from 7e909fc to 74ea3da Compare March 25, 2024 13:18
@hantangwangd hantangwangd requested a review from elharo March 25, 2024 13:24
Copy link
Contributor

@tdcmeehan tdcmeehan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice fix!

@tdcmeehan tdcmeehan requested a review from arhimondr March 25, 2024 13:28
@hantangwangd hantangwangd force-pushed the fix_recoverable_execution branch from 74ea3da to 7d1760f Compare March 25, 2024 15:58
@elharo
Copy link
Contributor

elharo commented Mar 25, 2024

Possibly flaky:

[ERROR] TestPrestoNativeJoinQueries>AbstractTestNativeJoinQueries.testAntiJoin:86->AbstractTestQueryFramework.assertQuery:159 Execution of 'actual' query failed: SELECT * FROM lineitem WHERE linenumber = 3 OR orderkey NOT IN (SELECT orderkey FROM orders WHERE (orderkey + custkey) % 2 = 0)
[ERROR] TestPrestoNativeJoinQueries>AbstractTestNativeJoinQueries.testBucketedInnerJoin:61->AbstractTestQueryFramework.assertQuery:159 Execution of 'actual' query failed: SELECT b.name, c.name FROM customer_bucketed b, customer c WHERE b.name=c.name

@tdcmeehan tdcmeehan merged commit 600681d into prestodb:master Mar 25, 2024
@hantangwangd hantangwangd deleted the fix_recoverable_execution branch March 25, 2024 23:29
@hantangwangd
Copy link
Member Author

Possibly flaky:

[ERROR] TestPrestoNativeJoinQueries>AbstractTestNativeJoinQueries.testAntiJoin:86->AbstractTestQueryFramework.assertQuery:159 Execution of 'actual' query failed: SELECT * FROM lineitem WHERE linenumber = 3 OR orderkey NOT IN (SELECT orderkey FROM orders WHERE (orderkey + custkey) % 2 = 0) [ERROR] TestPrestoNativeJoinQueries>AbstractTestNativeJoinQueries.testBucketedInnerJoin:61->AbstractTestQueryFramework.assertQuery:159 Execution of 'actual' query failed: SELECT b.name, c.name FROM customer_bucketed b, customer c WHERE b.name=c.name

Seems unrelated.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Status: ✅ Done

Development

Successfully merging this pull request may close these issues.

Flaky TestHiveRecoverableExecution.testInsertBucketedTable Flaky Test: TestHiveRecoverableExecution.testCreateBucketedTable

3 participants