Skip to content
This repository was archived by the owner on May 9, 2024. It is now read-only.

Add an issue bucketing script for query execution tests #368

Merged
merged 1 commit into from
Apr 6, 2023

Conversation

kurapov-peter
Copy link
Contributor

This is a basic version that collects test statuses in presence of failures. It captures the return codes of standalone tests and creates a yaml report. The report is coarse-grained since each test may include many queries, but it gives some coverage picture. The intended use is to simplify status checking for non-standard build/run options (like exec.heterogeneous.allow_cpu_retry = false). For example, I've collected the stats for ArrowBasedExecuteTest on Intel GPU (we have 211 tests in total there, each includes multiple queries):

status number of tests
passes 116
assertion/driver failure 82
incorrect result 13

Bucketing granularity can be easily improved by capturing the stderr. The help should be self-explanatory. Let me know if anything is unclear.

Verified

This commit was signed with the committer’s verified signature.
kurapov-peter Petr Kurapov
@kurapov-peter kurapov-peter requested a review from leshikus April 6, 2023 19:48
Copy link
Contributor

@leshikus leshikus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@leshikus leshikus merged commit fe5fb29 into main Apr 6, 2023
@leshikus leshikus deleted the pakurapo/bucketing branch April 6, 2023 22:22
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants