Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BUG][Spark] Fix flaky test suite org.apache.spark.sql.delta.DeltaSourceDeletionVectorsSuite #1982

Closed
2 of 8 tasks
vkorukanti opened this issue Aug 17, 2023 · 2 comments
Closed
2 of 8 tasks

[BUG][Spark] Fix flaky test suite org.apache.spark.sql.delta.DeltaSourceDeletionVectorsSuite #1982

vkorukanti opened this issue Aug 17, 2023 · 2 comments
Labels
bug Something isn't working

Comments

@vkorukanti
Copy link
Collaborator

Bug

Which Delta project/connector is this regarding?

  • Spark
  • Standalone
  • Flink
  • Kernel
  • Other (fill in here)

Describe the problem

Failed test subsequent DML commands are processed correctly in a batch - INSERT->UPDATE - List((ignoreChanges,true)) *** FAILED ***

CI runs: https://github.com/delta-io/delta/actions/runs/5835475304/job/15827113787, https://github.com/delta-io/delta/actions/runs/5868780845/job/15912353282?pr=1950

Willingness to contribute

The Delta Lake Community encourages bug fix contributions. Would you or another member of your organization be willing to contribute a fix for this bug to the Delta Lake code base?

  • Yes. I can contribute a fix for this bug independently.
  • Yes. I would be willing to contribute a fix for this bug with guidance from the Delta Lake community.
  • No. I cannot contribute a bug fix at this time.
@vkorukanti vkorukanti added the bug Something isn't working label Aug 17, 2023
@vkorukanti
Copy link
Collaborator Author

@larsk-db Could you please a take look at this test?

@larsk-db
Copy link
Contributor

Here's another one: https://github.com/delta-io/delta/actions/runs/5900171391/job/16003921031?pr=1987

This was referenced Aug 18, 2023
Closed
Closed
@larsk-db larsk-db mentioned this issue Aug 25, 2023
Merged
vkorukanti pushed a commit that referenced this issue Sep 7, 2023
@larsk-db
[Spark][2.4] Fix flaky test suite DeltaSourceDeletionVectorsSuite
68608d7 
## Description
(Cherrypick of 67c4b98 to branch-2.4)

- Remove flakiness from a previously flaky test case, by ensuring that the stream is stopped processing before running new DML commands on the source table.

Fixes #1982

## How was this patch tested?
Testing-only PR.

## Does this PR introduce _any_ user-facing changes?
No.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[Spark] Fix flaky test suite DeltaSourceDeletionVectorsSuite larsk-db/delta
2 participants
@vkorukanti @larsk-db