Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: copy into table collect files twice some times. #17300

Merged
merged 2 commits into from
Jan 16, 2025

Conversation

youngsofun
Copy link
Member

@youngsofun youngsofun commented Jan 16, 2025

I hereby agree to the terms of the CLA available at: https://docs.databend.com/dev/policies/cla/

Summary

add a log by the way which will be helpful for debugging #17284

Tests

  • Unit Test
  • Logic Test
  • Benchmark Test
  • No Test - Explain why

Type of change

  • Bug Fix (non-breaking change which fixes an issue)
  • New Feature (non-breaking change which adds functionality)
  • Breaking Change (fix or feature that could cause existing functionality not to work as expected)
  • Documentation Update
  • Refactoring
  • Performance Improvement
  • Other (please describe):

This change is Reviewable

@github-actions github-actions bot added the pr-bugfix this PR patches a bug in codebase label Jan 16, 2025
Copy link

what-the-diff bot commented Jan 16, 2025

PR Summary

  • Introduction of new files_collected field
    The Binder structure now features a new field named files_collected. This field, which is initially set as false, serves to monitor whether files have been gathered or not.

  • Updating bind_copy_into_table_from_location method
    This method has been updated to adjust the value of files_collected. Once all files are successfully collected, files_collected is then updated to true.

  • Introduction of files_collected field in CopyIntoTablePlan struct
    Just like the change made to the Binder structure, a new files_collected field is now also part of the CopyIntoTablePlan structure. This further streamlines the process of tracking file collection.

  • Modification of collect_files method
    A major modification has been implemented in the collect_files method. Now, whenever the system checks if files_collected is true, it immediately exits if it detects that all necessary files have already been captured. This removes the redundancy in file collection and improves the efficiency of the entire process.

@youngsofun youngsofun requested a review from sundy-li January 16, 2025 03:37
@youngsofun youngsofun added this pull request to the merge queue Jan 16, 2025
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Jan 16, 2025
@youngsofun youngsofun added this pull request to the merge queue Jan 16, 2025
@BohuTANG BohuTANG removed this pull request from the merge queue due to a manual request Jan 16, 2025
@BohuTANG BohuTANG merged commit 9b38e65 into databendlabs:main Jan 16, 2025
70 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
pr-bugfix this PR patches a bug in codebase
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants