Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor(query): only call once finish callback #17293

Merged
merged 4 commits into from
Jan 16, 2025

Conversation

sundy-li
Copy link
Member

@sundy-li sundy-li commented Jan 15, 2025

I hereby agree to the terms of the CLA available at: https://docs.databend.com/dev/policies/cla/

Summary

fix(query): only call once finish callback of unload_spill_meta

Tests

  • Unit Test
  • Logic Test
  • Benchmark Test
  • No Test - Explain why

Type of change

  • Bug Fix (non-breaking change which fixes an issue)
  • New Feature (non-breaking change which adds functionality)
  • Breaking Change (fix or feature that could cause existing functionality not to work as expected)
  • Documentation Update
  • Refactoring
  • Performance Improvement
  • Other (please describe):

This change is Reviewable

@github-actions github-actions bot added the pr-refactor this PR changes the code base without new features or bugfix label Jan 15, 2025
Copy link

what-the-diff bot commented Jan 15, 2025

PR Summary

  • Enhancement of 'PipelineBuilder'

    The 'PipelineBuilder' has been improved to use a method known as 'take_spill_files'. This enables it to unload (or remove) specific temporary files (spill files) saved during some operations.

  • Condition Check in 'main_pipeline'

    A new condition has been added in the 'main_pipeline', specifically to check for any non-empty temporary files in a remote location. This check happens before calling the unload method, hence it can prevent unnecessary operations when there are no files to remove.

  • Introduction of 'take_spill_files' method in 'QueryContext'

    A new method, 'take_spill_files', has been added to 'QueryContext', tasked with retrieving these temporary files. This replaces the previous method, demonstrating a better approach to the retrieval process.

  • Modification of 'unload_spill_meta'

    The method 'unload_spill_meta' has been updated to accept a list of strings representing remote temporary files. This refinement allows for more targeted unloading, increasing the efficiency of the operation.

  • Code Clean up in 'mod.rs'

    A commented-out line about Python runtime initialization has been removed from the 'mod.rs' file. This change helps to clean up and maintain the tidiness of the codebase.

@sundy-li sundy-li requested a review from zhang2014 January 15, 2025 15:00
@sundy-li sundy-li added this pull request to the merge queue Jan 16, 2025
@BohuTANG BohuTANG removed this pull request from the merge queue due to a manual request Jan 16, 2025
@BohuTANG BohuTANG merged commit e9fc0a1 into databendlabs:main Jan 16, 2025
70 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
pr-refactor this PR changes the code base without new features or bugfix
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants