Skip to content

Conversation

@andrefurlan-db
Copy link
Contributor

@andrefurlan-db andrefurlan-db commented Feb 16, 2024

This is a first set of fixes for cloud fetch.

  • Throw when failed to download file
  • Retry properly while downloading file
  • Add a bunch of debug logs
  • Added URL redacting
  • Prevent thread issues

TODO:

  • check rows context to avoid duplicates
  • http connection pools for cloud storage, proxies, etc.

Backported to version 2

backport to version 2

Signed-off-by: Andre Furlan <[email protected]>
@andrefurlan-db andrefurlan-db marked this pull request as ready for review February 16, 2024 02:23
@andrefurlan-db andrefurlan-db requested review from benc-db and jackyhu-db and removed request for arikfr February 16, 2024 02:25
@andrefurlan-db andrefurlan-db changed the title fixes for cloud fetch fixes for cloud fetch - part un Feb 16, 2024
Signed-off-by: Andre Furlan <[email protected]>
@andrefurlan-db andrefurlan-db merged commit 6a348ec into databricks:2.9.4 Feb 16, 2024
andrefurlan-db added a commit to andrefurlan-db/databricks-sql-python that referenced this pull request Feb 21, 2024
* fixes for cloud fetch

Signed-off-by: Andre Furlan <[email protected]>
---------

Signed-off-by: Andre Furlan <[email protected]>
Co-authored-by: Raymond Cypher <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants