fix: issue with path is returned from download_files is always set t… #731
Conversation
There was a problem hiding this comment.
Pull Request Overview
This PR fixes a bug where the download_files method in the parquet input handler was not returning the downloaded file path, causing the code to always use the original remote path instead of the local downloaded path.
- Removes unnecessary local
pathvariable assignment - Updates
download_filescall to not capture return value since it modifiesself.pathdirectly - Ensures
pd.read_parquet()uses the correct path after download
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
| await self.download_files(self.path) | ||
| # Use pandas native read_parquet which can handle both single files and directories | ||
| return pd.read_parquet(path) | ||
| return pd.read_parquet(self.path) |
There was a problem hiding this comment.
The fix assumes that download_files modifies self.path in-place, but this behavior is not evident from the code shown. If download_files doesn't update self.path to point to the downloaded local file, this will still read from the original remote path. Consider either ensuring download_files updates self.path or capturing its return value if it returns the local path.
application_sdk/inputs/parquet.py
Outdated
| if self.input_prefix and self.path: | ||
| path = await self.download_files(self.path) | ||
| await self.download_files(self.path) | ||
| # Use pandas native read_parquet which can handle both single files and directories | ||
| return pd.read_parquet(path) | ||
| return pd.read_parquet(self.path) |
There was a problem hiding this comment.
The fix assumes that download_files modifies self.path in-place, but this behavior is not evident from the code shown. If download_files doesn't update self.path to point to the downloaded local file, this will still read from the original remote path. Consider either ensuring download_files updates self.path or capturing its return value if it returns the local path.
Changelog
path was always set to none as the underlying methods inside
download_filesis returning NoneAdditional context (e.g. screenshots, logs, links)
Checklist
Copyleft License Compliance