Skip to content

Conversation

@wendigo
Copy link
Contributor

@wendigo wendigo commented May 20, 2024

Description

Additional context and related issues

Release notes

( ) This is not user-visible or is docs only, and no release notes are required.
( ) Release notes are required. Please propose a release note for me.
( ) Release notes are required, with the following suggested text:

# Section
* Fix some things. ({issue}`issuenumber`)

@cla-bot cla-bot bot added the cla-signed label May 20, 2024
@github-actions github-actions bot added the bigquery BigQuery connector label May 20, 2024
@wendigo wendigo force-pushed the serafin/do-not-transfer-v2 branch from b1f45be to d35d036 Compare May 21, 2024 07:48
@wendigo wendigo requested review from ebyhr and vlad-lyutenko May 21, 2024 07:49
@wendigo wendigo changed the title Wip: Do not slice ListVector Do not slice ListVector May 21, 2024
@wendigo
Copy link
Contributor Author

wendigo commented May 21, 2024

I don't know why are we copying memory in the first place to just read a vector.

@hashhar
Copy link
Member

hashhar commented May 21, 2024

I don't know why are we copying memory in the first place to just read a vector.

Probably for explicit ownership of the underlying buffers otherwise maybe the contents change between when it's sent back vs when it's actually read?

@wendigo
Copy link
Contributor Author

wendigo commented May 21, 2024

@hashhar this is read only memory. I don't think this is the reason

@wendigo
Copy link
Contributor Author

wendigo commented May 21, 2024

And we seems to do it only for lists, but not for other types and it doesn't make sense because they are encoded in the similiar way

@wendigo
Copy link
Contributor Author

wendigo commented May 21, 2024

I think that we were using much more memory than needed

@wendigo
Copy link
Contributor Author

wendigo commented May 21, 2024

@ebyhr @hashhar please review

@wendigo wendigo merged commit 24a6da9 into master May 22, 2024
@wendigo wendigo deleted the serafin/do-not-transfer-v2 branch May 22, 2024 01:51
@github-actions github-actions bot added this to the 449 milestone May 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bigquery BigQuery connector cla-signed

Development

Successfully merging this pull request may close these issues.

4 participants