Add additional checks in chunking functionality #671
Conversation
✅ Deploy Preview for agent-public-docs canceled.
|
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
dhurley
requested review from
oliveromahony,
aphralG,
Dean-Coakley,
craigell,
olli-holmala and
spencerugbo
| return [][]byte{buf} | ||
| } | ||
|
|
||
| chunks := make([][]byte, 0) |
There was a problem hiding this comment.
Why do we no longer set the capacity for chunks?
There was a problem hiding this comment.
Donal answered here at the office, so the reason was that the tool was complaining about our capacity potentially overflowing.
FWIW, I did the math on this one. The capacity was len(buf)/lim+1 meaning the only way we exceed the maximum length of an integer (i.e. overflow) is if the buffer (=buf) had 2^63 elements in it (so 9,223,372,036,854,775,807 elements) on a 64-bit system. Since buf is a byte slice, that means you would have 2^63 bytes, which translates to ~9,223 petabytes, meaning you definitely would have run into problem far earlier than when entering this.
sdk/checksum/checksum.go
Outdated
| log.Error("Unable to chuck payload, chunk size is too small") | ||
| return [][]byte{} | ||
| } | ||
|
|
||
| chuckSize := bufSize / lim | ||
|
|
||
| if chuckSize > math.MaxInt64-1 { | ||
| log.Error("Unable to chuck payload, data too large") | ||
| return [][]byte{} | ||
| } |
There was a problem hiding this comment.
nit/opt:
chuck -> chunk
chuckSize -> chunkSize
Proposed changes
Add additional checks in chunking functionality
Checklist
Before creating a PR, run through this checklist and mark each as complete.
CONTRIBUTINGdocumentmake install-toolsand have attached any dependency changes to this pull requestREADME.md)