-
Notifications
You must be signed in to change notification settings - Fork 205
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fastGet observations #407
Comments
This is great work! Clearly articulated and good methodical research. Thanks for sharing. Can I ask, since this does indicate an issue with fastGet packet size negotiation in ssh2, would you mind also posting it to the ssh2 project. I suspect this information would prove very useful there. Finally, I would like to add some of this to the README (in the trouble shooting section). Is that OK with you? |
Thank you. I posted it on the Please feel free to use it for the README. Thank you for your work on this repo! |
@amitkirdatt Thank you for creating this issue, I was able to replicate. I was very skeptical to believe that |
this hit me pretty hard. |
Didn't work in case of fastPut, files got corrupted on sftp, tried with multiple chunk sizes. info: SFTP :: Outbound: Sending CHANNEL_DATA (r:0, 86) {"timestamp":"2024-08-01T05:21:49.771Z"} |
Node version: v14.19.1
OS: MacOS 12.1
ssh2-sftp-client: 8.0.0
Issue: We were seeing files being corrupted when using
fastGet
.Resolution: We needed to adjust the chunkSize specific to the sftp server we are connecting to
We ended up solving the problem - Just sharing our experience:
Code using ssh2-sftp-client
The text file we are downloading is about ~86MB. We noticed that the file was consistently corrupted (data is missing).
To narrow down the issue we decided to whip up a quick test using
ssh2
We noticed the same corruption.
We then fired up the native sftp client in debug mode and noticed that the server rmax value was
32768
(same as the default chunkSize)We then added debug logging to ssh2
In the debug we noticed the following:
It looked liked the DATA was being broken up into 2 parts: 1 part was 31952 and the 2nd part was 816, which when added up is
32768
. We figured this was causing the issue.We then specified the chunkSize of 31952 for
fastGet
.We didn't see the DATA being broken up into two parts like before (31952 and 816).
The files are now downloading correctly.
I am sharing our experience in the hopes that somebody finds it useful.
ssh2-sftp-client updated code that worked for us:
ssh2 code:
The text was updated successfully, but these errors were encountered: