Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

res_stream_threshold_ is too high, causing incomplete data transmission #243

Closed
rmclassic opened this issue Oct 11, 2021 · 8 comments · Fixed by #245
Closed

res_stream_threshold_ is too high, causing incomplete data transmission #243

rmclassic opened this issue Oct 11, 2021 · 8 comments · Fixed by #245
Assignees
Labels
bug Something isn't working

Comments

@rmclassic
Copy link

Using single header library, when sending bigger text payloads to the clients, the connection is sometimes closed before the transmit is completed.

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
 42  606k   42  255k    0     0   255k      0  0:00:02 --:--:--  0:00:02  442k
curl: (18) transfer closed with 358690 bytes remaining to read
@The-EDev The-EDev added the bug Something isn't working label Oct 11, 2021
@The-EDev
Copy link
Member

Could you please test this with 1MiB or more data?

@rmclassic
Copy link
Author

I tried to reproduce the problem with string payload and did not succeed. turned out it happens with invalid data in json::wvalue object, then the server closes the connection with return code 200 instead of returning 500

@The-EDev
Copy link
Member

Could you elaborate on what kind of invalid data you're passing to json::wvalue, since normally you would get a compiler error for that..

@rmclassic
Copy link
Author

I thought it's because i assigned a char[12] to a std::string and then jsonified it. but the problem still persists. now i am just jsonifying int, double, and other numeric types.

@The-EDev
Copy link
Member

hmmm, the issue doesn't happen when I do the same.. Which version of Crow are you running?

@rmclassic
Copy link
Author

I'm using Crow/0.3
Does crow change it's behavior depending on response payload size?
because even if i concatenate the stringified JSON with another big random string, it works perfectly
even for payloads around 6MB

@rmclassic
Copy link
Author

Found the issue, the res_stream_threshold_ is set to 1MB, and is too much, so the payload is not completed. i can lower the res_stream_threshold_ to a lower value until the library is fixed
Thanks

@The-EDev
Copy link
Member

I was going to suggest that (hence the "Could you please test this with 1MiB or more data?"), I'm glad you found it, I'll add an app option to lower it.

@The-EDev The-EDev changed the title Data loss on huge payloads and request rates res_stream_threshold_ is too high, causing incomplete data transmission Oct 11, 2021
@The-EDev The-EDev self-assigned this Oct 12, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants