-
Notifications
You must be signed in to change notification settings - Fork 223
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Some data frame opcodes not being handled at all #137
Comments
which crates did you say are working? and can you give examples of the websocket traffic or how to get it? I'm fairly sure we've handled all the opcodes. Thanks! |
1. I wasn't specifically meaning other crates, but libraries in other languages and their respective websocket they're using. As they don't seem to suffer from this and it being a websocket problem only. Although, could you by chance at least add the unknown/invalid opcodes into the errors themselves? As it would help a bit identifying this, since it affects my users as well. |
@acdenisSK yeah that sounds super reasonable, I'll get crackin, thanks! After I push this update can you paste the new logs to see which opcodes are not being handled? |
a |
The wireshark data (May contain some unnecessary fuzz but should contain the data you're looking for). Also something to note, this doesn't seem to be an issue when i use ethernet instead of wifi. But still not sure about the culprit. |
So I've looked into this a bit since my bots that use serenity sometimes encounter this stream of errors. I added some logging to this library, and I've found that when it breaks it seems to be parsing data as a websocket frame. By that I mean that a standard websocket frame as described here is just arbitrarily filled with text data from a previous packet, but is still parsed as a frame. For example, debug printing a frame I get this:
When parsing data as ASCII, I get the following string:
When you take into account the way a websocket packet is framed, you can add on
So it looks like somewhere along the line, the length of a packet is being read in wrong. At least that would be my guess. |
I can now reproduce this consistently through a virtual machine by limiting the network speed. It consistently fails on large payloads with a "Resource temporarily unavailable" error. What I take from this is that it tries to grab a big payload all at once, but it hasn't actually received the entire payload. This causes it to skip since the read would be blocking. The problem with this is that before it tried to read the data, it already had finished reading the header. This means that in the next call, it tries to read the beginning of the data as another header and then failures pile up from there. The bug is on this line: https://github.com/cyderize/rust-websocket/blob/master/src/dataframe.rs#L89 I don't know which way would be best for handling this sort of error since I am not familiar with your codebase or standards. Some thoughts are using lazy static to store a header that was read but not used, or using a reader that can peek ahead. It should also be noted that although it is most common for it to fail catastrophically on large payloads, the same bug can occur in the read_header method as well. I haven't look at other places in the project, but I think it's safe to say any section that can potentially return an error after reading bytes can have this bug since there is no method to recover those lost bytes. |
Thank you for the research @Roughsketch I'll take a look and reproduce. |
We work on the same project, Roughsketch was testing on the sync version. |
@zeyla @acdenisSK @Roughsketch did you fix it it your fork https://github.com/serenity-rs/rust-websocket ? |
I believe it was fixed at least for our use case. From what I remember I just made it buffer data instead of throwing out parts of packets. |
What about uuid? |
What about it? It's been a long time so I don't remember all the details, but from what I can see on the commits the UUID is used to keep track of the |
To be specific, serenity (the crate i'm the current maintainer of) has sometimes after a while when booting up, lots of spam regarding unknown/invalid data frame opcodes.
But the thing is, the other api wrappers for discord are working just fine without anything interrupting them with data frame opcodes.
So by my assumption, there has to be some opcodes not being handled by this crate at all.
The text was updated successfully, but these errors were encountered: