-
Notifications
You must be signed in to change notification settings - Fork 33
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Issues with TLS Stream Dump #184
Comments
The wireshark HTTP follow code only support TCP filters, see http_follow, and the exported_pdu layer didn't register any tcp.* variables. So it's not possible to follow.
It's the first exported layer, which is added by udpdump. You can see the client address and the proxy address at the second exported_pdu layer which is generated by g3proxy.
I haven't checked the details yet, but packet loss is expected as it's UDP based dump protocol. |
Thank you for the link. I will investigate more how it is trying to restructure. I noticed you were including “synthetic” TCP data (such as sequence number etc) in the packet you were sending. I was seeing some tcp variables being registered (sequence etc) register in wireshark. I wonder if it can be aligned to allow it to be fully followed. I also tried modifying the wireshark side udpdump source code so it does not add the second layer of exported_pdu. It had no issues identifying the TCP data you send. The pcap capture feature you have written is an extremely useful feature. Having more in depth data and being able to follow the streams would really be great.
This is my mistake. Local machine testing. I was thinking the server would be the remote server.
That would make sense. It is possible to do pcap over IP using a TCP pipe. PolarProxy does this (sending decrypted TLS over TCP). https://www.netresec.com/?page=Blog&month=2022-05&post=Real-time-PCAP-over-IP-in-Wireshark ————————- Some background - we were leveraging ICAP but many sites have started using websockets which the ICAP protocol was not built to handle. So we started going down the PCAP route. Thank you again @zh-jq - your work on this project is much appreciated. |
It may be possible if those tcp.* variables especially tcp.stream get registered in the exported_pdu code. May be we can try and then contribute back to wireshark.
The udpdump added exported_pdu layer is also needed when you have multiple g3proxy process sending the traffic to the same wireshark instance. The src addr and dst addr may conflict within g3proxy processes on different hosts, the correct way to calculate the stream id is by using Don't forget to set the
It's still the same problem, the The way to add more info is to add more fields in the exported_pdu layer, which need to be defined by wireshark first. I'm not sure whether they will accept it.
I prefer to use UDP to do this, as packet loss is also allowed in internal processing. It's may be possible to replace the generated exported_pdu layer with IP+TCP, but this way we will lose the ability to set dissectors hints other than tcp.port.
I think that the exported_pdu code in wireshark could be improved to match our needs. It will be good if you can help to do that.
It's fine if you don't use the block&modify feature. You may also use ICAP for websocket upgrade request and then use udpdump to dump the traffic. |
We have been running into a few issues with the TLS stream dump functionality. I followed the udp dump/exported_pdu instructions in wireshark.
Wireshark is not able to fully parse the different streams (TCP/HTTP/etc). For example when you right click on an HTTP request then select follow-HTTP. It is unable to reassemble and follow the stream. Even if the dump has been running for a while with many requests wireshark identifies newer requests as an early tcp.stream (for example 1 or 2). This leads me to believe it's not able to properly discern between different TCP streams.
The source and destination are of the proxy server and the udpdump receiver. Ideally these would be of the connecting client and remote server. Or at least between the proxy and the remote server.
I believe due to issue 1 we are seeing HTTP [Malformed Packet] errors often.
Thanks for any help/suggestions!
The text was updated successfully, but these errors were encountered: