[Feature Request]: Man-In-The-Middle Mode / Browser Remote Control

[Jonas-Metzger]

Preflight Checklist

• I have searched the issue tracker for a feature request that matches the one I want to file, without success.

What package is this feature request for?

rrweb

Problem Description

I'd like to create the following type of web-service: you can connect to a "proxy-server" via https, which runs a (headless) chrome instance that you instruct to connect to some target site. The DOM of the chrome instance is efficiently forwarded to the end-user via the rrweb stack, and the users interactions with that DOM are forwarded back to the proxy-server, who executes them. The resulting DOM changes are sent back to the user, who incorporates them into their current DOM. The user should feel as if they're directly interacting with the target site, except at a different URL. Importantly however, the proxy-server directly connects to the target site, and thus can verify (and e.g. attest to third parties) that a given DOM is "authentic", i.e. can be directly generated by interacting with the target server.

Proposed Solution

It seems that the rrweb stack is really close to enabling that functionality, but not quite there. The current assumption is that the recording user directly connects to the target site, and the replay happens on a copy that is disconnected from the target server. I would need the recording to happen on a (real-time) copy disconnected from the target server, the replay to happen on the site that is connected to the target server, with the resulting DOM events forwarded back to the copy.

Alternatives Considered

I am currently forwarding the whole viewport of a headless chrome instance via xpra-html5, but the resulting experience is full of lags. That's why I'm looking for a solution to only share the DOM, and propagate the interactions forward and resulting DOM changes back, in real-time

Additional Information

No response

[Ontopic]

Interesting idea. I've myself experimented with using a headless Electron instance to achieve something similar.

I guess the main "problem" I see with your approach, is that as far as I'm aware, there is no way to "diff" between 2 snapshots. So that would mean that with every interaction, the whole page would need to be re-rendered on the "player"-side.

I'll run some experiments myself in this direction and let you know.

Edit: Of course there is incremental rendering, just mean that I'm not sure there is a mechanism in place, to stream just the changes "live" to another client

[Ontopic]

If you were to "fork" and install the Chrome extension in your headless browser, and add a socket connection to intercept events in https://github.com/rrweb-io/rrweb-chrome-extension/blob/master/src/inject.js, I guess that would make a nice starting point?

[YunFeng0817]

Thanks, Jonas and Ontopic, your ideas are very great. I think #976 is half close to the feature and we may implement the feature after #976 is merged.

[Ontopic]

Thank you Mark!

As a last suggestion for inspiration for Jonas and rrweb-team, https://github.com/GoogleChromeLabs/carlo
It is sadly unmaintained, but might work for Jonas' use-case still.