Add an interface for Pluggable Transports

[nbeirne]

This Pull Requests adds support for a "PluggableTransports" transport layer. It defines a public interface which makes it easy for an implementer to create a new pluggable transport. According to the pluggable transport spec we should provide a socket-like interface for ease of implementing the transport itself. I tried to mirror the asio functions as closely as possible, minus the async operations which are handled in the transport link.

Implementing and Using a New Pluggable Transport

To create a new transport the implementer must do a few things.

1. Create new Transport and Client objects which implement your preferred obfuscation technique. Any configuration should be contained within the Transport object.
2. Compile openvpn3 with -DOPENVPN_PLUGGABLE_TRANSPORTS and extend OpenVPNClient to implement the new_transport_factory method.
3. Configure the OpenVPNClient with the config.usePluggableTransports option.

Why Not Use an External Transport

In theory it is possible to implement this same functionality with minimal changes to OpenVPN 3 itself, and to use an external transport. This approach would have a few notable downsides:

1. Everyone who wants to use a pluggable transport would have to implement something equivalent to ptcli.hpp and ptlink.hpp as an external transport. That's a lot of code just to get obfuscation support.
2. They would also have to implement the protocol switching logic in their OpenVPNClient.

More Details

The client and link objects are based on the TCP client and link. In this version of the code they handle interfacing with the rest of the OpenVPN code (socket_protect, packet alignment, conforming to the send/receive interface) and also queuing data for the synchronous 'send' calls. The actual 'send' call is provided by the implementer, and may be as simple as a opening a regular TCP socket. While this somewhat complicates the implementation internal to OpenVPN, it simplifies the code for the implementer.

In theory the client and link can subclass their TCP counterparts so that more code is shared. I consider the refactor this would involve to be out-of-scope for this pull request, at least at the moment.

Some Meta Business

I am submitting this pull request for review through Github for initial review. I will send a patch to the mailing list after I have a few eyes on it.

I structured this PR to be read commit-by-commit. It might be easier to digest if you look at the diff for each commit. If and when these changes are merged it would be possible to squash them into a single commit - but that's really up to the final reviewers.

[nbeirne]

In theory a listen function should also be included here, to make a server. I can implement this functionality as well, but I didn't want to make this pull request larger.

[nbeirne]

native_handle is required to implement socket_protect.

[xtianxian]

Hi @nbeirne I need help because I'm stucked. I have this code to initialize Obfs4Transport and setUsePluggableTransports.
But what's next? Do I use the obfs local server:port as socks? but openvpn3 doesn't support socks5
So how can I make openvpn3 use the Obfs4Transport that I initialized? Thank you in advance

config.setUsePluggableTransports(true);

private String initObfs4Transport() {
    new Obfs4Transport().register();
    Properties options = new Properties();
    String address = "xxx.xxx.xxx.xxx:1234";
    String torBridgeLine = "obfs4 xxx.xxx.xxx.xxx:1234 key-not-used cert=ApWvCPD2uhjeAgaeS4Lem5PudwHLkmeQfEMMGoOkDJqZoeCq9bzLf/q/oGIggvB0b0VObg iat-mode=0";
    Obfs4Transport.setPropertiesFromBridgeString(options,torBridgeLine);
    Transport transport = Dispatcher.get().getTransport(this, DispatchConstants.PT_TRANSPORTS_OBFS4, options);
    transport.init(this,options);
    if (transport != null) {
        Connection ptConn = transport.connect(address);// transport.connect(options.getProperty(Obfs4Transport.OPTION_ADDRESS));
        if (ptConn != null) {
            Log.d("obfs4Log", ptConn.getLocalAddress() + ":" + ptConn.getLocalPort());
        }
    }
    return null;
}

[nbeirne]

I can't help much with your specific obfs4 setup since they can vary so much, and it depends on the implementation. I used this implementation of obfs4, which does not require a local socks5 connection. I wrapped it in a c++ class and was able to use this version of openvpn3 to configure it as a transport.

To integrate the transport follow these steps:

1. Make a c++ class follow this interface. This is where the external library sat in my version. In theory this is what Obfs4Transport in your implementation should be.
2. Make some object conform to this factory, which should generate your Connection. Notice that OpenVPNClient is already a PluggableTransports::Factory. When you compile with the -DOPENVPN_PLUGGABLE_TRANSPORTS flag it should warn about missing methods.
3. Set the setUsePluggableTransports flag to true, which it appears you are doing already.

Does all this make sense?

[pokamest]

Any news on this?
Should we wait for this PR will be merged?

[ordex]

Hi, at the moment there is a larger ongoing discussion about whether it makes sense to integrate PT into OpenVPN directly or not.
So far the idea is that such component should rather live outside of the core to avoid adding complexity and debt to the main codebase.

This said, @pokamest if you are aware of what is being used on the server side to communicate through interface, please let us know. As of now OpenVPN3 is a client only library, so we don't even know against what this code should be tested.

[pokamest]

Hi!
As I understand it's enough to run pluggable transport daemon as separate process on server side, configure it work with OpenVPN server.
For example, I configured the following with Cloak PT plugin in AmneziaVPN client.
On server side I'm running cloak server (ck-server), which configured to work with OpenVPN and ShadowSocks servers.
On client side I'm running cloak client (ck-client.exe) separately for OpenVPN protocol, because OpenVPN still not supporting pluggable transport.
But ShadowSocks supporting PT, so it's enough to setup and configure cloak plugin in ShadowSocks client instead of running separate ck-client process.
It important when you running PT on iOS and Android - it's ok to have separate PT processes on client side for desktops, but you should embed plugin for mobiles.
Take a look to https://www.tunnelbear.com/blog/tunnelbear-implements-pluggable-transports-with-openvpn3/
They mentioned https://github.com/OperatorFoundation openvpn2 fork which supporting PT, but it's not necessary on server side in my opinion, it's enough to run PT plugins as separate processes on server side.

[nbeirne]

That is correct. As long as the specific PT implementation on the client and server are protocol-compatible it should be fine.

At TunnelBear I used OperatorFoundation's implementation of obfs4 in Go with this code and a light c++ interface to make
them work together nicely. You're correct about the idea of embedding plugins for mobile devices - at some point Android was making it very hard to run openvpn as a daemon which is part why TB did this.

I no longer work at TunnelBear but I'd like to advocate for this merge since it's incredibly useful to a fairly small number of people.

[ordex]

... at some point Android was making it very hard to run openvpn as a daemon which is part why TB did this.

Actually there are apps (notably "OpenVPN for Android") that run several processes in background. I always imagined that the PT daemon could be one of those. That would be an easy way to ship a PT daemon on Android along with OpenVPN.

The PT component would be well suited for running outside OpenVPN, without the need to touch its codebase. What's wrong with that? As far as I understand this is already happening on the server side (even though nobody has seen such deployment yet), so the client can just do the same.

[schwabe]

One big blocker to merging this is the availibility of a setup to use this in. And with that I mean some how/tutorial how to setup both client and server side and client side. It is nice that Tunnelbear has some setup that works for them but if we as open source project should support this interface, we need something that everyone can run/use.

[outspace]

1. I used a shapeshifter-dispatcher with obfs4 and a clean openvpn2 server behind it as a server.
2. Compiled that PR with -DOPENVPN_PLUGGABLE_TRANSPORTS flag.
3. Wrote two classes, light c++ interface, and factory.
4. Shapeshifter-obfs4-OpenVPN-Transport-Plugin-Cgo was used as a plugin.
5. Set usePluggableTransports to true.
6. Setup obfs4 cert key to the plugin.

I'm getting PT_SIZE_ERROR.

I updated GO plugin to the new version of obfs4 (v2, v3), and the result was the same.
If I specify the wrong cert key for the obfs4 plugin, then It's not even trying to make a connection. That's why I think that I am on the correct way.

I would appreciate if someone could give me some feedback.

[outspace]

Problem was in Shapeshifter-obfs4-OpenVPN-Transport-Plugin-Cgo, after rewriting Obfs4_read It works perfectly

[nbeirne]

I want to close this due to almost two years of inactivity. I left TunnelBear shortly after opening this PR, but I will forward it to my former colleagues to see if they want to pick it up.

@outspace is correct on the setup steps. The shapeshifter implementation had a bug on read, which was fixed in this commit.

@schwabe If there is still interest in merging this, what needs to be done? Probably a rebase on the current development version and also documentation, but what else?

[pokamest]

AmneziaVPN still using this PR in the fork https://github.com/amnezia-vpn/openvpn3
It working not very smooth with Cloak plugin, there are some bugs causing crashes on Android and iOS.
We've stopped developing OpenVPN + Cloak, but we are open to discussion.

[schwabe]

I have not really looked at the code again if it is now coming with a proper setup to make this code actually useful without the properitary parts but I don't really see a movement here. While I think the idea is good it is very hard for us to accept code that we cannot test and maintain.

[nbeirne]

That's fair. I think if I were to redo this today I'd name it differently and make it more generally useful - it's really just a way to replace the socket transport read/write code with a customizable implementation (in this case, a connection which encrypts data with Obfs4 before sending it off. In Amnesia's case a it looks like they were using Cloak). The default transport could even be the plain unix socket to support normal openvpn3 connections.

Anyway, I'm going to close it now. Who knows, I might revisit this sometime in the future.