ROS# UWP TCP Client

[tsdavs]

• I am at the right place and my issue is directly related to ROS#. General technical questions I would post e.g. at ROS Answers or Stack Overflow. For library-specific questions I would look for help in the corresponding library forums.
• I have thoroughly read the Contributing Guideline and writing this issue is the right thing to do in my case.

---

I have a question!

• I searched the Wiki, open and closed issues for an answer. I tried my best to find the answer by myself without success. I believe that the discussion we will have in this issue, and the solutions we might find, will help me, and likely other community members who have a similar problem.

Here is my question:
Hi All,

My team and I have successfully built and deployed the @dwhit fork of the ROS# project for the Hololens. Unfortunately, we have run into some issues with socket communication. The project in Unity uses a subscriber class and prints to a UI element. Ultimately the plan is to view this on the Hololens as a kind of HUD element. Currently, the program functions as expected on the Unity IDE but when compiled and built onto the Hololens (or the Hololens sim), a connection is established to the Rosbridge server but no data is passed through. We believe that this may be because the issues outlined here. Should we consider a custom websocket solution like @jaguar243 metioned here? Does anyone have any advice?

---

• My OS is: Windows 10
• My Unity Version is: 2018.3.0f2
• My ROS Distribution is: Melodic
• My Build Target Platform is: Hololens

Thanks!

Tim

[irek-janusz]

Hi Tim,

I have exactly the same problem as you described. Connection is established, but you can't subscribe to any topic (even when the message type is as simple as standard String). Everything seems to be working fine while testing in the Unity Editor.

Did you come up with any solution to this problem? One of the alternative solutions one could implement could be to use Node-Red with MQTT. There is ros-contrib package you can get for Node-Red which will allow to publish/subscribe (I need to further investigate if it is possible to publish/subscribe topic with complex message types, or just the simple ones). Once the data is captured in Node-Red, I believe it could be broadcasted using MQTT (I previously worked with Hololens and MQTT and the setup is quite efortless, it also work as the opposite to Hololens and ROS).

Regards,
Irek

[MartinBischoff]

Hi @tsdavs @Xend1992 please make a development build and check the log file for any concrete error messages that we can work with.
From your reports I did not understand whether there is no communication because of the Websocket Client problems or because of problems with JSON .NET. For both should be alternatives available that work on your platform.
There are two Websocket Client implementations available in ROS# already. You can add your own by implementing the IProtocol interface.
Here is one JSON .NET implemantion which is compatible with ROS# and which might be better suited for our Unity projects than the existing one...

@dwhit do you have any further advice?

[MartinBischoff]

Please help me with your platforms guys. Is this issue a duplicate of #184 ?

[AsWeSee]

Hi @MartinBischoff. I had the exact same issue as described on top and your advice to use JSON .NET implemantion helped.

Short manual to fix the issue:

1. Delete from RosSharp/Plugins Newtonsoft.Json but keep Newtonsoft.Json.Bson
2. Import json asset from the asset store.
   There will be errors about possible runtime crashes, but in practice it works fine.

[MartinBischoff]

Thank your for this info @AsWeSee . It will likely be useful for others having the same problem.

I'm think about replacing replace the official Newtonsoft JSON .NET by the JSON .NET for Unity version.

As the ROS# libraries can also be used for non-Unity projects, I'd prefer keeping the official version. Also using Newtonsoft.BSON and JSON .NET for Unity in parallel is not so nice, since it seems the latter can handle both JSON and BSON...

[berkayalpcakal]

Hello @tsdavs, @Xend1992, @AsWeSee,

There has been an improvement in ROS# with #292 which adds a new Json serialization tool. You can get rid of Newtonsoft serialization tool and use .NET Json serialization API instead. Could you please try and see if it works with UWP?

I used to have a chance to work with HoloLens, and I successfully setup a proper two way communication bewteen ROS and HoloLens using this fork of ROS# (and this is my repo). Now I do not have the device, so I cannot do the further tests with the new serialization tool.

[dnlwbr]

Hi @berkayalpcakal, do you mean that #292 could have made the UWP fork obsolete? Unfortunately dwith's fork is not quite up to date and was last changed in May 2019. I'm no expert, so I don't know if there have been any important changes since then.

[berkayalpcakal]

Hi @dnlwbr, I do not think that #292 made UWP fork obsolete. #292 just provides an additional serialization tool so that you can get rid off the Newtonsoft dependency. As far as I remember the biggest modifications in that fork are related to threading related lines of codes, which are required to get ROS# running on a UWP platform.

Even if the UWP fork is old, you can still take advantage of it. I suggest that you clone UWP fork, then search for the use of preprocessor "WINDOWS_UWP" in the entire library solution. You can compare and see what are the major modifications to get ROS# running on a UWP platform.

I would like to adapt the work in the UWP fork into current ROS#, but I do not have a UWP platform to test it, so it would be nice if you comment your experiences here.

[dnlwbr]

I can't say much about it yet, but it seems that using Reflection is a problem with UWP builds, see dwhit#3 (comment). There is a fork that works around this problem (EricVoll/ros-sharp@6583bfd). If you plan to integrate UWP functionality into the current ROS#, you may want to pay attention to this.

[helenol]

Hi @berkayalpcakal , I couldn't find your e-mail but we're in Zurich and we can loan you a Hololens 1 device. :) (And would love a solution to the UWP problems!) Could you please get in touch with me by e-mail? eloleyni @ microsoft.com

[MartinBischoff]

Thank you @helenol for the offer to loan a Hololens. 😉 Our problem is not purchasing one. 😉

If we decided for officially supporting UWP (or other currently incompatible platforms) here, community would request us to maintain that in the future.
For the UWP, changes are necessary which effectively result in two different ROS# versions. This would effectively double the testing and maintenance effort for all code changes we make from then on.
In case you find a way for us to support UWP without increasing effort or harming the compatibility, performance and ease of use of other platforms, please let us know.
I want to keep ROS# as simple as it is. I hope you understand.

@dwhit kindly provided a ROS# UWP version for some time on his fork. You, as well as everyone else here is kindly invited to provide platform-specific ROS# forks.

[EricVoll]

@MartinBischoff
We performed some minor changes to dwhit's fork and had ROS# running on the HoloLens 1. The maintenance should not be too bad I think. This commit on my fork is all that was needed to run his fork on the HL and I am sure that there are nicer ways with a bit more time to think about them. The commit also removes some of the compiler-directives that differentiated between UWP/Editor/Windows platform builds etc.

I can help out if you want to.

[MartinBischoff]

Here is the link to @EricVoll UWP fork of ROS#: https://github.com/EricVoll/ros-sharp