WebVR Support - THREE.js & WebVR boilerplate

[sidequestlegend]

A basic example using WebVR to display the VNC canvas in VR with THREE.js.

[CendioOssman]

It's a fun experiment, but I have to ask what the point would be? VNC is fundamentally 2D so I don't quite see what this adds.

[kanaka]

@ShaneHarris that's really neat, do you have any video of it in action? Is this just a fun project or do you have something you are using this for? I'm not sure it's appropriate for the main tree, however, I would be happy to see a link to your repository added to the list here: https://github.com/novnc/noVNC/wiki/Projects-and-companies-using-noVNC

[sidequestlegend]

@kanaka yes i think your right its probably a bit out there for the main tree, would be awesome to add to the list. No video as of yet ill try to get one uploaded to youtube! I do have this demo of the screen distortion for large resolutions -

To be honest i took a notion one evening and threw it together in a few hours code bashing so not really devoted currently. It just hooks into the main no vnc canvas and uses that as a texture for a mesh in THREE.js. I guess it's inspired by products like Big Screen VR and Virtual Desktop but i haven't even spared a thought to fundamentals like mouse & keyboard input or advanced configuration.

I would like to see if its possible to hook straight into the separate differential tiles sent from the remote frame buffer and apply the tiles directly to separate meshes in THREE.js. This might give a bit of a performance boost to cut out the middle man so to speak.

What is unique about this approach is the ability to visualise much higher resolutions, the above demo uses a 4k resolution on a laptop that supports a native max of 1360x768. I can enlarge the screen in VR and wrap it around a sphere geometry to compensate for the higher resolution.

[DirectXMan12]

I would like to see if its possible to hook straight into the separate differential tiles sent from the remote frame buffer and apply the tiles directly to separate meshes in THREE.js. This might give a bit of a performance boost to cut out the middle man so to speak.

You could pass a custom version of Display to the RFB object. Some of the encodings might be more amenable to that than others.

[sidequestlegend]

@DirectXMan12 yes indeed i noticed most of the frame buffer and canvas guts going on in there I think that is probably the best way to hook into the direct feed.

I have been looking at input too. It seems there are a few options but its not quite there yet. Here are some things i have considered:

Leap Motion - https://www.youtube.com/watch?v=ckAGpmf21a8
I have one of these sitting there gathering dust. It may be a good option but the lack of haptic feedback makes it tricky. I believe this example uses a like for like physical keyboard aligned between VR and the real world to remove that problem, but this seems inaccessible. This gesture device seems to offer other alternatives too - https://www.youtube.com/watch?v=VmEy9jxwte4 im not sure if its any more accessible than the previous though...

There is always the cheap and cheerful webcam feed used in the RiftSketch project. https://www.youtube.com/watch?v=SKPYx4CEIlM

I wonder how many computers i could VNC at once in VR with this library. Any idea what the limitations would be there?

@kanaka I have created a new repo and i think a better approach is to add the noVNC repo as a submodule. its bare at the minute but ill be pushing the first commit this afternoon.

https://github.com/shaneharris/construct

[samhed]

Added to list of projects in wiki commit 4c509cb