Skip to content
This repository has been archived by the owner on Jul 19, 2024. It is now read-only.

Latest commit

 

History

History
51 lines (37 loc) · 4.02 KB

README.md

File metadata and controls

51 lines (37 loc) · 4.02 KB
Raw

Mixed Remote View Compositor

Overview

Mixed Remote View Compositor(MRVC) provides the ability for developers to incorporate near real-time viewing of HoloLens experiences from within a viewing application. This is achieved through low level Media Foundation components that use a lightweight network layer to transmit the data from the device to a remote pc viewing application. Through the Media Foundation capture pipeline, the sink components capture and encode the live camera data with its associated data. The data is then transmitted to the remote viewing application that will be decoded and displayed with respect to the device transformations.

There are two methods of operation:

  1. Low latency mixed reality capture (MRC)

    • Provides the ability to create a MRC companion app experience with lower latency (~150ms)
    • Similar to Device Portal and Companion app experiences for mixed reality capture
    • Getting Started

  2. Mixed Remote View Compositing

    • Provides the ability to composite the mixed reality capture on a remote machine, allowing the HoloLens user experience to remain at full frame rate
    • Capture HoloLens camera images (PV camera) with its transformation data and transmits data to a remote application.
    • Using an additional network connection, synchronize scene and input state data to reflect any changes to the world objects.
    • Composite a rendered frame of the scene object with PV camera image to re-create a mixed reality image.
    • Getting Started

What files are included?

Builds - This folder will be populated with the libraries built from the Source project. see Source for build instructions.

Samples - sample code

  • LowLatencyMRC

    • Unity - sample application that demonstrates using library to capture MRC on the HoloLens
    • UWP - sample viewing application to view live MRC video from the HoloLens
    • Learn More...

  • MRVC

    • Sample Unity project that will produce both the HoloLens experience application and the remote viewing application using the same codebase.
    • Demonstrates how to use the plug-in and other additional files to demonstrate serializing data to send to a remote application.
    • Learn More...

    Note: the network layer code was designed for use with the media pipeline only. It's overloaded use in this sample is provided as an example to demonstrate sending scene and input state data to the remote application. Consider using a different networking framework to sync scene and input states.

Source - Native C++ files used to build components for Win32 and UWP applications. These include the Unity plug-in wrapper, Network and Media Foundation components. Learn More...

Additional Resources

Developers should already be familiar with and have previous experience with developing HoloLens and/or Unity Applications. Here are resources to get started:

For more technical background on Media Foundation: