ProblemAnalysis.tex

\section{Problem Analysis}

Together with the companies from the Industrial Reality Hub mentioned in the Introduction, Saxion wants to investigate how virtual reality
can be rendered in the cloud in a safe and efficient manner. This involves looking at state-of-
the art technology in the field of virtual reality, cloud computing, rendering and machine learning for one
complete CloudVR pipeline. There are a multiple research directions in the overarching project (Multi-User Experiences, GPU Scaling, etc), however this report will focus on the following:

\subsection{Latency}
Current market players such as Google Stadia \parencite{stadia}, GeForce Now \parencite{geforcenow} and Xbox xCloud \parencite{xcloud} already offer cloud gaming services that stream games over the internet. Powerful servers are used for rendering games that are then streamed to users in real time. A bottleneck with this technology is the latency (delay). This is because user input is first sent to a server, which renders these new images, after which they are sent back to the users. All of this has to happen without compromising the user experience. The mentioned platforms all use network optimization. Low latency is very important for VR, where head movements should be converted to images in under 20 \acrfull{ms}, to prevent motion sickness \parencite{valvevrlatency}. The research for techniques for reducing latency is one of the spearheads of the CloudVR project. The following research directions are relevant here:

\paragraph{Network optimization}
As with the platforms described above, network optimization is one of the techniques
which needs to be investigated. The question is to what extent an optimized network
can reduce latency and how it relates to the quality of the network connection.
\paragraph{Two-step rendering}
One of the options to bypass latency is to render in two steps.
The delay is not so much reduced, but avoided. The server renders next to
\acrshort{rgb} also positions and BRDF variables for each pixel. Afterwards on the
user's (less powerful) hardware adjustments are made so that the image corresponds to the current position of the user.
By sending additional data, the user's local client can extrapolate the correct information and construct a frame that represents the correct head position in the last frame, meanwhile it is waiting for the  correct next frame from the server.
\paragraph{Behavioral prediction}
Another possibility to reduce latency is by predicting
user input through machine learning. This will mainly revolve around it
analyzing head movements to find out what behavior can be expected. With
this information we can render any part of the virtual world before it
is viewed by users. If this information is then forwarded from the
cloud to the location of the \acrshort{vr} experience, what information is displayed can be selected on the spot.

\subsection{Security}
If an application contains sensitive data, it is advisable to keep the data in a secure location. Previously this was impossible with VR applications, due to their high demand for computing power which meant that VR applications could only be run on a powerful, local computer. This in turn means that the sensitive data (e.g. A 3D model created from CAD drawings) is available directly on the machine and thus could be extracted from the \acrshort{gpu} for example. In a cloud \acrshort{vr} setup the sensitive data would remain on the (secure) server and only the results will be streamed to the local (unsecured) device, preventing unauthorized access since the data is never streamed directly to the local device, only the visual results. Researching how to maximize security for the clients data is the secondary major focus of this report.

\subsection{Architecture for a cloud VR system}
One of the questions to be answered is what the CloudVR architecture should look like
in terms of hardware and software. The major questions in this topic are whether to use an existing cloud computing service provider or an in-house server and which technologies for Latency reduction and Security fit best in the chosen architecture set up.