Meet Spaceport

Spaceport 1.1

Volumetric video is a field of media production technique that captures a three-dimensional space such as a place, person, or any object. Spaceport developed by Ant Media creates an end-to-end solution to capture dynamic scenes and offers a truly three-dimensional viewing experience with 6 Degrees of Freedom. It supports video playback on

• VR Goggles
• Web Browsers
• Mobile Devices

This document shows how you can install Spaceport and it is laid out in a fashion that allows you to use it as a reference while you’re building Spaceport. It is intended for readers who have prior exposure to Azure Kinect Sensor SDK and ROS.

1. Initial Configuration

Before moving forward, you have to have already installed ROS and Ubuntu 18.04. ROS Melodic Morenia is supported until April 2023, when Ubuntu 18.04 will reach EOL as well. So, you can install ROS Noetic by then. If you want to perform a quick installation, you can follow this documentation. However, keep in mind that not all details are mentioned here. Please, for a detailed description visit here.

Step 1: The first step is adding the ROS repository to your ubuntu sources.list

sudo sh -c 'echo "deb http://packages.ros.org/ros/ubuntu $(lsb_release -sc)main" > /etc/apt/sources.list.d/ros-latest.list'

Step 2: Add Official ROS repo keyring

sudo apt-key adv --keyserver 'hkp://keyserver.ubuntu.com:80' --recv-key C1CF6E31E6BADE8868B172B4F42ED6FBAB17C654

Step 3: And finally, update your packages list

sudo apt update

Step 4: Install ROS Melodic desktop package on Ubuntu 18.04

sudo apt install ros-melodic-desktop-full

Step 5: Let’s configure our environment. This is a very important step, once we have it done, working with ROS will be smooth.

echo "source /opt/ros/melodic/setup.bash" >> ~/.bashrc source ~/.bashrc

We are almost there!

Step 6: To install this tool and other dependencies for building ROS packages, run:

sudo apt install python-rosdep python-rosinstall python-rosinstall-generator python-wstool build-essential

Step 7: With the following, you can initialize rosdep.

sudo apt install python-rosdep sudo rosdep init rosdep update

Now, you have ROS Melodic installed and ready to use!

Step 8:At this point, we must have everything in place. Let’s try some ROS commands to make sure the installation has finished successfully.

roscore

Your terminal must be stuck here, therefore, you can not execute anything else while you have the roscore process there. Roscore is running and ready to serve other ROS processes!

3. Getting Transformation Matrixes

To get started you need to:

• Plug 6 Azure Kinect Sensor and complete the Initial Configuration Steps on each machine
• All of the computers you will use should be on the same network
• Print the calibration pattern “pattern.jpg” on a piece of paper (inside of spaceport/registration”)

Step 1: Choose a computer that will act as a server (this computer may also be a client at the same time), we recommend that it is the most powerful of the available machines.

Our default setup: hal: Nvidia GTX 1070 – i7 8700K

marvin1: Nvidia GTX 1060 – i5 7400

marvin2: Nvidia GTX 1060 – i5 7400

Step 2: Run the roscore on the chosen machine (hal).

Step 3: Use the sendjob script under the script folder to start all publishers. In a nutshell, the sendjob script will connect to your devices and run publishers on each device at the same time to avoid sync issues.

cd VolumetricVideoRegistration/scripts bash sendjob.sh

Step 4: In order to run the clients you need the printed calibration pattern. Once printed, attach it to something rigid and place it in a position where it is visible to all sensors.

Step 5: Create a serial.txt. The format should be as follows

null = for reference camera 0 = looking to the same side as the reference camera 1 = looking to the opposite side with the reference camera

Step 6: Then run the startRegistration.sh on the chosen machine.

cd /VolumetricVideoRegistration/src/registraiton bash startReg.sh

Step 7: Press S to get the image and then Press Q to skip the next step

Step 8: Find contours with the trackbar for both image and then press space

Note: After you found contours you can press s and then capture a new frame for the same camera pairs. Thus you can expand registration quality.

Step 9: Back to Step 7 until all camera pairs are registered

Result: You are going to get the transformation matrices for each camera in a similar way below.

000286592912_to_000009594512.txt 000286592912_to_000018594512.txt 000286592912_to_000023294512.txt 000286592912_to_000274192912.txt 000286592912_to_000288392912.txt

Once the Transformation matrices of all cameras have been calculated, move these values into binaries/spaceport/TransDir.

4. Capture Volumetric Video

You are ready to capture volumetric video. To start the capture process you can use volumetricCapture script under the binaries/spaceport folder. But before running it; you have to change the serial names with your Azure Kinect Sensor Serials in the script.

And then:

cd VolumetricVideoRegistration/binaries/spaceport/ bash volumetricCapture.sh

Seeing the following screen means the capture process has started. Simultaneously, your video will be saved to disk. Use ctrl + s to safely terminate the capture process.

Then you have to start volumetricProcess.sh to create the Spaceport Volumetric Container.

cd VolumetricVideoRegistration/binaries/spaceport/

bash volumetricProcess.sh

When the process is completed you can use the playVideo.sh script to replay your volumetric video on web browsers.

cd scripts

bash playVideo.sh