Spaceport 1.0 : How To Capture Volumetric Video?

6 min readFeb 2, 2021

Today is a perfect day for Spaceport Volumetric Video Capturing And Streaming. We are proud to announce Spaceport 1.0 with Registration & Capture & Playback improvements. Before we publish this release, a big thank you to the community and to the users that have helped to announce the major version of Spaceport with their commits and feedback.

A quick recap: In the first article of this series, we gave a rough overview of the Spaceport Volumetric Video Capturing and Streaming infrastructure and application process. In the next articles, we highlighted the Volumetric Video Container Structure, Volumetric Video Live Streaming, and major use cases involved in the Volumetric Video evolution. This article is going to explore the fundamentals of volumetric video applications bringing this new video technique to fruition and we are going to learn how we use applications inside of Spaceport to capture volumetric video and then play it.

1 — Initial Configuration

Before moving forward, you have to have already installed ROS and Ubuntu 18.04. ROS Melodic Morenia is supported until April 2023, when Ubuntu 18.04 will reach EOL as well. So, you can install ROS Noetic by then.

If you want to perform a quick installation, you can follow these steps. However, keep in mind that not all details are mentioned in the steps here. For a detailed description visit here.

Step 1: The first step is adding the ROS repository to your ubuntu sources.list

sudo sh -c 'echo "deb http://packages.ros.org/ros/ubuntu $(lsb_release -sc) main" > /etc/apt/sources.list.d/ros-latest.list'

Step 2: Add Official ROS repo keyring

sudo apt-key adv --keyserver 'hkp://keyserver.ubuntu.com:80' --recv-key C1CF6E31E6BADE8868B172B4F42ED6FBAB17C654sudo apt update

Install ROS Melodic desktop package on Ubuntu 18.04 Step 4:

sudo apt install ros-melodic-desktop-full

Step 5: Let’s configure our environment. This is a very important step, once we have it done, working with ROS will be smooth.

echo "source /opt/ros/melodic/setup.bash" >> ~/.bashrc source ~/.bashrc

We are almost there!

Step 6: To install this tool and other dependencies for building ROS packages, run:

sudo apt install python-rosdep python-rosinstall python-rosinstall-generator python-wstool build-essential

With the following, you can initialize Step 7:rosdep.

sudo apt install python-rosdep sudo rosdep init rosdep update

Now, you have ROS Melodic installed and ready to use!

Step 8: At this point, we must have everything in place. Let’s try some ROS commands to make sure the installation has finished successfully.

roscore

Your terminal must be stuck here, therefore, you can not execute anything else while you have the roscore process there. Roscore is running and ready to serve other ROS processes!

2 — Spaceport Network Setup

To run Spaceport on multiple machines, all the devices must be on the same local network. While working on multiple machines, you need only one roscore running. Choose one device for it- we will call it master. The next part of the tutorial is explained over 3 computers and 6 camera setup. For convenience, the devices are assigned sequential IP addresses. You can customize the devices and network settings for yourself In this scenario, hal is both master and client. marvin1 and marvin2 are clients.

Network Options for configuration are:

hal (192.168.1.51) (master) marvin1(192.168.1.50) (client) marvin2(192.1.168.1.52) (client)

Step 1: We have to tell what the hostname means. To do it edit /etc/hostnames file with sudo privileges for all computers:

sudo gedit /etc/hosts

Add the following lines to the host file:

Step 2: For all devices open the .bashrc file. Add the following line:

gedit ~/.bashrc export ROS_MASTER_URL=http://hal:11311

Step 3: Basic check — Please Try to ping each machine from itself.

ping hal

In the terminal, you should see similar lines below:

PING hal(192.168.1.50) 56(84) bytes of data. 64 bytes from hal(192.168.1.50): icmp_seq=1 til=64 time=0.309ms 64 bytes from hal (192.168.1.50): icmp_seq=1 til=64 time=0.233ms 64 bytes from hal (192.168.1.50): icmp_seq=1 til=64 time=0.293ms 64 bytes from hal (192.168.1.50): icmp_seq=1 til=64 time=0.296ms

Step 4: Then please try to ping between machines

Now, you are ready. After completing this tutorial you should be able to configure your devices to work together and exchange data with each other.

3 — Getting Transformation Matrixes

To get started you need to:

Plug 6 Azure Kinect Sensor and c omplete the Initial Configuration Steps on each machine
All of the computers you will use should be on the same network
Print the calibration pattern “pattern.jpg” on a piece of paper

Step 1:Choose a computer that will act as a server (this computer may also be a client at the same time), we recommend that it is the most powerful of the available machines.

Our default setup:

hal: Nvidia GTX 1070 — i7 8700K

marvin1: Nvidia GTX 1060 — i5 7400
marvin2: Nvidia GTX 1060 — i5 7400

Step 3: You can connect to all devices one by one with ssh and then run the publishers. Or use the sendjob script under the script folder. In a nutshell, the sendjob script will connect to your devices and run publishers on each device.

cd VolumetricVideoRegistration/binaries/spaceport-ros/scripts/ bash sendjob.sh

Step 4: In order to run the clients you need the printed calibration pattern. Once printed, attach it to something rigid and place it in a position where it is visible to all sensors.

Step 5: Create a serial.txt. The format should be as follows

null = for reference

camera0 = looking to the same side as the reference camera

1 = looking to the opposite side with the reference camera

Step 6: Then run the startRegistration.sh on the chosen machine.

cd /VolumetricVideoRegistration/src/registraiton bash startReg.sh

Step 7: Press S to get the image and then Press Q to skip the next step

Step 8: Find contours with the trackbar for both image and then press space

Step 9: Back to Step 7 until all camera pairs are registered

Result: You are going to get the transformation matrices for each camera in a similar way below.

000286592912_to_000009594512.txt 000286592912_to_000018594512.txt 000286592912_to_000023294512.txt 000286592912_to_000274192912.txt 000286592912_to_000288392912.txt

Once the Transformation matrices of all cameras have been calculated, move these values into binaries/spaceport/TransDir.

3 — Capture Volumetric Video

You are ready to capture volumetric video. To start the capture process you can use script under the startCapture binaries/spaceport folder. But before running it; you have to change the serial names with your Azure Kinect Sensor Serials in the script.

And then:

cd VolumetricVideoRegistration/binaries/spaceport/ bash startKinfu.sh

Seeing the following screen means the capture process has started. Simultaneously, your video will be saved to disk. Use ctrl + s to safely terminate the capture process.

When the consumer process is completed you can use the playVideo script to replay your volumetric video on web browsers.

cd scripts bash playVideo.sh

Conclusion

Spaceport is ready for simultaneously volumetric capture and multi-viewpoint cloud alignment with several Azure Kinect Cameras. Hope you enjoyed the blog post. Contact us to discuss cooperation opportunities and to try out the demo.

contact@antmedia.io https://spaceport.tv

Originally published at https://antmedia.io on February 2, 2021.