(Pocket-lint) – People are still learning from the coronavirus pandemic, including that there is no substitute for being together in a room with someone.
While video conferencing apps such as Zoom, Google Meet, and Microsoft Teams try to help you virtually connect with those you love and work with, the experience still feels rather limited. So, in an attempt to push the boundaries of remote collaboration, Google has come up with Project Starline.
What is Google Project Starline?
- Early video calling system that renders in 3D, no glasses required
Google CEO Sundar Pichai took the stage at Google I/O 2021 to announce that his company kicked off a project “several years ago” that builds on different areas of computer science and relies on custom-built hardware and highly specialised equipment. Called Project Starline, it’s an early system that essentially allows you to video call someone and experience them in hyper-realistic 3D.
How does Google Project Starline work?
There are three components to Project Starline:
- Cameras and depth sensors: Specialised equipment that captures a person from multiple perspectives
- Computer science advances: Custom software, including novel compression and streaming algorithms
- Light field display: Custom hardware that renders a realistic representation of someone in 3D
Project Starline essentially uses high-resing cameras and custom depth sensors to capture a user’s shape and appearance from multiple perspectives, and then all that is fused together by software to create an extremely detailed, real-time 3d model. Google said it’s applying research in computer vision, machine learning, spatial audio, and real-time compression. The effect is the feeling of a person sitting just across from you.
The resulting data is also huge – many gigabits per second.
So, to send this 3D imagery over existing networks, Google developed novel compression and streaming algorithms that reduce the data by a factor of more than 100. Google also developed a light field display that shows you the realistic representation of someone sitting right in front of you in three dimensions.
As you move your head and body, Google’s system can adjust the images you see in the light field display to match your perspective. It creates a sense of volume and depth without the need for additional glasses or headsets. “You can talk naturally gesture and make eye contact,” CEO Sundar Pichai described at I/O 2021. “It’s as close as we can get to the feeling of sitting across from someone.”
When will Project Starline be available?
- It’s currently available in a few of Google’s offices
Google said it’s already spent thousands of hours testing Project Starline in its own offices. Although there are no plans to commercially release a product for consumers, it said there’s been excitement from enterprise partners, and it is planning to expand access to partners in healthcare and media.
Want to know more?
Check out Google’s blog post for more details.
Writing by Maggie Tillman.