Soon after the COVID-19 pandemic hit, many offices quickly and unexpectedly moved to the cloud. Now, these “offices” only really exist in the abstract, relying on a patchwork of real-time messaging services like Slack, e-mail, and a suddenly endless stream of video conferences. The quick switch didn’t allow time for the idea of a truly digital office to take shape and deliver on the lofty promises of virtual reality spaces in which we could all hang out and handle our daily tasks.
During the keynote at its Ignite conference today, however, Microsoft provided an early look at its vision of its holographic space for human collaboration. The platform, called Mesh, would enable users to wear augmented reality headsets (such as Microsoft’s own HoloLens gear) and offer virtual collaborative opportunities across a variety of apps from games to Office.
The idea goes beyond some of the more straightforward VR office concepts we’ve seen from companies like Facebook. Mesh envisions a true mixed reality application, in which holographic avatars can interact with objects that appear in your physical spaces. Imagine standing in your living room while a large interactive globe spins in front of you and a co-worker or teacher’s holographic avatar interacts with it.
From the video demos, it’s clear that Microsoft doesn’t envision the digital elements to fool participants into thinking they’re physically there. The other people’s avatars have clear indicators of their digital nature, whether it’s a generally cartoony appearance or a vaguely translucent nature consistent with what pop culture often presents as a hologram.
Though the project is in its early stages, Microsoft is offering companies a version of the AltspaceVR app, which is an enterprise-grade meeting solution for companies that want to hold virtual meetings. Eventually, however, the Mesh tech will roll up directly into Microsoft’s own work platform Teams.
The challenge with mixing realities
For a straightforward VR experience, apps can keep the process relatively simple. The program can plunk users in virtual seats around a digital space and use the headgear’s internal motion sensors to track participants’ head movements. Bringing precisely mapped digital objects and team members into a real-world space, however, offers a more in-depth challenge.
For photorealistic “holoportation” as Microsoft calls it, outside sensors need to capture real-time movement information. Creating a truly accurate hologram requires a complex setup like Microsoft’s Mixed Reality Capture Studio, which involves controlled lighting and high-tech sensors. Mesh comes with a scaled-back version of this tech that allows users to create less-detailed, customizable avatars that work right out of the box.
In addition to tracking the people, the system also needs to keep tabs on the location to allow people to interact with objects that may only exist in the cloud. Spatial maps handle this task by providing a detailed, three-dimensional picture of the space you’re in, as well as the other objects around you.
If you’ve ever used a VR headset or even something like the Xbox Kinect, then you’ve already experienced something similar. Mesh just envisions it on a much larger scale.
What can we actually do with this tech?
In many ways, Mesh feels like the natural extension of HoloLens. Microsoft’s head-mounted smart display already overlaid digital directions and objects onto real-world settings. Mesh can bring more people into the mix and allow them to manipulate digital versions of those objects right along with you.
As with HoloLens in general, Microsoft obviously considers industry its main customer base for the moment, but it’s looking beyond the work environment to other settings, including theaters. MIcrosoft is working with the aquatic research organization, OceanX, to add “holographic laboratories” to their ships in order to aid in its scientific pursuits.
The company also suggests a situation in which people from various locations attend one central performance. The company has already begun work with the company behind Cirque Du Soleil to provide access to shows via mixed reality.
Will we all have to wear a Hololens?
For now, Microsoft has stated its commitment to making Mesh available across different devices. If you want to experience the three-dimensional aspects of the technology, you’ll need an AR or VR headset, but the company hopes to make the experiences equally accessible for people using smartphones or tablets.
This device-inclusive attitude has been a part of Microsoft’s Mixed Reality vision for years now and it seems to be sticking with it, even as the concept and implementations evolve.
For now, the Mesh implementations are still only available in specific settings, most of which won’t roll out to the general public for some time. But, if Microsoft remains committed to the project and other developers jump on board to develop for the platform, it could provide an early look at what remote collaboration looks like as physical offices struggle to bounce back in the wake of the pandemic. Before long, you may be able to say “this virtual meeting could have been an email,” directly to the organizer’s holographic face.
