The Evolution and Future of Immersive, Real-Time Technologies

by  Joel Pennington

Up until now, we in A/E/C have spent our careers working through a layer of abstraction—a two-dimensional screen. We design and build something that is three dimensional, but the 2D screen acts as the interface between us and what we’re designing. Removing the 2D layer of abstraction has proven to be an effective solution and is revolutionizing the industry.

We’re entering a new era in technology as industries will move away from the WIMP (Windows, icons, menus, pointer) paradigm to a fundamental 3D experience in the computing platform. To that end, VIM AEC is building the next generation of immersive technology products with the goal of making them more accessible.

How did we get here?
A key breakthrough in real-time immersion began in 1968 with University of Utah computer scientist Ivan Sutherland’s Sword of Damocles, funded to test the idea of presenting the user with a perspective image which changes as he moves. In 1985, NASA’s Dr. Michael McGreevy noticed RadioShack’s new Citizen Watch Co portable TVs were LCD-based. He decided to put these lightweight screens on astronaut’s helmets, so they could engage in virtual reality training programs.

A few years later, new prototypes like Autodesk’s Cyberspace enabled new users to visualize a building before construction, with the aim of helping reduce ambiguity and bridge the gap owners often feel when trying to make sense of 2D drawings. Virtual Reality also hit a stride through established corporations like SEGA and Nintendo.

Unfortunately, the bubble burst, and the immersive technology failed because the ancillary technology required to make it work well wasn’t ready yet. Nintendo sold three-quarters of a million VR units before pulling its Virtual Boy product as users complained of motion sickness.

By leveraging technology from a booming smart phone market a couple decades later, the next generation of VR immersive technology includes Google Cardboard (2014); Oculus Rift (2013); and HTC Vive (2015). HTC’s business leadership was so bullish on VR that it sold its phone business to focus entirely on VR. Facebook bought Oculus and continues to drive for consumer adoption, while the main gains in VR adoption have been in the enterprise space. This is because consumers are fickle and driven by how quickly and easily they can be entertained by a device; the lowest energy needed is currently via a smart phone. When a VR or AR provider wants consumers to access their experience, it requires so much activation energy by the consumer, that its experience needs to be 10 times better than simply watching something on a smart phone. This is a very high bar to pass, and a factor in why consumer VR and AR has not taken off.

In 2012–2013, a good VR experience meant paying an expensive Unity or Unreal artist for a creation that allows a designer and a customer to look at rooms together. Today, we can have a collaborative, multi-user experience where people are either remotely or physically together, collaborating in the same digital space. They’re using tools that allow them to section the model, change finishes, make notes, do RFIs, and more.

At VIM AEC, we’re testing ideas where, with Magic Leap, we can give someone an immersive experience in life-size and see an augmentation of BIM in the real world. It’s more accessible for people who aren’t comfortable going into a myopic experience like virtual reality.

Time and money
In the A/E/C industry, two important considerations are time and money. For every $25 million spent on a project with access to immersive technology, often in AR, these are potential ROIs:

• Site analysis, up to $10,000
• Designer view options, $10,000
• Design mock-up visualization, $100,000
• Construction documents, $10,000
• Coordination and detailing, $10,000
• Pre-con visualization, $25,000
• Pre-fab visualization, $100,000
• Pre-fab assembly, $100,000
• Layout, up to $1M
• Installation, $100,000+
• Verification and quality control, $50,000
• Commission and testing, $25,000
• Operations, $1M+
• Maintenance, $1M+
• Emergency first responders’ safety analysis, $1M+

At Skanska’s Tampa office, value engineering work specialists using Autodesk went from a table-based, Excel-type experience to VR. When these specialists put owners in a room and allowed them to change the floor or the ceiling finish, they could see the cost change immediately. Business development account managers began using VR for winning projects— increasing their win rate by up to 50 percent, all because they could offer VR. The result was $400 million in extra revenue.

In adjacent industries, Bell Helicopter used VR in the design review, multiplied their speed by ten, and designed better helicopters. In shipbuilding, Newport News used AR to reduce inspections from 36 hours to 90 minutes, resulting in savings of $80 million a year.

In A/E/C, VR is very impactful early in design. But AR takes over quickly and lives on throughout the life of that building, helping it operate more efficiently and reducing the lifetime costs.

The technologies under the hood
The majority of AR and VR apps and experiences rely on two core pieces of technology, Unity and Unreal. Neither is designed to work with parametric design data, BIM, or CAD files, but they’ve been successful because access to their technology is readily available. There are more than seven million third-party software developers split between these two game engines. Anyone can go onto Upwork or Fiverr and, within minutes, hire developers ready to start building real-time experiences.

Unfortunately, trying to convert parametric models or solids into what the game engines need—lightweight, clean polygons— is a big task. It takes money, time, and expertise, as well as the potential loss of data. The industry has discovered it can take hundreds of hours to develop products before the experience is even usable.

At VIM AEC, we are building a new data platform that understands design intent from parametric design tools like Revit. This allows our customers to automatically receive lightweight, clean polygons for use in the game engine quickly and effectively. With this breakthrough, new experiences will be possible without having to spend a lot of time manipulating data.

Where we’re headed
In 1982, Autodesk had the first CAD application for the IBM PC, but it took more than a decade before they became a real player thanks to Intel and Microsoft, who weren’t angling to democratize the A/E/C platform but simply make computers more accessible. Luckily companies like Autodesk leveraged that new platform well. At VIM AEC, we have heeded this as a warning and make keeping up with adjacent technologies and their impact on A/E/C a priority.

In the video game and film industry, concept artists are building in virtual reality rather than desktop design software because it’s a fundamentally better way to work. Instead of a 2D screen, they’re building in 3D for 3D-like sculptors. Concept artists love this because they don’t have to be perfect with their lines; they can just be creative with no filter.

You might say that A/E/C has rules—unlike a video game or movie, we have to actually construct a building. Unreal engine and Rhino know this, which is why they have funded Gabriel Sorento’s Mindesk, a real-time VR parametric modeler connected directly to Rhino that allows designers to conceptually build with rules. Because enterprise solutions for VR and AR are key, you can bet Google is right behind them, along with Facebook, Microsoft and some startups.

I am excited about how we can enable the next generation of designers to design and build. As responsible human beings, we have to manage the consequences of this technology ourselves. We can’t count on government rules to fix it for us, but I do have hope that it will solve real problems in the enterprise business space. If it reduces the time of design and construction and gives owners more information so buildings run more efficiently, then maybe the good we’re doing outweighs potential consequences.

Editor’s note: This is an adaptation of Joel’s presentation at the 2019 Leadership Summit on Technology & Applied Innovation, sponsored by Design Futures Council.

Joel Pennington is head of product at VIM AEC. He oversees product vision, strategy, design and development.