JSG: In my opinion, the most interesting topic at this moment is the futures of digital collaboration. Two colliding themes are happening. One is moving fast and starting rapidly, driven by COVID-19. The other is moving slowly and is on a long path to maturity.

The one that’s moving fast obviously is the theme of remote communication collaboration brought-upon overnight by COVID-19 – as everybody learns to work from home full-time! Many of us have acclimated to remote collaboration technologies over many years, but this has been a huge wake-up call for universal multi-industry fluency of remote collaboration. What if you don’t have the option to meet in real life?

I observe, culturally, that the AEC industry loves to meet – in all the best and worst of ways. My mother and father had an architectural practice in the 70s through the 90s. I’ve never worked as an architect personally, but I have spent these past 5 years building technologies for the AEC industry. Coming into this industry from the perspective of startups and tech companies, I observed that the AEC industry honors ‘The Theater of the Meeting’ as a powerful heartbeat in the design process. Participants sometimes spend as much as 30 to 40% of their office time preparing for these high-stakes meetings. As a % of workload for a knowledge worker, this is at once incredibly silly and inefficient - and also incredibly beautiful and human.  A refreshing departure of the email-and Slack-centric channels used by the tech industry. Now, COVID happens, and people have to figure out how to go through these motions without the benefit of a real physical office. So, that’s one of the vectors.

The other vector, which I observe as both a technologist and a father, and I chuckle when I say this, is Minecraft. Do you know it? 

DI: I’m aware of it. I’m a boomer, not a player. :( In his interview in my book, Managing Design, Arol Wolford poses Minecraft as the future platform for collaboration.

JSG: Minecraft is a funny thing, because on one hand, the aesthetic it puts forth is pixilated, low-fi art style. Even I, as a technologist and a gamer, initially looked at it and thought to myself, “Not interested. Looks like some kitschy digital experiment.” Then, you scratch the surface and you’re like, “Wow, there’s a fair amount of depth and complexity here.” And then, you dive a bit deeper, and see that it’s a wonderful creative tool, and the foundation for a generation of future collaborators. Every single one of my friends’ kids, every one of my son’s peers, that’s all they do. Every day, they watch people Minecrafting (on YouTube) and then build insanely complex creations of their own… sometimes individually, sometimes together – all in Minecraft.

I contend that Minecraft is a crisp expression about the future of design communication, and also, the future of education. It is difficult to overstate. It’s probably the most sophisticated piece of software of any kind I’ve ever encountered in my life. And the reason I say that is because in moments, my son can meet any random kid he’s never met before and, in 12 seconds – they’re building together.  One is on an iPad, another on a computer, maybe a third kid is in VR, and a fourth kid is on his Android phone and they are all in-there – in 3D – together – doing the most sophisticated collaborative BIM construction I’ve ever seen, all at the speed of light.

So – that’s the other, perhaps more hopeful vector toward futures of AEC collaboration. These two vectors are going to collide as this generation hits the workforce. They have been learning how to build collaboratively for decades in a very sophisticated way using the most enlightened tools humankind can conceive.

Meanwhile, old guys like us are chasing it. We’re trying to figure out how we can bolt together off-the-shelf and homebrewed solutions to collaborate remotely, virtually, and in the real world. It’s my belief these two things will collide in 7 to 10 years into this supernova that will change the way things are designed and built. COVID gave us a turbocharge to get us there. Now, those two lines are going to collide at a very specific future point and change the way buildings are built.

DI: That’s an emotional and technological idea. More so as accelerated by this crisis. It is exciting, as I think about it, many of these kids may not be playing games much longer. They may seize this opportunity and be in business, starting up on a shoestring. That potential is exciting.

JSG: Maybe this generation will hit the workforce and having been running on a treadmill informed by this COVID moment and we will meet at a future generation of immersive design.

Two other important themes are happening at this very moment:  Augmented reality (AR) and Virtual reality (VR). A lot of people say, “Oh, yeah. We use VR. We have VR Lab. It’s kind of passé’. We have a VR guy.” It’s almost denigrated with false praise, but AR, in my opinion, is a different breed of cat. Are you fluent in the continuum between AR and VR?

DI: My former firm, pioneered some stuff where we put a target in a model and held up the model on our iPad to augment reality. That’s the extent of my augmented reality knowledge.

JSG: There’s a bridge drug to AR - a set of solutions, kind of a magic window - where you hold your phone up in the real world and you’re looking at the phone and it projects a three-dimensional thing, which is cool. Not to dismiss those, but when I refer to AR, I’m referring to wearable eyewear with transparent lenses that can project digital content into your visual system such that it’s seamlessly blended with the real world. It’s virtual reality technology with the added feature of transparent displays.

We’re right on the precipice. In fact, a HoloLens 2 from Microsoft has a reasonable shot at being version 1.0 of augmented reality in the enterprise. At $3,500, it’s not positioned as a consumer device, but I think it has a reasonable shot at credibly allowing you bring digital information into the real world in a very natural way, and even to use your hands to interact, as opposed to specialized controllers.

DI: Like Google Glass - second generation?

JSG: No. Google Glass was eyewear with a little monocle, with low resolution and, intentionally, a very small field of view. Google Glass tried to optimize something you could wear that would be socially acceptable and wear all day. Google Glass was much more about providing a heads-up display like what you see fixed on an automotive windshield. AR blends digital content together with reality much more seamlessly.

Microsoft’s approach has been several orders of magnitude more ambitious. Their goal is to hijack your entire ocular system and field of view, so your wetware is looking at the real world. But you’re projecting digital objects into the physical world in a way that’s pinned accurately into the physical world. You could have a digital object sitting on a table and the table’s real. Your actual optical wetware is looking through the lenses at the table, but the digital system is projecting that building on the table into your eyes, so when you walk around it, it stays pinned to the table. The virtual and the physical interplay and a way that is organic and beautiful.

We’re in the earliest versions of that, but the added thing that’s super rad is that HoloLens 2 brings perfect 10 finger hand tracking, so now you can reach out and interact with these objects.

That isn’t some science fiction-y thing. It’s shipping in volume at this moment.   And – the technology is not just Microsoft’s. Apple’s working on it. Facebook’s working on it. Google’s working on it. They haven’t announced their plans, but there are multiple billion-dollar efforts flying in formation that are going to release, let’s say in years. Within the decade, it will look like glasses I’m wearing, will be able to see and track my hands and project 3D objects into the lenses.

Another trend worth tracking is Sculptural Sketching.  The only reason why 2D sketching is the primary way we interact with one another - two people, using a pencil, sitting over a paper and sketching in two dimensions, this awkward kind of flattening 3D things into 2D – it’s all we’ve had for millennia. We spent thousands of years sketching in 2D is because until this moment, it’s been easier to draw in 2D than it was to sculpt in 3D. For 3D, you had to have clay or stone. It was kind of a pain in the ass.

We are at a point where, eventually most design professionals will wear digital AR-enhanced glasses all day every day. These will allow them to sculpturally sketch… together! They’ll do it in a way so easy, so organic, and so flexible that it will become easier than drawing on a white board.  It can happen anywhere. If you so choose, it could eventually become the foundation for the building model itself. In this way, we find ourselves at an incredibly unique moment in time.

As proof of this movement, we’re just now to see major companies invest in this new set of “immersive authoring” tools. It’s more interesting than the tools we use today, which are more about walking through the model. Examples of this next generation of Sculptural Sketching is Google Tilt Brush, Google Blocks, Microsoft Maquette, Sony Dreams, and others. These tools are designed to allow you to create 3D content in an immersive and expressive way. Super-powerful.

DI: We recently migrated our publications from their former lives as dead tree documents. The DI media group website has gone live. www.di-mediagroup.com. We are transitioning to be remotely web accessible, with more of a constant feed dynamic. So, I hope we can include links to these videos in this piece. 

JSG: You should absolutely do that. The HoloLens 2 demo is quite current and contemporary, as the hardware is just now becoming available. HoloLens 2 Demo: https://youtube/e-n90xrVXh8?t=235

Another must-see clip was released five years ago associated with the awesome Future of Storytelling event.  It shows legendary Disney animator Glen Keane diving-in to the then-new domain of room-scale VR. Glen Keane – Step into the Page: https://youtu.be/GSbkn6mCfXE

Interestingly, the rig he’s using in this video, which was shot back in 2015, includes a high-end PC and external sensors, likely costing $4,000 or more. And it probably took them half a day to rig up that prototype and configuration he uses in that video. But now, that hardware and software solution is $400, and is ready for creative expression (using the same tool – TiltBrush) in about 2 minutes. No PC is needed. You just get the Oculus Quest for $400, put on your wifi, put it on your noggin, and you’re doing sculptural sketching. So, in five years the cost has reduced by 10X, the cable has disappeared, the PC is now optional, and the setup is accessible to anyone, regardless of their technical acumen.

DI: There’s no shortage of new ways of working, visualizing and collaborating, all with great potential. But as Dave Gilmore says, since many of the folks running firms are “pale, male, and stale”, we need help implementing. Most people my age are looking for their places in the pasture. We’ll be gone, but the younger folks - your son and his cohorts - have a vested interest in saving their firms when we come back from all this. But they’ll need help solving the money problem - and demonstrating new value propositions for architecture. 

Every other industry says, “Technology? Solution! Let’s invest. Let’s use it.” But in architecture, we see it as, “Oh, the technology problem. It’s an expense. We can’t afford it.” We have a blind spot for technology. So, we still have to get over that hump. Maybe when the new generation comes and we’re gone, it’ll finally be here.

JSG: There’s another thing we should consider, that feeds into the remote conversation around COVID. In five years, you’ll have eyewear that looks like mine or yours that can project content in three-dimensional space you can use to interact in an elegant way with your hands. Collaboratively, we’ll all be able to see a shared thing and work on it together.

Everything you can do together “co-present”, you can do just as easily together “distributed”. And, you can do it synchronously in the same time zone, but also asynchronously.

So, you can imagine I could drop-in and work on a form and tinker and leave that as a ghost, or an “echo” for you to drop into an experience and interact or iterate at another time, in another place.

This was the vision of our Visual Vocal project. In retrospect, maybe our first project was a bit too ambitious for the day. But this notion of being able to do something synchronously and asynchronously, something copresent and something distributed, on two axes, gives tremendous dynamic range for this new sculptural sketching pattern.

The third axis, which is thrilling and terrifying at the same time is the axis of humans and machines playing together. Particularly when you’re talking about geometric form, there are certain things humans are way worse at than machines are. So, hence the Grasshopper generation, people who grew up building sculptural forms through math. That will be just a control inside this thing where, as you’re crafting, certain people will be able to sculpt in a way that is not just their hand.  The output is informed by machine learning. That will be a whole new brush or tool. I don’t know what that tool is called in sculpture, but that’s a whole new implement you’ll be able to sculpt with in this new world.

DI: It’s called a chisel.

JSG: A “chisel”. That’s perfect. That will be the sort of machine-learning-enhanced chisel you’ll be able to place into these forms that has all the goodness of Grasshopper, but all the accessibility of AR and sculptural sketching. If we had those tools today in the context of COVID, we wouldn’t even hit a speed bump. We’d be able to just do this method.

Another thing comes to mind - holoportation, which is kind of holographic teleportation. Microsoft did a paper in 2016 that was the first time this was shown. Microsoft Holoportation: https://www.microsoft.com/en-us/research/video/holoportation-virtual-3d-teleportation-real-time/

Think of this as the next generation of videoconferencing. You and I would be whole-body-projected virtually into the same space together. We’d be interacting and I’d see you. I’d see your social cues and you’d see mine. Right now, all we’ve got is a Zoom meeting. I can at least see if you’re laughing, but there’s more possible. Granted, in the video, there’s thousands of dollars worth of hardware making it possible. The day after tomorrow, this will be built into every office and every home office in the world. So, how will Holoportation change architecture? Incredibly, I think.

DI: We had a visioning session in my former firm. We were redoing the office. We asked, “What are we going to do for AV, telecom, and collaboration?”  I said, “I want to be able to walk into the room and just start talking and interacting. I don’t want to hook up any cords. I don’t want to reserve any meetings. I don’t want to have to do anything. Just sound, sight, or motion-activated, and we’re seeing and working together and sharing data. We interact and get to work. You’re saying we can do that?

JSG:  Yes.

DI: OK. I’ve heard all this. I’m excited. What do I do to start? I’m a practitioner in a mid-sized firm. Where do I start? What do I do first? 

JSG: I get asked that a lot. Controversially, I don’t usually recommend three things you should start doing Monday morning at work because I think those things have a high disruption factor. You need to get people to agree to do it. Without buy in, you pick a tool and move on and life gets in the way. It doesn’t stick.

Rather than starting with WORK, maybe start with PLAY. Very smart, creative teams are doing a tremendous amount of experimentation with virtual worlds that forecast the same types of tools we’ll be using together very soon.

Let me share two ideas of a possible place to start, both from the domain of immersive entertainment. Both offer best-in-class VR, and maybe even allow you to scratch the itch of hanging-out (hiking, even!) during the COVID-19 pandemic?

Entry-Level:

System: Oculus Quest
(Self-contained – no PC needed $400)

Experience: Google TiltBrush ($20)

Advanced-Level:

System: Add a gaming-grade PC (must be at a specific level of performance – good guidelines are here), connect the PC to your Quest via USB cable using Oculus Link. 

(Note: Some games are too powerful to run on the headset alone, they require an attached videogame-level PC.)

Software/Experience: No Man’s Sky, Purchased via Steam (requires a PC – doesn’t run on the Quest alone).

In modern, state-of-the-art VR systems, there are games/experiences that allow you to solve problems collaboratively, and/or “Be Together Distantly” – a pattern of immense value, particularly during the COVID-19 pandemic.  However, among these, No Man’s Sky, in particular, provides a broad array of experiences, from creation to crafting to interplanetary exploration, each with excellent in-game tutorials of the in-VR UI and interaction approach.

I contend you’ll learn more faster if you find something that is joyful, exploratory, and/or creative than something that makes you think, “I really NEED to learn this.”  As a bonus, you can convince a friend to join you in VR and create and explore together!

DI: That’s always my answer too. Get your head right first. I don’t care how cool a thing is, if you don’t want to, or aren’t willing to get over that hurdle, paradigm, or mindset, you’re doomed to fail. So, you have to start with your head - and by tinkering. 

JSG: Sounds like we’re in orbits around the same planets, so, I hope these conversations continue. 

DI: John, I can’t thank you enough. I had way more fun than I thought I was going to have. Dave said you had a lot of energy. That’s clear.

JSG: The honor has been mine.  I’m such a fan and supporter of the work of Design Intelligence, and the critical role you play in elevating the conversations about design futures.”

---

John SanGiovanni has more than two decades of experience as entrepreneur and inventor. He has authored more than 20 patents in the areas of augmented reality, wearable hardware systems, and UI, many of which have subsequently been licensed or purchased by Apple, Microsoft, and others. John has led product teams which have shipped more than 30 top-tier Apps for iOS/Android, 10 of which achieved top-5 in their respective App Store category and won major digital/interactive awards. Earlier in his career, John worked in various technology/entertainment roles for the Walt Disney Company, and served as a Technical Evangelist for Microsoft Research, where he led external research funding in mobile technology and AR, and co-invented several UI metaphors now used on more than a billion devices worldwide. John cofounded and served as executive / board director at Zumobi (mobile app platform) and Visual Vocal (AR/VR spatial communication platform) and cofounded the nonprofit TeamXbot, focused on technology mentorship of inner-city teens.  SanGiovanni frequently consults, writes, and speaks on technology trends, and has been honored as an IIDA Design Pioneer and featured at the MIT Investment Forum, Microsoft Research Summit, and TEDx Seattle.