This is the final post in our series of Grand Design Challenges for 2025. You can read Part 1, part 2, part 3, and part 4 here.
Spatial Computing / Extended Reality
There is an awkward early-teen clumsiness to Extended Reality's (XR) interactions and visual aesthetic. Both are gangly, ill-fitting, and struggling to define what they are. The current state of the interaction model feels much like the days of touch interfaces trying to move beyond mimicking mouse interactions and become touch-first interactions. The design of 2D digital experiences has converged to have standard patterns, interaction models, and components, but this progress is just beginning in 3D space.
Unlike other platforms, spatial computing has no clear line between video games and non-video games. The visual expression is a split between video-game and Material Design. When you are on a personal computer (desktop, laptop, smartphone), it is obvious what type of experience you are in, but not in XR. This diffused personality heavily influenced by video games is confusing and inappropriate for non-consumer applications. Hopefully, this is just future shock and will resolve over time and with better design for two reasons. Firstly, because it is a sign of a mature and established paradigm; the other is that non-consumer applications for the foreseeable future will dominate XR.
Presenting both 2D and 3D content in 3D space is a dynamic that is exceptionally specific to XR. The dimension duality poses an entirely new problem space for designers. Representing 2D content in 3D space is critical to spatial computing because the physical world (3D space) is filled with 2D content. The challenge is beyond imposing the same rules of the physical world on content or established video-game solutions (which have merit and a place in some experiences). The dynamics of spatial computing are different. Replicating the physical world in a digital form is prohibitively expensive and neuters the “digital-ness” of digital content. Being digital is a unique phenomenon with freedoms and advantages not possible in the physical world, which designers should exploit. Conversely, designers must respect and utilize the affordances of the physical world; otherwise, why mix realities? Spatial computing is alien to us, and the grand challenge is to tame this feral space.
Focal Points:
- What are the best practices for how and when digital worlds interact with physical environments, which includes affordances to guide users as experiences transition across the AR <-> MR <-> VR spectrum?
- Is there a unified interaction model for spatial computing such that there are base or core interactions that every experience has across all devices, or is the breadth of experiences and device specifications so broad that bespoke interactions are the only option?
- What is the visual aesthetic for industrial or enterprise experiences?
- How do developers and designers reduce the production costs of creating and maintaining spatial computing experiences?
