Definition: Spatial computing is a form of human-computer interaction in which a system captures and understands its physical environment and anchors digital content at real locations and objects. Instead of working on a flat screen, users interact with information and 3D models in the space: with gaze, hand, voice or touch.
Simon Greenwold coined the term at the MIT Media Lab in 2003. He used it to describe systems that store and process relations to real objects and spaces. So spatial computing is not a device and not a single technology. It is a concept that combines sensor technology, computer vision, real-time rendering and new forms of input.
In B2B mechanical engineering, companies use spatial computing to: show machines at their original size at the customer site, show service information directly on the system, do design reviews in a shared space, train on the virtual object and plan layouts in the real hall.
Advantage: Complex technology becomes easy to understand at the place where it will stand and operate later.
English: Spatial Computing
German: Räumliches Computing, raumbezogene Datenverarbeitung
How It Works
From the screen into the space
A conventional computer knows only pixel coordinates on a display. A spatial computing system also knows the position of walls, floors, tables, machines and people. For this, four steps run continuously in parallel:
- Capture the environment: Cameras, depth sensors and LiDAR measure the space. Inertial sensors capture the movements of the device (see Sensors).
- Determine the position: Methods such as SLAM build a map of the environment and locate the device in it in real time.
- Understand the space: Software detects planes, edges, objects and distinctive image features. You can attach content to these anchors, for example with markerless tracking or marker tracking.
- Show and operate content: The graphics are rendered in the correct perspective. Input comes from hand tracking, eye tracking, voice, controllers or touch.
Spatial anchors
The core of the concept is the anchor: a saved position in the real space to which a digital object is bound. When the user moves, the object stays at its location. Persistent anchors last longer than one session. So on the next day, a technician finds the same overlay at the same position on the machine.
Difference from AR, VR and XR
Spatial computing is the umbrella term for the interaction principle. Augmented reality, mixed reality and virtual reality are forms of representation that use this principle to different degrees. Extended reality (XR) groups these forms of representation together. Spatial computing, on the other hand, describes how a system brings together space, objects and users, independent of the output device.
| Term | Focus |
|---|---|
| Spatial computing | Concept: the computer understands and uses the real space |
| Augmented reality | Digital content over the real view |
| Mixed reality | Digital content reacts to real objects |
| Virtual reality | Fully digital environment |
| Extended reality | Collective term for AR, MR and VR |
Platforms and Device Classes
Spatial computing is not tied to a headset. Depending on the use, a tablet or a permanently installed screen is sufficient:
- Smartphones and tablets: With ARKit and ARCore, mobile devices capture planes and anchors. For sales and trade shows, this is often the lowest entry barrier.
- Headsets: Mixed reality headsets capture the space with several cameras and allow hands-free operation. Apple explicitly calls the Vision Pro a "spatial computer". For details about the operating system, see the entry visionOS. Devices with Android XR and Meta Quest also use spatial anchors. In the industrial environment, the HoloLens was common for a long time.
- Web: With the WebXR Device API, you can create spatial experiences in the browser without an app installation.
- Kiosks and large screens: Cameras and depth sensors capture visitors in front of a display. Content reacts to position and gestures, for example at the booth.
One model, many output channels: The same optimized 3D model can run in the web viewer, on the tablet, in the headset and on the kiosk. The spatial layer is added depending on the device. The data basis stays the same.
B2B Applications
Sales: machines in the customer's space
A sales engineer places a packaging line at its original size in the customer's hall. Distances to walls, columns and escape routes are visible immediately. Questions about the installation area are answered on site instead of with drawings weeks later.
Trade shows: large systems without transport
Instead of shipping a system that weighs tons, the company shows it digitally at the booth: in the headset, on tablets or on a large screen. Visitors walk around the machine and open protective covers virtually.
Service: information where the work happens
A service technician sees maintenance steps, torques or warnings directly on the related assembly. With live data from the IoT, you can show operating states on the real system. In combination with a digital twin, you can compare the target state with the actual state.
Training: learning on the virtual object
Operators practice setup procedures or troubleshooting on a spatially displayed model before the real machine is available. Errors have no consequences for material or the system.
Design review: decide together in the space
Design, production and the customer examine a draft at a 1:1 scale. Accessibility for maintenance, ergonomics at the control panel and collisions with existing infrastructure become visible early.
Challenges
- Data preparation: CAD data is too heavy for real-time display and must be optimized through a 3D pipeline.
- Spatial data and data privacy: Cameras capture halls, systems and people. Companies must clarify where spatial data is processed (see data privacy and data security).
- Device variety: Platforms differ in anchors, input and management. Open standards such as OpenXR decrease the effort.
- Environment: Reflective surfaces, bad light and little structure make spatial capture more difficult.
Frequently Asked Questions (FAQ)
What is the difference between spatial computing and augmented reality?
Augmented reality is a form of representation: digital content over the real view. Spatial computing is the higher-level principle by which a system understands the space and anchors content in it. AR is a common form of it, but VR applications with spatial capture and sensor-based kiosk systems are also part of it.
Does spatial computing always need a headset?
No. Tablets and smartphones with ARKit or ARCore can do spatial capture and anchors. Headsets give the advantage of free hands and a larger field of view. For sales and trade shows, a tablet is often sufficient. For service with both hands on the tool, a headset is useful.
Is spatial computing the same as visionOS or Apple Vision Pro?
No. Apple uses the term to position the Vision Pro, but the concept is older and independent of manufacturers. visionOS is an operating system that implements spatial computing. Other platforms do this too.
Which data does a company need to get started?
The basis is 3D models of the products, usually derived from CAD data. They are reduced, given materials and exported to formats such as glTF or USDZ. For service applications, documentation and, if applicable, machine data are added.
How secure is the captured spatial data?
This depends on the device, the platform and the application. Many systems process camera images locally and give apps only derived information such as planes or anchors. Before use in production areas, companies should check which data leaves the device and where it is stored.
Industry Standards and Technologies
- OpenXR – open Khronos standard for XR applications, including extensions for spatial anchors and planes
- WebXR Device API – W3C standard for AR and VR in the browser
- OpenUSD – Alliance for OpenUSD, scene description for spatial 3D content
- glTF 2.0 – open format for real-time 3D content
- ISO/IEC 18039 – reference model for mixed and augmented reality
- Spatial Computing (Greenwold, MIT 2003) – thesis in which the term was coined
Related Topics
Forms of representation: - Augmented reality – digital content in the real view - Mixed reality – merging of real and digital - Virtual reality – fully virtual environment - Extended reality (XR) – collective term - Hologram – representation with a spatial effect
Platforms: - visionOS – Apple's operating system for spatial computing - HoloLens – mixed reality headset - ARKit – Apple's AR framework - ARCore – Google's AR platform - Android XR – Google's platform for XR headsets
Tracking and sensor technology: - SLAM – mapping and localization - LiDAR – laser-based depth measurement - Sensors – capture of movement and environment - Eye tracking – gaze capture - Markerless tracking – tracking without markers - Marker tracking – tracking with reference markers - Hand tracking – gesture recognition
3D data: - 3D model – digital geometry - 3D pipeline – from CAD to real-time display - Real-time rendering – real-time graphics - USDZ – 3D format for Apple platforms - WebGL – 3D in the browser
Applications: - Digital twin – digital replica of the system - IoT – connected machine data - Immersive experience – immersion in content - Immersion – degree of immersion - 3D product configurators – interactive variants
