Project Overview
Discover through play!
The Microsoft Mixed Reality Toolkit (MRTK 3) is an open-source framework designed to accelerate cross-platform mixed reality development. With MRTK 3 introducing a range of new features, we created the Microsoft Playground—an interactive walkthrough that helps developers experience these capabilities firsthand and seamlessly integrate them into their own projects.
Launched during the Mixed Reality Dev Days Online Hackathon in 2022, Playground transforms feature discovery into an intuitive, hands-on experience, ensuring developers can grasp, apply, and recall new tools effectively.
The Goal
Our objective was to showcase MRTK 3’s features through natural interaction, triggered by user actions rather than passive explanations. By embedding features into an engaging flow, we ensured they were intuitive, memorable, and easy to implement.
Research
Our objective was to make learning MRTK 3 more engaging, intuitive, and actionable. Instead of presenting features in isolation, we designed an experience where features are introduced through real-time interactions, making them easy to understand, recall, and implement.
Guided discovery – Features are introduced organically as users interact with the environment.
Contextual learning – Features appear in real-world use cases, reinforcing practical application.
Streamlined access – Developers can quickly reference and integrate features directly from the Playground.
Hands-on interaction – Instead of passive instruction, users learn by doing, ensuring deeper retention.
Storyline
Since we want to tell a story, this is essentially a linear narrative that guides the user through the experience. At a high level, we can break this down into three steps, which we will delve into further later.
Guiding Principles for Design
Show, Don’t Tell – Users should experience features instead of reading about them.
Progressive Learning – Features should be introduced gradually rather than all at once.
Intuitive Interactions – Users should naturally discover key functionalities without needing extensive instructions.
Collect Turbines
Build & Power
Optimize and Activate Robot
Design and UX Implementation
To address the dynamic challenges of AR environments, we designed a system that adapts to spatial variations while encouraging natural user movement.
Instead of explicit instructions, interactive objects guide users through room mapping, helping the headset capture missing data. Testing showed users often remained stationary, unsure where to move. Strategic visual cues solved this by prompting movement organically, ensuring a seamless, intuitive onboarding experience.
This “show, don’t tell” approach reduces cognitive load, fosters engagement, and makes learning feel effortless and natural.
Feature Tooltip
Setting up the experience
After the room scan, a window appears in an imperfect location, prompting users to relocate it to a suitable wall. This interaction reinforces spatial awareness and hands-on learning, leading smoothly to the next phase—placing the wind farm.
Users collect and place turbines on the floating wind farm. Upon opening a window to activate them, some turbines malfunction.
By gazing at alerts, users identify faulty parts and replace them using a wrist-mounted toolbox, restoring full functionality. This step ensures hands-on problem-solving before transitioning to the final phase.
After repairs, the control panel displays five dials, each showing different power levels. Users must adjust turbine orientation and dials to optimize airflow and power output, demonstrating MRTK3’s UX controls and real-time interaction mechanics.