AiRPlay
Inclusive AR Development
Overview
Role
Tools
Professors
Duration
[problem]
Individuals with mobility disabilities face significant barriers to full participation in traditional sports
Sports often require mobility abilities that exclude those using aids (e.g., wheelchairs, prosthetics). Limited participation from these activities undermines shared experiences with peers and can harm self-esteem and mental well-being.
Traditional sports lack adaptive frameworks to accommodate these diverse abilities.
so...
How might we build an inclusive exercise and sports systems for people of varying mobility abilities?
LONG-TERM GOALS:
- supports co-play between users of all physical abilities
- is intuitive, fun, and accessible
- can be deployed in real-world school or community settings
[solution]
AiRPlay is a projection-based augmented reality game system, designed for inclusive physical entertainment and exercise.
*Diagram of AiRPlay system and its components.
A major design element the system relies on is the player's peripersonal circle. Using the player detection algorithm that we built, we draw rough circles around each player and use that as part of the core game mechanic. Players can expand their peripersonal circle by stretching their arms and/or legs; this action can be reimagined as a “kick” in certain games, which is normally not possible among wheelchair users in traditional sports games. This effectively allows players of different mobilities to participate in games together!
[system components]
Let's break it down...
Projector
projects game on the ground (e.g. hockey field)
Camera, ROS2
camera feed is used as input for player detection algorithms; ROS2 acts as a communication bridge by sending realtime player position and radius data to Unity
Unity Engine
renders players' peripersonal circles given ROS2 data; runs general game logic
Kick Button
allows users to expand their peripersonal circle if they are unable to expand with their arms and legs
Web App
controls game settings and sets up players and scoring
[development goals]
How can we further develop the existing AiRPlay system so it is ready to be integrated into school systems?
When I joined the AiRPlay team, there was an early air hockey game running on a basic projection system. But it was buggy, hard to use, and required a developer just to get it working — making it impractical for schools and community centers to adopt.
PREVIOUS STATE
*Tabletop version of the previous state of the game system featuring air hockey. (An onsite version existed but I don't have a good video of it :C)
Buggy player detection
High latency between player movement and system response
Lack of gamification (e.g. sound effects, scoring, etc.)
Zero onboarding
Developer-only operation, requiring command-line setup
MY IMPACT GOALS AS A DEVLOPER
To address these pain points, I focused on improvements that expanded gameplay, improved detection, enhanced UX, and made the system more user-friendly and scalable.
Expand Gameplay
Improve Player Detection
Enhance User Experience
Enable Convenient Operation
Create Documentation
Key Highlights
01
Polishing and System Improvements
I brought the original Air Hockey game from an early prototype closer to a finalized experience that could be deployed in real-world school and community settings through gamification polish and bug fixes.
Key Improvements
- Fixed player detection bugs and latency for smoother, more accurate gameplay
- Added gamification elements: rules, scoring, sound effects, and a timer to encourage competition and playfulness
✦ Outcomes
- Increased stability and replay value — making the game more fun and accessible
02
New Game: Shapes Catcher
To expand iGYM's gameplay variety and promote inclusive, social play, I developed a new game — Shape Catcher — from scratch. I integrated it into the existing system using Unity and ROS2, while refactoring the codebase for future games.
*Players must match their peripersonal circle to falling shapes — encouraging movement and coordination.
Process
We collaborated closely with the design team to brainstorm and evaluate game concepts that aligned with our goals and constraints:
- Encourage users to play together (cooperative or competitive) over solo experiences
- Give players of all mobilities equal advantages
- Maximize use of the peripersonal circle
- Ensured ideas were technically feasible within our system's capabilities and timeline
*Ongoing list of games that were brainstormed during a paper prototyping session.
✦ Outcomes
- Added a second engaging game option, increasing the replay value and adaptability of the system for different settings.
- Improved the system's overall codebase for easier future expansion.
03
Web App Integration
To make iGYM usable by teachers and community staff, I helped integrate a web app that allows users to customize game settings without the need for developer control. We were also able to finally project a tutorial/onboarding process instead of just the game, allowing users to feel more guided throughout the experience.
*Internally testing new web app controls. That's me in the wheelchair!
*Customizable game parameters controlled by web app settings
✦ Outcomes
- Enabled staff to operate the system independently with no coding required
- Reduced setup and troubleshooting time for play sessions
- Made the system more scalable for schools and community centers
[reflection]
How Working as a Developer Shaped My Perspective as a Designer
Working as a developer on AiRPlay gave me a rare inside look at how design teams work — and showed me what good collaboration between designers and developers should look like. This experience shaped how I approach design today: with a deep respect for technical feasibility, clear specs, and real user needs.