Creating a place where kids of all abilities can play together

As a power wheelchair user, James Falahee had become used to being left out.

“I think that the chance for disabled people to be able to participate in sports is very important, because throughout most of my life, I have been forced to sit on the sidelines and watch my friends play sports and games,” he told The University Record.

While there are invaluable adaptive sporting opportunities for those with mobile disabilities, such as powerchair football, these opportunities aren’t designed to allow disabled and able-bodied people to play together. Now, thanks to an interdisciplinary team of University of Michigan researchers, Falahee can play with and compete against his friends, regardless of anyone’s physical abilities.

iGYM is an augmented reality system that emulates a life-sized version of air hockey. The field, goals, and ball are projections that interact with players. Players can pass the ball to their teammates and score goals.

The project is funded by U-M’s Exercise and Sport Science Initiative (ESSI) program, and is led by Roland Graf, an Associate Professor of art and design at the Penny W. Stamps School of Art & Design. Graf, Sun Young Park, an Assistant Professor of information at the School of Information, and Michael Nebeling, an Assistant Professor of information at the School of Information, designed the initial concept of the project, but they needed someone to create the actual system. They then recruited ECE Assistant Professor, Hun-Seok Kim.

“The initial idea was to make a real physical robot, an autonomous ball, that actually moved around,” Kim says, “but that turned out to be a very challenging process.”

The team decided to create an augmented reality system instead. Two regular projectors are mounted on the ceiling next to a camera. A computer vision algorithm then compares each image frame to the reference image without any players, allowing the algorithm to detect when a player is on a certain area of the field.

“Based on that information, we can project the virtual balls and goals, and we simulate the physics between the ball and the person,” Kim says. “And that’s how the gameplay works.”

While this creates the actual gameplay, the team also wanted to implement parameters for a more inclusive competition. These parameters – such as the friction of the ball to the playing surface, the size and speed of the ball, the goal size, and the bounciness of the ball – could be adjusted for each individual so that the players’ skill levels would be more evenly matched.

To do this, the team first had each player play the game in single-player mode. Single-player mode resembles squash where the is no goal and the player only plays the ball. This mode allowed the team to profile each person’s play style and skill. They were then able to adjust the parameters for each player during group games to ensure a more level gameplay – much like handicaps in golf or bowling.

“We found that having this ability to adjust the game parameters made it more enjoyable, more accessible, and more inclusive for everyone,” Kim says.

The most difficult parameter was dealing with what Kim’s team coined as the “peripersonal circle.” The peripersonal circle is an individual’s immediate area of motion. An able-bodied person can easily stretch this area by lunging or pivoting, but someone with a mobile disability may be more confined. This creates an unfair advantage for the able-bodied person.

To address this, Kim and his team created a button the disabled player can press that expands their peripersonal circle, emanating a kick. They created many different kinds of buttons to suit different players’ abilities. Some buttons were large; some were small. Some buttons could be pressed by fingers or hands; others could be attached to the inside of one’s knees. The player would get whichever button best suited their abilities.

Another challenge was accommodating shadows, which can easily be mistaken for another object on the field.

“We are currently using an algorithm that compares the current frame with the reference in the different domains where there’s no shadow,” Kim says. “By combining different domains, we have a way to eliminate the shadow from the background.”

The aesthetics of the game can also be easily adjusted. You can change the color of the lines, the layout of the field, or the type of ball. You can also add audio of a crowd cheering every time a goal is scored or stadium chants to hype up the players. The team hopes to expand the program to simulate other games like bowling, ping pong, an interactive obstacle course, or life-size Pac-Man.

The game itself has been well-received by kids and adults alike, with many participants returning frequently.

“We had a case where these two brothers had never played a physical game before, because their abilities are quite different,” Kim said. “We saw some teary eyes from the parents, and it was a very rewarding moment to see them having a lot of fun.”

The team spent eight months developing the program, and it’s been up and running for over a year. Kim has used the past year to refine the system and has improved many of its algorithms.

“We can now predict where the player will be in the next image frame based on the history of the player in the previous frames,” Kim said. “This allows us to limit the search range for detecting a player, which makes the movement of the peripheral circle smoother and the overall system faster, because you’re reducing the number of computations required for image processing.”

The team hopes to release their program to the public so schools, rec-centers, and gyms all over the world can install and operate the system themselves, but there’s still more work to be done before this can become a reality. One of the biggest constraints to the system is that the ceiling needs to be high enough for the projectors to create a large enough play space.

“Currently, the system is very customized to our specific environment,” Kim says. “We are planning to automate the process, so once you enter the coordinates of different equipment – like the camera and the projector – the algorithm will automatically calculate the parameters for the game.”

Kim and his team published their work, “iGYM: An Interactive Floor Projection System for Inclusive Exergame Environments,” with a focus on the peripheral circle, and it won the Best Paper award at CHI-PLAY 2019, a subconference of the international Computer Human Interface Conference. CHI-PLAY focuses on research in games and future of digital play and accepts fewer than 30% of submitted papers.

iGYM is currently open and available to the public for use. It’s located in the design studio on North campus, and you can book time to use it simply by contacting [email protected].