Invention by local high school girls could lead to changes in the way hospital staff communicate

HOLMDEL — Three local high school students have captured worldwide recognition for their invention: A robot that uses AI to read American Sign Language and play corresponding piano keys.

“We wanted to sort of bridge the gap between … people who use Sign Language on a day-to-day basis and people who speak spoken language,” said Aditi Gopalakrishnan, 16 of Piscataway.

The musically inclined robot is only a preliminary phase of a bigger plan. The three girls want to pave the way for a future where deaf people can more effectively communicate with medical professionals without the need of a human translator.

"I volunteer in hospitals, so I've encountered patients who have language barriers," Gopalakrishnan said, noting that sometimes people who can translate are not always available.

Gopalakrishnan explained that their current model could work in the medical field as a fully-equipped, multi-language AI translator, allowing hospital staffers to communicate with people who are deaf or use Sign Language.

Maya Baireddy, 15, of Holmdel, and Julia Chan, 15, of Bridgewater make up the other members of the JAM session team; the three initials of their first names combining to create the J, A and M.

RoboCup Junior is an international competition that encourages high-school-aged students to explore their interest in technology. The JAM team represented the United States.

On July 16, the three competed in the “onstage,” or performance, challenge which took place in Eindhoven, Netherlands.

For their performance, the USA team created a green monster with large googly eyes that would play keys on a keyboard that corresponded to Sign Language letters the girls formed with their hands. The girls used training data and AI tools to have the robot recognize the letters as they held up a hand to the camera on the back of the contraption.

Out of 24 teams and 21 countries participating, JAM session earned second place.

Gopalakrishnan, Baireddy and Chan attend Woodbridge Academy Magnet School, High Technology High School in Middletown, and Bridgewater Raritan High School, respectively.

They worked on the project at the Storming Robots robotics center in Branchburg, Somerset County, where their mentor recommended that they take part in the RoboCup Junior competition.

The three teens eventually won first place in the national competition in May, at the Wardlaw-Hartridge School in Edison. That punched their tickets as the only U.S. representative in the Netherlands.

Gopalakrishnan spearheaded the sign language detection algorithm; Baireddy worked on the frame, costume, and 3D printing the mechanisms that pressed the keys; and Chan worked on the motors, which allowed for the precise movement of the mechanisms that pressed the keys.

The team had been working on the project since last December, but never lost interest in robotics.

"Probably the opposite," said Chan.

Baireddy added, "I actually enjoyed it more after this," and the other girls agreed vehemently.

The girls emphasized that through the competitions they learned the importance of communication with each other, time management and scheduling, as well as the importance of having spare parts to help avoid possible mechanical failures.

"Spares are very, very very important," Chan.

They said that Murphy's Law -- the idea that anything that could go wrong will -- was a big part of the process. Now, they can laugh about it.

Parts broke and even the frame fell apart at one point in the national preparations. The girls used those failures to better prepare for the international competition.

They now want to turn their good showing into something that spreads. The trio intend to start a group that encourages young girls to find mentorship in robotics.

They noted that STEM fields are male-dominated, about 80 percent of competitors in the Junior and Major leagues of the RoboCup competitions were male.

"We've been thinking about starting a group that helps people, especially younger girls," said Gopalakrishnan. "We've all been at a point in our robotics journey where we wanted a mentor."

This article originally appeared on Asbury Park Press: Invention by local high school girls could lead to changes in the way hospital staff communicate