Join our daily and weekly newsletters for the latest updates and exclusive content on industry-leading AI coverage. Learn More
Nvidia has unveiled a new AI platform for teaching people how to use American Sign Language to help bridge communication gaps.
The Signs platform is creating a validated dataset for sign language learners and developers of ASL-based AI applications.
It so happens that American Sign Language is the third most prevalent language in the United States —
but there are vastly fewer AI tools developed with ASL data than data representing the country’s most common languages, English and Spanish.
Nvidia, the American Society for Deaf Children and creative agency Hello Monday are helping close this gap with Signs, an interactive web platform built to support ASL learning and the development of accessible AI applications.
Sign language learners can access the platform’s validated library of ASL signs to expand their vocabulary with the help of a 3D avatar that demonstrates signs — and use an AI tool that analyzes webcam footage to receive real-time feedback on their signing. Signers of any skill level can contribute by signing specific words to help build an open-source video dataset for ASL.
The dataset — which Nvidia aims to grow to 400,000 video clips representing 1,000 signed words — is being validated by fluent ASL users and interpreters to ensure the accuracy of each sign, resulting in a high-quality visual dictionary and teaching tool.
“Most deaf children are born to hearing parents. Giving family members accessible tools like Signs to start learning ASL early enables them to open an effective communication channel with children as young as six to eight months old,” said Cheri Dowling, executive director of the American Society for Deaf Children, in a statement. “And knowing that professional ASL teachers have validated all the vocabulary on the platform, users can be confident in what they’re learning.”
Nvidia teams plan to use this dataset to further develop AI applications that break down communication barriers between the deaf and hearing communities. The data is slated to be available to the public as a resource for building accessible technologies including AI agents, digital human applications and video conferencing tools. It could also be used to enhance Signs and enable ASL platforms across the ecosystem with real-time, AI-powered support and feedback.
Whether novice or expert, volunteers can record themselves signing to contribute to the ASL dataset.
Supporting ASL education and exploring language nuance
During the data collection phase, Signs already provides a powerful platform for ASL language acquisition, offering opportunities for individuals to learn and practice an initial set of 100 signs so they can more effectively communicate with friends or family members who use ASL.
“The Signs learning platform could help families with deaf children quickly search for a specific word and see how to make the corresponding sign. It’s a tool that can help support their everyday use of ASL outside of a more formal class,” Dowling said. “I see both kids and parents exploring it — and I think they could play with it together.”
While Signs currently focuses on hand movements and finger positions for each sign, ASL also incorporates facial expressions and head movements to convey meaning. The team behind Signs is exploring how these non-manual signals can be tracked and integrated in future versions of the platform.
They’re also investigating how other nuances, like regional variations and slang terms, can be represented in Signs to enrich its ASL database — and working with researchers at the Rochester Institute of Technology’s Center for Accessibility and Inclusion Research to evaluate and further improve the user experience of the Signs platform for deaf and hard-of-hearing users.
“Improving ASL accessibility is an ongoing effort,” said Anders Jessen, founding partner of Hello Monday/DEPT, which built the Signs web platform and previously worked with the American Society for Deaf Children on Fingerspelling.xyz, an application that taught users the ASL alphabet. “Signs can serve the need for advanced AI tools that help transcend communication barriers between the deaf and hearing communities.”
The dataset behind Signs is planned for release later this year.
Start learning or contributing with Signs at signs-ai.com, and learn more about Nvidia’s trustworthy AI initiatives. Attendees of Nvidia GTC, a global AI conference taking place March 17-21 in San Jose, will be able to participate in Signs live at the event. See notice regarding software product information.
Michael Boone, manager of Trustworthy AI Product at Nvidia, said in a message to GamesBeat, “There are millions of deaf individuals globally. NVIDIA built this platform to bridge the communication gap between Deaf and hearing communities, and capture data for future AI accessibility tools.”
Asked how it works, Boone said Signs has two components: teach and record. In the teaching module, learners are taught the correct hand movement and positioning of words and given real-time instructive feedback. In the record section, proficient American Sign Language signers can record and contribute a learned word or short phrase to help build the dataset to advance accessibility technology for the deaf.
“Users of the platform can learn and improve the hand positioning and movement for a word or short phrase in American Sign Language,” Boone said.
As far as who benefits, he said, “This free AI platform benefits ASL learners of all levels, while also empowering the Deaf community through contributions that improve the platform.”
By strengthening communication across communities and creating a robust, publicly available dataset, Signs will advance research and technology development for the entire ecosystem.
“Ultimately, our goal is to connect families, friends, and communities by making ASL learning more accessible, while simultaneously enabling the creation of more inclusive AI technologies,” he said.
Source link