👌 AI is Learning to Translate Speech into Sign Language

Limitless Mind, a Chinese startup, is developing an AI that can translate between spoken and sign languages. The lightweight model, optimized for use with smart glasses and smartphones, enables deaf and hearing people to communicate without requiring special training. The project is still in the prototype stage and is currently seeking investors.

While many apps can convert speech into text, not all people with hearing loss read fluently—for many, signs are their first and most intuitive form of communication. However, gestures vary across regions like dialects, and full understanding depends on hand movement, facial expression, head tilt, and signing speed.

The researchers collected a dataset of about 12,000 annotated videos covering different "dialects" of sign language, each labeled with 3D coordinates for hand, body, and facial movements. With this data, they trained the model to recognize gestures in video and convert them into text—and to do the reverse, turning text or speech into gestures performed by a virtual 3D avatar.

Limitless Mind isn't the only team working on AI sign language translation. Cloud-based solutions have already been developed by Chinese companies Baidu and Alibaba, and Google recently unveiled SignGemma for American Sign Language.

@hiaimediaen