ETRI Develops Bidirectional Medical Sign Language Interpretation Technology

A bidirectional medical ‘sign language interpretation’ technology has been developed to lower the barriers for hearing-impaired individuals accessing hospitals.

The Electronics and Telecommunications Research Institute (ETRI), in collaboration with Chungnam National University Hospital, the Korea Deaf Association, and EQ4ALL, announced on the 14th that they have developed a bidirectional medical sign language service kiosk for the hearing-impaired.

Image of a kiosk for sign language interpretation of the health screening pre-questionnaire. Provided by Korea Electronics and Telecommunications Research Institute.

The kiosk operates by enabling communication between an avatar on the screen and the hearing-impaired person using sign language. Through this service, hearing-impaired individuals can understand the medical questionnaires they need to fill out during health checkups in sign language and respond in sign language as well.

In medical settings, important medical information such as infectious disease control and vaccination guidance has been provided through various channels since the COVID-19 pandemic. However, there is a limitation in that many hearing-impaired individuals have difficulty accessing this information.

For the hearing-impaired, the medical field is considered the area where sign language interpretation services are most urgently needed. On the other hand, it is difficult for medical institutions to assign sign language interpreters, and the fact that medical staff wear masks, preventing hearing-impaired patients from seeing lip movements and facial expressions, further creates a communication barrier between hearing-impaired patients and medical staff.

The kiosk was developed to solve these problems. When hearing-impaired individuals fill out pre-health checkup questionnaires, the avatar on the screen conveys the content of the questionnaire written in Korean into sign language, assisting in completing the questionnaire and thereby improving the accessibility of medical services for the hearing-impaired.

In particular, the kiosk provides a bidirectional function that recognizes the hearing-impaired person’s sign language and converts it into Korean. When the hearing-impaired respond in sign language, it automatically analyzes and converts the response into Korean text, which is then entered into the questionnaire. This is expected to be an important tool enabling hearing-impaired individuals to communicate their health status more accurately to medical staff.

The research team plans to develop a bidirectional sign language-Korean conversational interpreter to facilitate smooth communication between hearing-impaired individuals and medical staff in the future. They also envision expanding these functions to various public service sectors to contribute to improving the quality of life for information-vulnerable groups.

After verifying issues through hospital demonstrations, the team plans to transfer related technologies to barrier-free kiosk companies and non-face-to-face reservation system enterprises needed by people with disabilities, laying the groundwork for the nationwide spread of bidirectional medical sign language interpretation technology (kiosk).

Previously, in 2021, the research team developed an avatar sign language system to assist hearing-impaired individuals in hospital access, which was installed at Chungnam National University Hospital, enhancing convenience for people with disabilities during the COVID-19 pandemic. In 2020, they also developed COVID-19 guidelines for people with disabilities in avatar sign language and distributed them to hospitals.

Lee Tae-jin, head of ETRI’s Media Research Division, said, “In the age of artificial intelligence, essential social information such as medical, daily life, and safety information should be equally accessible to everyone,” adding, “Continuous interest and effort in developing warm ICT technologies for vulnerable groups are necessary.”