Handwriting Analysis with a Single Image... UNIST and Seoul National University Develop World's First Optical Tactile Sensor

Joint Research Team Develops High-Resolution Polymer and Luminescent Composite Sensor Mimicking Skin Sensory Structure

From Fingerprint and Braille Recognition to Handwriting Analysis... Published in Nature Communications

An optical tactile sensor, attracting attention as a next-generation biometric technology, has been developed.

This sensor surpasses the limitations of conventional optical sensors by being able to analyze dynamic forces with a single image, making it highly promising for applications such as handwriting analysis, surface inspection, and anti-counterfeiting.

A joint research team led by professors Lee Jisuk, Ko Hyunhyeop, and Kim Donghyuk from the Department of Energy and Chemical Engineering at UNIST (President Park Jongrae), along with Professor Kim Jungwook from Seoul National University, has developed an optical tactile sensor capable of real-time analysis of moving touch signals.

From the left, Professor Lee Jiseok, Baek Dahye, Researcher Ryu Chaeyoung, Kim Donghyuk, Professor Ko Hyunhyup, (top row) Son Changil, Kim Jinyoung, Researcher Park Seojeong.

While existing sensors could only measure static forces, the research team implemented a technology that can separate and analyze both static and dynamic forces. In particular, in the field of handwriting analysis, they visually represented changes in writing speed and pressure, opening new possibilities for personal identification through machine learning analysis.

The core of this technology lies in upconversion nanoparticles. By utilizing these, dynamic forces can be measured with high resolution, and near-infrared absorption enables precise detection of externally applied forces.

The research team applied machine learning techniques to analyze the data collected by the sensor with greater precision. Using machine learning algorithms, they separated vertical pressure and frictional shear force from dynamic touch signals and accurately identified the direction of the force. The validity of force transmission paths and signal changes within the sensor was also verified using finite element analysis (FEA).

The sensor structure amplifies force by mimicking the sensory organs of human skin. It can simultaneously distinguish vertical pressure and frictional shear force with a single optical image. The sensor can detect even minute forces of 0.05N generated by a gentle press and demonstrated a rapid response time of 9.12ms.

The developed sensor can be applied not only to handwriting recognition but also to fingerprint and Braille recognition, among other fields. In fact, the research team applied this sensor to a system that converts Braille to voice and showcased dynamic biometric systems and anti-counterfeiting applications.

Optical tactile sensor mimicking human skin sensory organs.

Professor Lee Jisuk explained, "This is the first study to simultaneously visualize static pressure and dynamic frictional force by mimicking the structure of skin sensory organs and to separate and analyze the two forces in real time using machine learning. Upconversion nanoparticles are less expensive and produce clearer signals with less noise compared to conventional materials."

First author Son Changil emphasized, "Even with this simple sensor structure, it is highly likely to be used as a dynamic pressure quantification sensor for high-sensitivity handwriting detection in the future." Co-first author Ryu Chaeyoung expressed expectations, saying, "It will contribute to the development of AI learning-based sensors applicable to robotics."

The research results were published in the world-renowned journal Nature Communications on September 12. The research was supported by the Samsung Future Technology Development Program and the National Research Foundation of Korea.