The robot has learned to read Braille twice as fast as humans.

Scientists have developed a robotic sensor that recognizes Braille font twice as fast as most people. To achieve this, it uses not only unusual tactile sensors but also a highly effective neural network.

Robotics is actively developing, and the tasks facing it require the creation of algorithms endowed with some human-like properties. For example, to automate delivery, it is necessary to improve robots’ ability to navigate in space. And in order to explore the resources of the oceans, robots need to be taught to conduct «tactile probing,» i.e., to identify objects through touch. Scientists have already developed «magnetic skin» that allows this: with its help, the algorithm is able to at least differentiate between animate and inanimate objects.

Thanks to the sensitivity of human fingertips, a person receives a vast amount of information about the surrounding world. The softness of the skin allows regulating interaction with the environment, such as the force with which people grip objects. Creating a technology that possesses both of these properties simultaneously is difficult. However, researchers from the University of Cambridge (United Kingdom) have succeeded in doing so at a high level.

In an article published in the journal IEEE Robotics and Automation Letters, researchers described a robotic sensor that recognizes Braille font more accurately and faster than its counterparts. Its predecessors read letter by letter and detached from the surface after each symbol — people do not perceive Braille text in this way.

The new technology works with the font almost like a human, but twice as fast. A neural network model called YOLO v8 was used for Braille font recognition, which is relatively resource-efficient.

According to the scientists, the robot processes information from the camera and sensory sensors simultaneously, but when it moves, the image blurs, making it difficult to recognize letters. To avoid this problem, researchers trained the model on artificially blurred texts written in Braille font. Thanks to computer vision technology, the algorithm was able to detect and classify each symbol.

The resulting robot was tested by quickly moving it along the Braille text lines. The sensor read 315 words per minute with 87% accuracy. This is twice as fast as most people who understand the font, with almost the same number of errors. Scientists noted that using artificial blurring, such a result can be considered remarkable.

Initially, the technology was not intended as an assistive tool: reading Braille font was simply an effective way to test the sensor’s sensitivity. Similar developments are useful, for example, for prosthetics with artificial skin, as well as for applications that determine surface texture. However, the final algorithm can become an assistant for visually impaired individuals: researchers have managed to achieve a balance of speed and accuracy close to human levels.