Praying mantis eyes inspire breakthroughs in machine vision

By Stephen Beech via SWNS

The eyes of the praying mantis have inspired state-of-the-art vision for robots, self-driving cars and home security systems.

American scientists have sharpened the 'eyesight' of machines by mimicking nature and taking advanced computing to the edge.

They explained that self-driving cars occasionally crash because their visual systems can’t always process static or slow-moving objects in 3D space.

In that regard, they’re like the monocular vision of many insects, whose compound eyes provide great motion-tracking and a wide field of view - but poor depth perception.

However, the praying mantis is one of few insects with the ability to perceive 3D space.

Now engineers are replicating their visual systems to make machines see better.

They explained that a praying mantis’ field of view also overlaps between its left and right eyes, creating "binocular vision" with depth perception in 3D space.

Researchers at the University of Virginia combined that with state-of-the-art optoelectrical engineering and innovative “edge” computing - processing data in or near the sensors that capture it - to develop artificial compound eyes that overcome limitations in the way machines currently collect and process visual data.

They say current limitations include accuracy issues, data processing lag times and the need for substantial computational power.

Study first author Byungjoon Bae said: “After studying how praying mantis eyes work, we realized a biomimetic system that replicates their biological capabilities required developing new technologies."

The team’s meticulously designed “eyes” mimic nature by integrating microlenses and multiple photodiodes, which produce an electrical current when exposed to light.

The researchers used flexible semiconductor materials to emulate the convex shapes and faceted positions within mantis eyes.

Doctoral candidate Bae said: “Making the sensor in hemispherical geometry while maintaining its functionality is a state-of-the-art achievement, providing a wide field of view and superior depth perception.

“The system delivers precise spatial awareness in real-time, which is essential for applications that interact with dynamic surroundings.”

He said such uses include low-power vehicles and drones, self-driving vehicles, robotic assembly, surveillance and security systems, and smart home devices.

Among the research team’s important findings on the lab’s prototype system was a potential reduction in power consumption by more than 400 times compared to traditional visual systems.

Rather than using cloud computing, Bae says the system can process visual information in real-time, nearly eliminating the time and resource costs of data transfer and external computation while minimizing energy usage.

He added: “The technological breakthrough of this work lies in the integration of flexible semiconductor materials, conformal devices that preserve the exact angles within the device, an in-sensor memory component, and unique post-processing algorithms."

The team says the key is that the sensor array continuously monitors changes in the scene, identifying which pixels have changed and encoding this information into smaller data sets for processing.

The approach, described in the journal Science Robotics , mirrors how insects perceive the world through visual cues, differentiating pixels between scenes to understand motion and spatial data.

The team says praying mantis eyes are special because, like us, they use stereopsis - seeing with both eyes to perceive depth - as well as their hemispherical compound eye geometries and motion parallax to understand their surroundings.

Study first author Professor Kyusang Lee said “The seamless fusion of these advanced materials and algorithms enables real-time, efficient and accurate 3D spatiotemporal perception."

He added: “Our team’s work represents a significant scientific insight that could inspire other engineers and scientists by demonstrating a clever, biomimetic solution to complex visual processing challenges."

The post Praying mantis eyes inspire breakthroughs in machine vision appeared first on Talker .