Chinese robot dogs get foot-eyes for unmatched off-road skills

Quadruped robots often struggle with off-road exploration in unstructured environments due to their sensorless feet, which provide limited information about foot interactions with the surroundings.

To address this issue, Chinese researchers have introduced Foot Vision, an innovative vision-based sensorized foot designed to enhance the capabilities of quadruped robots.

The team’s robot design is comprised of an internal camera, a transparent foot sole with visible marks, and a compliant ankle.

Researchers at the Southern University of Science and Technology in Shenzhen aimed for a foot design that is affordable, lightweight, and adaptable to both rigid and deformable surfaces.

The details of the team’s research were published in the journal IEEE Robotics and Automation Letters .

Enhanced robot stability

Contact perception is vital for the stability of legged robots. Traditional methods estimate ground reaction forces (GRFs) to prevent slippage and maintain control. However, these methods often struggle with accurate terrain adaptation, especially on rough surfaces.

According to researchers, current approaches using dynamics models and joint torques are complex and sensitive to changes, while plane-fitting methods introduce errors on uneven terrain. Sensorized feet with inertial measurement units (IMUs), force F/T sensors, or FSRs have limitations such as high cost, fragility, and poor performance under dynamic conditions.

To address these challenges, the team sought to design a vision-based tactile sensor similar to those used in robotic manipulation, but specifically as a low-cost, lightweight, and robust sensor for legged robots.

Researchers claim that Foot Vision represents the first integration of a vision-based sensor into a dynamic-legged robot. It uniquely combines the abilities to sense 6D contact forces, estimate surface inclinations, and perceive foot-terrain interactions, all using a single camera.

Innovative foot design

The team designed Foot Vision to enhance the contact perception capabilities of quadruped robots and ensure stable locomotion. The vision-based sensorized foot is tailored for integration into a Unitree Go1 quadruped robot, adhering to specific design specifications.

These include maintaining compatible dimensions and weight, maximizing the foot sole area to reduce sinkage on deformable terrains, ensuring a straightforward and robust sensing system, and positioning a compliant ankle lower for stability without obstructing the visual system.

Foot Vision weighs 120 g and is 60 mm in height, featuring a flat circular foot sole with a 100 mm diameter. The foot sole, made of transparent acrylic and a central hollow 3D-printed plate, uses black countersunk screws as visual markers.

An fisheye camera with a 180° FOV captures foot-terrain interaction. According to researchers, the ankle, made of TPU, deforms radially under force without obstructing the camera’s view, and the entire system is sealed with waterproof fabric to prevent dust and water ingress.

Dynamic terrain sensing

Experiments showed that Foot Vision can accurately and dynamically predict contact forces and torques in real-time (100 Hz) with an accuracy of over 95 percent.

When installed on a quadruped robot , the sensorized foot effectively detects 6D contact forces on various terrains and estimates the local surface inclination. The transparent foot sole lets the in-foot camera identify terrain types and directly measure soil flow distribution.

According to researchers, experimental results confirm that this vision-based sensorized foot significantly enhances the contact perception capabilities of legged robots in diverse environments.

Future improvements

While Foot Vision shows promise, there is room for improvement. Prolonged use may cause material fatigue in the compliant ankle, affecting contact force accuracy.

Researchers plan to use more durable materials and refine the fabrication process. Enhancing the tread pattern with grouser pads will improve traction, and a light shield will minimize LED reflections, preserving visibility.

Furthermore, integrating Foot Vision data into the state estimator will improve the accuracy of contact and inclination estimation. The team also plans to optimize foot pose estimation and control algorithms for better locomotion stability.