RoboGrocery: MIT’s automated soft robot packs groceries with record accuracy

An MIT team has developed a new robot designed to automate the task of bagging items at grocery stores.

The new soft robotic system, developed by researchers at the institute’s Computer Science and Artificial Intelligence Laboratory (CSIL), consists of soft-touch sensors, motor-based proprioception, enhanced vision technologies, and a novel algorithm.

Named RoboGrocery, the system can manage a constant flow of random objects moving on a conveyor belt.

According to researchers, its soft robotic arm uses multiple sensors to measure an object’s size and firmness. This helps it transform the vague idea of a “well-packed container” into clear, measurable goals.

The details of the team’s research were published in the journal IEEE Xplore.

Advanced robotic packing

Traditional robotic bin-packing has focused on rigid, rectangular items, often struggling with objects of different shapes, sizes, and stiffness.

“The challenge here is making immediate decisions about whether to pack an item or not, especially since we make no assumptions about the object as it comes down the conveyor belt,” said Annan Zhang, a Ph.D. student at MIT CSAIL and the study’s lead author, in a statement published on The Robot Report .

MIT’s RoboGrocery overcomes these limitations with a unique combination of RGB-D cameras , closed-loop control servo motors, and soft tactile sensors . The RGB-D cameras capture depth and color data, allowing precise identification of objects’ shapes and sizes on the conveyor belt.

According to the researchers, the servo motors provide accurate control and feedback, enabling the gripper to adjust its hold based on each object’s properties.

Additionally, the gripper’s integrated sensors measure pressure and deformation, offering insights into an object’s stiffness and fragility. This advanced system allows RoboGrocery to handle a diverse array of items effectively.

Precision item sorting

To test the RoboGrocer’s efficacy, researchers randomly placed ten different, realistic grocery items on a conveyor belt and repeated this process three times. They evaluated the system by counting how often heavy items were placed on top of delicate ones.

MIT CSAIL reported that the soft robotic system significantly outperformed traditional methods, performing nine times fewer item-damaging actions than a sensorless baseline, which used pre-programmed motions, and 4.5 times fewer than a vision-only approach that lacked tactile sensing.

In a scenario where grapes and a can of soup arrived on the conveyor, the RGB-D camera identified and sized them. The gripper gently picked up the grapes, and tactile sensors noted their delicacy, placing them in a buffer. The gripper then picked up the soup, detected as not delicate, and packed it directly into the bin. Finally, the grapes were carefully placed on top.

According to researchers, the system, managed by a microprocessor, demonstrated robust performance with various items, including bread, chips, soup cans, and ice cream containers.

While the system has been successful, there’s still room for improvement. The researchers noted that the current method for determining an item’s delicacy is rather basic.

“Currently, our grasping methods are quite basic, but enhancing these techniques can lead to significant improvements. For example, determining the optimal grasp direction to minimize failed attempts and efficiently handle items placed on the conveyor belt in unfavorable orientations,” said Zhang.

“A cereal box lying flat might be too large to grasp from above, but standing upright, it could be perfectly manageable,” he explained.