Tight Robot Packing in the Real World
A Complete Manipulation Pipeline with Robust Primitives
Jingjin Yu, Abdeslam Boularias, and Kostas E. Bekris
Abstract: — Many order fulfillment applications in logistics, such as packing, involve picking objects from unstructured piles before tightly arranging them in bins or shipping containers. Desirable robotic solutions in this space need to be lowcost, robust, easily deployable and simple to control. The current work proposes a complete pipeline for solving packing tasks for cuboid objects, given access only to RGB-D data and a single robot arm with a vacuum-based end-effector, which is also used as a pushing or dragging finger. The pipeline integrates perception for detecting the objects and planning so as to properly pick and place objects. The key challenges correspond to sensing noise and failures in execution, which appear at multiple steps of the process. To achieve robustness, three uncertainty-reducing manipulation primitives are proposed, which take advantage of the end-effector’s and the workspace’s compliance, to successfully and tightly pack multiple cuboid objects. The overall solution is demonstrated to be robust to execution and perception errors. The impact of each manipulation primitive is evaluated in extensive realworld experiments by considering different versions of the pipeline. Furthermore, an open-source simulation framework is provided for modeling such packing operations. Ablation studies are performed within this simulation environment to evaluate features of the proposed primitives..
Fig 1: Left: Pipeline in terms of control, data flow (green lines) and failure handling (red lines). The blocks identify the modules of the system. Sensing receives an RGBD image of initial bin and object CAD models to return a grasp point. Based on the picking surface, the object is either transferred to the target bin or is handled by the Toppling module, which flips the object and places it back in initial bin. When the object is transferred, a robust Placement module places the object at the target pose. The Packing module validates and corrects the placement to achieve tight packing. Right: a) Instance segmentation. b) Pose estimation and picking point selection are provided by sensing, c) Picking d) Toppling e) Placement and f) Packing.
Experiments
V1- Our method: The complete pipeline with all the primitives achieves the highest accuracy and success rate.
V2- No corrective actions: The experiment corresponds to the use of V1 without the packing module of Fig 1, that performs corrective actions.
V3- No push-to-place actions: This version is V2 without the use of the robust placement module (Fig 1) that does push actions to achieve robust placements.
V4- No toppling actions: These experiments used V2 without considering toppling actions to deal with objects not exposing a valid surface that allows the target placement.
V5- No push-to-place, toppling, pose-estimation: The naive baseline that solely uses a pose-unaware grasping module that reports locally graspable points and drops the grasped object at an end-effector pose raised from the center of the desired object position, with no adjustment in orientation.
Fig 2: Left: The final set of object poses in the target poses at the end of every experiment. Different column represents different versions. The top row is the best case, and the bottom row is the worst case. Right: the blue bar represents the fraction of successful object transfers, the orange bar represents the percentage of unoccupied volume within the ideal target placement volume.
Video recording of all Experiments
Experiment set V1: OUR APPROACHExperiment set V2: W/O CORRECTIVE ACTIONS
Experiment set V3: W/O PUSH TO PLACE PRIMITIVE
Experiment set V4: W/O TOPPLING ACTION
Experiment set V5: W/O POSE ESTIMATION
Rutgers University New Brunswick New Jersey USA