Sequencing Diversified Policy Variants For Targeted Object Retrieval From Cluttered Shelves
After using open loop approaches to object retrieval from shelves, I found that the majority of failures were from the robot being unable to react. Oftentimes the object was easily graspable if it were able to nudge neighboring objects out of the way slightly. Or as it was grasping, the object would slip or be nudged and a simple readjustment would fix it. Other times the object was not in a graspable configuration and a simple reorientation would be the remedy. This experience built my interest in closed loop grasping approaches.
Beginning based off of ClutterDexGrasp I was able to recreate their results on MuJoCo with a UR5e with a pinch gripper using FastTD3 and slight changes to their observations and rewards. As I added in a shelf and the goal of not knocking over any object, ClutterDexGrasp's monolithic approach failed almost entirely. Breaking the task into parts of pushing then grasping with an overhaul of the ClutterDexGrasp training pipeline with new rewards and removing the curriculum allowed it to begin to learn something on this harder setup. The success rates on individually pushing and grasping only hovered around 60-70%, failing consistently on harder scenes. After intensively watching the policy failure modes, I thought of the idea of sequencing variants of the policies with mcts on the given scene for both grasping and pushing separately. This increased success to 95~98% on each scene.
So far the distillation technique from ClutterDexGrasp has not succeeded on this harder long horizon manipulation task that has multiple distinct stages. We are currently looking to distill a task planner and individual distilled policies for each stage.