This repo contains the official implementation of GD2P. It consists of:
-
A dataset of pushing/pulling hand poses and corresponding object meshes.
-
Pre-trained models for generating pushing/pulling hand poses and motin planning scripts.
-
Code for dataset generation, visualization, model training, nonprehensile motion planning (arm + hand).
gd2p
├── assets // Allegro/LEAP hand URDFs, table, and box assets
├── data // Dataset, models, and outputs (downloaded separately)
├── docs // Documentation
└── gd2p // Source code
├── dataset_generation // Generate hand poses + Isaac Gym filtering
├── generative_model // BPS-conditioned diffusion model (train + sample)
├── pushing_motion_planning // Nonprehensile motion planning (arm + hand)
├── utils // Shared utilities
└── visualization // Visualization tools
Installation instructions here.
Please run all commands from the root directory of this repository.
Define contact candidates generation. We generated /assets/allegro_hand_description/contact_points_precision_whole_hand.json as our candidates, emphasizing the distribution all over the hand to facilitate pushing and pulling. To generate new configurations:
python gd2p/dataset_generation/scripts/generate_allegro_contact_points_precision_push.py
Then, to check the defined contact candidates, run
python gd2p/visualization/scripts/visualize_hand_model.py
We can generate hand poses now with
python gd2p/dataset_generation/scripts/generate_hand_config_dicts.py
python gd2p/dataset_generation/scripts/generate_grasp_config_dicts.py
Visualize the optimized hand pose by
python gd2p/visualization/scripts/visualize_config_dict.py --meshdata_root_path ${MESHDATA_ROOT_PATH} --input_config_dicts_path ${PUSH_CONFIG_DICTS_PATH} --object_code_and_scale_str ${OBJECT_CODE_AND_SCALE_STR} --idx_to_visualize 0
We'll evaluate the generated hand poses with Issacgym to test whether it leads to a success.
python gd2p/dataset_generation/scripts/eval_grasp_config_dict.py --meshdata_root_path ${MESHDATA_ROOT_PATH} --input_grasp_config_dicts_path data/dataset/NEW/grasp_config_dicts --output_evaled_grasp_config_dicts_path None --object_code_and_scale_str ${OBJECT_CODE_AND_SCALE_STR} --max_grasps_per_batch 500 --save_to_file False --use_gui True --debug_index 0
Isaac Gym evaluation: each hand pose is simulated to check whether it successfully pushes/pulls the object.
In the below script, define the pushing direction and distance, then run the script with the command to get the joint angles (Arm + Hand) saved locally for real robot deployment.
python gd2p/pushing_motion_planning/scripts/run_pushing_motion_planning.py \
--skip_grasp_planning_config_dict data/dataset/NEW/evaled_grasp_config_dicts/core-bottle-1a7ba1f4c892e2da30711cdbdbc73924_0_2218.npy \
--object_name core-bottle-1a7ba1f4c892e2da30711cdbdbc73924 \
--object_mesh_path <path/to/object_mesh.obj> \
--num_grasps 200
This saves the planned trajectories (q_trajs.npy, qd_trajs.npy, T_trajs.npy) for real-robot deployment.
For a dataset object, --object_mesh_path is the object mesh scaled to object_scale (the mesh stays in the object/Oy frame; the motion planner's frames rotate it upright). For example:
import trimesh
m = trimesh.load("data/meshdata/<object_code>/coacd/decomposed.obj", force="mesh")
m.apply_scale(<object_scale>) # e.g. 0.1553 for ..._0_1553
m.export("object_scaled.obj")Render one planned trajectory to a GIF (headless PyBullet). --dynamic_object True (default) simulates the real push (the object slides/tips on contact); --dynamic_object False pins the object to inspect only the arm+hand motion:
python gd2p/pushing_motion_planning/utils/render_rollout.py \
--q_trajs_path q_trajs.npy \
--object_mesh_path object_scaled.obj \
--traj_idx 0 \
--output_gif rollout.gif
Example motion-planning rollout: the FR3 arm + Allegro hand executing a planned push on a dataset object (PyBullet).
We release a pre-trained diffusion checkpoint for the Allegro hand (see download). To train your own BPS-conditioned diffusion sampler on the generated data:
# 1. Build the BPS pushing dataset (.h5) from sim point clouds + evaluated grasp configs
python gd2p/generative_model/scripts/create_pushing_dataset.py
# 2. (Optional) create train/val/test splits
python gd2p/generative_model/scripts/create_train_val_test_split.py
# 3. Train the diffusion sampler
python gd2p/generative_model/scripts/train_bps_sampler_pushing_model.py
See here for details about creating the dataset for your own hand.
See here for additional useful details about this codebase (highly recommended).
@misc{li2025learninggeometryawarenonprehensilepushing,
title={Learning Geometry-Aware Nonprehensile Pushing and Pulling with Dexterous Hands},
author={Yunshuang Li and Yiyang Ling and Gaurav S. Sukhatme and Daniel Seita},
year={2025},
eprint={2509.18455},
archivePrefix={arXiv},
primaryClass={cs.RO},
url={https://arxiv.org/abs/2509.18455},
}
Our implementation is adapted from the open-source codebase of Get a Grip. We thank the authors for making the code publicly available to the research community.
