ShapeForce is a low-cost, plug-and-play soft robotic wrist that converts external forces and torques into measurable deformations, which are then estimated via marker-based pose tracking and converted into force-like signals for contact-rich robotic manipulation. This repository supports both rule-based (search-and-control) policies and imitation learning policies.

• Core Idea: Inspired by human perception, contact-rich tasks rely on relative force changes rather than precise force magnitudes. ShapeForce captures contact information through deformation of its compliant core, eliminating the need for expensive six-axis force-torque sensors.
• Hardware: 3D-printed compliant core (TPU) + rigid connectors (PLA) + wrist-mounted RGB camera + ChArUco marker board (see Fabrication Details)

• Robot: UFactory xArm7 (or xArm Python SDK compatible)
• Camera: Intel RealSense D435/D435i (wrist-mounted)
• Teleoperation: GELLO (for imitation learning data collection)
• OS: Ubuntu 22.04, Python 3.9+, PyTorch 2.x, CUDA 12.x

git clone https://github.com/shapeforce/Deformable-Force-Sensor.git
cd Deformable-Force-Sensor

conda create -n shapeforce python=3.9 && conda activate shapeforce
pip install -r scripts/requirements.txt
pip install xarm-python-sdk pyrealsense2

export PYTHONPATH="${PYTHONPATH}:$(pwd)/third_party:$(pwd)"

Set robot IP in third_party/3rdparty/xarm7/xarm7_interface/__init__.py.

Calibration: Run camera intrinsic calibration (aruco_detect/capture_images.py + calibration.py), then hand-in-eye calibration (hand-eye-calibration/hand_in_eye/). Update exst_name in third_party/utils_repo/peg_insertion/return_transformation.py to your calibration result directory.

Use ShapeForce force-like signals to drive search and force-control policies.

export PYTHONPATH="$(pwd)/third_party:$(pwd)"
cd scripts/search_and_control_policies
python peg_insertion.py   # or clean_board.py, screw.py, etc.

Scripts with _gt suffix use a commercial force-torque sensor as baseline.

Use ACT or Diffusion Policy with ShapeForce force-like signals, images, and joint states.

cd gello_software
git submodule init && git submodule update
pip install -r requirements.txt && pip install -e . && pip install -e third_party/DynamixelSDK/python

# Launch robot and GELLO
python experiments/launch_nodes.py --robot=<your_robot>
python experiments/run_env.py --agent=gello --use-save-interface

python gello/data_utils/demo_to_gdict.py --source-dir=<path_to_collected_pkls>

Output goes to _conv/multiview/train/none/ (or similar). Note the path for dataset_dir in constants.py.

Edit saywhen_learning_base/ModelTrain/constants.py:

TASK_CONFIGS = {
    'train_gello': {
        'dataset_dir': '/path/to/your/_conv/multiview/train/none/',  # path from demo_to_gdict
        'episode_len': 4000,
        'train_ratio': 0.95,
        'camera_names': ['wrist']
    },
    # ...
}

cd saywhen_learning_base/ModelTrain

# Fix sys.path in model_train_act.py / model_train_diff.py to point to robomimic
# e.g. sys.path.append('/path/to/Deformable-Force-Sensor/saywhen_learning_base/robomimic-r2d2/robomimic')

pip install -r ../robomimic-r2d2/requirements.txt
pip install -e ../robomimic-r2d2

# ACT
python model_train_act.py --ckpt_dir ./ckpt/your_task --task_name train_gello --batch_size 64 --num_steps 9000

# Diffusion Policy
python model_train_diff.py --ckpt_dir ./ckpt/your_task_diff --task_name train_gello --num_steps 30000

# Edit ckpt_dir and model_name in the inference script
python model_inference_test_gello_act.py   # ACT
python model_inference_test_gello_diff.py # Diffusion Policy

├── aruco_detect/           # ChArUco detection, pose estimation, calibration
├── hand-eye-calibration/  # Hand-in-eye calibration
├── scripts/
│   ├── calculate_transformation.py   # Real-time deformation visualization
│   ├── K_calculate_6d.py             # Stiffness matrix calibration (optional)
│   └── search_and_control_policies/  # Rule-based task scripts
├── third_party/utils_repo/peg_insertion/
│   ├── force_insert.py    # Core search & force control logic
│   └── return_transformation.py  # ShapeForce deformation signal
├── gello_software/        # GELLO teleoperation & data collection
└── saywhen_learning_base/ModelTrain/  # ACT & Diffusion Policy training

@article{zhu2025shapeforce,
  title={ShapeForce: Low-Cost Soft Robotic Wrist for Contact-Rich Manipulation},
  author={Zhu, Jinxuan and Yan, Zihao and Xiao, Yangyu and Guo, Jingxiang and Tie, Chenrui and Cao, Xinyi and Zheng, Yuhang and Shao, Lin},
  journal={arXiv preprint arXiv:2511.19955},
  year={2025}
}

• Project Page
• Paper