DynamicManip

DynamicManip is a source-only research pipeline for dynamic robot manipulation. It combines a lightweight five-task RoboTwin-compatible environment, stage-aware DP3 with Taylor inference, and a maintained static-to-dynamic DynamicAug path.

This repository intentionally contains no third-party robot/object assets, raw or augmented demonstrations, processed training data, policy checkpoints, evaluation outputs, or developer-machine presets. These files are generated or imported locally and are ignored by Git.

The source release includes:

The public DynamicAug subset currently exposes the validated whack_a_mole_modular_mesh augmentation path. The environment and policy retain all five task entrypoints.

The validated runtime uses Python 3.10, Torch 2.12.1+cu130, CUDA 13.0, PyTorch3D 0.7.8, cuRobo 0.7.8.post1.dev0, and SAPIEN 3.0.0b1 on an RTX 5090.

Clone the source repository:

git clone git@github.com:liaohr9/DynamicManip.git
cd DynamicManip

Create the base environment:

conda env create -f environment.yml
conda activate dynamicmanip

Install a CUDA/Torch/PyTorch3D/cuRobo stack appropriate for your GPU, then install the project layers:

python -m pip install -r requirements-runtime.txt
python -m pip install -r requirements-dynamic-aware.txt
python -m pip install -e augmentation
export PYTHON_BIN="$(command -v python)"

See docs/INSTALL.md for the complete version and verification notes.

Binary assets are not distributed under this repository's MIT license. Obtain environment and DynamicAug asset directories that you are authorized to use, then run:

bash scripts/prepare_assets.sh \
  --environment-assets /path/to/environment/assets \
  --augmentation-assets /path/to/DynamicAug/assets

The importer rejects Git LFS pointers and unknown conflicts, deduplicates identical files, writes ignored assets/, and creates augmentation/assets -> ../assets. The audited source revisions produce 405 files / 1.255 GB. See docs/ASSETS.md.

Source-only gates do not require runtime assets:

bash scripts/verify_release.sh

Full integration gates require the robotics environment and imported assets:

bash scripts/verify_integration.sh

List the fixed task set:

bash scripts/run_task.sh list

Collect one task:

RUN_ID=collect_$(date +%Y%m%d_%H%M%S) EXPERT_DATA_NUM=200 \
  bash scripts/run_task.sh collect block_ball 4

Raw environment episodes remain under ignored data/ and are never hosted by this project.

Process, train, and evaluate one collected run:

RUN_ID=<same_run_id> EXPERT_DATA_NUM=200 \
  bash scripts/run_task.sh process block_ball 4

RUN_ID=<same_run_id> EXPERT_DATA_NUM=200 \
  bash scripts/run_task.sh train-eval block_ball 4 0

Convert 200 stage-labelled HDF5 episodes with an episode-level 160/20/20 split:

python policy/Dynamic_Aware_DP3/scripts/process_stage_data.py \
  --input-dir data/dynamic_augmented/v1/mole_whacking \
  --output-zarr policy/Dynamic_Aware_DP3/data/mole_whacking-stage-200.zarr \
  --manifest policy/Dynamic_Aware_DP3/data/mole_whacking-stage-200.split.json \
  --num-episodes 200 --val-count 20 --test-count 20

Train 100 epochs:

policy/Dynamic_Aware_DP3/scripts/train_dynamic_aware.sh \
  dynamic_aware_full 100 0

Run held-out standard/Taylor inference:

python policy/Dynamic_Aware_DP3/scripts/eval_dynamic_aware_offline.py \
  --exp-dir <experiment_seed_dir> \
  --checkpoint <checkpoint_path>

Run matched environment evaluations:

EVAL_TEST_NUM=20 policy/Dynamic_Aware_DP3/scripts/eval_dynamic_aware_env.sh standard 100 0
EVAL_TEST_NUM=20 policy/Dynamic_Aware_DP3/scripts/eval_dynamic_aware_env.sh taylor 100 0

Each environment run writes dynamic_inference.jsonl beside its result summary. Validated metrics are summarized in docs/RESULTS.md.

Import an environment episode without generating data:

scripts/augment_episode.sh \
  --source-h5 /path/to/episode0.hdf5 \
  --demogen-task whack_a_mole_modular_mesh \
  --import-only

Export the first frame for mask preparation:

scripts/augment_episode.sh \
  --source-h5 /path/to/episode0.hdf5 \
  --demogen-task whack_a_mole_modular_mesh \
  --stage extract

The default import is a symlink under ignored augmentation/data/imported/. Pass --copy only when an independent HDF5 copy is required.

The existing liaohr9/DynamicManip-Resources Hugging Face repository remains private while its asset/data distribution terms are reviewed. Authorized collaborators may use:

export HF_TOKEN=hf_your_read_token
bash scripts/download_resources.sh training

The public source release does not require this repository: users can collect, augment, and process their own episodes. Never place a token in a script or Git commit.

• Git history is source-only.
• assets/, data/, Zarr, HDF5, checkpoints, logs, and evaluation outputs are ignored.
• No environment-collected raw episode is hosted.
• Third-party license and provenance details are in THIRD_PARTY_NOTICES.md and PROVENANCE.md.
• Contribution and security guidance are in CONTRIBUTING.md and SECURITY.md.

The repository root is MIT licensed. DynamicAug and upstream DP3 retain their own MIT copyright notices beside their source. Binary assets are explicitly outside the root license grant.

See CITATION.cff and THIRD_PARTY_NOTICES.md before publishing derived work.