Harness-S

Overview · Harness Evolution · Installation · Quick Start · Harness Pool · Reproduction · Architecture

Harness-S trains one agent policy using complementary trajectories from a fixed sibling harness pool. When every base-harness attempt fails, a validation-based rule selects auxiliary harnesses for fresh attempts. GRPO learns from the replacement group in each trajectory's own generating context. The trained policy can execute under the original harness or use task-specific routing at inference.

Harness-S framework. Directional evolution builds a reusable sibling pool; fresh sibling rollouts rescue all-failure tasks and train one shared policy; learned behaviors transfer back to the original harness H0, with optional Rank-1 routing at inference. Click the figure for the original PDF.

The figure shows the paper's full method, including behavior profiles and LLM voting. The public recipes default to validation-based rules with profile calibration disabled; the model-based path is optional.

This first release focuses on AppWorld: the frozen 20-member pool, exact task partitions, portable execution assets, rule-based rescue, optional behavior profiling and voting router, training scripts and evaluation scripts. Plain GRPO, H0 rescue and Harness-S share the same sampler, scorer, optimizer, validation and checkpoint lifecycle. Offline harness search has already produced the included pool; regenerating that search is outside this release's launch commands.

• 20 reusable harnesses: H0 and 19 evolved siblings, with prompts, skills, context, memory, hooks and tool-view settings.
• Selective rescue: retain the base group when it contains a success; otherwise replace all eight trajectories with fresh auxiliary attempts.
• Low-cost selection: local task similarity and validation success rates select Top-8 siblings; profile calibration and LLM voting are off by default.
• One training path: established Dressage/Slime backend, policy-token masks, token importance correction, and no reference-model KL.
• Portable recipes: explicit environment variables, single seed, 32 concurrent tasks and one GPU per inference instance by default.
• Limited checkpoints: keep the validation-best and latest restart point during training; best and final after completion.

Sibling harnesses start from a common H0 and share the same execution foundation. They vary the prompts, skills, context, memory, hooks, and tool views that support the policy. The paper identifies three properties of this evolution:

Properties of harness evolution (Qwen3.5-4B, AppWorld). Left: repaired and regressed tasks relative to H0. Middle: individual success, sibling oracle union coverage, and H0 pass@K at matched rollout counts. Right: paired outcomes for fixed harnesses as the policy learns. Click the figure for the original PDF.

• Locality: a harness edit can repair some failures while introducing regressions on previously solved tasks. Different repair directions develop different strengths.
• Complementarity: siblings recover one another's errors. Their oracle union covers more tasks than individual variants or repeated H0 sampling at the same rollout count; this measures the pool's potential coverage, while a router must select harnesses before observing outcomes.
• Nonstationarity: a fixed harness's benefit changes with the policy. An advantage over H0 can turn into a disadvantage, and sibling rankings can reverse across checkpoints.

Directional Harness Evolution diagnoses failure causes and branches into targeted repairs. Joint validation coverage selects the next parents, while every executable, protocol-compliant child is retained in the pool. Rescue training uses that fixed pool to explore alternative behaviors on failed tasks and transfer them into the policy. This release includes the resulting AppWorld pool; rerunning the offline evolution search is outside its launch commands.

The control commands require Python 3.10+. Training requires Linux, NVIDIA GPUs, Docker with GPU support, Node.js 22.19+ (or 24+), and a separately installed AppWorld environment and task data. The paper used eight H800 GPUs.

git clone https://github.com/DeepExperience/Harness-S.git
cd Harness-S
bash scripts/bootstrap.sh
bash scripts/build_image.sh
cp .env.example .env
# Edit .env with your own model, AppWorld and Node paths. Router API settings are optional.
source .env

bootstrap.sh retrieves the pinned Slime commit and applies the included compatibility patch. The adapted Dressage runtime and its Apache license are included under third_party/. The Docker image supplies the GPU training stack. Model weights, AppWorld databases and credentials are external inputs.

Convert your Hugging Face policy checkpoint using the pinned Slime converter before training. See setup and conversion. Use the matching architecture recipe for 4B or 9B.

Check the archive and prepare a run without allocating GPUs:

harness-s check-pool
harness-s prepare --config configs/appworld/harness_s_4b.json \
  --run-dir outputs/appworld-harness-s

bash scripts/train_appworld.sh --config configs/appworld/harness_s_4b.json \
  --run-dir outputs/appworld-harness-s --dry-run

Start or resume training:

bash scripts/train_appworld.sh --config configs/appworld/harness_s_4b.json \
  --run-dir outputs/appworld-harness-s

Choose grpo_4b.json for plain GRPO or h0_rescue_4b.json for fresh H0 resampling. Matching *_9b.json recipes are included. No default recipe requires a router API key. Each command uses the same training implementation.

Evaluate the best committed policy under H0, or rank one harness per task before its three attempts:

bash scripts/evaluate_appworld.sh --run-dir outputs/appworld-harness-s \
  --checkpoint best --harness h0
bash scripts/evaluate_appworld.sh --run-dir outputs/appworld-harness-s \
  --checkpoint best --harness rank1

Logs, selection evidence, optional profile/routing API returns, trajectories, metrics and restart records stay in the ignored run directory. The launcher starts an independent Ray head inside the training container and cleans up its own processes by PID. It does not stop or join the host Ray cluster.

Default selection retrieves the five nearest validation instructions using local TF-IDF cosine similarity, then ranks harnesses by their mean success on those same tasks. Ties use overall validation success, then harness ID. With no positive text similarity it uses overall validation success. Rescue excludes H0; rank-1 evaluation includes H0. Test outcomes never enter selection. Validation rollouts still run to update these statistics; the defaults remove profile-generation and voting API costs.

To enable the optional LLM/profile path, see configuration. This is a different selection strategy from the public rule-based default; historical experiment scores do not validate the new default.

The AppWorld pool contains the complete archive of H0 + 19 auxiliaries. Each member includes harness.yml, state.json and any skills/. The catalog records source identities and public-file checksums. Public YAML templates are rendered to absolute paths at preparation time.

The paper describes a 20-member routing pool. Some historical training runs used an H0-plus-nine subset; its exact membership is retained separately. The default public recipe uses all 20. It does not silently substitute the historical subset or claim new measurements for the expanded setup.

Task IDs are disjoint; sibling variants can share task families across partitions. The paper's main-table averages over three independent runs are distinct from this repository's default single-run launch.

This extraction has CPU contract tests and release checks. The portable GPU launcher has not yet completed an end-to-end training run. Its defaults also change inference placement to one GPU per engine; historical runs often used two. Do not treat this release as a bitwise replay or a fresh confirmation of the reported scores. See the implementation mapping and remaining validation.

configs/appworld/        # 4B / 9B recipes for all three training variants
data/appworld/           # Frozen task IDs, order and family assignments
docs/figures/            # Paper figures: original PDFs and README PNG previews
harness_pool/appworld/   # Public H0 + 19 sibling implementations
src/harness_s/           # Shared orchestration, routing and AppWorld adapter
runtime/appworld/        # Fixed DSH composition, components and MCP replay
scripts/                # Setup, training, evaluation and release checks
third_party/            # Adapted Dressage, pinned Slime patch and notices
tests/                  # CPU method and artifact contract tests

python -m pip install -e '.[test]'
pytest -q
python scripts/check_release.py

The execution and learning backend builds on Dressage, Slime, DeepSeek Harness, and AppWorld, SWE-Gym and DBBench (AgentBench). See third-party notices for versions and licensing.

The manuscript is titled Harness-S: Learning from Sibling Harnesses for Generalizable Agent Training. A verified public paper identifier and complete BibTeX entry will be added with the paper release.