✨ SCOPE: Selective Context Preference Optimization ✨

SCOPE teaches language models when to trust external signals. It mines problems that a base model answers correctly without added context but incorrectly under a plausible misleading signal, then reuses each preference pair across four matched conditions: clean, misleading, correct, and irrelevant.

The optimization remains standard full-completion sigmoid DPO. This repository provides the complete public pipeline for LoRA-DPO training, adapter merging, MIST inference, exact-match scoring, paired bootstrap confidence intervals, and release-data validation.

flowchart LR
    A["MIST-Train<br/>matched preference quartets"] --> B["SCOPE<br/>LoRA + DPO"]
    B --> C["Merged model"]
    C --> D["MIST-Bench<br/>4 matched conditions"]
    D --> E["Accuracy · Overall · SC2W"]

git clone https://github.com/worldbench/SCOPE.git
cd SCOPE

conda create -n scope python=3.10 -y
conda activate scope
pip install -r requirements.txt

Training and evaluation require CUDA-capable GPUs. Multi-GPU evaluation is controlled by --tensor_parallel_size.

python -m unittest discover -s tests -v

from datasets import load_dataset

train = load_dataset("worldbench/MIST-Train", split="train")
benchmark = load_dataset("worldbench/MIST-Bench", split="test")

If the datasets are downloaded beside this repository, validate their schema and matched-condition structure:

python scripts/validate_data.py \
  --train_dir ../hf-train \
  --mist_file ../hf-mist/mist_1000.jsonl

Set the base model once, then launch the paper recipe:

export BASE_MODEL="your-hugging-face-model"

accelerate launch scope/train.py \
  --model_name_or_path "$BASE_MODEL" \
  --dataset worldbench/MIST-Train \
  --output_dir outputs/scope

For a short formatting/data probe before a full run:

accelerate launch scope/train.py \
  --model_name_or_path "$BASE_MODEL" \
  --dataset worldbench/MIST-Train \
  --output_dir outputs/scope_probe \
  --max_items 8 \
  --selftest

Use --train_file ../hf-train/train.jsonl instead of --dataset for local data. Add --disable_thinking when the selected chat template should not enable thinking mode. Resume an interrupted run with --resume_from_checkpoint <CHECKPOINT>.

python -m scope.merge_lora \
  --base "$BASE_MODEL" \
  --adapter outputs/scope \
  --output outputs/scope_merged

Run MIST with vLLM:

python -m mist.evaluate \
  --model outputs/scope_merged \
  --dataset worldbench/MIST-Bench \
  --output_dir outputs/mist_scope

Use --data_file ../hf-mist/mist_1000.jsonl for a local benchmark. Before a full 1,000-item run, use --smoke_items 10 to exercise inference, parsing, scoring, and report generation end to end.

The evaluator writes:

Reported metrics include accuracy for all four conditions, balanced Overall accuracy, and SC2W ↓: the fraction of clean-correct items that become wrong under misleading context. Confidence intervals use paired item-level bootstrap resampling.

README.md                    # Setup and end-to-end usage guide
requirements.txt            # Runtime dependencies
scope/
├── train.py              # Full-completion DPO + LoRA
└── merge_lora.py         # Merge an adapter into its base model
mist/
├── evaluate.py           # vLLM inference and report generation
├── answer_utils.py       # MC, numeric, and boolean parsing
└── metrics.py            # Accuracy, SC2W, and bootstrap CIs
scripts/
└── validate_data.py      # Public dataset integrity checks
tests/
├── test_answer_utils.py  # Parser and answer-matching tests
└── test_metrics.py       # Metric and bootstrap tests

@article{scope2026,
  title   = {Learning When to Trust via Selective Context Preference Optimization},
  author  = {Sun, Xian and Chow, Wei and Wang, Yingshuo and Liu, Junhao and Gao, Wei and Wu, Qing and Kong, Lingdong},
  journal = {arXiv preprint arXiv:2608.06377},
  year    = {2026}
}