This document explains how to use the gui_grpo.sh and gui_grpo_qwen3.sh scripts under examples/ for GRPO (Group Relative Policy Optimization) reinforcement learning training of GUI agents.

Both scripts are built on the EasyR1 framework and train vision-language models (VLMs) to perform GUI tasks by outputting structured action instructions (click, type, scroll, etc.).

cd EasyR1
bash set_up.sh

Clone the GUI-Libra-81K-RL dataset from Hugging Face:

git lfs install
git clone https://huggingface.co/datasets/GUI-Libra/GUI-Libra-81K-RL

The dataset contains three folders:

We recommend using the downsampled/ (40K) subset for RL training, as it provides better training efficiency while maintaining strong performance.

Each sample should contain the following fields:

The ground_truth structure:

{
    "gt_action": "click",
    "gt_bbox": [x1, y1, x2, y2],
    "gt_point_2d": [x, y],
    "gt_input_text": "search query",
    "image_width": 1920,
    "image_height": 1080,
    "image_width_new": 1920,
    "image_height_new": 1080,
    "is_grounding": true
}

• gt_action: Action type — click, longpress, type, write, scroll, openapp, swipe, terminate, answer, select
• gt_bbox / gt_point_2d: Normalized coordinates in range [0, 1000]
• gt_input_text: Required for text-based actions (type, write, etc.)
• is_grounding: true for grounding tasks (click/longpress); false for reasoning tasks

Edit the variables at the top of the training script:

# Model path: base model or SFT checkpoint
MODEL_PATH=Qwen/Qwen2.5-VL-3B-Instruct

# Training set: path to parquet file(s)
TRAIN_FILES=path/to/GUI-Libra-81K-RL/data/downsampled/train.parquet

# Validation set
VAL_FILES=path/to/GUI-Libra-81K-RL/data/validation/test_androidcontrol_mind2web_dataset.parquet

# Qwen2.5-VL
bash examples/gui_grpo.sh

# Qwen3-VL
bash examples/gui_grpo_qwen3.sh

The model should produce the following structured output:

<think>
Step-by-step reasoning about the GUI task...
</think>
<answer>
{
  "action_type": "click",
  "action_description": "Click the search button",
  "value": "None",
  "point_2d": [512, 256]
}
</answer>

• point_2d: Screen coordinates [x, y] in [0, 1000]; use "none" or [-100, -100] if not applicable
• For Qwen3-VL, use <thinking>...</thinking> instead of <think>...</think>

The reward consists of three components:

1. Format: Whether the output follows the required JSON structure (action_type, action_description, value, point_2d)
2. Accuracy: Whether the predicted action matches the ground_truth
   • Click/longpress: coordinate error within threshold scores 1
   • Text actions: F1 score with gt_input_text >= 0.5 scores 1
3. Reasoning usage: Whether reasoning content (<think> or <thinking>) is present
   • For non-grounding tasks: format score is set to 0 if no reasoning is used
   • For grounding tasks: format score is set to 0 if reasoning is used

Overall score: (1 - format_weight) * accuracy + format_weight * format_score (default format_weight=0.1)

Try the following adjustments:

• Lower worker.rollout.gpu_memory_utilization (e.g., 0.6)
• Lower worker.actor.micro_batch_size_per_device_for_update
• Lower worker.actor.micro_batch_size_per_device_for_experience
• Enable worker.actor.offload.offload_params=true

After training, merge checkpoints into Hugging Face format:

python3 scripts/model_merger.py \
    --local_dir checkpoints/easy_r1/<experiment_name>/global_step_<step>/actor

• EasyR1 Documentation
• GUI-Libra Paper
• GRPO Algorithm