HAG

pip install -r requirements.txt

Edit config/model_api_config.py to configure API keys and base URLs:

MODEL_API_CONFIG = {
    "your-model-name": (
        "your_api_key",
        "your_base_url"
    ),
}

Edit config/value_constraints.json to specify allowed values for each demographic dimension.

python generate_agents.py --topic "Your topic here" --total_target 100 --model "your-model-name"

Edit run_batch_experiments.sh to configure topics and models, then run:

bash run_batch_experiments.sh

Edit eval/run_stats_evaluation.sh to configure topic prefix, models, and ground truth stats, then run:

bash eval/run_stats_evaluation.sh

This script compares generated user statistics with ground truth benchmark statistics using statistical metrics (JSD, KL divergence, etc.).

Edit eval/run_llm_evaluation.sh to configure topic prefix, models, theme context, and evaluation model, then run:

bash eval/run_llm_evaluation.sh

This script evaluates the sociological consistency of generated users using LLM-based micro and meso level assessments.

Edit eval/run_evaluation_summary.sh to configure topics and models, then run:

bash eval/run_evaluation_summary.sh

This script aggregates evaluation results from multiple topics and models into Excel files for easy comparison.

The method requires a database at database/wvs_users.json in World Values Survey format. This database serves as the source for matching and generating users.

Benchmark datasets are stored in benchmark/ directory and organized by source:

• Amazon (benchmark/amazon/): Product review datasets
• Bluesky (benchmark/bluesky/): Social media topic datasets
• IMDb (benchmark/IMDb/): Movie review datasets

To keep output paths stable and safe, this repo normalizes topic_prefix as:

• Replace any non-alphanumeric characters with _ (A-Z, a-z, 0-9 are kept)
• Trim leading/trailing _
• Truncate to the first 10 characters
• If the result is empty, use default

Example: "Joker(2019)" → "Joker_2019"

Generated files are saved to:

• topic-agent/{topic_prefix}/: Distribution trees, matched/generated users, per-blueprint statistics
• results/{topic_prefix}/: Final users and field distribution statistics
• eval/eval_results/{topic_prefix}/: Evaluation results (statistical and LLM-based)