This repository contains the full codebase, datasets, and evaluation pipelines accompanying the paper:

Challenges of Evaluating LLM Safety for User Welfare
IASEAI 2026

The project introduces a methodology for user-welfare safety evaluation, focusing on how LLM advice can pose different risks to different users depending on their personal context and vulnerability. Unlike universal safety evaluations (cybersecurity, jailbreak robustness, sycophancy, etc.), user-welfare safety concerns context-dependent harms in domains like personal finance and health.

This repository includes:

• Reddit-inspired advice-seeking question dataset
• Professionally curated user vulnerability profiles
• A systematic context-blind vs. context-aware safety evaluation pipeline
• Prompt-enriched datasets for realistic user disclosure (RQ2)
• Scripts for collecting LLM responses (GPT-5, Claude Sonnet 4, Gemini 2.5 Pro)
• LLM-as-Judge scoring pipeline for safety evaluation
• Survey results (likelihood & relevance rankings, Borda aggregation)
• End-to-end reproducible code to regenerate all prompts, responses, and safety scores.

context_eval/
│
├── eval_dataset/
│   ├── eval_dataset.csv
│   ├── evaluation_prompts.csv
│   ├── evaluation_prompts.json
│   ├── llm_responses.csv
│   ├── ai_rankings.csv
│   ├── relevance_rankings.csv
│   ├── likelihood_rankings.csv
│   ├── clause_cache.csv
│   ├── clause_cache_usage.csv
│   └── variant_usage.csv
│
├── output/
│   ├── llm_response_f00.csv
│   ├── llm_response_f01.csv
│   └── ... (one file per request_id)
│
├── scores/
│   ├── safety_scores_f00.csv
│   ├── safety_scores_f01.csv
│   └── ... (per-request contextualized + apparent scores)
│
├── survey_results/
│   ├── relevance_survey_*.xlsx
│   ├── likelihood_survey_*.xlsx
│   ├── relevance_*_borda.csv
│   ├── relevance_survey_ranking_finance.xlsx
│   ├── relevance_survey_ranking_health.xlsx
│   └── ... (all Borda aggregation outputs)
│
├── src/
│   ├── config.py
│   ├── prompts.py
│   ├── utils_pipeline.py
│   ├── prompt_assembly.py
│   ├── get_llm_responses.py
│   ├── run_llm_responses_per_prompt.py
│   ├── score_pipeline.py
│   ├── logger.py
│   ├── patch_baseline_scores_per_request.py
│   ├── patch_missing_safety_scores.py
│   ├── patch_llm_response_missing.py
│   └── patch_scores_for_baseline.py
│
└── README.md

Reddit-inspired advice-seeking questions compiled via:

• Reddit scraping
• Advice-seeking classification
• Theme classification
• GPT-assisted synthesis

Example rows:

Each row represents a fully assembled evaluation prompt:

• prompt_id
• request_id
• topic, theme
• vulnerability_profile_level (low, medium, high)
• vulnerability_profile
• ranking_type (baseline | relevance | likelihood)
• context_level (0, 1, 3, 5)
• context_variant (five phrasings)
• final_prompt

Includes all generated outputs for:

• GPT-5
• Claude Sonnet 4
• Gemini 2.5 Pro

Temperature = 1.0 for diversity.

Detailed outputs also exist under:

/output/llm_response_{request_id}.csv

Each file includes:

• apparent_safety_score (context-blind)
• contextualized_safety_score (context-aware)
• likelihood, severity, safeguard adequacy (1–7)
• evaluator reasoning (chain-of-thought)
• applied scoring logic (Appendix G)

Missing or baseline scores patched via:

• patch_missing_safety_scores.py
• patch_baseline_scores_per_request.py

This directory contains all datasets and utilities related to ranking context factors.

What users say they would voluntarily disclose when asking for advice.

What domain professionals recommend should be disclosed for safe, responsible advice.

The repository includes the script:

src/ai_ranking.py

This script generates AI-produced rankings of context factors.
These rankings were not used in the final experiments, but the file is included for completeness and potential future extensions.

Final stable rankings are computed using Borda count and stored in:

• relevance_*_borda.csv
• relevance_survey_ranking_*.xlsx

1. Converts profile factors into first-person clauses
2. Ranks them using:
   • expert relevance order
   • user likelihood order
3. Creates prompts with 1, 3, or 5 factors
4. Generates 5 linguistic variants per prompt
5. Caches outputs for reproducibility (clause cache)

• Sends each prompt to multiple LLMs
• Stores outputs in central CSV + per-request files

Implements LLM-as-Judge:

• Evaluates likelihood of harm
• Severity of harm
• Safeguard adequacy
• Computes final safety score under explicit rules
• Produces structured justification and numeric ratings

@inproceedings{contexteval2026,
  title={Challenges of Evaluating LLM Safety for User Welfare},
  booktitle={IASEAI},
  year={2026}
}