Why and How LLMs Hallucinate: Connecting the Dots with Subsequence Associations

This repository contains the source code for Why and How LLMs Hallucinate: Connecting the Dots with Subsequence Associations by Yiyou Sun, Yu Gai, Lijie Chen, Abhilasha Ravichander, Yejin Choi, and Dawn Song.

This project investigates hallucinations in large language models (LLMs) by analyzing subsequence associations in model outputs. It provides tools for benchmarking, evaluation, and visualization of hallucinated content using the HALoGEN dataset.

To set up the required environment, install dependencies from requirements.txt:

python3 -m nltk.downloader averaged_perceptron_tagger punkt
pip install -r requirements.txt
python3 get_embeddings.py

You can test the hallucination analysis using the demo script:

python demo.py

The results will be stored in:

halu_results/{model_name}/customize

We use a preprocessed subset of HALoGEN prompts and corresponding hallucination subsequences. The preprocessed data is available in:

./halu_results

(The preprocess script is python data_preprocess.py)

To evaluate a model on the benchmark dataset: 1. Add your OpenAI API key to ./key.py. 2. Run the following command:

python run_benchmark.py \
    --halo_type {dataset} \
    --model_name {model_name} \
    --num_perturbations 1024 \
    --min_level 2 \
    --search_beam 20 \
    --completion_modes openai-mask openai-token bert random \
    --test_sentence_num 25 \
    --eval_level_range quad

If you use our codebase, please cite our work:

@article{sun2025sat,
  title={Why and How LLMs Hallucinate: Connecting the Dots with Subsequence Associations},
  author={Sun, Yiyou and Gai, Yu and Chen, Lijie and Ravichander, Abhilasha and Choi, Yejin and Song, Dawn},
  journal={arxiv},
  url={https://arxiv.org/abs/2504.12691},
  year={2025}
}