Authors: Sapir Harary, Eran Hirsch, Aviv Slobodkin, David Wan, Mohit Bansal and Ido Dagan.

This repository contains the code for the paper "PrefixNLI: Detecting Factual Inconsistencies as Soon as They Arise".

This repository provides the implementation of our PrefixNLI Controlled Decoding introduced in the paper, which integrates prefix-level entailment signals from the MiniTruePrefixes model to reduce hallucinations during autoregressive text generation.

The core model, MiniTruePrefixes, is a fine-tuned LLaMA-1B-Instruct entailment model trained to assess whether a text prefix is entailed by a given premise. It is specifically optimized for evaluating factual consistency in summarization tasks.

When used during decoding, this model enables token-level factuality control, allowing the generator to detect and mitigate hallucinations as they emerge.

To install dependencies, run:

pip install -r requirements.txt

Below is a minimal example showing how to use PrefixNLI Controlled Decoding
to generate faithful summaries with prefix-level entailment guidance.

python controlled_decoding.py \
    --lm_model meta-llama/Llama-3.2-1B-Instruct \
    --entailment_model sapirharary/MiniTruePrefixes \
    --dataset_name xsum \
    --dataset_split test \
    --gpu 0 \
    --output_csv results.csv

This example runs controlled decoding on the XSum dataset using meta-llama/Llama-3.2-1B-Instruct as the generator and sapirharary/MiniTruePrefixes as the entailment model.

During generation, the entailment model evaluates each partial prefix and penalizes unfaithful continuations during decoding. The generated summaries and timing information are saved in results.csv.

The MiniTruePrefixes model expects its input in the following chat format:

{"role": "user", "content": f"premise: {SOURCE_TEXT} hypothesis: {PREFIX_TEXT}"}

Where:

• SOURCE_TEXT — the source document.
• PREFIX_TEXT — the summary prefix being evaluated for entailment.

@misc{harary2025prefixnlidetectingfactualinconsistencies,
      title={PrefixNLI: Detecting Factual Inconsistencies as Soon as They Arise}, 
      author={Sapir Harary and Eran Hirsch and Aviv Slobodkin and David Wan and Mohit Bansal and Ido Dagan},
      year={2025},
      eprint={2511.01359},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2511.01359}, 
}