AliMark [ICML 2026]

This is the official repository of our ICML 2026 paper: AliMark: Enhancing Robustness of Sentence-Level Watermarking Against Text Paraphrasing.

Both the camera-ready version and a more organized code repository will be released soon. Stay tuned!

Before playing around with AliMark, you should first set up the Python environment by running the following commands:

conda create -n alimark python=3.13
conda activate alimark
pip install -r requirements.txt

We have uploaded three datasets used in our experiments, including Booksum, C4, and Natural Questions (NQ). Each dataset contains 500 text samples. They are organized in JSON format, with each row comprising a prompt and a natural_text. See the following as an example:

{
    "prompt": "The Minister's Vigil Dimmesdale mounts the scaffold.", 
    "natural_text": "The pain in his breast causes him to scream aloud, and ..."
}

During the experiment, the prompt will be fed into the LLM to generate watermarked text (as the positive sample), while the natural_text will be used to form the human text counterpart (as the negative sample).

All the following scripts are based on almost the same set of watermarked text generation hyperparameters defined in the .sh files. We provide a brief description here:

--watermark_algorithm                       # name of our watermarking algorithm
--watermark_model                           # the LLM to be watermarked
--watermark_embedder                        # sentence/text embedder from sentence-transformers
--watermark_embedding_dim                   # the corresponding embedding dimension of the embedder
--watermark_block_size                      # block size M
--watermark_num_next_sentence_candidates    # next sentence candidate budget Q

--min_new_sentences                         # the number of new sentences generated by the LLM
--dataset_name                              # name of the dataset from the ./dataset/ folder
--vllm_gpu_mem_util                         # the GPU memory utilization rate of vLLM

Some other arguments related to LLM generation configuration (e.g., temperature, top_p, and repetition_penalty) are hard-coded in watermark/alimark.py, since they are the same across all possible experiments in our study.

After fixing these generation hyperparameters, you can seamlessly start experimenting with the full AliMark pipeline. All the outputs will be stored as JSON files in Pandas Dataframe format (orient="index") under the ./_result folder.

You can generate watermarked text with AliMark by running:

bash script/1_run_generation.sh

This script also generates the corresponding unwatermarked text and save the human text into the result file.

We adopt four paraprasers: [1] Pegasus, [2] Parrot, [3] DIPPER, and [4] GPT-3.5. [1] and [2] are weak paraphrasers since they perform sentence-to-sentence paraphrasing, while [3] and [4] are much stronger ones, taking the entire text as input and performing text-to-text paraphrasing.

Make sure you have set up the OpenRouter API key as an environment variable to enable the GPT-3.5 paraphraser:

export OPENROUTER_API_KEY="your_api_key_here"

Then you can run the paraphrasing (attack) script:

bash script/2_run_attack.sh

Once both the watermarked text generation and the paraphrasing attacks have completed, you can run the detection algorithm:

bash script/3_run_detection.sh

Here, the detection uses the default detection hyperparameters in our paper (i.e., alpha=0.5, beta=1.5, and single-step re-structuring). You can alternatively modify these hyperparameters within watermark.detect_watermark() in the 3_detection.py file to customize the detection.

With the watermark scores for all the abovementioned texts ready, you can evaluate the watermarking performance, such as AUROC, TPR@1%, TPR@5% under no attack and under paraphrasing attacks:

bash script/4_run_evaluation.sh

In case you are interested in the detailed implementation with our AliMark, feel free to take a closer look at the ./watermark folder:

• ./watermark/
  • ber_map/ folder: stores the mean and standard deviation values of two random bit sequences of different block size M and number of blocks N'.
  • secret_key/ folder: stores both the secret bit sequence and secret vectors.
  • adaptive_bit_sequence_alignment.py: implements the ABSA module (e.g., block edit rate).
  • alimark.py: implements the AliMark class with the main generation and detection workflow.
  • restructurer.py: implememnts the RS module.
  • utils.py: manage the secret keys.

Please consider citing our paper if you find it useful:

@inproceedings{li2026alimark,
    title={AliMark: Enhancing Robustness of Sentence-Level Watermarking Against Text Paraphrasing},
    author={Yuexin Li and Wenjie Qu and Linyu Wu and Yulin Chen and Yufei He and Tri Cao and Bryan Hooi and Jiaheng Zhang},
    booktitle={Forty-third International Conference on Machine Learning},
    year={2026},
    url={https://openreview.net/forum?id=jQmlwZSPuw}
}

This project is built upon the foundational work of MarkLLM, SemStamp and PMark. We sincerely thank the authors and contributors of these repositories for their excellent open-source work.