[ICLR 2026] Code Aesthetics with Agentic Reward Feedback

Paper Link👁️

• [2026.01.23]: Release the OpenDesign benchmark.

• [2026.01.12]: Release the AesCode dataset.

• [2025.10.29]: Release the AesCoder-4B model.

• [2025.10.27]: Release the Project Page and the Arxiv version.

Large Language Models (LLMs) have become valuable assistants for developers in code-related tasks. While LLMs excel at traditional programming tasks such as code generation and bug fixing, they struggle with visually-oriented coding tasks, often producing suboptimal aesthetics. In this paper, we introduce a new pipeline to enhance the aesthetic quality of LLM-generated code. We first construct AesCode-358K, a large-scale instruction-tuning dataset focused on code aesthetics. Next, we propose agentic reward feedback, a multi-agent system that evaluates executability, static aesthetics, and interactive aesthetics. Building on this, we develop GRPO-AR, which integrates these signals into the GRPO algorithm for joint optimization of functionality and code aesthetics. Finally, we develop OpenDesign, a benchmark for assessing code aesthetics. Experimental results show that combining supervised fine-tuning on AesCode-358K with reinforcement learning using agentic reward feedback significantly improves performance on OpenDesign and also enhances results on existing benchmarks such as PandasPlotBench. Notably, our AesCoder-4B surpasses GPT-4o and GPT-4.1, and achieves performance comparable to large open-source models with 480B-685B parameters, underscoring the effectiveness of our approach.

• Release paper and project page
• Release our AesCoder model
• Release AesCode dataset
• Release OpenDesign benchmark code

If you find this codebase useful for your research, please use the following entry.

@misc{xiao2025codeaestheticsagenticreward,
      title={Code Aesthetics with Agentic Reward Feedback}, 
      author={Bang Xiao and Lingjie Jiang and Shaohan Huang and Tengchao Lv and Yupan Huang and Xun Wu and Lei Cui and Furu Wei},
      year={2025},
      eprint={2510.23272},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2510.23272}, 
}