We study how autonomous coding agents fail during ordinary development work and design safeguards, from constraint enforcement to failure transparency and safe-halt behaviors, for deploying them responsibly.
Case Western Reserve University · Computer and Data Sciences
reSAID Lab
Responsible Software and AI Design Lab
The reSAID Lab studies the principles and practices for specifying, building, evaluating, and maintaining AI-enabled software systems. We combine empirical software engineering, formal methods, program analysis, and systems-oriented experimentation to understand how LLMs, coding agents, and ML components behave in real development settings. Our goal is to develop rigorous methods and practical tools that improve reliability, fairness, safety, and maintainability across the lifecycle of responsible AI software.
Projects
Testing and analysis for hidden failure modes in large language and vision-language models, from social bias under black-box access to reasoning-level backdoors.
Methods that make language-model reasoning more deliberate, structured, and inspectable, separating high-level planning from low-level action generation.
Simulation-based analysis of long-term fairness and safety in ML-enabled systems whose decisions reshape their own future inputs through feedback loops.
SMT-based verification of individual fairness in neural networks, using input partitioning and sound neural pruning to produce certificates or counterexamples for real-world models.
We study how technical debt appears and evolves in machine learning software, mining self-admitted technical debt at scale to guide the maintenance of ML systems.
News
Talk
Zhihao Dou presents PTA-GRPO at ICML 2026
Zhihao Dou presented our paper on plan-then-action reasoning with guidance-aware reinforcement learning (PTA-GRPO) at ICML 2026 in Seoul, South Korea. Related paper
Service
Guest Editor of ASE Journal special issue on trustworthy LLMs
Sumon Biswas is serving as Guest Editor of the Springer Automated Software Engineering journal's special issue on Software Engineering for and with Trustworthy Large Language Models, connected to the lab's LLMTrust workshop.
Service
LLMTrust workshop held at FSE 2026
Sumon Biswas organized the International Workshop on Trustworthy Large Language Models for Software Engineering (LLMTrust), co-located with FSE 2026 — a successful edition with great participation and lively discussion.
Publication
ReShift backdoor paper accepted to ECCV 2026
Our paper proposing ReShift, a reasoning-level backdoor framework for Vision–Language Models, was accepted to ECCV 2026 in Malmö, Sweden. Related paper
Service
Co-Chair of the AAAI Fall Symposium on Trustworthy Agentic Systems (TAS 2026)
Sumon Biswas is serving as Co-Chair of the AAAI Fall Symposium on Trustworthy Agentic Systems (TAS 2026), November 5-7, 2026, in Arlington, Virginia. Link
People
Sumon Biswas
Principal Investigator
Assistant Professor, Department of Computer and Data Sciences, Case Western Reserve University
Ph.D. Students
M.S. Students
Undergraduate Researchers
High School
Recent Publications
What Breaks When LLMs Code? Characterizing Operational Safety Failures of Agentic Code Assistants
41st IEEE/ACM International Conference on Automated Software Engineering (ASE) 2026
@inproceedings{hasan2026breaks,
title={What Breaks When LLMs Code? Characterizing Operational Safety Failures of Agentic Code Assistants},
author={Hasan, Alif Al and Biswas, Sumon},
booktitle={Proceedings of the 41st IEEE/ACM International Conference on Automated Software Engineering (ASE)},
year={2026}
}
arXiv
ReShift: Aha-Moment-Driven Reasoning-Level Backdoor Attacks on Vision–Language Models
European Conference on Computer Vision (ECCV) 2026
@inproceedings{dou2026reshift,
title={ReShift: Aha-Moment-Driven Reasoning-Level Backdoor Attacks on Vision--Language Models},
author={Dou, Zhihao and Zhao, Qinjian and Gao, Zhiqiang and Biswas, Sumon},
booktitle={Proceedings of the European Conference on Computer Vision (ECCV)},
year={2026}
}
arXiv
Plan Then Action: High-Level Planning Guidance Reinforcement Learning for LLM Reasoning
43rd International Conference on Machine Learning (ICML) 2026
@inproceedings{dou2026plan,
title={Plan Then Action: High-Level Planning Guidance Reinforcement Learning for LLM Reasoning},
author={Dou, Zhihao and Zhao, Qinjian and Wan, Zhongwei and Zhang, Dinggen and Wang, Weida and Raiyan, Towsif
and Chen, Benteng and Pan, Qingtao and Ouyang, Yang and Gao, Zhiqiang and Zhang, Shufei and Biswas, Sumon},
booktitle={Proceedings of the 43rd International Conference on Machine Learning},
year={2026}
}
arXiv
Bias Testing and Mitigation in Black Box LLMs using Metamorphic Relations
In minor revision, ACM Transactions on Software Engineering and Methodology (TOSEM) 2026
@article{salimian2026bias,
title={Bias Testing and Mitigation in Black Box LLMs using Metamorphic Relations},
author={Salimian, Sina and Uddin, Gias and Biswas, Sumon and Leung, Henry},
journal={arXiv preprint arXiv:2512.00556},
year={2026}
}
arXiv
FairSense: Long-Term Fairness Analysis of ML-Enabled Systems
47th IEEE/ACM International Conference on Software Engineering (ICSE) 2025
@inproceedings{she2025fairsense,
title={FairSense: Long-Term Fairness Analysis of ML-Enabled Systems},
author={She, Yining and Biswas, Sumon and K{\"a}stner, Christian and Kang, Eunsuk},
booktitle={Proceedings of the 47th International Conference on Software Engineering},
pages={782--794},
year={2025},
doi={10.1109/ICSE55347.2025.00159}
}
DOI
Are Prompt Engineering and TODO Comments Friends or Foes? An Evaluation on GitHub Copilot
46th IEEE/ACM International Conference on Software Engineering (ICSE) 2024
@inproceedings{obrien2024prompt,
title={Are Prompt Engineering and TODO Comments Friends or Foes? An Evaluation on GitHub Copilot},
author={OBrien, David and Biswas, Sumon and Imtiaz, Sayem and Abdalkareem, Rabe and Shihab, Emad and Rajan, Hridesh},
booktitle={Proceedings of the 46th International Conference on Software Engineering},
pages={1--13},
year={2024},
doi={10.1145/3597503.3639176}
}
DOI
Data
Towards Safe ML-Based Systems in Presence of Feedback Loops
International Workshop on Dependability and Trustworthiness of Safety-Critical Systems with Machine Learned Components, ESEC/FSE (SE4SafeML) 2023
@inproceedings{biswas2023safe,
title={Towards Safe ML-Based Systems in Presence of Feedback Loops},
author={Biswas, Sumon and She, Yining and Kang, Eunsuk},
booktitle={Proceedings of the International Workshop on Dependability and Trustworthiness of Safety-Critical Systems with Machine Learned Components at ESEC/FSE},
year={2023},
doi={10.1145/3617574.3617861}
}
DOI
Fix Fairness, Don't Ruin Accuracy: Performance Aware Fairness Repair using AutoML
31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (FSE) 2023
@inproceedings{nguyen2023fair,
title={Fix Fairness, Don't Ruin Accuracy: Performance Aware Fairness Repair using AutoML},
author={Nguyen, Giang and Biswas, Sumon and Rajan, Hridesh},
booktitle={Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering},
pages={502--514},
year={2023},
doi={10.1145/3611643.3616257}
}
DOI
Towards Understanding Fairness and its Composition in Ensemble Machine Learning
45th IEEE/ACM International Conference on Software Engineering (ICSE) 2023
@inproceedings{gohar2023ensemble,
title={Towards Understanding Fairness and its Composition in Ensemble Machine Learning},
author={Gohar, Usman and Biswas, Sumon and Rajan, Hridesh},
booktitle={Proceedings of the 45th International Conference on Software Engineering},
pages={1533--1545},
year={2023},
doi={10.1109/ICSE48619.2023.00133}
}
DOI
Fairify: Fairness Verification of Neural Networks
45th IEEE/ACM International Conference on Software Engineering (ICSE) 2023
@inproceedings{biswas2023fairify,
title={Fairify: Fairness Verification of Neural Networks},
author={Biswas, Sumon and Rajan, Hridesh},
booktitle={Proceedings of the 45th International Conference on Software Engineering},
pages={1546--1558},
year={2023},
doi={10.1109/ICSE48619.2023.00134}
}
DOI
