Phu Mon Htut (Phu)

These days I mostly work on safeguards for agentic and multi-agent systems, across different attack surfaces and attack types. In these systems, malicious instructions can arrive through user input, but also through tool outputs, tool descriptions or messages from other agents. I also work on catching unsafe actions and hallucinated tool calls. Our recent paper, MAS-Guard, proposes guardrails that work for both single agents and multi-agent systems arXiv '26. Before Oracle, I led the prompt-injection, toxic-content and hallucination detectors that shipped in Amazon Bedrock Guardrails.

A lot of my work has been about evaluation, both for production systems and in published research. At AWS I worked with human annotators to collect high-quality seed data for Bedrock Guardrails and for RAG post-training of Amazon Titan models, and built pipelines that generate synthetic data from it for fine-tuning and evaluation. At Oracle I run the evaluation and auto-research pipeline for our agent safeguards: synthetic data and agent environments (like AgentDojo) for measuring vulnerabilities, stress tests before release, and agent loops that improve the guardrails against those metrics arXiv '26.

I've also worked on benchmarks such as BBQ for social bias in question answering ACL Findings '22, (QA)² for questions with questionable assumptions ACL '23, and, with my intern, open-domain QA with conflicting retrieved contexts NAACL Findings '25. Beyond building them, I've looked at how good our benchmarks are, using item response theory to compare test sets ACL '21 and to cluster examples across benchmarks Insights '22. During my PhD I helped build jiant, a toolkit for training and evaluating NLU models, and at AWS I co-organized the IWSLT 2023 shared task on formality control in translation IWSLT '23.

I like figuring out why a model, or a whole agentic system around it, behaves the way it does, finding its failure modes, and fixing them. During my time on the Amazon Titan foundation model team at AWS, I explored methods for estimating P("I know") and used them in ablation experiments to design fine-tuning data that teaches a model to say it doesn't know, ask a follow-up question, or call retrieval (some of it made its way into the released Titan models), building on work by Kadavath et al. (2022) showing that models largely know what they know. With PhD interns, I've also worked on long-context hallucination detection NAACL Findings '25, estimating black-box uncertainty by probing a model with varied rephrasings EMNLP Findings '25, and checking whether vision-language models' reasoning steps are grounded in the image, and repairing those that aren't EACL '26.

My PhD work had a similar flavor. I studied when and why intermediate-task transfer helps pretrained models ACL '20 AACL '20, what BERT knows about grammar EMNLP '19 and whether its attention heads track syntax arXiv '19, and replicated a grammar induction result to understand why it worked EMNLP '18.