Mickel Liu

I am a Ph.D. student at Paul G. Allen School of Computer Science & Engineering, University of Washington, co-advised by Prof. Natasha Jaques and Prof. Luke Zettlemoyer. I recently joined Meta FAIR as a visiting researcher.

I did my Master's at the School of Computer Science, Peking University, mentored by Prof. Yizhou Wang and Prof. Yaodong Yang.

Email  /  Short CV  /  Google Scholar  /  Github /  Twitter 

We used self-play reinforcement learning and hidden Chain-of-Throught to discover more diverse adversarial attacks and to align safer language model.

We present the BeaverTails dataset for safety research in large language models. Our findings show that modeling decoupled human preferences for helpfulness and harmlessness improves LLM safety without sacrificing performance.

Building on the previous BeaverTails dataset, we introduce an RLHF algorithm with safety constraints. Using the Lagrangian method, Safe RLHF fine-tunes the balance between harmlessness and helpfulness. Our three-round fine-tuning shows better mitigation of harmful responses and improved performance over existing value-aligned algorithms.
This work has been graciously promote-tweeted by Ahsen (@_akhaliq).

During my time at Baichuan, I have participated in the open-sourcing of our LLMs, which were trained from scratch on 2.6 trillion tokens, and credited as an author to in the corresponding technical report.
Baichuan 2 matches or exceeds the performance of other open-source models in its class across various public benchmarks, including MMLU, CMMLU, GSM8K, and HumanEval. Notably, Baichuan2-13B-Chat achieved the highest score on the SuperCLUE-agent benchmark among all open-sourced models.

Active3DPose Presents a multi-agent reinforcement learning (MARL) scheme for proactive Multi-Camera Collaboration in 3D Human Pose Estimation in dynamic human crowds. Aerial cameras are decentralized, self-interest agents, but need to collaborate to complete the given task. We proposed a reward structure inspired by the solution concept of Shapley value that helps facilitating the process of collaboration. The simulation environment is built using UnrealEngine and we used distributive RL framework Ray RLlib to train our agents.

We introduce the Multi-Agent Tracking Environment (MATE), a novel multi-agent environment simulates the target coverage control problems in the real world. MATE hosts an asymmetric cooperative-competitive game consisting of two groups of learning agents — "cameras" and "targets" — with opposing interests. This process of co-evolution between cameras and targets helps to realize a less exploitable camera network.

We introduce a framework designed to expedite SafeRL research endeavors. Our framework encompasses an collection of algorithms spanning different RL domains and places heavy emphasis on safety elements.