[FG 2026] LibreFace 2.0: A Generalizable Facial Expression Analysis Toolkit Leveraging Synthetic Data

Xulang Guan*, Ashutosh Chaubey*, Maksim Siniukov, Annabelle Hsieh, Zongjian Li, Mohammad Soleymani

[WACV 2024] LibreFace: An Open-Source Toolkit for Deep Facial Expression Analysis

Di Chang, Yufeng Yin, Zongjian Li, Minh Tran, Mohammad Soleymani

Institute for Creative Technologies, University of Southern California

• Introduction
• Getting started with Python installation
• Usage
  • Using commandline
  • Using Python script
• Getting Started with Derivative Tools
• Gaze estimation
• Synthetic Data for LibreFace 2.0
• Training models
• Results and Accuracy
• TODO
• License
• Citation
• Contact
• Acknowledgments

This is the official implementation of our WACV 2024 Application Track paper: LibreFace: An Open-Source Toolkit for Deep Facial Expression Analysis. The recent update (May 2026) incorporates the code and checkpoints for our FG 2026 paper "LibreFace 2.0: A Generalizable Facial Expression Analysis Toolkit Leveraging Synthetic Data". LibreFace is an open-source and comprehensive toolkit for accurate and real-time facial expression analysis with both CPU-only and GPU-acceleration versions. LibreFace eliminates the gap between cutting-edge research and an easy and free-to-use non-commercial toolbox. We propose to adaptively pre-train the vision encoders with various face datasets and then distillate them to a lightweight ResNet-18 and RepVGG models in a feature-wise matching manner. LibreFace 2.0 additionally supports gaze estimation using a MediaPipe landmark-based MLP pipeline. We conduct extensive experiments of pre-training and distillation to demonstrate that our proposed pipeline achieves comparable results to state-of-the-art works while maintaining real-time efficiency. LibreFace system supports cross-platform running, and the code is open-sourced in C# (model inference and checkpoints) and Python (model training, inference, and checkpoints).

• Python 3.9
• You should have cmake installed in your system.
  • For Linux users - sudo apt-get install cmake. If you run into trouble, consider upgrading to the latest version (instructions).
  • For Mac users - brew install cmake.

You can first create a new Python 3.9 environment using conda and then install this package using pip from the PyPI hub:

conda create -n libreface_env python=3.9
conda activate libreface_env
pip install --upgrade libreface

You can use this package through the command line using the following command.

libreface --input_path="path/to/your_image_or_video"

Note that the above script would save results in a CSV at the default location - sample_results.csv.

libreface --input_path="path/to/your_image_or_video" --output_path="path/to/save_results.csv"

libreface --input_path="path/to/your_image_or_video" --device="cuda:0"

libreface --input_path="path/to/your_image_or_video" --temp="your/temp/path"

libreface --input_path="path/to/your_video" --batch_size=256 --num_workers=2 --device="cuda:0"

libreface --input_path="sample_disfa.png"

libreface --input_path="sample_disfa.avi" --output_path="my_custom_file.csv" --device="cuda:0"

Here’s how to use this package in your Python scripts.

import libreface 
detected_attributes = libreface.get_facial_attributes(image_or_video_path)

import libreface 
libreface.get_facial_attributes(image_or_video_path,
                                output_save_path = "your_save_path.csv")

import libreface 
libreface.get_facial_attributes(image_or_video_path,
                                device = "cuda:0") # can be "cpu" or "cuda:0", "cuda:1", ...

import libreface 
libreface.get_facial_attributes(image_or_video_path,
                                temp_dir = "your/temp/path")

import libreface 
libreface.get_facial_attributes(video_path,
                                batch_size = 256,
                                num_workers = 2)

import libreface 
libreface.get_facial_attributes(image_or_video_path,
                                weights_download_dir = "your/directory/path")

import libreface

aligned_image_path, _, _ = libreface.get_aligned_image("path/to/your_image.png", temp_dir="./tmp")
gaze = libreface.estimate_gaze(aligned_image_path, device="cpu")
print(gaze)  # {'gaze_yaw': <float>, 'gaze_pitch': <float>}

import libreface

gaze_df = libreface.estimate_gaze_video(aligned_frames_path_list,
                                        device="cuda:0",
                                        batch_size=256,
                                        num_workers=2)
# gaze_df has columns "gaze_yaw" and "gaze_pitch", one row per frame.

We offer several derivative tools on the .NET platform to facilitate easier integration of LibreFace into various systems, in addition to pytorch code. These works are based on ONNX platform weights exported from the model weights of this project.

• NuGet Package: We have released a NuGet Package named LibreFace. This NuGet Package contains the ONNX weight files, and its source code is located in this directory of the OpenSense repository. For how to integrate it, you can refer to the documentation that comes with the Package or the source code of the OpenSense Component below. This Package is cross-platform compatible and is recommended to be used with an ONNX Runtime Execution Provider with acceleration features. For non-.NET developers, you can access the ONNX weight files inside the package by changing the extension from .nupkg to .zip.

• OpenSense Component: We have also wrapped LibreFace into a component available in OpenSense. With this work, other components in OpenSense can be used in conjunction with LibreFace in Zero-Code setup for real-time or non-real-time inference. Its source code is mainly stored in this directory of the OpenSense repository. Running this component by default requires CUDA support, but other ONNX Providers can be used when compiling from the source code. A OpenSense Pipeline we were using for testing can be used as an example and can be downloaded from here. Please set the camera you want to use before running it.

Gaze yaw and pitch are returned by default in the get_facial_attributes output (keys gaze_yaw and gaze_pitch, or columns of the same name for video).

The figure above shows a Gaze360 example with three panels: biased (raw) model output (red, yaw=+11.07°, pitch=+11.43°), unbiased (bias-corrected) prediction (green, yaw=+4.51°, pitch=+6.44°), and ground truth (blue, yaw=-7.68°, pitch=+2.61°). Bias correction subtracts the systematic offset measured on the held-out test set (+6.55° yaw, +5.05° pitch).

If you only need gaze and want to skip AU and expression inference, call estimate_gaze (image) or estimate_gaze_video (list of aligned frames) directly. Both expect an already aligned face image — use libreface.get_aligned_image to produce one from a raw image.

Real AU datasets like DISFA and BP4D lack demographic diversity, so LibreFace 2.0 augments training with synthetic data: Stable Diffusion 3.5 generates diverse identities (age, gender, race), and LivePortrait retargets real AU motion from DISFA/BP4D driving videos onto them. This yielded 240,000+ synthetic frames across 708 identities, mixed with real data during training to drive the fairness gains seen on RAF-AU and AffWild2.

LibreFace's underlying AU intensity, AU detection, and facial expression recognition models can be trained from scratch in Python — dataset preparation, per-task training/inference commands, and config flags are documented in TRAINING.md.

We performed a variety of evaluations across several demographic axes to observe how our model performs on different groups of people.

Average Pearson Correlation Coefficient (PCC) across the 12 AUs of DISFA (5-fold cross-validation), comparing LibreFace 2.0 against prior open-source and academic baselines.

Beyond raw accuracy, LibreFace 2.0 narrows the performance gap across demographic groups. On AffWild2, it achieves both the highest F1 and the lowest standard deviation across gender and racial groups, indicating more consistent, fairer AU detection across subpopulations than OpenFace 2.0 or LibreFace 1.0.

Our code is distributed under the USC research license. See LICENSE.txt file for more information.

If you use LibreFace, make sure to cite both of our works below:

@InProceedings{Chang_2024_WACV,
    author    = {Chang, Di and Yin, Yufeng and Li, Zongjian and Tran, Minh and Soleymani, Mohammad},
    title     = {LibreFace: An Open-Source Toolkit for Deep Facial Expression Analysis},
    booktitle = {Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)},
    month     = {January},
    year      = {2024},
    pages     = {8205-8215}
}

@INPROCEEDINGS{11556988,
  author={Guan, Xulang and Chaubey, Ashutosh and Siniukov, Maksim and Hsieh, Annabelle and Li, Zongjian and Soleymani, Mohammad},
  booktitle={2026 IEEE 20th International Conference on Automatic Face and Gesture Recognition (FG)},
  title={LibreFace 2.0: A Generalizable Facial Expression Analysis Toolkit Leveraging Synthetic Data},
  year={2026},
  volume={},
  number={},
  pages={1-10},
  keywords={Modeling;Printing;Facial expressions;Aging;Conferences;Signal detection;Action units;Training;Computers;Faces},
  doi={10.1109/FG67764.2026.11556988}}

If you have any questions, please raise an issue or email Ashutosh Chaubey (achaubey@usc.edu) or Mohammad Soleymani (soleymani@ict.usc.edu).

Our code follows several awesome repositories, KD_SRRL, XNorm, and LivePortrait. We appreciate them for making their codes available to public.

This work is sponsored by the U.S. Army Research Laboratory (ARL) under contract number W911NF-14-D-0005. The content of the information does not necessarily reflect the position or the policy of the Government, and no official endorsement should be inferred.