// about

Khanh Nguyen

  • Build efficient inference engines for neural nets across various devices.
  • Optimize VLAs, VLMs, and LLMs for edge deployment.
  • Interested in searching and studying efficient neural net architectures.
Edge AI Model optimization Robotics VLA C / C++

Mentor

An Thai Le

Assistant Professor at VinUniversity and Director of Foundation AI at VinRobotics, working on robots that plan, learn, and act reliably with limited compute and data.

Domain

AI Software Engineer

Inference engineering · Model optimization · Edge AI for robotics · AI SDK development · Efficient VLA inference

Research

M.S. in AI Convergence

Multimodal learning & affective behavior analysis - 6 peer-reviewed papers (CVPRW, ECCVW, RSSW)

// benchmark

Making capable AI models run fast on real hardware.

I build lean C/C++ inference engines that bring VLMs, VLAs, and LLMs to CPUs, accelerators, and robots. Inspired by llama.cpp: a few lines of simple code, big results on your own hardware. The two projects I'm proudest of are vla.cpp and vla.simd - compare their measured latency across devices in the explorer. Three companion pages go deeper: VLA Hub benchmarks every model they serve on every device, VLA Arch illustrates each model's architecture as block diagrams, and XPU Arch diagrams the CPUs, GPUs, NPUs and systems on chip behind VLA Hub.

VLA Hub VLA Arch XPU Arch
latency explorer
Latency ms / call
Actions / s
Memory
Image
Views
Action chunk
Setup
Same model on other devices

    // projects

    Pinned projects

    vla.cpp architecture: a Python robot client sends requests over Protobuf and ZeroMQ to the C++ runtime, which runs on llama.cpp with CUDA and CPU backends and loads a GGUF model bundle

    vla.cpp

    The open-source project I'm P.I.C of at VinRobotics - bringing Vision-Language-Action models to local hardware, in the spirit of llama.cpp for robotics.

    C++ggmlInferenceVLA

    K.D. Nguyen, H.T. Ho, C.T. Nguyen, T.Q. Duong, L.D. Le, D.M.H. Nguyen, V.A. Ngo, A.T. Le

    Code ↗ arXiv ↗ Project Page ↗
    vla.simd kernels: matmul, convolution and softmax built from tiling and a SIMD micro-kernel, with its tile layout and register budget

    vla.simd

    Efficient CPU inference for language-conditioned manipulation - shared SIMD micro-kernels (AVX2 / NEON) plus IMPACT, a language-conditioned policy that supplies 30+ actions per second on a Raspberry Pi 5.

    C++SIMDCPU inferenceVLA

    K.D. Nguyen, H.M. Truong, A.T. Le

    Code ↗ arXiv ↗ Project Page ↗
    FoldQuantVLA folding: channel scaling and an orthogonal rotation shrink activation outliers and are folded into the weights, so (WSRᵀ)(RS⁻¹X) = WX before rounding

    FoldQuantVLA

    Native low-bit quantization of VLA models - W8A8 and W4A4 projections run on the INT8 / INT4 tensor cores through custom TensorRT plugins, 1.20-1.33× faster than float TensorRT on Jetson AGX Orin across GR00T and π0.5.

    QuantizationTensorRTINT4VLA

    H.T. Ho, K.D. Nguyen, Q.D. Nguyen, T.Q. Duong, N. Le, M. Guo, V.A. Ngo, A.T. Le

    Code ↗ arXiv ↗ Project Page ↗
    A PyTorch Linear and ReLU model beside the ggml C++ graph builder that ggml-compiler generates for it

    ggml-compiler

    Compiles PyTorch and ONNX models into ggml programs that run on CPU and CUDA without Python - LLMs, VLMs and the robot policies built on them, each checked against llama.cpp and vla.cpp.

    CompilerggmlPyTorchONNXGGUF
    Code ↗
    // news

    My updates

    ICRA 2027 logo: robots in front of Seoul landmarks above the words SEOUL 2027
    Reviewer for ICRA 2027, the IEEE International Conference on Robotics and Automation, held in Seoul on May 24-28, 2027.
    An SO-101 robot arm putting a roll of tape into a drawer, driven by a VLA running on a Raspberry Pi
    Deployed VLA in real-time on Raspberry Pi with vla.simd.
    Khanh D. Nguyen speaking to a full classroom at SOICT, Hanoi University of Science and Technology
    Speaker at SOICT Summer School 2026 hosted by Hanoi University of Science and Technology, topic Edge AI in Robotics.
    ReadItDown window with a Markdown file, its rendered preview and a terminal side by side
    Released ReadItDown, a minimal Markdown viewer with a folder explorer and built-in terminal, on the web and on Linux, macOS and Windows.
    Model diagram: a speaker behavior extractor with a Multimodal Bottleneck Transformer feeds a listener reaction generator, whose Transformer decoder samples a mixture of Gaussian models to produce multiple facial reactions
    3rd place at the REACT 2024 Challenge held with IEEE FG 2024, generating multiple appropriate facial reactions (paper).
    # Khanh D. Nguyen > AI Software Engineer - inference engineering, model optimization, and edge AI for robotics. [GitHub](https://github.com/khanhnd61-vr) · [LinkedIn](https://www.linkedin.com/in/khanhnd185/) · [Google Scholar](https://scholar.google.com/citations?user=5AdBep8AAAAJ) · [Hugging Face](https://huggingface.co/khanhnd61) · [Website](https://khanhnd61-vr.github.io/) ## About - Build efficient inference engines for neural nets across various devices. - Optimize VLAs, VLMs, and LLMs for edge deployment. - Interested in searching and studying efficient neural net architectures. Topics: Edge AI, Model optimization, Robotics, VLA, C / C++ ### Domain AI Software Engineer: inference engineering · model optimization · edge AI for robotics · AI SDK development · efficient VLA inference ### Research M.S. in AI Convergence: multimodal learning & affective behavior analysis - 6 peer-reviewed papers (CVPRW, ECCVW, RSSW) ### Mentor [An Thai Le](https://anindex.github.io/) - Assistant Professor at VinUniversity and Director of Foundation AI at VinRobotics, working on robots that plan, learn, and act reliably with limited compute and data. ## Focus Making capable AI models run fast on real hardware. I build lean C/C++ inference engines that bring VLMs, VLAs, and LLMs to CPUs, accelerators, and robots. Inspired by [llama.cpp](https://github.com/ggml-org/llama.cpp): a few lines of simple code, big results on your own hardware. The two projects I'm proudest of are [vla.cpp](https://github.com/VinRobotics/vla.cpp) and [vla.simd](https://github.com/cair-vinuni/vla.simd). Three companion pages go deeper: [VLA Hub](https://khanhnd61-vr.github.io/vla-hub/) benchmarks every model they serve on every device, [VLA Arch](https://khanhnd61-vr.github.io/vla-arch/) illustrates each model's architecture as block diagrams, and [XPU Arch](https://khanhnd61-vr.github.io/xpu-arch/) diagrams the CPUs, GPUs, NPUs and systems on chip behind VLA Hub. ## Projects ### vla.cpp The open-source project I'm P.I.C of at VinRobotics - bringing Vision-Language-Action models to local hardware, in the spirit of llama.cpp for robotics. - GitHub: https://github.com/VinRobotics/vla.cpp - arXiv: https://arxiv.org/abs/2606.08094 - Project page: https://vlacpp.github.io/ - Tags: C++, ggml, Inference, VLA - Authors: K.D. Nguyen, H.T. Ho, C.T. Nguyen, T.Q. Duong, L.D. Le, D.M.H. Nguyen, V.A. Ngo, [A.T. Le](https://anindex.github.io/) ### vla.simd Efficient CPU inference for language-conditioned manipulation - shared SIMD micro-kernels (AVX2 / NEON) plus IMPACT, a language-conditioned policy that supplies 30+ actions per second on a Raspberry Pi 5. - GitHub: https://github.com/cair-vinuni/vla.simd - arXiv: https://arxiv.org/abs/2609.24274 - Project page: https://vla-simd.github.io/ - Tags: C++, SIMD, CPU inference, VLA - Authors: K.D. Nguyen, [H.M. Truong](https://tmhoanggg.github.io/), [A.T. Le](https://anindex.github.io/) ### FoldQuantVLA Native low-bit quantization of VLA models - W8A8 and W4A4 projections run on the INT8 / INT4 tensor cores through custom TensorRT plugins, 1.20-1.33× faster than float TensorRT on Jetson AGX Orin across GR00T and π0.5. - GitHub: https://github.com/cair-vinuni/FoldQuantVLA - arXiv: https://arxiv.org/abs/2609.24433 - Project page: https://review-artifact-27f4.github.io/foldquantvla - Tags: Quantization, TensorRT, INT4, VLA - Authors: H.T. Ho, K.D. Nguyen, Q.D. Nguyen, T.Q. Duong, N. Le, M. Guo, V.A. Ngo, [A.T. Le](https://anindex.github.io/) ### ggml-compiler Compiles PyTorch and ONNX models into ggml programs that run on CPU and CUDA without Python - LLMs, VLMs and the robot policies built on them, each checked against llama.cpp and vla.cpp. - GitHub: https://github.com/khanhnd61-vr/ggml-compiler - Tags: Compiler, ggml, PyTorch, ONNX, GGUF ## News - 2026-10-28: Reviewer for [ICRA 2027](https://2027.ieee-icra.org/), the IEEE International Conference on Robotics and Automation, held in Seoul on May 24-28, 2027. - 2026-09-16: Deployed VLA in real-time on Raspberry Pi with [vla.simd](https://vla-simd.github.io/). - 2026-08-11: Speaker at [SOICT Summer School 2026](https://soict.hust.edu.vn/summer-school) hosted by Hanoi University of Science and Technology, topic [Edge AI in Robotics](https://khanhnd61-vr.github.io/modelopt101/). - 2026-07-02: Released [ReadItDown](https://github.com/khanhnd61-vr/readitdown), a minimal Markdown viewer with a folder explorer and built-in terminal, on the web and on Linux, macOS and Windows. - 2024-05-31: 3rd place at the [REACT 2024 Challenge](https://sites.google.com/cam.ac.uk/react2024/) held with [IEEE FG 2024](https://fg2024.ieee-biometrics.org/), generating multiple appropriate facial reactions ([paper](https://brosdocs.net/fg2024/CT10.pdf)). ## Companion pages - [VLA Hub](https://khanhnd61-vr.github.io/vla-hub/): measured latency and memory for open VLA models on GPUs, NPUs and CPUs, run with vla.cpp and vla.simd - [VLA Arch](https://khanhnd61-vr.github.io/vla-arch/): each model's architecture as block diagrams - [XPU Arch](https://khanhnd61-vr.github.io/xpu-arch/): block diagrams of the CPUs, GPUs, NPUs and systems on chip behind VLA Hub - [ReadItDown](https://khanhnd61-vr.github.io/readitdown/): a minimal, fast Markdown viewer - try it online, or download the desktop app for Linux, macOS and Windows