I am a Ph.D. student at KAIST, advised by Professor Joon Son Chung. My research focuses on multimodal learning and generative modeling for natural human–AI interaction, particularly joint audio–visual generation and efficient streaming conversational agents. I am also interested in evaluating the quality, cross-modal consistency, and contextual appropriateness of generated speech and facial behavior.
🔥 News
- 2026.10: 🎉 Three papers are released on arXiv.
- 2026.03: 🎉 One paper is accepted at IEEE Transactions on Multimedia.
- 2025.12: 🎉 One paper is released on arXiv.
- 2025.03: 🎉 One paper is accepted as an oral presentation at ICASSP 2025.
- 2025.03: 🏆 Received the Best Teaching Assistant Award from KAIST EE.
📝 Publications
Selected Publications
JAVIS: Joint Audio-Visual Interaction and Streaming for Real-Time Conversational Agents
Junseok Ahn, Jaehyun Park, Jeongsoo Choi, Joon Son Chung, Ji-Hoon Kim
- Real-time audio-visual agent generating speech and face from one representation on a shared LLM. ~200× lower initial latency than a strong baseline, with better lip sync and expression appropriateness.
TAVID: Text-Driven Audio-Visual Interactive Dialogue Generation
Ji-Hoon Kim*, Junseok Ahn*, Doyeop Kwak, Joon Son Chung, Shinji Watanabe
- Jointly generates interactive talking/listening faces and conversational speech from text and reference images. Shown effective in face realism, listening responsiveness, interaction fluency and speech quality.
VoiceDiT: Dual-Condition Diffusion Transformer for Environment-Aware Speech Synthesis
Jaemin Jung*, Junseok Ahn*, Chaeyoung Jung, Tan Dat Nguyen, Youngjoon Jang, Joon Son Chung
- Generates environment-aware speech from text and visual prompts. Outperforms prior models on real-world data in audio quality and modality integration.
- Oral presentation
VoxSim: A perceptual voice similarity dataset
Junseok Ahn, Youkyum Kim, Yeunju Choi, Doyeop Kwak, Ji-Hoon Kim, Seongkyu Mun, Joon Son Chung
- Perceptual voice similarity dataset of ~70k human ratings on 41k VoxCeleb utterance pairs. Trained models generalize to out-of-domain VCC2018.
- Best Student Paper Award Nominee
Faces that Speak: Jointly Synthesising Talking Face and Speech from Text
Youngjoon Jang*, Ji-Hoon Kim*, Junseok Ahn, Doyeop Kwak, Hong-Sun Yang, Yoon-Cheol Ju, Il-Hwan Kim, Byeong-Yeol Kim, Joon Son Chung
- Jointly synthesizes talking faces and speech from text in a unified framework. Produces natural faces with text-matched speech and generalizes to unseen identities.
Slowfast Network for Continuous Sign Language Recognition
Junseok Ahn*, Youngjoon Jang*, Joon Son Chung
- Two-pathway SlowFast network capturing spatial and dynamic features for continuous sign language recognition via bi-directional fusion and pathway enhancement. Outperforms prior state of the art on PHOENIX14, PHOENIX14-T and CSL-Daily.
Other Publications
arXiv 2026DIAL-S2ST: Learning Latency Control for Simultaneous Speech-to-Speech Translation, Jeongsoo Choi, Junseok Ahn, Joon Son ChungarXiv 2026X-Dubbing: Speech-to-Speech Translation under Temporal and Visual Constraints, Jeongsoo Choi, Ji-Hoon Kim, Junseok Ahn, Joon Son ChungIEEE TMM 2026Deep Understanding of Sign Language for Sign to Subtitle Alignment, Youngjoon Jang, Jeongsoo Choi, Junseok Ahn, Joon Son Chung
* Equal contribution
📖 Education
- 2023.03 - now, Integrated M.S./Ph.D. in Electrical Engineering, KAIST. Advisor: Joon Son Chung. Expected graduation: Feb. 2028
- 2019.03 - 2023.02, B.S. in Electrical Engineering, KAIST. Summa Cum Laude; ranked 2nd among KAIST’s 2023 bachelor’s graduates
🎖 Honors and Awards
- 2025.03 Best Teaching Assistant Award, KAIST School of Electrical Engineering (EE488, Fall 2024)
- 2024.09 Best Student Paper Award Nominee, Interspeech 2024
- 2019 - 2021 Dean’s List, KAIST (Spring 2019, Spring 2020, Spring 2021)
💻 Research Experience
- 2025.04 - now, Graduate Researcher, IITP-funded Project: Sign Language Video Anonymization, KAIST Multimodal AI Lab.
- Avatar-based anonymization for sign language video, preserving hand motion, body pose and facial expression; 3D motion parameter prediction integrated with 3D Gaussian Splatting avatar rendering, targeting real-time motion transfer.
- 2023.07 - 2023.12, Graduate Researcher, Samsung Research–KAIST Collaboration, KAIST Multimodal AI Lab.
- Led data curation, perceptual annotation and quality control for VoxSim (Interspeech 2024); developed WavLM- and ECAPA-TDNN-based models for perceptual speaker similarity and synthesized speech quality assessment.
- 2022.03 - 2022.12, Undergraduate Researcher (URP), KAIST Multimodal AI Lab.
- Language-invariant speaker embeddings via adversarial training with gradient reversal; evaluated cross-lingual speaker verification.
- 2021.09 - 2022.02, Student Researcher (Intern), Samsung Electronics System LSI.
- MATLAB simulator for a 5G NR PUSCH MIMO receiver; compared zero-forcing, LMMSE and maximum-likelihood receivers by bit error rate.
🔍 Academic Service
- Conference Reviewer: SLT 2026, EMNLP 2026, ICML 2026, ICASSP 2026, ASRU 2025, NeurIPS 2025
📚 Teaching
- EE738 Speech Recognition Systems, Teaching Assistant (Spring 2023, Spring 2024, Spring 2026; Head TA in Spring 2024)
- EE488 Deep Learning for Visual Understanding, Teaching Assistant (Fall 2023, Fall 2024; Head TA in Fall 2024)
