琥珀青葉@KohakuLab

@KBlueleaf

Graduate student in Taiwan. Leader of KohakuLab. Researcher/Dev in ComfyOrg

Taiwan

kblueleaf.net

Joined May 2021

1,005 Following

5,212 Followers

Replies Reposts Media

Pinned
琥珀青葉@KohakuLab
@KBlueleaf
Sep 29
TT-VidT is accepted at #NeurIPS2026 🎉 A temporal-decoupled methodology in video pretraining to provide real "motion" info extracted from the video and only requires small scale training to reach high quality results. Huge thanks to all collaborators, @NVIDIAAI and @spellbrush
3
琥珀青葉@KohakuLab
@KBlueleaf
Nov 3
After encounter tons of bad training dynamic in tons of different tasks and most all of them are about residual stream or hidden state exploding I feel like I should fully read EDM2 again...
arxiv.org
Analyzing and Improving the Training Dynamics of Diffusion Models
Diffusion models currently dominate the field of data-driven image synthesis with their unparalleled scaling to large datasets. In this paper, we identify and rectify several causes for uneven and...
1
琥珀青葉@KohakuLab
@KBlueleaf
Nov 3
An issue can be fixed by monitor few hundred to few k step with few line fix will not "cause your model training become None" and using a thing that hide the issue is not "better than None" when you can find and fix at very early
2
琥珀青葉@KohakuLab
@KBlueleaf
Nov 3
Another example on precision issue is not dtype issue. Thhe y axis is the cosine sim of gradient vs fp64 reference. Why we have such issue? bcuz the mantissa is too few to bring the information when difference is not large enough. github.com/KohakuBlueleaf…
2
琥珀青葉@KohakuLab
@KBlueleaf
Nov 1
github.com/KohakuBlueleaf… Tokenomos have added V100 support UwUb
琥珀青葉@KohakuLab
@KBlueleaf
Nov 1
Replying to @vincentzed_cuda
Tokenomos use this idea with improved multi request scheduler and reach few more time speed in small model github.com/KohakuBlueleaf…

琥珀青葉@KohakuLab

琥珀青葉@KohakuLab’s posts