PipeTransformer: Automated Elastic Pipelining for Distributed Training of Transformers
Chaoyang He
,
Shen Li
(Facebook AI Research),
Mahdi Soltanolkotabi
,
Salman Avestimehr
Accepted to
ICML 2021
(International Conference on Machine Learning
202
1)
[
Arxiv
] [
Homepage
] [
Slides
] [
Animation
] [
Open Source Code
] [
DistML.ai
]
--
Chaoyang He
https://chaoyanghe.com