Skip to content

Commit ce9f454

Browse files
mi804mi804
authored
Add optional MiniMax-H3 Training Adapter (#1678)
* Add optional MiniMax-H3 Training Adapter preset to training scripts and a news entry * Drop the Training Adapter preset block from full fine-tuning scripts * Mention the self-generated MiniMax-H3 dataset in the Training Adapter news entry
1 parent 7822d17 commit ce9f454

19 files changed

Lines changed: 85 additions & 29 deletions

‎README.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -40,6 +40,8 @@ See also:
4040
4141
> Currently, the development personnel of this project are limited, with most of the work handled by [Artiprocher](https://github.com/Artiprocher) and [mi804](https://github.com/mi804). Therefore, the progress of new feature development will be relatively slow, and the speed of responding to and resolving issues is limited. We apologize for this and ask developers to understand.
4242
43+
- **September 8, 2026** We trained a [MiniMax-H3 Training Adapter](https://www.modelscope.cn/models/DiffSynth-Studio/MiniMax-H3-TrainingAdapter) for the CFG-distilled MiniMax-H3 base, a DeCFG LoRA training adapter offered in FL2VA and Ref2VA versions. The training data comes from the self-generated [MiniMax-H3-Self-Generated-Dataset](https://www.modelscope.cn/datasets/DiffSynth-Studio/MiniMax-H3-Self-Generated-Dataset). Based on this adapter, we trained two toy models, [MiniMax-H3-Songyu-LoRA](https://www.modelscope.cn/models/mibei0804/MiniMax-H3-Songyu-LoRA) (character identity, FL2VA) and [MiniMax-H3-Ref2VA-FirstFrame-Lineart](https://www.modelscope.cn/models/mibei0804/MiniMax-H3-Ref2VA-FirstFrame-Lineart) (lineart first-frame control, Ref2VA), as references for inference and fine-tuning.
44+
4345
- **September 1, 2026** We have integrated [SenseNova-U1.5](https://www.modelscope.cn/models/SenseNova/SenseNova-U1.5-8B-MoT), SenseTime's unified multimodal model, for which we provide text-to-image generation, image editing, low VRAM inference, and training support. For details, please refer to the [documentation](/docs/en/Model_Details/SenseNova-U1.md) and [example code](/examples/sensenova_u1/).
4446

4547
- **August 31, 2026** We have integrated [Qwen-Video-Edit](https://modelscope.cn/models/yunpeng1998/Qwen-Video-Edit), a video editing model developed by open-source community contributor [yunpeng1998](https://github.com/yunpeng1998) based on the image editing model Qwen-Image-Edit. This serves as an excellent example of exploring and expanding model capabilities.

‎README_zh.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -40,6 +40,8 @@
4040
4141
> 目前本项目的开发人员有限，大部分工作由 [Artiprocher](https://github.com/Artiprocher) 和 [mi804](https://github.com/mi804) 负责，因此新功能的开发进展会比较缓慢，issue 的回复和解决速度有限，我们对此感到非常抱歉，请各位开发者理解。
4242
43+
- **2026年9月8日** 我们为 CFG 蒸馏的 MiniMax-H3 基座训练了 [MiniMax-H3 Training Adapter](https://www.modelscope.cn/models/DiffSynth-Studio/MiniMax-H3-TrainingAdapter)，这是一个 DeCFG LoRA 训练 Adapter，提供 FL2VA 与 Ref2VA 两个版本。训练数据来自自生成数据集 [MiniMax-H3-Self-Generated-Dataset](https://www.modelscope.cn/datasets/DiffSynth-Studio/MiniMax-H3-Self-Generated-Dataset)。基于该 Adapter，我们训练了两个 toy 模型，[MiniMax-H3-Songyu-LoRA](https://www.modelscope.cn/models/mibei0804/MiniMax-H3-Songyu-LoRA)（角色身份，FL2VA）与 [MiniMax-H3-Ref2VA-FirstFrame-Lineart](https://www.modelscope.cn/models/mibei0804/MiniMax-H3-Ref2VA-FirstFrame-Lineart)（线稿首帧控制，Ref2VA），供推理与微调参考。
44+
4345
- **2026年9月1日** 我们接入了 [SenseNova-U1.5](https://www.modelscope.cn/models/SenseNova/SenseNova-U1.5-8B-MoT)，这是商汤科技开源的统一多模态模型，我们为其提供了文生图、图像编辑、低显存推理和训练支持。详情请参考[文档](/docs/zh/Model_Details/SenseNova-U1.md)和[示例代码](/examples/sensenova_u1/)。
4446

4547
- **2026年8月31日** 我们接入了 [Qwen-Video-Edit](https://modelscope.cn/models/yunpeng1998/Qwen-Video-Edit)，这是开源社区用户 [yunpeng1998](https://github.com/yunpeng1998) 基于图像编辑模型 Qwen-Image-Edit 训练的视频编辑模型，是探索模型能力拓展的优秀案例。

‎examples/minimax_h3/model_training/lora/MiniMax-H3-FL2VA.sh‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@ modelscope download --dataset DiffSynth-Studio/diffsynth_example_dataset --inclu
22

33
# Optional
44
# 1. Fuse the DeCFG training adapter into the DiT while training, for a better optimization landscape on this CFG-distilled base. Training only -- do not load it at inference.
5-
# modelscope download --model DiffSynth-Studio/MiniMax-H3-TrainingAdapter --local_dir ./models/DiffSynth-Studio/MiniMax-H3-TrainingAdapter
5+
# modelscope download --model DiffSynth-Studio/MiniMax-H3-TrainingAdapter --include model.safetensors --local_dir ./models/DiffSynth-Studio/MiniMax-H3-TrainingAdapter
66
# --preset_lora_path "./models/DiffSynth-Studio/MiniMax-H3-TrainingAdapter/model.safetensors" \
77
# --preset_lora_model "dit"
88
# 2. Add `--training_cfg_scale 4` to both stages below to enable CFG-aware training. Both stages must use the same value because the unconditional embeddings are cached in stage 1.

‎examples/minimax_h3/model_training/lora/MiniMax-H3-FP8-Pruned-FL2VA.sh‎

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,10 @@
11
modelscope download --dataset DiffSynth-Studio/diffsynth_example_dataset --include "minimax_h3/MiniMax-H3-Pruned-FL2VA/*" --local_dir ./data/diffsynth_example_dataset
22

3+
# Optional: fuse the DeCFG training adapter into the DiT while training, for a better optimization landscape on this CFG-distilled base. This DiT uses the ComfyUI qkv layout, so take the model_for_comfy_dit variant. Training only -- do not load it at inference.
4+
# modelscope download --model DiffSynth-Studio/MiniMax-H3-TrainingAdapter --include model_for_comfy_dit.safetensors --local_dir ./models/DiffSynth-Studio/MiniMax-H3-TrainingAdapter
5+
# --preset_lora_path "./models/DiffSynth-Studio/MiniMax-H3-TrainingAdapter/model_for_comfy_dit.safetensors" \
6+
# --preset_lora_model "dit"
7+
38
# T2VA - stage 1: encode text/video/audio and cache pipeline inputs (text encoder loaded here only)
49
accelerate launch examples/minimax_h3/model_training/train.py \
510
--dataset_base_path data/diffsynth_example_dataset/minimax_h3/MiniMax-H3-Pruned-FL2VA \
@@ -35,10 +40,6 @@ accelerate launch examples/minimax_h3/model_training/train.py \
3540
--use_gradient_checkpointing \
3641
--find_unused_parameters \
3742
--task "sft:train"
38-
# Optional: fuse the DeCFG training adapter into the DiT while training, for a better optimization landscape on this CFG-distilled base. This DiT uses the ComfyUI qkv layout, so take the model_for_comfy_dit variant. Training only -- do not load it at inference.
39-
# modelscope download --model DiffSynth-Studio/MiniMax-H3-TrainingAdapter --local_dir ./models/DiffSynth-Studio/MiniMax-H3-TrainingAdapter
40-
# --preset_lora_path "./models/DiffSynth-Studio/MiniMax-H3-TrainingAdapter/model_for_comfy_dit.safetensors" \
41-
# --preset_lora_model "dit"
4243

4344
# input_image / end_image take the first and last frame of the training video
4445
# FL2VA - stage 1

‎examples/minimax_h3/model_training/lora/MiniMax-H3-FP8-Pruned-Ref2VA.sh‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,10 @@
11
modelscope download --dataset DiffSynth-Studio/diffsynth_example_dataset --include "minimax_h3/MiniMax-H3-Pruned-Ref2VA/*" --local_dir ./data/diffsynth_example_dataset
22

3+
# Optional: fuse the DeCFG training adapter into the DiT while training, for a better optimization landscape on this CFG-distilled base. This DiT uses the ComfyUI qkv layout, so take the model_for_comfy_dit variant. Training only -- do not load it at inference.
4+
# modelscope download --model DiffSynth-Studio/MiniMax-H3-TrainingAdapter --include model_ref2va_for_comfy_dit.safetensors --local_dir ./models/DiffSynth-Studio/MiniMax-H3-TrainingAdapter
5+
# --preset_lora_path "./models/DiffSynth-Studio/MiniMax-H3-TrainingAdapter/model_ref2va_for_comfy_dit.safetensors" \
6+
# --preset_lora_model "dit"
7+
38
# Ref2VA - stage 1: encode reference/text/video/audio and cache (text encoder + processor loaded here only)
49
accelerate launch examples/minimax_h3/model_training/train.py \
510
--dataset_base_path data/diffsynth_example_dataset/minimax_h3/MiniMax-H3-Pruned-Ref2VA \

‎examples/minimax_h3/model_training/lora/MiniMax-H3-Fun-Controlnet-Union.sh‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,10 @@
11
modelscope download --dataset DiffSynth-Studio/diffsynth_example_dataset --include "minimax_h3/MiniMax-H3-Fun-Controlnet-Union/*" --local_dir ./data/diffsynth_example_dataset
22

3+
# Optional: fuse the DeCFG training adapter into the DiT while training, for a better optimization landscape on this CFG-distilled base. Training only -- do not load it at inference.
4+
# modelscope download --model DiffSynth-Studio/MiniMax-H3-TrainingAdapter --include model.safetensors --local_dir ./models/DiffSynth-Studio/MiniMax-H3-TrainingAdapter
5+
# --preset_lora_path "./models/DiffSynth-Studio/MiniMax-H3-TrainingAdapter/model.safetensors" \
6+
# --preset_lora_model "dit"
7+
38
# stage 1 (data process)
49
accelerate launch examples/minimax_h3/model_training/train.py \
510
--dataset_base_path data/diffsynth_example_dataset/minimax_h3/MiniMax-H3-Fun-Controlnet-Union \

‎examples/minimax_h3/model_training/lora/MiniMax-H3-Int8-ConvRot-FL2VA.sh‎

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,10 @@
11
modelscope download --dataset DiffSynth-Studio/diffsynth_example_dataset --include "minimax_h3/MiniMax-H3-FL2VA/*" --local_dir ./data/diffsynth_example_dataset
22

3+
# Optional: fuse the DeCFG training adapter into the DiT while training, for a better optimization landscape on this CFG-distilled base. This DiT uses the ComfyUI qkv layout, so take the model_for_comfy_dit variant. Training only -- do not load it at inference.
4+
# modelscope download --model DiffSynth-Studio/MiniMax-H3-TrainingAdapter --include model_for_comfy_dit.safetensors --local_dir ./models/DiffSynth-Studio/MiniMax-H3-TrainingAdapter
5+
# --preset_lora_path "./models/DiffSynth-Studio/MiniMax-H3-TrainingAdapter/model_for_comfy_dit.safetensors" \
6+
# --preset_lora_model "dit"
7+
38
# T2VA - stage 1: encode text/video/audio and cache pipeline inputs (text encoder loaded here only)
49
accelerate launch examples/minimax_h3/model_training/train.py \
510
--dataset_base_path data/diffsynth_example_dataset/minimax_h3/MiniMax-H3-FL2VA \
@@ -35,10 +40,6 @@ accelerate launch examples/minimax_h3/model_training/train.py \
3540
--use_gradient_checkpointing \
3641
--find_unused_parameters \
3742
--task "sft:train"
38-
# Optional: fuse the DeCFG training adapter into the DiT while training, for a better optimization landscape on this CFG-distilled base. This DiT uses the ComfyUI qkv layout, so take the model_for_comfy_dit variant. Training only -- do not load it at inference.
39-
# modelscope download --model DiffSynth-Studio/MiniMax-H3-TrainingAdapter --local_dir ./models/DiffSynth-Studio/MiniMax-H3-TrainingAdapter
40-
# --preset_lora_path "./models/DiffSynth-Studio/MiniMax-H3-TrainingAdapter/model_for_comfy_dit.safetensors" \
41-
# --preset_lora_model "dit"
4243

4344
# input_image / end_image take the first and last frame of the training video
4445
# FL2VA - stage 1

‎examples/minimax_h3/model_training/lora/MiniMax-H3-Int8-ConvRot-Pruned-FL2VA.sh‎

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,10 @@
11
modelscope download --dataset DiffSynth-Studio/diffsynth_example_dataset --include "minimax_h3/MiniMax-H3-Pruned-FL2VA/*" --local_dir ./data/diffsynth_example_dataset
22

3+
# Optional: fuse the DeCFG training adapter into the DiT while training, for a better optimization landscape on this CFG-distilled base. This DiT uses the ComfyUI qkv layout, so take the model_for_comfy_dit variant. Training only -- do not load it at inference.
4+
# modelscope download --model DiffSynth-Studio/MiniMax-H3-TrainingAdapter --include model_for_comfy_dit.safetensors --local_dir ./models/DiffSynth-Studio/MiniMax-H3-TrainingAdapter
5+
# --preset_lora_path "./models/DiffSynth-Studio/MiniMax-H3-TrainingAdapter/model_for_comfy_dit.safetensors" \
6+
# --preset_lora_model "dit"
7+
38
# T2VA - stage 1: encode text/video/audio and cache pipeline inputs (text encoder loaded here only)
49
accelerate launch examples/minimax_h3/model_training/train.py \
510
--dataset_base_path data/diffsynth_example_dataset/minimax_h3/MiniMax-H3-Pruned-FL2VA \
@@ -35,10 +40,6 @@ accelerate launch examples/minimax_h3/model_training/train.py \
3540
--use_gradient_checkpointing \
3641
--find_unused_parameters \
3742
--task "sft:train"
38-
# Optional: fuse the DeCFG training adapter into the DiT while training, for a better optimization landscape on this CFG-distilled base. This DiT uses the ComfyUI qkv layout, so take the model_for_comfy_dit variant. Training only -- do not load it at inference.
39-
# modelscope download --model DiffSynth-Studio/MiniMax-H3-TrainingAdapter --local_dir ./models/DiffSynth-Studio/MiniMax-H3-TrainingAdapter
40-
# --preset_lora_path "./models/DiffSynth-Studio/MiniMax-H3-TrainingAdapter/model_for_comfy_dit.safetensors" \
41-
# --preset_lora_model "dit"
4243

4344
# input_image / end_image take the first and last frame of the training video
4445
# FL2VA - stage 1

‎examples/minimax_h3/model_training/lora/MiniMax-H3-Int8-ConvRot-Pruned-Ref2VA.sh‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,10 @@
11
modelscope download --dataset DiffSynth-Studio/diffsynth_example_dataset --include "minimax_h3/MiniMax-H3-Pruned-Ref2VA/*" --local_dir ./data/diffsynth_example_dataset
22

3+
# Optional: fuse the DeCFG training adapter into the DiT while training, for a better optimization landscape on this CFG-distilled base. This DiT uses the ComfyUI qkv layout, so take the model_for_comfy_dit variant. Training only -- do not load it at inference.
4+
# modelscope download --model DiffSynth-Studio/MiniMax-H3-TrainingAdapter --include model_ref2va_for_comfy_dit.safetensors --local_dir ./models/DiffSynth-Studio/MiniMax-H3-TrainingAdapter
5+
# --preset_lora_path "./models/DiffSynth-Studio/MiniMax-H3-TrainingAdapter/model_ref2va_for_comfy_dit.safetensors" \
6+
# --preset_lora_model "dit"
7+
38
# Ref2VA - stage 1: encode reference/text/video/audio and cache (text encoder + processor loaded here only)
49
accelerate launch examples/minimax_h3/model_training/train.py \
510
--dataset_base_path data/diffsynth_example_dataset/minimax_h3/MiniMax-H3-Pruned-Ref2VA \

‎examples/minimax_h3/model_training/lora/MiniMax-H3-Int8-ConvRot-Ref2VA.sh‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,10 @@
11
modelscope download --dataset DiffSynth-Studio/diffsynth_example_dataset --include "minimax_h3/MiniMax-H3-Ref2VA/*" --local_dir ./data/diffsynth_example_dataset
22

3+
# Optional: fuse the DeCFG training adapter into the DiT while training, for a better optimization landscape on this CFG-distilled base. This DiT uses the ComfyUI qkv layout, so take the model_for_comfy_dit variant. Training only -- do not load it at inference.
4+
# modelscope download --model DiffSynth-Studio/MiniMax-H3-TrainingAdapter --include model_ref2va_for_comfy_dit.safetensors --local_dir ./models/DiffSynth-Studio/MiniMax-H3-TrainingAdapter
5+
# --preset_lora_path "./models/DiffSynth-Studio/MiniMax-H3-TrainingAdapter/model_ref2va_for_comfy_dit.safetensors" \
6+
# --preset_lora_model "dit"
7+
38
# Ref2VA - stage 1: encode reference/text/video/audio and cache (text encoder + processor loaded here only)
49
accelerate launch examples/minimax_h3/model_training/train.py \
510
--dataset_base_path data/diffsynth_example_dataset/minimax_h3/MiniMax-H3-Ref2VA \

0 commit comments

Comments
 (0)