蚂蚁 inclusionAI 开源 dFactory:面向 dLLM 的高效微调工具
蚂蚁 inclusionAI 在 GitHub 开源 dFactory,用于 dLLM 的高效微调,支持 LLaDA2.0-mini(16B)和 LLaDA2.0-flash(100B)两款模型,集成(连续)监督微调(block-diffusion、full attention)等方法。项目采用 Apache-2.0 许可,提供 MoE 专家权重合并与拆分脚本,训练需先将权重转为 merged-expert 格式。
performance. Before starting, you must download the standard weights and convert them. 1. Download the original model: We provide a helper script to download the weights from the Hugging Face Hub. 2. Convert to the merged format: Run the following script to create the merged checkpoint required for training. The directory /path/to/save/merged model is what you will use for the training script. For more details, see MoE Expert Merging and Splitting Utilities 2. Prepare Training Data Before train
匹配字段正文


