About the scheduler used for finetuning.

[aHapBean]

I noticed in the inference script (cli_demo.py) the following comment:

    # 2. Set Scheduler.
    # Can be changed to `CogVideoXDPMScheduler` or `CogVideoXDDIMScheduler`.
    # We recommend using `CogVideoXDDIMScheduler` for CogVideoX-2B.
    # using `CogVideoXDPMScheduler` for CogVideoX-5B / CogVideoX-5B-I2V.

So, when fine-tuning CogVideoX-2B, should I set the scheduler to CogVideoXDPMScheduler or CogVideoXDDIMScheduler? I couldn't find any documentation on this. In the released training code, CogVideoXDPMScheduler is used by default for training.

If I use CogVideoXDPMScheduler for fine-tuning and switch to CogVideoXDDIMScheduler for inference, will the results be worse compared to using CogVideoXDDIMScheduler for both fine-tuning and inference?

[aHapBean]

In addition to this, I would like to inquire whether it is acceptable to fine-tune a 2B model using 'bf16' and conduct inference with 'bf16', even though the official recommendation is to use 'fp16'.

I have attempted to perform inference with 'bf16' on the released CogVideoX-2B model, and the results appear to be reasonable.

[liyz15]

They should be identical for training.

[zRzRzRzRzRzRzR]

Can use bf16 for 2B inference, but the version of our model weights released is FP16. During training, you should use CogVideoXDDIMScheduler for training, as we mainly work on 5B, and the 5B model uses DPM. But the 2B model should also work normally with DPM.

[aHapBean]

Thank you for your response. It's clear to me now and I will proceed as recommended.