Coati SFT Training Process stack in "[extension] Compiling or loading the JIT-built cpu_adam kernel during runtime now"

[linmou]

When I run Chat/examples/train_sft.sh several times, the process seems stacked somewhere.
No error reports, it just stops after showing "[extension] Compiling or loading the JIT-built cpu_adam kernel during runtime now". Any idea about why and where it stacks?

[Haoran1234567]

My problem has been resolved. The solution is to change the ColossalAI installation method from the original 'CUDA_EXT=1 pip install colossalai 'Replace with Download From Source,' CUDA '_ EXT=1 pip install .’. I hope it can help everyone

[linmou]

Thanks for your help. But I tried 'CUDA_EXT=1 pip install git+https://github.com/hpcaitech/ColossalAI.git', and another error occurs 'cannot import name 'ZeroDDP' from 'colossalai.nn.parallel', which colossal version did you install ?