You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
pretrain llama2-7b can resume when using "zero2" plugin, but can not resume when using "gemini" plugin, when using "gemini" plugin, the resume process will stuck, the cuda memory do not change in "nvtop" monitor.
Environment
16 * 8 * H100
torch 2.0.0
The text was updated successfully, but these errors were encountered:
馃悰 Describe the bug
pretrain llama2-7b can resume when using "zero2" plugin, but can not resume when using "gemini" plugin, when using "gemini" plugin, the resume process will stuck, the cuda memory do not change in "nvtop" monitor.
Environment
16 * 8 * H100
torch 2.0.0
The text was updated successfully, but these errors were encountered: