910B芯片Swift多模態(tài)模型分布式訓(xùn)練實踐
一、環(huán)境準(zhǔn)備1.前置條件首先準(zhǔn)備好訓(xùn)練機(jī)器和數(shù)據(jù),筆者采用了32節(jié)點910BNPU,300萬網(wǎng)頁訓(xùn)練數(shù)據(jù)。2.環(huán)境配置環(huán)境安裝,首先安裝多模態(tài)訓(xùn)練框架msswift,安裝torchnpu及deepspeed。安裝msswift(當(dāng)前推薦從源碼安裝,待發(fā)版后可直接pip安裝)gitclonehttps:github.commodelscopeswift.gitcdswiftpipinstalle'.[llm]'安裝torchnpupipinstalltorchnpudecorator安裝deepspeedpipinstalldeepspeed完整python依賴版本見文末附錄3.環(huán)境驗...