Skip to content

Support use safetensors with paddle.MmapStorage to load model files #2730

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: develop
Choose a base branch
from

Conversation

zeroRains
Copy link
Contributor

@zeroRains zeroRains commented Jul 7, 2025

pcard-71500

使用带有paddle.MmapStorage的safetensors,加速模型文件的加载。

单卡测试:ernie-4_5-21b-a3b-bf16-paddle

方法 文件加载时间 文件加载时间+模型参数导入模型时间 结论
baseline 28 44.64527976885438
MmapSotrage 4 22.69623290002346 加载时间缩短为原来的一半

8卡测试:ernie-4_5-300b-a47b-bf16-paddle

方法 文件加载时间 文件加载时间+模型参数导入模型时间 结论
baseline 175 222.6731412112713
MmapSotrage (get_slice) 168 226.66718888282776 基本不变
MmapSotrage (get_tensor) 86 131.25562098249793 加载时间缩短为原来的一半

前置PR:

提前使用方式:

  1. 编译最新develop的paddle
  2. 编译这个仓库git clone -b pp https://github.com/zeroRains/safetensors.git
  3. 拉取本pr的commit

Copy link

paddle-bot bot commented Jul 7, 2025

Thanks for your contribution!

@CLAassistant
Copy link

CLAassistant commented Jul 7, 2025

CLA assistant check
All committers have signed the CLA.

@zeroRains zeroRains force-pushed the loading branch 3 times, most recently from 371ac7b to 52b4ac6 Compare July 7, 2025 12:31
Change-Id: I8f6faff3d86b682ccdccc31b38eb1d6b1db5e8a1
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants