fix torchao memory check#12539

jiqing-feng wants to merge 4 commits intohuggingface:mainhuggingface/diffusers:mainfrom

jiqing-feng:memjiqing-feng/diffusers:memCopy head branch name to clipboard