Fix removing "cpu" from frozenset in bitsandbytes.py to allow better ROCm support. (huggingface#36975)

anadon · MekkCyber · zucchini-nlp · commit b52cacf3417b · 2025-05-14T18:43:06.000+02:00
* Fix removing "cpu" from frozenset in bitsandbytes.py to allow better ROCm support. Related to bitsandbytes-foundation/bitsandbytes#1573 and huggingface#36949 , this resolves a bug in allowing ROCm/HIP support in bitsandbytes. * Related to bitsandbytes-foundation/bitsandbytes#1573 and huggingface#36949 , this resolves a bug in the biteandbytes integration, allowing ROCm/HIP support in bitsandbytes. --------- Co-authored-by: Mohamed Mekkouri <93391238+MekkCyber@users.noreply.github.com>
diff --git a/src/transformers/integrations/bitsandbytes.py b/src/transformers/integrations/bitsandbytes.py
@@ -496,7 +496,9 @@ def _validate_bnb_multi_backend_availability(raise_exception):
                 "You have Intel IPEX installed but if you're intending to use it for CPU, it might not have the right version. Be sure to double check that your PyTorch and IPEX installs are compatible."
             )
 
-        available_devices.discard("cpu")  # Only Intel CPU is supported by BNB at the moment
+        available_devices = frozenset(
+            [device for device in available_devices if device != "cpu"]
+        )  # Only Intel CPU is supported by BNB at the moment
 
     if not available_devices.intersection(bnb_supported_devices):
         if raise_exception:

Original file line number	Diff line number	Diff line change
`@@ -496,7 +496,9 @@ def _validate_bnb_multi_backend_availability(raise_exception):`
`496`	`496`	`"You have Intel IPEX installed but if you're intending to use it for CPU, it might not have the right version. Be sure to double check that your PyTorch and IPEX installs are compatible."`
`497`	`497`	`)`
`498`	`498`
`499`		`- available_devices.discard("cpu") # Only Intel CPU is supported by BNB at the moment`
	`499`	`+ available_devices = frozenset(`
	`500`	`+ [device for device in available_devices if device != "cpu"]`
	`501`	`+ ) # Only Intel CPU is supported by BNB at the moment`
`500`	`502`
`501`	`503`	`if not available_devices.intersection(bnb_supported_devices):`
`502`	`504`	`if raise_exception:`