Enable CPU/XPU native and ipex path #1628
Conversation
Signed-off-by: jiqing-feng <jiqing.feng@intel.com>
Signed-off-by: jiqing-feng <jiqing.feng@intel.com>
Signed-off-by: jiqing-feng <jiqing.feng@intel.com>
Signed-off-by: jiqing-feng <jiqing.feng@intel.com>
Signed-off-by: jiqing-feng <jiqing.feng@intel.com>
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
|
I am cleaning the CPU and XPU tests, process 50% |
Signed-off-by: jiqing-feng <jiqing.feng@intel.com>
Signed-off-by: jiqing-feng <jiqing.feng@intel.com>
Signed-off-by: jiqing-feng <jiqing.feng@intel.com>
| quant_state.blocksize, | ||
| quant_state.shape, | ||
| quant_state.dtype, | ||
| ) |
There was a problem hiding this comment.
Is there reason why this change can't be in bitsandbytes/backends/cpu/ops.py?
There was a problem hiding this comment.
Signed-off-by: jiqing-feng <jiqing.feng@intel.com>
Signed-off-by: jiqing-feng <jiqing.feng@intel.com>
Signed-off-by: jiqing-feng <jiqing.feng@intel.com>
Signed-off-by: jiqing-feng <jiqing.feng@intel.com>
Signed-off-by: jiqing-feng <jiqing.feng@intel.com>
Signed-off-by: jiqing-feng <jiqing.feng@intel.com>
Signed-off-by: jiqing-feng <jiqing.feng@intel.com>
Signed-off-by: jiqing-feng <jiqing.feng@intel.com>
Signed-off-by: jiqing-feng <jiqing.feng@intel.com>
Signed-off-by: jiqing-feng <jiqing.feng@intel.com>
docs/source/installation.mdx
Outdated
| @@ -316,15 +316,29 @@ pip install -e . # `-e` for "editable" install, when developing BNB (otherwise | |||
| > [!TIP] | |||
| > Intel CPU/XPU backend only supports building from source; for now, please follow the instructions below. | |||
|
|
|||
| It does not need compile CPP codes, all required ops are in [intel_extension_for_pytorch](https://pytorch-extension.intel.com/), please follow the instruction to install ipex. | |||
| It requires [intel_extension_for_pytorch](https://pytorch-extension.intel.com/), please follow the instruction to install ipex. | |||
There was a problem hiding this comment.
I would expect IPEX to be optional. Especially so for CPU on Windows or for Linux/macOS on aarch64.
There was a problem hiding this comment.
Yes, I have updated the limitation.
Signed-off-by: jiqing-feng <jiqing.feng@intel.com>
|
Hi @matthewdouglas . Please check if there are anything missed before merging. We can discuss the tests in this issue #1637 |
Signed-off-by: jiqing-feng <jiqing.feng@intel.com>
bitsandbytes/cextension.py
Outdated
| if not ipex_cpu: | ||
| logger.warning( | ||
| "The installed version of bitsandbytes was compiled without IPEX support. " | ||
| "You can install ipex by running `pip install intel_extension_for_pytorch`to get better performance if you use the Intel CPU.", | ||
| ) |
There was a problem hiding this comment.
This seems like extra noise that we'd want to avoid.
Something to point out is that we still plan to ship libbitsandbytes_cpu in our wheels, so for most users, it's going to load a CPU, CUDA, or eventually ROCm or Metal library and we'll hit this logging line. At most we should really only raise this warning when:
- We're on a platform with IPEX CPU support. My understanding is this is limited to Linux x86-64.
- We expect the user to be using CPU, i.e. no CUDA, XPU, or MPS accelerators available.
On torch >= 2.6 we could just usetorch.accelerator.is_available()and on older versions I think we can overlook privateuse1 backends like HPU or Ascend NPU. - There's some expectation of IPEX being beneficial. We don't want to prompt users to install it if e.g. it needs AVX512 or AMX support to be effective. This is something I can't speak to directly but defer to Intel folks to determine.
Any other thoughts @Titus-von-Koeller ?
There was a problem hiding this comment.
You were right. I also agree that the log should only exist if no devices like cuda/xpu are available and the CPU is an Intel product.
There was a problem hiding this comment.
I have changed it, please review again. Thanks!
Signed-off-by: jiqing-feng <jiqing.feng@intel.com>
Signed-off-by: jiqing-feng <jiqing.feng@intel.com>
|
Hi @matthewdouglas . You can see some CPU ops are implemented by pure python, which works for all devices. Could we move these pure python implementations to the default folder? As some of these ops could be reused by XPU. |
Signed-off-by: jiqing-feng <jiqing.feng@intel.com>
jiqing-feng
changed the title
|
Hi @matthewdouglas . I removed pure pytorch ops to the default folder and it works well. The CIs are all passed. Please continue to review this PR. |
Signed-off-by: jiqing-feng <jiqing.feng@intel.com>
Signed-off-by: jiqing-feng <jiqing.feng@intel.com>
Signed-off-by: jiqing-feng <jiqing.feng@intel.com>
Signed-off-by: jiqing-feng <jiqing.feng@intel.com>
Signed-off-by: jiqing-feng <jiqing.feng@intel.com>
This PR enables ipex and other optimizations including:
Also, it fixed the parameter patch for cpu.
It could pass all transformers tests
After this PR merged, I will update the installation guide.
@matthewdouglas @Titus-von-Koeller