[Feat] Allow symmetric_no_clipping_error for KleidiAI kernels, update Readme and validate Kleidi INT4 quantization path #2570
Conversation
…ated quantizer API - Switched to quantize_() with Int8DynamicActivationIntxWeightConfig - Validated the move of packed_linear_int8_dynamic_activation_intx_weight_layout.py in torchao/dtypes/uintx - Fixed handling of SYMMETRIC_NO_CLIPPING_ERR mapping type - Validated INT4 path on a 2-layer nn.Sequential model with torch.int4 weights - Compared SYMMETRIC vs SYMMETRIC_NO_CLIPPING_ERR across PerAxis and PerGroup granularities
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/ao/2570
Note: Links to docs will display an error until the docs builds have been completed. This comment was automatically generated by Dr. CI and updates every 15 minutes. |
|
Hi @gausah-arm! Thank you for your pull request and welcome to our community. Action RequiredIn order to merge any pull request (code, docs, etc.), we require contributors to sign our Contributor License Agreement, and we don't seem to have one on file for you. ProcessIn order for us to review and merge your suggested changes, please sign at https://code.facebook.com/cla. If you are contributing on behalf of someone else (eg your employer), the individual CLA may not be sufficient and your employer may need to sign the corporate CLA. Once the CLA is signed, our tooling will perform checks and validations. Afterwards, the pull request will be tagged with If you have received this in error or have any questions, please contact us at cla@meta.com. Thanks! |
gausah-arm
changed the title
gausah-arm
changed the title
meta-cla
bot
added
the
CLA Signed
|
@digantdesai @jerryzh168 @metascroy can you please review this PR |
|
@metascroy - can you please take a look? |
gausah-arm
force-pushed
the
kai_quantization
branch
from
28c603c to
e9746df
Compare
|
@nikhil-arm can you fix the linter error (ruff) and then we can land. You could also add the new mapping type to the tests here to get CI coverage: https://github.com/pytorch/ao/blob/main/torchao/experimental/tests/test_int8_dynamic_activation_intx_weight.py#L56 |
Thankyou so much! |
Thankyou! I have fixed the lint error and done with the changes around test. Please review. |
This PR validates the support for the INT4 quantization path using KleidiAI by checking the integration to use the new
quantize_()API andInt8DynamicActivationIntxWeightConfig. The PR addresses breakages like Symmetric_no_clipping_err caused by API refactors and ensures compatibility with the current layout.