Skip to content

[NVFP4][WIP] Add NVFp4 Support #287

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Draft
wants to merge 16 commits into
base: main
Choose a base branch
from
Draft

[NVFP4][WIP] Add NVFp4 Support #287

wants to merge 16 commits into from

Conversation

dsikka
Copy link
Collaborator

@dsikka dsikka commented Mar 31, 2025 •
edited
Loading

Summary

  • Introduce FloatArg classes to manage float args data (min/min/rounding table)
  • Introduce new global_scale parameter
  • Update calculate_qparams, quant, dequant to handle global scale operations
  • Add NVFP4 QuantScheme to match the expected group_size expected

brian-dellabetta
brian-dellabetta reviewed Apr 1, 2025
View reviewed changes
src/compressed_tensors/quantization/quant_args.py
FP8_E4M3_DATA = FloatArgs(
exponent=4,
mantissa=3,
bits=8,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think we usually use num_bits right? I changed AWQ Modifier to it based on what i saw elsewhere in llm comressor

Suggested change
bits=8,
num_bits=8,

Copy link
Collaborator Author

@dsikka dsikka Apr 1, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR isn’t ready for review lol. It’s set to Draft

@dsikka dsikka requested a review from anmarques April 2, 2025 20:05
@mgoin mgoin self-requested a review April 24, 2025 19:18
@dsikka dsikka mentioned this pull request May 1, 2025
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@brian-dellabetta brian-dellabetta brian-dellabetta left review comments

@anmarques anmarques Awaiting requested review from anmarques

@mgoin mgoin Awaiting requested review from mgoin

At least 2 approving reviews are required to merge this pull request.

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@dsikka @brian-dellabetta