add rtol for certain tasks for phi-4 models to match measured behavior #26

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Open

derekk-nm wants to merge 2 commits into main from dmk/phi-4_rtol

Contributor

derekk-nm commented Apr 24, 2025

SUMMARY:
these rtol settings get the accuracy workflow passing for the phi-4 models. Research will need to review if these settings are acceptable.

TEST PLAN:

microsoft/phi-4: https://github.com/neuralmagic/nm-cicd/actions/runs/14627328323
RedHatAI/phi-4-FP8-dynamic: https://github.com/neuralmagic/nm-cicd/actions/runs/14630743574
RedHatAI/phi-4-quantized.w4a16: https://github.com/neuralmagic/nm-cicd/actions/runs/14627189243
RedHatAI/phi-4-quantized.w8a8: https://github.com/neuralmagic/nm-cicd/actions/runs/14627144471

derekk-nm added 2 commits

April 23, 2025 20:17


          add rtol for certain tasks to match measured behavior

43ced54


          tweak one rtol

4cf66f4

derekk-nm requested a review from dhuangnm

April 24, 2025 01:05

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet