NXP backend: Resolve limitations of uncertain tensor formats. #14576

MartinPavella · 2025-09-25T08:01:36Z

Summary

This PR resolves format related issues by inferring the format (NCHW/NHWC) for all nodes before partitioning. These formats are then used by the NeutronPartitioner to accurately determine which nodes are supported on Neutron.

Test plan

Unit tests provided, and correct function is tested by nearly every test in the nxp backend.

cc @kimishpatel

pytorch-bot · 2025-09-25T08:01:40Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/14576

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 2 New Failures, 1 Unrelated Failure

As of commit 974c05d with merge base f0e8ea8 ():

NEW FAILURES - The following jobs have failed:

pull / test-samsung-models-linux / linux-job (gh)
RuntimeError: Command docker exec -t ee1644598c7f12de22728044f546c5de7db36b25f5ea4784d9dbe1b85c80a229 /exec failed with exit code 1
Test CUDA Builds / export-voxtral-cuda-artifact / linux-job (gh)
RuntimeError: Command docker exec -t 523099fc706ef42c8ae3045617a4f5f0f5abd135b64349ec0a2d2b2406025b28 /exec failed with exit code 2

FLAKY - The following job failed but was likely due to flakiness present on trunk:

pull / unittest-arm-backend-with-no-fvp (test_pytest_models) / linux-job (gh) (detected as infra flaky with no log or failing log classifier)

This comment was automatically generated by Dr. CI and updates every 15 minutes.

MartinPavella · 2025-09-25T08:01:53Z

@pytorchbot label "module: nxp" "release notes: nxp"

kimishpatel · 2025-09-26T03:03:02Z

backends/nxp/TARGETS

    deps = [
        ":neutron_sdk",
        ":aten_passes",
+        ":_passes",


Can you add neutron_backend build to this https://github.com/pytorch/executorch/blob/main/.ci/scripts/unittest-buck2.sh. Thats the one that was failing last time. This way we get signal on the pr itself

facebook-github-bot · 2025-09-26T03:05:59Z

@kimishpatel has imported this pull request. If you are a Meta employee, you can view this in D83311730.

kimishpatel · 2025-09-26T03:06:36Z

i have imported it internally. There is an nxp test failing as well

digantdesai

Review automatically exported from Phabricator review in Meta.

MartinPavella · 2025-10-01T12:59:12Z

@kimishpatel I have fixed the failing nxp test, and added the nxp backend build to the unittest-buck.sh. The build is now failing with:

1: From load at backends/nxp/TARGETS:4
2: File not found: shim_et//arvr/tools/build_defs/oxx_python.bzl

These lines were in the file already before my changes. Do you know what is causing the issue/how to fix it?

kimishpatel · 2025-10-03T02:57:41Z

@kimishpatel I have fixed the failing nxp test, and added the nxp backend build to the unittest-buck.sh. The build is now failing with:

1: From load at backends/nxp/TARGETS:4 2: File not found: shim_et//arvr/tools/build_defs/oxx_python.bzl

These lines were in the file already before my changes. Do you know what is causing the issue/how to fix it?

I guess it is triggering your build in oss and we have never really had this test in oss so its trying to pull in internal files. I would say roll back that change for now just to be able to land.

MartinPavella · 2025-10-03T08:35:50Z

@kimishpatel I have fixed the failing nxp test, and added the nxp backend build to the unittest-buck.sh. The build is now failing with:
1: From load at backends/nxp/TARGETS:4 2: File not found: shim_et//arvr/tools/build_defs/oxx_python.bzl
These lines were in the file already before my changes. Do you know what is causing the issue/how to fix it?

I guess it is triggering your build in oss and we have never really had this test in oss so its trying to pull in internal files. I would say roll back that change for now just to be able to land.

@kimishpatel thanks for the response. I have removed the build as you said. Can you please now verify that your internal build passes?

kimishpatel · 2025-10-05T19:35:38Z

doing

MartinPavella · 2025-10-09T06:29:01Z

doing

@kimishpatel any updates? Is the build passing?

MartinPavella · 2025-10-13T05:22:18Z

doing

@kimishpatel any updates? Is the build passing?

@kimishpatel pinging in case you missed this.

StrycekSimon

Please update the numMacs related comment. Did the remove_geititem_pass.py just pass the linter with such long lines?

backends/nxp/_passes/remove_getitem_pass.py

backends/nxp/backend/ir/converter/node_converters/ops_converters/cat_converter.py

backends/nxp/tests/test_node_format_inference.py

backends/nxp/backend/node_format_inference.py

backends/nxp/tests/ir/converter/node_converter/test_softmax_converter.py

backends/nxp/backend/node_format.py

MartinPavella · 2025-10-14T08:23:08Z

Please update the numMacs related comment. Did the remove_geititem_pass.py just pass the linter with such long lines?

Yes, the linter passed. I have shortened the lines now anyway.

… node format. The pass `RemoveGetItemPass` replaces a `max_pool2d_with_indices` node with a `max_pool2d` node, that doesn't require a GetItem afterward. The new operator must, however, preserve the original node format. Therefore, a copy of the pass was created in `backends/nxp/_passes`, where it was modified. The new directory was created, because the pass doesn't follow the `NeutronEdgePass` interface.

Before, the format inference was done during conversion to NeutronIR (after partitioning), so the partitioner didn't yet know the formats. Now, the partitioner has the format data, which can be used to accurately select nodes for delegation.

…ode formats.

MartinPavella requested a review from kimishpatel September 25, 2025 08:01

meta-cla bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Sep 25, 2025

pytorch-bot bot added module: nxp Issues related to NXP Neutron NPU delegation and code under backends/nxp/ release notes: nxp Changes to the NXP Neutron backend delegate labels Sep 25, 2025

kimishpatel reviewed Sep 26, 2025

View reviewed changes

digantdesai approved these changes Sep 26, 2025

View reviewed changes

MartinPavella force-pushed the upstream/main-nxp/EIEX-392-resolve-limitations-of-uncertain-tensor-formats branch 2 times, most recently from bb48aad to e1893f5 Compare October 1, 2025 07:41

MartinPavella marked this pull request as draft October 1, 2025 12:22

MartinPavella force-pushed the upstream/main-nxp/EIEX-392-resolve-limitations-of-uncertain-tensor-formats branch 2 times, most recently from 2672277 to 1ecfe36 Compare October 3, 2025 07:15

MartinPavella marked this pull request as ready for review October 3, 2025 08:34

MartinPavella requested a review from robert-kalmar as a code owner October 3, 2025 08:34

StrycekSimon suggested changes Oct 14, 2025

View reviewed changes

MartinPavella force-pushed the upstream/main-nxp/EIEX-392-resolve-limitations-of-uncertain-tensor-formats branch from 1ecfe36 to 27444cf Compare October 14, 2025 08:26

MartinPavella force-pushed the upstream/main-nxp/EIEX-392-resolve-limitations-of-uncertain-tensor-formats branch from 27444cf to fa43ac2 Compare October 14, 2025 08:27

MartinPavella requested a review from StrycekSimon October 14, 2025 09:12

robert-kalmar approved these changes Oct 14, 2025

View reviewed changes

StrycekSimon approved these changes Oct 14, 2025

View reviewed changes

MartinPavella mentioned this pull request Oct 14, 2025

NXP backend: Add support for aten.permute_copy.default. #15099

Draft

MartinPavella added 7 commits October 15, 2025 14:55

NXP backend: Store inferred node format in the node.meta.

b137c60

NXP backend: Improve cat delegation by using inferred node formats.

f93aa76

NXP backend: Improve constant_pad_nd delegation by using inferred n…

449aef2

…ode formats.

NXP backend: Do not infer format for unknown nodes.

f57a930

Fix rebase.

974c05d

MartinPavella force-pushed the upstream/main-nxp/EIEX-392-resolve-limitations-of-uncertain-tensor-formats branch from fa43ac2 to 974c05d Compare October 15, 2025 12:55

NXP backend: Resolve limitations of uncertain tensor formats. #14576

Are you sure you want to change the base?

NXP backend: Resolve limitations of uncertain tensor formats. #14576

Conversation

MartinPavella commented Sep 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Test plan

Uh oh!

pytorch-bot bot commented Sep 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/14576

❌ 2 New Failures, 1 Unrelated Failure

Uh oh!

MartinPavella commented Sep 25, 2025

Uh oh!

kimishpatel Sep 26, 2025

Choose a reason for hiding this comment

Uh oh!

facebook-github-bot commented Sep 26, 2025

Uh oh!

kimishpatel commented Sep 26, 2025

Uh oh!

digantdesai left a comment

Choose a reason for hiding this comment

Uh oh!

MartinPavella commented Oct 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kimishpatel commented Oct 3, 2025

Uh oh!

MartinPavella commented Oct 3, 2025

Uh oh!

kimishpatel commented Oct 5, 2025

Uh oh!

MartinPavella commented Oct 9, 2025

Uh oh!

MartinPavella commented Oct 13, 2025

Uh oh!

StrycekSimon left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

MartinPavella commented Oct 14, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

MartinPavella commented Sep 25, 2025 •

edited

Loading

pytorch-bot bot commented Sep 25, 2025 •

edited

Loading

MartinPavella commented Oct 1, 2025 •

edited

Loading