Use CMake EXPORT #8954

swolchok · 2025-03-05T01:40:22Z

The CMake EXPORT feature makes target_compile_options/target_link_libraries etc. work nicely for both in-project builds and also projects that consume your project by installing it and finding it with find_package. We are currently working around our lack of EXPORT by (incompletely and therefore incorrectly!) repeating our configuration in tools/cmake/executorch-config.cmake.

I was previously stalled here by XNNPACK's lack of use of EXPORT, but I found that this can be worked around by building those targets with ExternalProject!

[ghstack-poisoned]

swolchok · 2025-03-05T01:40:23Z

Stack from ghstack (oldest at bottom):

pytorch-bot · 2025-03-05T01:40:26Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/8954

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 38 New Failures, 1 Cancelled Job, 3 Unrelated Failures

As of commit 200b7dd with merge base 986be2a ():

NEW FAILURES - The following jobs have failed:

Apple / build-benchmark-app / macos-job (gh)
RuntimeError: Command bash /Users/runner/work/_temp/exec_script failed with exit code 1
Apple / build-frameworks-ios / macos-job (gh)
RuntimeError: Command bash /Users/runner/work/_temp/exec_script failed with exit code 1
pull / android / build-llm-demo / linux-job (gh)
RuntimeError: Command docker exec -t aa0fe4d4a419aca8e86cf90fc97c20d0ebab8d62727e94af8d19f90cbc3cb39d /exec failed with exit code 1
pull / test-binary-size-linux / linux-job (gh)
RuntimeError: Command docker exec -t 632e9337ce7f5fe40daf9b8199bc40059645829f33484f4ace6fd1967df97bfe /exec failed with exit code 1
pull / test-binary-size-linux-gcc / linux-job (gh)
RuntimeError: Command docker exec -t bb719e3ca5f9b0c6ad61f143afc63d98cb6056b4edafe54964e56a77108dd2ba /exec failed with exit code 1
pull / test-custom-ops-linux / linux-job (gh)
RuntimeError: Command docker exec -t e696b38bc286665ad4c122b91b1654155282f7254ac60b031ec23d152517c4b0 /exec failed with exit code 1
pull / test-eval_llama-mmlu-linux / linux-job (gh)
RuntimeError: Command docker exec -t 2a3494e20ab31b341db7633d1c4fbb2386259f97f433e749379a03edb5ed3658 /exec failed with exit code 1
pull / test-llama-runner-linux-android / linux-job (gh)
RuntimeError: Command docker exec -t 2a532914156e4d7f173686e7b84c85d9532f40260711bae3469eb3ca519d5713 /exec failed with exit code 1
pull / test-llama-runner-qnn-linux (fp32, qnn_16a16w, qnn) / linux-job (gh)
RuntimeError: Command docker exec -t c4b80a6d0aa0b1c883890a099676932cc5ad62b095a0a2a40b381e56c2681aba /exec failed with exit code 1
pull / test-llama-runner-qnn-linux (fp32, qnn_8a8w, qnn) / linux-job (gh)
RuntimeError: Command docker exec -t e98ef57074cddca4e4a63d678697d186bb0dbbe89c60278511f194d8578b2217 /exec failed with exit code 1
pull / test-llava-runner-linux / linux-job (gh)
RuntimeError: Command docker exec -t ec3f5287cc7851a32e338e626def930e81cc635f45bd090db0633f80b8cfb85b /exec failed with exit code 1
pull / test-phi-3-mini-runner-linux / linux-job (gh)
RuntimeError: Command docker exec -t 9de4e03c34f667551c965390e912c6e51af72bf88cd6f00f5e9749d9443b2fa9 /exec failed with exit code 1
pull / test-selective-build-linux / linux-job (gh)
RuntimeError: Command docker exec -t 68d9fd76714bf5e5e0a53562e7d2073e6ec1e8a1efb6f84e5f4eb562a758ce9d /exec failed with exit code 1
pull / test-static-llama-qnn-linux / linux-job (gh)
RuntimeError: Command docker exec -t 89778577725ba69bc847cba0dcfe2e2672d449d144a21136a716b5a20b2b4fcd /exec failed with exit code 1
trunk / test-arm-backend (test_models_ethos-u55) / linux-job (gh)
RuntimeError: Command docker exec -t 743c417c467efd27032b91549c3c52a3033525e9b0e6f4dc1175e34d15598f43 /exec failed with exit code 1
trunk / test-arm-backend (test_models_ethos-u85) / linux-job (gh)
RuntimeError: Command docker exec -t 7d9538526a6e85bdeef04e2ba9b8d2726abdf745ab376a3c8e65626967893840 /exec failed with exit code 1
trunk / test-arm-backend (test_models_tosa) / linux-job (gh)
RuntimeError: Command docker exec -t 218a1acd508baba7d963ae0702b48bd907f4072dd44c5473ffbba27f1fbe5eb9 /exec failed with exit code 1
trunk / test-arm-backend (test_pytest_models_ethosu_fvp) / linux-job (gh)
RuntimeError: Command docker exec -t 330e6addfd2fd6d7d13111340212bfe303c92a3e46c0b76f4d5c5d66d22ec7e4 /exec failed with exit code 1
trunk / test-arm-backend (test_pytest_ops_ethosu_fvp) / linux-job (gh)
RuntimeError: Command docker exec -t 5963c4ddcf3d6ba36f54a20e6b1c8337149900b29447259b3347ac29f89ecf5d /exec failed with exit code 1
trunk / test-arm-backend (test_run_ethosu_fvp) / linux-job (gh)
RuntimeError: Command docker exec -t 29390abf62b772b3cd390d0e42f2204b4b47d14a4cba291d0017119ce5ec8ff8 /exec failed with exit code 1
trunk / test-arm-cortex-m-size-test (bare_metal) / linux-job (gh)
RuntimeError: Command docker exec -t 5036bcabd3fb0fc551d18fe2f28e5beb5c1fc696f29c2a9c1b4e5e866e3ad7b7 /exec failed with exit code 1
trunk / test-arm-cortex-m-size-test (zephyr-preset) / linux-job (gh)
RuntimeError: Command docker exec -t 60c1f53ca5157d1f7eba37ab307bd4d882303775512938035efa5a306846cec9 /exec failed with exit code 1
trunk / test-custom-ops-macos (cmake) / macos-job (gh)
Process completed with exit code 1.
trunk / test-llama-runner-qnn-linux (fp32, qnn_16a16w, qnn) / linux-job (gh)
RuntimeError: Command docker exec -t 6cf039addfbff9400840e3a140373d0e8f78c845a1ec79d152a21d034e2e9929 /exec failed with exit code 1
trunk / test-llama-runner-qnn-linux (fp32, qnn_8a8w, qnn) / linux-job (gh)
RuntimeError: Command docker exec -t ae832f31389be2c547eb2b21e29abd4d41e23cc8cfdd05e0706e2d0a2e95c163 /exec failed with exit code 1
trunk / test-llama-torchao-lowbit / macos-job (gh)
RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 1
trunk / test-qnn-model (fp32, dl3) / linux-job (gh)
RuntimeError: Command docker exec -t 26c2a8f57329dfd745c5ce22ba14f9f884f63c16fc34ed14792bcd5e308a243e /exec failed with exit code 1
trunk / test-qnn-model (fp32, ic3) / linux-job (gh)
RuntimeError: Command docker exec -t 3c2930f3d763882dc3fc2f5d76a953da40a0e76802f541dd00338d0c6ca5fe17 /exec failed with exit code 1
trunk / test-qnn-model (fp32, ic4) / linux-job (gh)
RuntimeError: Command docker exec -t 1d8f54b8f752d78383a98afb10997ecc34119b0e24ebff0b96e984852c73a78b /exec failed with exit code 1
trunk / test-qnn-model (fp32, mb) / linux-job (gh)
RuntimeError: Command docker exec -t 1ed86a4676d46a137ef442e6c9abff09654864ea0f39bdab127c59d9287b0093 /exec failed with exit code 1
trunk / test-qnn-model (fp32, mv2) / linux-job (gh)
RuntimeError: Command docker exec -t a1257b0068e4b98daf14b4a4e27de1d4fa4651e4a0532f19c721a3beb0c33132 /exec failed with exit code 1
trunk / test-qnn-model (fp32, mv3) / linux-job (gh)
RuntimeError: Command docker exec -t 5c7edc9aeaa7434285b38d1beea77caad4d3fe71ed8ca27b0820ec738f8f626b /exec failed with exit code 1
trunk / test-qnn-model (fp32, vit) / linux-job (gh)
RuntimeError: Command docker exec -t 1cf527198a4a131b99627af92a7b847b5680ab8c163986e4d5edc2025cb0119d /exec failed with exit code 1
trunk / test-qnn-model (fp32, w2l) / linux-job (gh)
RuntimeError: Command docker exec -t 5413e01d8630e6149734ab1e898f60204b820dd66b392fc1ed85f52f91820c62 /exec failed with exit code 1
trunk / test-qnn-optimum-model (fp32, albert) / linux-job (gh)
RuntimeError: Command docker exec -t e465eaf5aa703c8dcf975109542532a6d6fa594ab7500988496713ffd9559656 /exec failed with exit code 1
trunk / test-qnn-optimum-model (fp32, bert) / linux-job (gh)
RuntimeError: Command docker exec -t 9690bdccbd52e3cae42bcbed7e5fc92259773b81794eccc6d0b953aa51f114d8 /exec failed with exit code 1
trunk / test-qnn-optimum-model (fp32, distilbert) / linux-job (gh)
RuntimeError: Command docker exec -t c20d3bfee15b84dc896596751ac0dd2eed47ae2b6611be5a8e8514e0b3fa78bf /exec failed with exit code 1
trunk / test-selective-build-macos (cmake) / macos-job (gh)
RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 1

CANCELLED JOB - The following job was cancelled. Please retry:

trunk / test-models-macos (llama, portable) / macos-job (gh)
##[error]The operation was canceled.

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

pull / unittest / linux / linux-job (gh) (trunk failure)
examples/models/llama/tests/test_ring_attention.py::TestRingAttention::test_sliding_window_attention
pull / unittest-editable / linux / linux-job (gh) (trunk failure)
examples/models/llama/tests/test_ring_attention.py::TestRingAttention::test_sliding_window_attention
trunk / unittest-release / linux / linux-job (gh) (trunk failure)
examples/models/llama/tests/test_ring_attention.py::TestRingAttention::test_sliding_window_attention

This comment was automatically generated by Dr. CI and updates every 15 minutes.

The CMake EXPORT feature seems like it's supposed to make target_compile_options etc. work nicely for both in-project builds and also projects that consume your project by installing it and finding it with find_package. Using EXPORT would make it easy to have similar behavior to Buck's exported_preprocessor_flags, for example. Unfortunately, I am currently stalled here by XNNPACK's lack of use of EXPORT. I tried working around dependencies that don't use EXPORT by re-installing them with EXPORT ourselves, but XNNPACK has rather a lot so I'm sending this out to highlight the problem and ask if there's another way. I found some similar issues in TFLite: tensorflow/tensorflow#57658 tensorflow/tensorflow#87172 ghstack-source-id: e3e5b36 ghstack-comment-id: 2699501745 Pull Request resolved: #8954

github-actions · 2025-03-05T01:41:02Z

This PR needs a `release notes:` label

If your changes are user facing and intended to be a part of release notes, please use a label starting with release notes:.

If not, please add the topic: not user facing label.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "topic: not user facing"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

swolchok · 2025-07-14T21:20:37Z

have an update cooking, need to make sure check whether warnings I'm seeing about macOS version mismatch during install_executorch.sh are new and if so problematic

[ghstack-poisoned]

The CMake EXPORT feature seems like it's supposed to make target_compile_options etc. work nicely for both in-project builds and also projects that consume your project by installing it and finding it with find_package. Using EXPORT would make it easy to have similar behavior to Buck's exported_preprocessor_flags, for example. Unfortunately, I am currently stalled here by XNNPACK's lack of use of EXPORT. I tried working around dependencies that don't use EXPORT by re-installing them with EXPORT ourselves, but XNNPACK has rather a lot so I'm sending this out to highlight the problem and ask if there's another way. I found some similar issues in TFLite: tensorflow/tensorflow#57658 tensorflow/tensorflow#87172 ghstack-source-id: af3cc8d ghstack-comment-id: 2699501745 Pull Request resolved: #8954

[ghstack-poisoned]

The CMake EXPORT feature seems like it's supposed to make target_compile_options etc. work nicely for both in-project builds and also projects that consume your project by installing it and finding it with find_package. Using EXPORT would make it easy to have similar behavior to Buck's exported_preprocessor_flags, for example. Unfortunately, I am currently stalled here by XNNPACK's lack of use of EXPORT. I tried working around dependencies that don't use EXPORT by re-installing them with EXPORT ourselves, but XNNPACK has rather a lot so I'm sending this out to highlight the problem and ask if there's another way. I found some similar issues in TFLite: tensorflow/tensorflow#57658 tensorflow/tensorflow#87172 ghstack-source-id: f646a79 ghstack-comment-id: 2699501745 Pull Request resolved: #8954

[ghstack-poisoned]

The CMake EXPORT feature seems like it's supposed to make target_compile_options etc. work nicely for both in-project builds and also projects that consume your project by installing it and finding it with find_package. Using EXPORT would make it easy to have similar behavior to Buck's exported_preprocessor_flags, for example. Unfortunately, I am currently stalled here by XNNPACK's lack of use of EXPORT. I tried working around dependencies that don't use EXPORT by re-installing them with EXPORT ourselves, but XNNPACK has rather a lot so I'm sending this out to highlight the problem and ask if there's another way. I found some similar issues in TFLite: tensorflow/tensorflow#57658 tensorflow/tensorflow#87172 ghstack-source-id: c2f4cdd ghstack-comment-id: 2699501745 Pull Request resolved: #8954

swolchok · 2025-07-15T21:29:40Z

Clear TODO list from test failures:

Stack yet another PR to convert FXDiv to ExternalProject
mpsdelegate needs a path update (manually test build_apple_frameworks and the 4 apple presets)
extension_module needs a path update (manually test build_android_library)
pthreadpool, cpuinfo, XNNPACK, xnnpack-microkernels-prod need to be put back in executorch-config.cmake (manually test llama-runner)
The examples/models/llama and extension/llm/tokenizers builds need a bunch of fixing (again, manually test llama-runner)
neuron_backend has a bad path in INTERFACE_SOURCES (test by CI)

[ghstack-poisoned]

The CMake EXPORT feature seems like it's supposed to make target_compile_options etc. work nicely for both in-project builds and also projects that consume your project by installing it and finding it with find_package. Using EXPORT would make it easy to have similar behavior to Buck's exported_preprocessor_flags, for example. Unfortunately, I am currently stalled here by XNNPACK's lack of use of EXPORT. I tried working around dependencies that don't use EXPORT by re-installing them with EXPORT ourselves, but XNNPACK has rather a lot so I'm sending this out to highlight the problem and ask if there's another way. I found some similar issues in TFLite: tensorflow/tensorflow#57658 tensorflow/tensorflow#87172 ghstack-source-id: f60a1cd ghstack-comment-id: 2699501745 Pull Request resolved: #8954

swolchok · 2025-07-16T22:21:02Z

I've updated extension/llm/tokenizers to point at pytorch-labs/tokenizers#103 . We'll presumably need to fix the submodule reference after that gets merged.

[ghstack-poisoned]

The CMake EXPORT feature seems like it's supposed to make target_compile_options etc. work nicely for both in-project builds and also projects that consume your project by installing it and finding it with find_package. Using EXPORT would make it easy to have similar behavior to Buck's exported_preprocessor_flags, for example. Unfortunately, I am currently stalled here by XNNPACK's lack of use of EXPORT. I tried working around dependencies that don't use EXPORT by re-installing them with EXPORT ourselves, but XNNPACK has rather a lot so I'm sending this out to highlight the problem and ask if there's another way. I found some similar issues in TFLite: tensorflow/tensorflow#57658 tensorflow/tensorflow#87172 ghstack-source-id: a5b67f8 ghstack-comment-id: 2699501745 Pull Request resolved: #8954

[ghstack-poisoned]

The CMake EXPORT feature seems like it's supposed to make target_compile_options etc. work nicely for both in-project builds and also projects that consume your project by installing it and finding it with find_package. Using EXPORT would make it easy to have similar behavior to Buck's exported_preprocessor_flags, for example. Unfortunately, I am currently stalled here by XNNPACK's lack of use of EXPORT. I tried working around dependencies that don't use EXPORT by re-installing them with EXPORT ourselves, but XNNPACK has rather a lot so I'm sending this out to highlight the problem and ask if there's another way. I found some similar issues in TFLite: tensorflow/tensorflow#57658 tensorflow/tensorflow#87172 ghstack-source-id: 16c0343 ghstack-comment-id: 2699501745 Pull Request resolved: #8954

[ghstack-poisoned]

The CMake EXPORT feature seems like it's supposed to make target_compile_options etc. work nicely for both in-project builds and also projects that consume your project by installing it and finding it with find_package. Using EXPORT would make it easy to have similar behavior to Buck's exported_preprocessor_flags, for example. Unfortunately, I am currently stalled here by XNNPACK's lack of use of EXPORT. I tried working around dependencies that don't use EXPORT by re-installing them with EXPORT ourselves, but XNNPACK has rather a lot so I'm sending this out to highlight the problem and ask if there's another way. I found some similar issues in TFLite: tensorflow/tensorflow#57658 tensorflow/tensorflow#87172 ghstack-source-id: 2f0620d ghstack-comment-id: 2699501745 Pull Request resolved: #8954

Update

6d51bff

[ghstack-poisoned]

swolchok requested review from dbort, JacobSzwejbka, lucylq, manuelcandales, kimishpatel, shoumikhin, jackzhxng, iseeyuan, larryliu0820, digantdesai, mcr229, SS-JIA and cccclai as code owners March 5, 2025 01:40

facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Mar 5, 2025

swolchok marked this pull request as draft March 5, 2025 01:40

github-actions bot mentioned this pull request Mar 10, 2025

Weekly pr metrics report - 2025-03-01..2025-03-07 wdvr/pytorch#16

Open

This was referenced Mar 17, 2025

Weekly pr metrics report - 2025-03-01..2025-03-07 wdvr/pytorch#18

Open

Weekly pr metrics report - 2025-03-01..2025-03-07 wdvr/pytorch#20

Open

github-actions bot mentioned this pull request Mar 31, 2025

Weekly pr metrics report - 2025-03-01..2025-03-07 wdvr/pytorch#22

Open

github-actions bot mentioned this pull request Apr 7, 2025

Weekly pr metrics report - 2025-03-01..2025-03-07 wdvr/pytorch#26

Open

Update

b87d2fb

[ghstack-poisoned]

This was referenced Jul 14, 2025

[easy] cmake-format CMakeLists.txt #12450

Merged

[easy] cmake-format extension/training/CMakeLists.txt #12451

Merged

swolchok mentioned this pull request Jul 14, 2025

Convert pthreadpool to ExternalProject #12468

Open

swolchok changed the title ~~WIP: attempt to use CMake EXPORT~~ Use CMake EXPORT Jul 14, 2025

swolchok mentioned this pull request Jul 14, 2025

RFC: Create high level CMake targets #12138

Open

Update

497ea70

[ghstack-poisoned]

swolchok mentioned this pull request Jul 15, 2025

Allow ciflow/binaries label to trigger wheel builds on PRs #12505

Merged

swolchok changed the base branch from gh/swolchok/509/head to gh/swolchok/511/head July 15, 2025 20:43

swolchok mentioned this pull request Jul 15, 2025

Build cpuinfo as ExternalProject #12513

Open

Update

f8d2baf

[ghstack-poisoned]

swolchok added the release notes: build Changes related to build, including dependency upgrades, build flags, optimizations, etc. label Jul 15, 2025

swolchok mentioned this pull request Jul 16, 2025

Fixes in the CMake install setup for EXPORT pytorch-labs/tokenizers#103

Open

Update

62658a6

[ghstack-poisoned]

This was referenced Jul 16, 2025

format even more CMakeLists.txt files #12560

Open

Build FXDiv as ExternalProject #12561

Open

swolchok changed the base branch from gh/swolchok/511/head to gh/swolchok/514/head July 16, 2025 22:21

swolchok mentioned this pull request Jul 16, 2025

Remove bundled_program dep from mpsdelegate #12562

Open

stack on #12566

46e9698

[ghstack-poisoned]

swolchok changed the base branch from gh/swolchok/514/head to gh/swolchok/515/head July 16, 2025 23:50

swolchok mentioned this pull request Jul 16, 2025

Turn BUILD_TESTING off if EXECUTORCH_BUILD_TESTS is off #12566

Open

Update

0c7f574

[ghstack-poisoned]

swolchok changed the base branch from gh/swolchok/515/head to gh/swolchok/516/head July 17, 2025 00:52

swolchok mentioned this pull request Jul 17, 2025

Don't use iOS toolchain on macOS #12578

Open

Update

200b7dd

[ghstack-poisoned]

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Use CMake EXPORT #8954

Use CMake EXPORT #8954

Uh oh!

swolchok commented Mar 5, 2025 •

edited

Loading

Uh oh!

swolchok commented Mar 5, 2025 •

edited

Loading

Uh oh!

pytorch-bot bot commented Mar 5, 2025 •

edited

Loading

Uh oh!

github-actions bot commented Mar 5, 2025

Uh oh!

swolchok commented Jul 14, 2025

Uh oh!

swolchok commented Jul 15, 2025 •

edited

Loading

Uh oh!

swolchok commented Jul 16, 2025

Uh oh!

Uh oh!

Use CMake EXPORT #8954

Are you sure you want to change the base?

Use CMake EXPORT #8954

Uh oh!

Conversation

swolchok commented Mar 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

swolchok commented Mar 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pytorch-bot bot commented Mar 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/8954

❌ 38 New Failures, 1 Cancelled Job, 3 Unrelated Failures

Uh oh!

github-actions bot commented Mar 5, 2025

This PR needs a release notes: label

Uh oh!

swolchok commented Jul 14, 2025

Uh oh!

swolchok commented Jul 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

swolchok commented Jul 16, 2025

Uh oh!

Uh oh!

swolchok commented Mar 5, 2025 •

edited

Loading

swolchok commented Mar 5, 2025 •

edited

Loading

pytorch-bot bot commented Mar 5, 2025 •

edited

Loading

This PR needs a `release notes:` label

swolchok commented Jul 15, 2025 •

edited

Loading