-
Notifications
You must be signed in to change notification settings - Fork 605
Backend data separation test #10531
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Backend data separation test #10531
Conversation
Add backend data separation test with demo ExecutorBackend. Note: ExecutorBackend is a wrapper around a portable PTE file. It doesn't support delegated program-data separation in the sense of tagging tensors as external and placing them into the named data store as such. Rather, this test: 1. Creates a linear PTE file with data removed, using the portable flow. 2. Packages (1) into preprocessed blob as a delegate, using ExecutorBackend. Note: this discards the PTD portion. 3. Re-create the PTD portion via export_program (portable flow again). 4. Runs the delegated ExecutorBackend linear with portable linear.ptd file. Caveat: this means that LinearModule definition in export_program and export_delegated_program must stay in sync, as we get the PTE artifact from export_delegated_program (wrapped in ExecutorBackend), and the PTD artifact from export_program. Differential Revision: [D73679733](https://our.internmc.facebook.com/intern/diff/D73679733/) [ghstack-poisoned]
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/10531
Note: Links to docs will display an error until the docs builds have been completed. ❗ 1 Active SEVsThere are 1 currently active SEVs. If your PR is affected, please view them below: ✅ No FailuresAs of commit 61a7b79 with merge base de98d25 ( This comment was automatically generated by Dr. CI and updates every 15 minutes. |
This pull request was exported from Phabricator. Differential Revision: D73679733 |
Add backend data separation test with demo ExecutorBackend. Note: ExecutorBackend is a wrapper around a portable PTE file. It doesn't support delegated program-data separation in the sense of tagging tensors as external and placing them into the named data store as such. Rather, this test: 1. Creates a linear PTE file with data removed, using the portable flow. 2. Packages (1) into preprocessed blob as a delegate, using ExecutorBackend. Note: this discards the PTD portion. 3. Re-create the PTD portion via export_program (portable flow again). 4. Runs the delegated ExecutorBackend linear with portable linear.ptd file. Caveat: this means that LinearModule definition in export_program and export_delegated_program must stay in sync, as we get the PTE artifact from export_delegated_program (wrapped in ExecutorBackend), and the PTD artifact from export_program. Differential Revision: [D73679733](https://our.internmc.facebook.com/intern/diff/D73679733/) [ghstack-poisoned]
This pull request was exported from Phabricator. Differential Revision: D73679733 |
Add backend data separation test with demo ExecutorBackend. Note: ExecutorBackend is a wrapper around a portable PTE file. It doesn't support delegated program-data separation in the sense of tagging tensors as external and placing them into the named data store as such. Rather, this test: 1. Creates a linear PTE file with data removed, using the portable flow. 2. Packages (1) into preprocessed blob as a delegate, using ExecutorBackend. Note: this discards the PTD portion. 3. Re-create the PTD portion via export_program (portable flow again). 4. Runs the delegated ExecutorBackend linear with portable linear.ptd file. Caveat: this means that LinearModule definition in export_program and export_delegated_program must stay in sync, as we get the PTE artifact from export_delegated_program (wrapped in ExecutorBackend), and the PTD artifact from export_program. Differential Revision: [D73679733](https://our.internmc.facebook.com/intern/diff/D73679733/) [ghstack-poisoned]
Pull Request resolved: #10531 Add backend data separation test with demo ExecutorBackend. Note: ExecutorBackend is a wrapper around a portable PTE file. It doesn't support delegated program-data separation in the sense of tagging tensors as external and placing them into the named data store as such. Rather, this test: 1. Creates a linear PTE file with data removed, using the portable flow. 2. Packages (1) into preprocessed blob as a delegate, using ExecutorBackend. Note: this discards the PTD portion. 3. Re-create the PTD portion via export_program (portable flow again). 4. Runs the delegated ExecutorBackend linear with portable linear.ptd file. Caveat: this means that LinearModule definition in export_program and export_delegated_program must stay in sync, as we get the PTE artifact from export_delegated_program (wrapped in ExecutorBackend), and the PTD artifact from export_program. ghstack-source-id: 282158282 @exported-using-ghexport Differential Revision: [D73679733](https://our.internmc.facebook.com/intern/diff/D73679733/)
This pull request was exported from Phabricator. Differential Revision: D73679733 |
Add backend data separation test with demo ExecutorBackend. Note: ExecutorBackend is a wrapper around a portable PTE file. It doesn't support delegated program-data separation in the sense of tagging tensors as external and placing them into the named data store as such. Rather, this test: 1. Creates a linear PTE file with data removed, using the portable flow. 2. Packages (1) into preprocessed blob as a delegate, using ExecutorBackend. Note: this discards the PTD portion. 3. Re-create the PTD portion via export_program (portable flow again). 4. Runs the delegated ExecutorBackend linear with portable linear.ptd file. Caveat: this means that LinearModule definition in export_program and export_delegated_program must stay in sync, as we get the PTE artifact from export_delegated_program (wrapped in ExecutorBackend), and the PTD artifact from export_program. Differential Revision: [D73679733](https://our.internmc.facebook.com/intern/diff/D73679733/) [ghstack-poisoned]
Pull Request resolved: #10531 Add backend data separation test with demo ExecutorBackend. Note: ExecutorBackend is a wrapper around a portable PTE file. It doesn't support delegated program-data separation in the sense of tagging tensors as external and placing them into the named data store as such. Rather, this test: 1. Creates a linear PTE file with data removed, using the portable flow. 2. Packages (1) into preprocessed blob as a delegate, using ExecutorBackend. Note: this discards the PTD portion. 3. Re-create the PTD portion via export_program (portable flow again). 4. Runs the delegated ExecutorBackend linear with portable linear.ptd file. Caveat: this means that LinearModule definition in export_program and export_delegated_program must stay in sync, as we get the PTE artifact from export_delegated_program (wrapped in ExecutorBackend), and the PTD artifact from export_program. ghstack-source-id: 282178788 @exported-using-ghexport Differential Revision: [D73679733](https://our.internmc.facebook.com/intern/diff/D73679733/)
This pull request was exported from Phabricator. Differential Revision: D73679733 |
Add backend data separation test with demo ExecutorBackend. Note: ExecutorBackend is a wrapper around a portable PTE file. It doesn't support delegated program-data separation in the sense of tagging tensors as external and placing them into the named data store as such. Rather, this test: 1. Creates a linear PTE file with data removed, using the portable flow. 2. Packages (1) into preprocessed blob as a delegate, using ExecutorBackend. Note: this discards the PTD portion. 3. Re-create the PTD portion via export_program (portable flow again). 4. Runs the delegated ExecutorBackend linear with portable linear.ptd file. Caveat: this means that LinearModule definition in export_program and export_delegated_program must stay in sync, as we get the PTE artifact from export_delegated_program (wrapped in ExecutorBackend), and the PTD artifact from export_program. Differential Revision: [D73679733](https://our.internmc.facebook.com/intern/diff/D73679733/) [ghstack-poisoned]
Pull Request resolved: #10531 Add backend data separation test with demo ExecutorBackend. Note: ExecutorBackend is a wrapper around a portable PTE file. It doesn't support delegated program-data separation in the sense of tagging tensors as external and placing them into the named data store as such. Rather, this test: 1. Creates a linear PTE file with data removed, using the portable flow. 2. Packages (1) into preprocessed blob as a delegate, using ExecutorBackend. Note: this discards the PTD portion. 3. Re-create the PTD portion via export_program (portable flow again). 4. Runs the delegated ExecutorBackend linear with portable linear.ptd file. Caveat: this means that LinearModule definition in export_program and export_delegated_program must stay in sync, as we get the PTE artifact from export_delegated_program (wrapped in ExecutorBackend), and the PTD artifact from export_program. ghstack-source-id: 282187684 @exported-using-ghexport Differential Revision: [D73679733](https://our.internmc.facebook.com/intern/diff/D73679733/)
This pull request was exported from Phabricator. Differential Revision: D73679733 |
Pull Request resolved: #10531 Add backend data separation test with demo ExecutorBackend. Note: ExecutorBackend is a wrapper around a portable PTE file. It doesn't support delegated program-data separation in the sense of tagging tensors as external and placing them into the named data store as such. Rather, this test: 1. Creates a linear PTE file with data removed, using the portable flow. 2. Packages (1) into preprocessed blob as a delegate, using ExecutorBackend. Note: this discards the PTD portion. 3. Re-create the PTD portion via export_program (portable flow again). 4. Runs the delegated ExecutorBackend linear with portable linear.ptd file. Caveat: this means that LinearModule definition in export_program and export_delegated_program must stay in sync, as we get the PTE artifact from export_delegated_program (wrapped in ExecutorBackend), and the PTD artifact from export_program. ghstack-source-id: 282187684 @exported-using-ghexport Differential Revision: [D73679733](https://our.internmc.facebook.com/intern/diff/D73679733/)
Pull Request resolved: #10531 Add backend data separation test with demo ExecutorBackend. Note: ExecutorBackend is a wrapper around a portable PTE file. It doesn't support delegated program-data separation in the sense of tagging tensors as external and placing them into the named data store as such. Rather, this test: 1. Creates a linear PTE file with data removed, using the portable flow. 2. Packages (1) into preprocessed blob as a delegate, using ExecutorBackend. Note: this discards the PTD portion. 3. Re-create the PTD portion via export_program (portable flow again). 4. Runs the delegated ExecutorBackend linear with portable linear.ptd file. Caveat: this means that LinearModule definition in export_program and export_delegated_program must stay in sync, as we get the PTE artifact from export_delegated_program (wrapped in ExecutorBackend), and the PTD artifact from export_program. ghstack-source-id: 282187684 @exported-using-ghexport Differential Revision: [D73679733](https://our.internmc.facebook.com/intern/diff/D73679733/) Backend data separation test Add backend data separation test with demo ExecutorBackend. Note: ExecutorBackend is a wrapper around a portable PTE file. It doesn't support delegated program-data separation in the sense of tagging tensors as external and placing them into the named data store as such. Rather, this test: 1. Creates a linear PTE file with data removed, using the portable flow. 2. Packages (1) into preprocessed blob as a delegate, using ExecutorBackend. Note: this discards the PTD portion. 3. Re-create the PTD portion via export_program (portable flow again). 4. Runs the delegated ExecutorBackend linear with portable linear.ptd file. Caveat: this means that LinearModule definition in export_program and export_delegated_program must stay in sync, as we get the PTE artifact from export_delegated_program (wrapped in ExecutorBackend), and the PTD artifact from export_program. Differential Revision: [D73679733](https://our.internmc.facebook.com/intern/diff/D73679733/)
Pull Request resolved: #10531 Add backend data separation test with demo ExecutorBackend. Note: ExecutorBackend is a wrapper around a portable PTE file. It doesn't support delegated program-data separation in the sense of tagging tensors as external and placing them into the named data store as such. Rather, this test: 1. Creates a linear PTE file with data removed, using the portable flow. 2. Packages (1) into preprocessed blob as a delegate, using ExecutorBackend. Note: this discards the PTD portion. 3. Re-create the PTD portion via export_program (portable flow again). 4. Runs the delegated ExecutorBackend linear with portable linear.ptd file. Caveat: this means that LinearModule definition in export_program and export_delegated_program must stay in sync, as we get the PTE artifact from export_delegated_program (wrapped in ExecutorBackend), and the PTD artifact from export_program. ghstack-source-id: 282187684 @exported-using-ghexport Differential Revision: [D73679733](https://our.internmc.facebook.com/intern/diff/D73679733/) Backend data separation test Add backend data separation test with demo ExecutorBackend. Note: ExecutorBackend is a wrapper around a portable PTE file. It doesn't support delegated program-data separation in the sense of tagging tensors as external and placing them into the named data store as such. Rather, this test: 1. Creates a linear PTE file with data removed, using the portable flow. 2. Packages (1) into preprocessed blob as a delegate, using ExecutorBackend. Note: this discards the PTD portion. 3. Re-create the PTD portion via export_program (portable flow again). 4. Runs the delegated ExecutorBackend linear with portable linear.ptd file. Caveat: this means that LinearModule definition in export_program and export_delegated_program must stay in sync, as we get the PTE artifact from export_delegated_program (wrapped in ExecutorBackend), and the PTD artifact from export_program. Differential Revision: [D73679733](https://our.internmc.facebook.com/intern/diff/D73679733/)
Pull Request resolved: pytorch#10531 Add backend data separation test with demo ExecutorBackend. Note: ExecutorBackend is a wrapper around a portable PTE file. It doesn't support delegated program-data separation in the sense of tagging tensors as external and placing them into the named data store as such. Rather, this test: 1. Creates a linear PTE file with data removed, using the portable flow. 2. Packages (1) into preprocessed blob as a delegate, using ExecutorBackend. Note: this discards the PTD portion. 3. Re-create the PTD portion via export_program (portable flow again). 4. Runs the delegated ExecutorBackend linear with portable linear.ptd file. Caveat: this means that LinearModule definition in export_program and export_delegated_program must stay in sync, as we get the PTE artifact from export_delegated_program (wrapped in ExecutorBackend), and the PTD artifact from export_program. ghstack-source-id: 282187684 @exported-using-ghexport Differential Revision: [D73679733](https://our.internmc.facebook.com/intern/diff/D73679733/) Backend data separation test Add backend data separation test with demo ExecutorBackend. Note: ExecutorBackend is a wrapper around a portable PTE file. It doesn't support delegated program-data separation in the sense of tagging tensors as external and placing them into the named data store as such. Rather, this test: 1. Creates a linear PTE file with data removed, using the portable flow. 2. Packages (1) into preprocessed blob as a delegate, using ExecutorBackend. Note: this discards the PTD portion. 3. Re-create the PTD portion via export_program (portable flow again). 4. Runs the delegated ExecutorBackend linear with portable linear.ptd file. Caveat: this means that LinearModule definition in export_program and export_delegated_program must stay in sync, as we get the PTE artifact from export_delegated_program (wrapped in ExecutorBackend), and the PTD artifact from export_program. Differential Revision: [D73679733](https://our.internmc.facebook.com/intern/diff/D73679733/)
Stack from ghstack (oldest at bottom):
Add backend data separation test with demo ExecutorBackend.
Note: ExecutorBackend is a wrapper around a portable PTE file. It doesn't support delegated program-data separation in the sense of tagging tensors as external and placing them into the named data store as such. Rather, this test:
Caveat: this means that LinearModule definition in export_program and export_delegated_program must stay in sync, as we get the PTE artifact from export_delegated_program (wrapped in ExecutorBackend), and the PTD artifact from export_program.
Differential Revision: D73679733