Add Metal backend core ETMetal runtime. #15020

manuelcandales · 2025-10-10T21:01:37Z

This commit introduces the foundational Metal backend runtime.

Key features:

ETMetalStream for managing Metal devices, command queues, buffers, and synchronization.
ETMetalShaderLibrary for compiling Metal shader source and caching pipeline states.
ETMetalKernelFunction for kernel argument binding, dispatching, and synchronization with stream-managed encoders.
Added global buffer management and pointer tracking between host and Metal buffers.
Added global stream management utilities and synchronization helpers

This provides the necessary runtime primitives for executing compute shaders and MPSGraph workloads.

[ghstack-poisoned]

manuelcandales · 2025-10-10T21:01:39Z

Stack from ghstack (oldest at bottom):

pytorch-bot · 2025-10-10T21:01:41Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/15020

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure, 1 Unrelated Failure

As of commit b782bb5 with merge base 6e0c9f6 ():

NEW FAILURE - The following job has failed:

pull / test-multimodal-linux (gemma3-4b) / linux-job (gh)
RuntimeError: Command docker exec -t 0ed55fde44046bbf4aa7ab5b7717b19f069e8eb67772711e4f5ed23476834503 /exec failed with exit code 139

FLAKY - The following job failed but was likely due to flakiness present on trunk:

pull / unittest-arm-backend-with-no-fvp (test_pytest_models) / linux-job (gh) (detected as infra flaky with no log or failing log classifier)

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[ghstack-poisoned]

mergennachin

See inline

mergennachin · 2025-10-12T16:13:24Z

backends/apple/metal/runtime/shims/et_metal.mm

+    // Commit buffer and allow immediate reuse for better performance
+    [commandBuffer_ commit];
+    ET_LOG(Debug, "ETMetalStream::commitAndContinue: Committed buffer %p with continue", commandBuffer_);
+


Don't you need to release the buffer after commit?

[commandBuffer_ release]; commandBuffer_ = nil;

mergennachin · 2025-10-12T16:17:29Z

backends/apple/metal/runtime/shims/et_metal.mm

+    if (cps_) [cps_ retain];
+    if (func_) [func_ retain];


Why do you need these retain lines? Aren't these already owned by the class?

mergennachin · 2025-10-12T16:18:13Z

backends/apple/metal/runtime/shims/et_metal.mm

+        // Don't release encoder_ here - the stream owns it
+        // Only clean up our own references
+        if (cps_) {
+            [cps_ release];
+            cps_ = nil;
+        }
+        if (func_) {
+            [func_ release];
+            func_ = nil;
+        }
+
+        encoder_ = nil; // Clear reference without releasing


Just this?

cps_ = nil; func_ = nil; encoder_ =nil;

mergennachin · 2025-10-12T16:19:02Z

backends/apple/metal/runtime/shims/et_metal.mm

+                          resultsDictionary:results
+                        executionDescriptor:nil];
+
+            //synchronize(syncType);


Why commented out?

mergennachin · 2025-10-12T16:27:22Z

backends/apple/metal/runtime/shims/et_metal.h

+};
+
+// =======================
+// ETMetalShaderLibrary - ExecuTorch Metal shader library management


Add a bit more comments. ETMetalShaderLibrary

Compiles Metal Shading Language (MSL) source to MTLLibrary

Caches compiled pipeline states for reuse

Creates ETMetalKernelFunction lazily

Lifetime: Persistent (application lifetime)

mergennachin · 2025-10-12T16:33:18Z

backends/apple/metal/runtime/shims/et_metal.h

+};
+
+// =======================
+// ETMetalKernelFunction - ExecuTorch Metal kernel function execution


- Encodes kernel dispatches into command buffers - Lifetime: Per-kernel-invocation

mergennachin · 2025-10-12T16:34:14Z

backends/apple/metal/runtime/shims/et_metal.h

+};
+
+// =======================
+// ETMetalStream - Metal command buffer and synchronization management


ETMetalStream - Manages Metal device, command queue, and command buffers - Provides synchronization primitives (5 modes) - Implements kernel coalescing for batch optimization - Executes MPSGraph operations for AOTI-compiled models

mergennachin · 2025-10-12T16:38:54Z

backends/apple/metal/runtime/shims/et_metal.h

+extern "C" {
+#endif
+
+// Memory management functions for Metal


Docblock about the memory management aspect of this design, such as buffer lifecycle, thread safety etc.

mergennachin · 2025-10-12T16:41:55Z

backends/apple/metal/runtime/shims/et_metal.h

+
+// C++ only - expose the Metal buffer mapping
+#ifdef __OBJC__
+extern std::unordered_map<void*, MTLBuffer_t> ptr_to_mtl_buffer;


do you need a lock and thread safety to access ptr_to_mtl_buffer?

mergennachin · 2025-10-12T16:43:33Z

backends/apple/metal/runtime/shims/et_metal.mm

+void ETMetalStream::copy(id<MTLBuffer> srcBuffer, id<MTLBuffer> dstBuffer, size_t length,
+                        size_t srcOffset, size_t dstOffset, SyncType syncType) {


before copying check that offsets are within the buffer bounds

[ghstack-poisoned]

This commit introduces the foundational Metal backend runtime. Key features: - ETMetalStream for managing Metal devices, command queues, buffers, and synchronization. - ETMetalShaderLibrary for compiling Metal shader source and caching pipeline states. - ETMetalKernelFunction for kernel argument binding, dispatching, and synchronization with stream-managed encoders. - Added global buffer management and pointer tracking between host and Metal buffers. - Added global stream management utilities and synchronization helpers This provides the necessary runtime primitives for executing compute shaders and MPSGraph workloads. ghstack-source-id: ea4fbb5 ghstack-comment-id: 3392300041 Pull-Request: pytorch#15020

[ghstack-poisoned]

manuelcandales added 5 commits October 10, 2025 13:29

Update

6420712

[ghstack-poisoned]

Update

d036c07

[ghstack-poisoned]

Update

1a22c5e

[ghstack-poisoned]

Update

d6f0bc9

[ghstack-poisoned]

Update

7e11615

[ghstack-poisoned]

manuelcandales requested review from cccclai and shoumikhin as code owners October 10, 2025 21:01

meta-cla bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Oct 10, 2025

manuelcandales requested review from larryliu0820 and mergennachin and removed request for cccclai and shoumikhin October 10, 2025 21:03

manuelcandales added 2 commits October 11, 2025 15:47

Update

ca5f1e5

[ghstack-poisoned]

Update

7e971b0

[ghstack-poisoned]

mergennachin reviewed Oct 12, 2025

View reviewed changes

manuelcandales added 6 commits October 13, 2025 12:46

Update

c4c16aa

[ghstack-poisoned]

Update

ce0f085

[ghstack-poisoned]

Update

e391e17

[ghstack-poisoned]

Update

3572de8

[ghstack-poisoned]

Update

89d3f14

[ghstack-poisoned]

Update

7590e37

[ghstack-poisoned]

Update

bea144f

[ghstack-poisoned]

manuelcandales added the release notes: none Do not include this in the release notes label Oct 13, 2025

manuelcandales added 4 commits October 14, 2025 20:57

Update

971a762

[ghstack-poisoned]

Update

3425f17

[ghstack-poisoned]

Update

f46adc5

[ghstack-poisoned]

Update

16d863c

[ghstack-poisoned]

Base automatically changed from gh/manuelcandales/139/head to main October 15, 2025 02:58

Update

b782bb5

[ghstack-poisoned]

		void ETMetalStream::copy(id<MTLBuffer> srcBuffer, id<MTLBuffer> dstBuffer, size_t length,
		size_t srcOffset, size_t dstOffset, SyncType syncType) {

Add Metal backend core ETMetal runtime. #15020

Are you sure you want to change the base?

Add Metal backend core ETMetal runtime. #15020

Conversation

manuelcandales commented Oct 10, 2025

Uh oh!

manuelcandales commented Oct 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pytorch-bot bot commented Oct 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/15020

❌ 1 New Failure, 1 Unrelated Failure

Uh oh!

mergennachin left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

manuelcandales commented Oct 10, 2025 •

edited

Loading

pytorch-bot bot commented Oct 10, 2025 •

edited

Loading