books: hyperactor-book: hyperactor macros (#388)

Summary:
Pull Request resolved: #388

new section documenting the hyperactor macros

Reviewed By: mariusae

Differential Revision: D77595758

fbshipit-source-id: 714c31736f0386e51c69d3974ebf4d9a7033b374

Author: shayne-fletcher

books/hyperactor-book/src/SUMMARY.md

@@ -1,6 +1,13 @@
 # Summary
 
 - [Introduction](./introduction.md)
+- [Macros](macros/index.md)
+  - [`#[derive(Handler)]`](macros/handler.md)
+  - [`#[derive(HandleClient)]`](macros/handle_client.md)
+  - [`#[derive(RefClient)]`](macros/ref_client.md)
+  - [`#[derive(Named)]`](macros/named.md)
+  - [`#[export]`](macros/export.md)
+  - [`#[forward]`](macros/forward.md)
 - [Mailboxes and Routers](mailboxes/index.md)
   - [Ports](mailboxes/ports.md)
   - [MailboxSender](mailboxes/mailbox_sender.md)

books/hyperactor-book/src/introduction.md

@@ -4,4 +4,4 @@ This book describes the design and implementation of the hyperactor runtime.
 
 The goal is to provide a clear, structured explanation of how actors communicate safely and efficiently across distributed systems using hyperactor’s abstractions.
 
-Work in progress.
+We hope this becomes the book we wish we had when we started working with Monarch. Work in progress.

books/hyperactor-book/src/macros/export.md

@@ -0,0 +1,70 @@
+# `#[export]`
+
+The `#[hyperactor::export]` macro turns a regular `Actor` implementation into a remotely spawnable actor, registering its type information, `spawn` function, and supported message handlers for discovery and use across processes or runtimes.
+
+## What It Adds
+
+When applied to an actor type like this:
+
+```rust
+#[hyperactor::export(
+    spawn = true,
+    handlers = [ShoppingList],
+)]
+struct ShoppingListActor(HashSet<String>);
+```
+The macro expands to include:
+ - A `Named` implementation for the actor
+ - A `Binds<Self>` implementation that registers supported message types
+ - Implementations of `RemoteHandles<T>` for each type in the `handlers = [...]` list
+ - A `RemoteActor` marker implementation
+ - If `spawn = true`, a `RemotableActor` implementation and an inventory registration of the `spawn` function.
+
+This enables the actor to be:
+ - Spawned dynamically by name
+ - Routed to via typed messages
+ - Reflected on at runtime (for diagnostics, tools, and orchestration)
+
+## Generated Implementations (simplified)
+```rust
+impl RemoteActor for ShoppingListActor {}
+
+impl RemoteHandles<ShoppingList> for ShoppingListActor {}
+impl RemoteHandles<Signal> for ShoppingListActor {}
+
+impl Binds<ShoppingListActor> for ShoppingListActor {
+    fn bind(ports: &Ports<Self>) {
+        ports.bind::<ShoppingList>();
+    }
+}
+
+impl Named for ShoppingListActor {
+    fn typename() -> &'static str {
+        "my_crate::ShoppingListActor"
+    }
+}
+```
+If `spawn = true`, the macro also emtis:
+```rust
+impl RemotableActor for ShoppingListActor {
+    fn gspawn(...) -> ...
+}
+```
+Along with a registration into inventory:
+```
+inventory::submit!(SpawnableActor {
+    name: ...,
+    gspawn: ...,
+    get_type_id: ...,
+});
+```
+This allows the actor to be discovered and spawned by name at runtime.
+
+## Summary
+
+The `#[export]` macro makes an actor remotely visible, spawnable, and routable by declaring:
+ - What messages it handles
+ - What messages it handles
+ - How to bind those messages
+ - What its globally unique name is
+ - (Optionally) how to spawn it dynamically

books/hyperactor-book/src/macros/forward.md

@@ -0,0 +1,28 @@
+# `#[forward]`
+
+The `#[hyperactor::forward]` macro connects a user-defined handler trait implementation (like `ShoppingListHandler`) to the core `Handler<T>` trait required by the runtime.
+
+In short, it generates the boilerplate needed to route incoming messages of type `T` to your high-level trait implementation.
+
+## What it generates
+
+The macro expands to:
+```rust
+#[async_trait]
+impl Handler<ShoppingList> for ShoppingListActor {
+    async fn handle(&mut self, ctx: &Context<Self>, message: ShoppingList) -> Result<(), Error> {
+        <Self as ShoppingListHandler>::handle(self, ctx, message).await
+    }
+}
+```
+This avoids having to manually match on enum variants or duplicate message logic.
+
+## When to use it
+
+Use `#[forward(MessageType)]` when:
+
+- You’ve defined a custom trait (e.g., `ShoppingListHandler`)
+- You’re handling a message enum (like `ShoppingList`)
+- You want the runtime to route messages to your trait automatically.
+
+This is most often used alongside `#[derive(Handler)]`, which generates the corresponding handler and client traits for a user-defined message enum.

books/hyperactor-book/src/macros/handle_client.md

@@ -0,0 +1,65 @@
+# `#[derive(HandleClient)]`
+
+`#[derive(Handler)]` generates both the server-side handler trait (`ShoppingListHandler`) and the client-side trait definition (`ShoppingListClient`). However, it does not implement the client trait for any specific type.
+
+This is where `#[derive(HandleClient)]` comes in.
+
+## What It Adds
+
+`#[derive(HandleClient)]` generates the following implementation:
+
+```rust
+impl<T> ShoppingListClient for ActorHandle<T>
+where
+  T: ShoppingListHandler + Send + Sync + 'static`
+```
+
+This means you can call methods like `.add(...)` or `.list(...)` directly on an `ActorHandle<T>` without needing to manually implement the `ShoppingListClient` trait:
+
+In other words, `HandleClient` connects the generated `ShoppingListClient` interface (from `Handler`) to the concrete type `ActorHandle<T>`.
+
+## Generated Implementation (simplified)
+```rust
+use async_trait::async_trait;
+use hyperactor::{
+    ActorHandle,
+    anyhow::Error,
+    cap::{CanSend, CanOpenPort},
+    mailbox::open_once_port,
+    metrics,
+    Message,
+};
+
+#[async_trait]
+impl<T> ShoppingListClient for ActorHandle<T>
+where
+    T: ShoppingListHandler + Send + Sync + 'static,
+{
+    async fn add(&self, caps: &impl CanSend, item: String) -> Result<(), Error> {
+        self.send(caps, ShoppingList::Add(item)).await
+    }
+
+    async fn remove(&self, caps: &impl CanSend, item: String) -> Result<(), Error> {
+        self.send(caps, ShoppingList::Remove(item)).await
+    }
+
+    async fn exists(
+        &self,
+        caps: &impl CanSend + CanOpenPort,
+        item: String,
+    ) -> Result<bool, Error> {
+        let (reply_to, recv) = open_once_port(caps)?;
+        self.send(caps, ShoppingList::Exists(item, reply_to)).await?;
+        Ok(recv.await?)
+    }
+
+    async fn list(
+        &self,
+        caps: &impl CanSend + CanOpenPort,
+    ) -> Result<Vec<String>, Error> {
+        let (reply_to, recv) = open_once_port(caps)?;
+        self.send(caps, ShoppingList::List(reply_to)).await?;
+        Ok(recv.await?)
+    }
+
+```

books/hyperactor-book/src/macros/handler.md

@@ -0,0 +1,96 @@
+# `#[derive(Handler)]`
+
+The `#[derive(Handler)]` macro generates the infrastructure for sending and receiving typed messages in hyperactor. When applied to an enum like this:
+```rust
+#[derive(Handler)]
+enum ShoppingList {
+    // Fire-and-forget messages
+    Add(String),
+    Remove(String),
+
+    // Request-response messages
+    Exists(String, #[reply] OncePortRef<bool>),
+    List(#[reply] OncePortRef<Vec<String>>),
+}
+```
+... it generates **two key things**:
+
+### 1. `ShoppingListHandler` trait
+This trait defines a method for each variant, and a `handle` method to route incoming messages:
+```rust
+use async_trait::async_trait;
+use hyperactor::anyhow::Error;
+
+#[async_trait]
+pub trait ShoppingListHandler: hyperactor::Actor + Send + Sync {
+    async fn add(&mut self, ctx: &Context<Self>, item: String) -> Result<(), Error>;
+    async fn remove(&mut self, ctx: &Context<Self>, item: String) -> Result<(), Error>;
+    async fn exists(&mut self, ctx: &Context<Self>, item: String) -> Result<bool, Error>;
+    async fn list(&mut self, ctx: &Context<Self>) -> Result<Vec<String>, Error>;
+
+    async fn handle(&mut self, ctx: &Context<Self>, msg: ShoppingList) -> Result<(), Error> {
+        match msg {
+            ShoppingList::Add(item) => {
+                self.add(ctx, item).await
+            }
+            ShoppingList::Remove(item) => {
+                self.remove(ctx, item).await
+            }
+            ShoppingList::Exists(item, reply_to) => {
+                let result = self.exists(ctx, item).await?;
+                reply_to.send(ctx, result)?;
+                Ok(())
+            }
+            ShoppingList::List(reply_to) => {
+                let result = self.list(ctx).await?;
+                reply_to.send(ctx, result)?;
+                Ok(())
+            }
+        }
+    }
+}
+```
+Note:
+  - `Add` and `Remove` are **oneway**: no reply port
+  - `Exists` and `List` are **call-style**: they take a `#[reply] OncePortRef<T>` and expect a response to be sent back.
+
+### 2. `ShoppingListClient` trait
+
+Alongside the handler, the `#[derive(Handler)]` macro also generates a client-side trait named `ShoppingListClient`. This trait provides a convenient and type-safe interface for sending messages to an actor.
+
+Each method in the trait corresponds to a variant of the message enum. For example:
+```rust
+use async_trait::async_trait;
+use hyperactor::anyhow::Error;
+use hyperactor::cap::{CanSend, CanOpenPort};
+
+#[async_trait]
+pub trait ShoppingListClient: Send + Sync {
+    async fn add(&self, caps: &impl CanSend, item: String) -> Result<(), Error>;
+    async fn remove(&self, caps: &impl CanSend, item: String) -> Result<(), Error>;
+    async fn exists(&self, caps: &impl CanSend + CanOpenPort, item: String) -> Result<bool, Error>;
+    async fn list(&self, caps: &impl CanSend + CanOpenPort) -> Result<Vec<String>, Error>;
+}
+```
+
+#### Capability Parameter
+Each method takes a caps argument that provides the runtime capabilities required to send the message:
+- All methods require `CanSend`.
+- Methods with `#[reply]` arguments additionally require `CanOpenPort`.
+
+In typical usage, `caps` is a `Mailbox`.
+
+#### Example Usage
+```rust
+let mut proc = Proc::local();
+let actor = proc.spawn::<ShoppingListActor>("shopping", ()).await?;
+let client = proc.attach("client").unwrap();
+
+// Fire-and-forget
+actor.add(&client, "milk".into()).await?;
+
+// With reply
+let found = actor.exists(&client, "milk".into()).await?;
+println!("got milk? {found}");
+```
+Here, actor is an `ActorHandle<ShoppingListActor>` that implements `ShoppingListClient`, and `client` is a `Mailbox` that provides the necessary capabilities.

books/hyperactor-book/src/macros/index.md

@@ -0,0 +1,32 @@
+# Macros
+
+This section documents the macros provided by hyperactor for actor and message integration.
+
+These macros support a complete message-passing workflow: from defining message enums and generating client APIs, to routing messages and exporting actors for dynamic or remote use.
+
+- [`#[derive(Handler)]`](handler.md) — generate message handling and client traits for actor enums
+- [`#[derive(HandleClient)]`](handle_client.md) — implement the generated client trait for `ActorHandle<T>`
+- [`#[derive(RefClient)]`](ref_client.md) — implement the generated client trait for `ActorRef<T>`
+- [`#[derive(Named)]`](named.md) — give a type a globally unique name and port for routing and reflection
+- [`#[export]`](export.md) — make an actor remotely spawnable and routable by registering its type, handlers, and and optionally spawnable from outside the current runtime
+- [`#[forward]`](forward.md) — route messages to a user-defined handler trait implementation
+
+## Macro Summary
+
+- **`#[derive(Handler)]`**
+  Generates handler and client traits for a message enum.
+
+- **`#[derive(HandleClient)]`**
+  Implements the client trait for `ActorHandle<T>`.
+
+- **`#[derive(RefClient)]`**
+  Implements the client trait for `ActorRef<T>`.
+
+- **`#[derive(Named)]`**
+  Registers the type with a globally unique name and port.
+
+- **`#[export]`**
+  Makes an actor spawnable and routable via inventory.
+
+- **`#[forward]`**
+  Forwards messages to a user-defined handler trait implementation.

books/hyperactor-book/src/macros/named.md

@@ -0,0 +1,77 @@
+# `#[derive(Named)]`
+
+The `#[derive(Named)]` macro implements the `hyperactor::Named` trait for a type, making it identifiable at runtime through a globally unique string and stable hash.
+
+## The `Named` trait
+
+The `hyperactor::data::Named` trait is the foundation of type identification in hyperactor. It gives each type a globally unique identity based on its name used in routing.
+```rust
+pub trait Named: Sized + 'static {
+    fn typename() -> &'static str;
+    fn typehash() -> u64 { ... }
+    fn typeid() -> TypeId { ... }
+    fn port() -> u64 { ... }
+    fn arm(&self) -> Option<&'static str> { ... }
+    unsafe fn arm_unchecked(self_: *const ()) -> Option<&'static str> { ... }
+}
+```
+
+### Trait Methods
+
+#### `typename() -> &'static str`
+
+Returns the globally unique, fully-qualified type name for the type. This should typically look like:
+```rust
+"foo::bar::Corge<quux::garpy::Waldo>"
+```
+
+#### `typehash() -> u64`
+
+Returns a stable hash derived from `typename()`. This value is used for message port derivation.
+```rust
+cityhasher::hash(Self::typename())
+```
+
+#### `typeid() -> TypeId`
+
+Returns the Rust `TypeId` for the type (, which is only unique within a single binary).
+
+#### `port() -> u64`
+
+Returns a globally unique port number for the type:
+```rust
+Self::typehash() | (1 << 63)
+```
+Typed ports are reserved in the range 2^63 .. 2^64 - 1.
+
+### `arm(&self) -> Option<&'static str>`
+
+For enum types, this returns the name of the current variant, e.g., "Add" or "Remove".
+
+### `unsafe fn arm_unchecked(ptr: *const ()) -> Option<&'static str>`
+
+The type-erased version of `arm()`. Casts ptr back to `&Self` and calls `arm()`.
+
+Useful for dynamic reflection when the concrete type isn’t statically known
+
+### Runtime Registration
+
+In addition to implementing the `Named` trait, the macro registers the type’s metadata at startup using the `inventory` crate:
+```rust
+const _: () = {
+    static __INVENTORY: ::inventory::Node = ::inventory::Node {
+        value: &TypeInfo { ... },
+        ...
+    };
+    // Registers the type info before main() runs
+    #[link_section = ".init_array"]
+    static __CTOR: unsafe extern "C" fn() = __ctor;
+};
+```
+This allows the type to be discovered at runtime, enabling:
+- Message dispatch from erased or serialized inputs
+- Introspection and diagnostics
+- Dynamic spawning or reflection
+- Tooling support
+
+Types registered this way appear in the global `inventory::iter<TypeInfo>` set, which is how the hyperactor runtime locates known message types.