Merge commit '85c812484be6e8fd186aa4ee23674051e18d048c'

Author: slinkydeveloper

sdk-core/src/main/service-protocol/dev/restate/service/protocol.proto

@@ -151,6 +151,26 @@ message ClearStateEntryMessage {
   bytes key = 1;
 }
 
+// Completable: No
+// Fallible: No
+// Type: 0x0800 + 3
+message ClearAllStateEntryMessage {
+}
+
+// Completable: Yes
+// Fallible: No
+// Type: 0x0800 + 4
+message GetStateKeysEntryMessage {
+  message StateKeys {
+    repeated bytes keys = 1;
+  }
+
+  oneof result {
+    StateKeys value = 14;
+    Failure failure = 15;
+  };
+}
+
 // ------ Syscalls ------
 
 // Completable: Yes

sdk-core/src/main/service-protocol/service-invocation-protocol.md

@@ -7,7 +7,7 @@ The following specification describes the protocol used by Restate to invoke rem
 The system is composed of two actors:
 
 - Restate Runtime
-- Service endpoint, which is split into:
+- Service deployment, which is split into:
   - SDK, which contains the implementation of the Restate Protocol
   - User business logic, which interacts with the SDK to access Restate system calls (or syscalls)
 
@@ -16,15 +16,15 @@ either by user code or by _Runtime events_.
 
 Every state transition is logged in the _Invocation journal_, used to implement Restate's durable execution model. The
 journal is also used to suspend an invocation and resume it at a later point in time. The _Invocation journal_ is
-tracked both by Restate's runtime and the service endpoint.
+tracked both by Restate's runtime and the service deployment.
 
-Runtime and service endpoint exchange _Messages_ containing the invocation journal and runtime events through an HTTP
+Runtime and service deployment exchange _Messages_ containing the invocation journal and runtime events through an HTTP
 message stream.
 
 ## State machine and journal
 
 Every invocation state machine begins when the stream is opened and ends when the stream is closed. In the middle,
-arbitrary interaction can be performed from the Service endpoint to the Runtime and vice versa via well-defined
+arbitrary interaction can be performed from the Service deployment to the Runtime and vice versa via well-defined
 messages.
 
 The state machine is summarized in the following diagram:
@@ -63,14 +63,14 @@ Both runtime and SDKs transition the message stream through 2 states:
 
 There are a couple of properties that we enforce through the design of the protocol:
 
-- Runtime and service endpoint both have their view of the journal
+- Runtime and service deployment both have their view of the journal
 - The source of truth of the journal and its ordering is:
   - The runtime, when the invocation is not in _processing_ state
-  - The service endpoint, when the invocation is in _processing_ state
-- When in _replaying_ state, the service endpoint cannot create new journal entries.
-- When in _processing_ state, only the service endpoint can create new journal entries, picking their order.
+  - The service deployment, when the invocation is in _processing_ state
+- When in _replaying_ state, the service deployment cannot create new journal entries.
+- When in _processing_ state, only the service deployment can create new journal entries, picking their order.
   Consequently, it might have newer entries that the runtime is not aware of. It’s also the responsibility of the
-  service endpoint to make sure the runtime has the same ordered view of the journal it has.
+  service deployment to make sure the runtime has the same ordered view of the journal it has.
 - Only in processing state the runtime can send
   [`CompletionMessage`](#completable-journal-entries-and-completionmessage)
 
@@ -90,8 +90,8 @@ Each syscall defines a priori whether it replies with an ack or a completion, or
 
 ## Messages
 
-The protocol is composed by messages that are sent back and forth between runtime and service Endpoint. The protocol
-mandates the following messages:
+The protocol is composed by messages that are sent back and forth between runtime and the service deployment. The
+protocol mandates the following messages:
 
 - `StartMessage`
 - `[..]EntryMessage`
@@ -102,8 +102,8 @@ mandates the following messages:
 
 ### Message stream
 
-In order to execute a service method invocation, service endpoint and restate Runtime open a single stream between the
-runtime and the service endpoint. Given 10 concurrent service method invocations to a service endpoint, there are 10
+In order to execute a service method invocation, service deployment and restate Runtime open a single stream between the
+runtime and the service deployment. Given 10 concurrent service method invocations to a service deployment, there are 10
 concurrent streams, each of them mapping to a specific invocation.
 
 Every unit of the stream contains a Message serialized using the
@@ -115,9 +115,9 @@ in two modes:
 
 - Full duplex (bidirectional) stream: Messages are sent back and forth on the same stream at the same time. This option
   is supported only when using HTTP/2.
-- Request/Response stream: Messages are sent from runtime to service endpoint, and later from service endpoint to
-  runtime. Once the service endpoint starts sending messages to the runtime, the runtime cannot send messages anymore
-  back to the service endpoint.
+- Request/Response stream: Messages are sent from runtime to service deployment, and later from service deployment to
+  runtime. Once the service deployment starts sending messages to the runtime, the runtime cannot send messages anymore
+  back to the service deployment.
 
 When opening the stream, the request method MUST be `POST` and the request path MUST have the following format:
 
@@ -284,6 +284,7 @@ descriptions in [`protocol.proto`](dev/restate/service/protocol.proto).
 | ------------------------------- | -------- | ----------- | -------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------- |
 | `PollInputStreamEntryMessage`   | `0x0400` | Yes         | No       | Carries the invocation input message(s) of the invocation. Note: currently the runtime always sends this entry completed, but this may change in future.         |
 | `GetStateEntryMessage`          | `0x0800` | Yes         | No       | Get the value of a service instance state key.                                                                                                                   |
+| `GetStateKeysEntryMessage`      | `0x0804` | Yes         | No       | Get all the known state keys for this service instance. Note: the completion value for this message is a protobuf of type `GetStateKeysEntryMessage.StateKeys`.  |
 | `SleepEntryMessage`             | `0x0C00` | Yes         | No       | Initiate a timer that completes after the given time.                                                                                                            |
 | `InvokeEntryMessage`            | `0x0C01` | Yes         | Yes      | Invoke another Restate service.                                                                                                                                  |
 | `AwakeableEntryMessage`         | `0x0C03` | Yes         | No       | Arbitrary result container which can be completed from another service, given a specific id. See [Awakeable identifier](#awakeable-identifier) for more details. |
@@ -292,18 +293,22 @@ descriptions in [`protocol.proto`](dev/restate/service/protocol.proto).
 | `OutputStreamEntryMessage`      | `0x0401` | No          | No       | Carries the invocation output message(s) or terminal failure of the invocation.                                                                                  |
 | `SetStateEntryMessage`          | `0x0800` | No          | No       | Set the value of a service instance state key.                                                                                                                   |
 | `ClearStateEntryMessage`        | `0x0801` | No          | No       | Clear the value of a service instance state key.                                                                                                                 |
+| `ClearAllStateEntryMessage`     | `0x0802` | No          | No       | Clear all the values of the service instance state.                                                                                                              |
 
 #### Awakeable identifier
 
 When creating an `AwakeableEntryMessage`, the SDK MUST expose to the user code an id, required to later complete the
 entry, using either `CompleteAwakeableEntryMessage` or some other mechanism provided by the runtime.
 
-The id format is a [Base64 URL Safe string](https://datatracker.ietf.org/doc/html/rfc4648#section-5) encoding a byte
-array that concatenates:
+The id format is a string starts with `prom_1` concatenated with a
+[Base64 URL Safe string](https://datatracker.ietf.org/doc/html/rfc4648#section-5) encoding of a byte array that
+concatenates:
 
 - `StartMessage.id`
 - The index of the Awakeable entry, encoded as unsigned 32 bit integer big endian.
 
+An example of a valid identifier would look like `prom_1NMyOAvDK2CcBjUH4Rmb7eGBp0DNNDnmsAAAAAQ`
+
 ## Suspension
 
 As mentioned in [Replaying and processing](#replaying-and-processing), an invocation can be suspended while waiting for
@@ -329,8 +334,8 @@ To notify a failure, the SDK can either:
 
 - Close the stream with `ErrorMessage` as last message. This message is used by the runtime for accurate reporting to
   the user.
-- Close the stream without `EndMessage` or `SuspensionMessage` or `ErrorMessage`. This is equivalent to sending
-  an `ErrorMessage` with unknown reason.
+- Close the stream without `EndMessage` or `SuspensionMessage` or `ErrorMessage`. This is equivalent to sending an
+  `ErrorMessage` with unknown reason.
 
 The runtime takes care of retrying to execute the invocation after such failures occur, following a defined set of
 policies. When retrying, the previous stored journal will be reused. Moreover, the SDK MUST NOT assume that every
@@ -398,5 +403,5 @@ A possible implementation could be the following. Given a user requests a state
   - If `partial_state` is set, generate a `GetStateEntryMessage` without a `result`, and wait for the runtime to send a
     `Completion` back (same logic as without eager state)
 
-In order for the aforementioned algorithm to work, set and clear state operations must be reflected on the local
-`state_map` as well.
+In order for the aforementioned algorithm to work, set, clear and clear all state operations must be reflected on the
+local `state_map` as well.