You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
This repository was archived by the owner on Sep 30, 2024. It is now read-only.
telemetrygateway: update schema to accommodate managed services (#60991)
Expands the Telemetry Gateway service schema to accommodate events from managed services, as proposed in [RFC 897](https://docs.google.com/document/d/1OberM_s04DdkCmo8WUN9mrjn5agNNoGtVKYESQyiuxo/edit?tab=t.0#heading=h.aai9y6ehm647). The idea is that managed services _backends_ will publish events to the same gRPC Telemetry Gateway service that single-tenant Sourcegraph do today. The changes include:
1. A new RPC, `RecordEvent`, for publishing events one at a time. This will be much easier to work with for new services, as the bidirectional streaming RPC is hard to hold right and is overkill for low-volume exporters.
2. A new `Identifier`, `ManagedService`, which managed services can use to identify as e.g. `service_id: ssc, service_environment: prod`.
3. A new `EventUser` identifier type, `optional string sams_external_id`, for associating an event with a SAMS user. We only expect this from managed services.
4. Docstring changes for clearer references to "managed services" and "single-tenant Sourcegraph".
Each service will implement its own mechanisms for how simple events are submitted by its clients, enriched by the backend, and then published to Telemetry Gateway. Pretty much everything else can be treated the same way as in Sourcegraph instances, but specific to the exporting service's context.
**This change does _not_ include any implementation yet** - in particular, the new `RecordEvent` RPC will return an unimplemented error. This is implemented in https://github.com/sourcegraph/sourcegraph/pull/61022
@@ -64,7 +67,7 @@ This page contains generated documentation for telemetry event data that gets ex
64
67
| parameters |[EventParameters](#telemetrygateway-v1-EventParameters)|| <p>Parameters of the event.</p> |
65
68
| user |[EventUser](#telemetrygateway-v1-EventUser)| optional | <p>Optional user associated with the event.</p><p>This field should be hydrated by the Sourcegraph server, and not provided</p><p>by clients.</p> |
66
69
| feature_flags |[EventFeatureFlags](#telemetrygateway-v1-EventFeatureFlags)| optional | <p>Optional feature flags configured in the context of the event.</p> |
67
-
| marketing_tracking |[EventMarketingTracking](#telemetrygateway-v1-EventMarketingTracking)| optional | <p>Optional marketing campaign tracking parameters.</p><p>🚨 SECURITY: This metadata is NEVER exported from an instance, and is only</p><p>exported for events tracked in the publicSourcegraph.com instance.</p> |
70
+
| marketing_tracking |[EventMarketingTracking](#telemetrygateway-v1-EventMarketingTracking)| optional | <p>Optional marketing campaign tracking parameters.</p><p>🚨 SECURITY: This metadata is NEVER exported from single-tenant Sourcegraph</p><p>instances, and is only exported for events tracked in the public</p><p>Sourcegraph.com instance and managed services.</p> |
68
71
| interaction |[EventInteraction](#telemetrygateway-v1-EventInteraction)| optional | <p>Optional metadata identifying the interaction that generated the event.</p> |
69
72
70
73
@@ -156,9 +159,9 @@ This page contains generated documentation for telemetry event data that gets ex
156
159
### EventMarketingTracking
157
160
Marketing campaign tracking metadata.
158
161
159
-
🚨 SECURITY: This metadata is NEVER exported from private Sourcegraph
162
+
🚨 SECURITY: This metadata is NEVER exported from single-tenant Sourcegraph
160
163
instances, and is only exported for events tracked in the public
161
-
Sourcegraph.com instance.
164
+
Sourcegraph.com instance and managed services.
162
165
163
166
164
167
| Field | Type | Label | Description |
@@ -186,7 +189,7 @@ Sourcegraph.com instance.
186
189
| Field | Type | Label | Description |
187
190
| ----- | ---- | ----- | ----------- |
188
191
| version |[int32](#int32)|| <p>Version of the event parameters, used for indicating the "shape" of this</p><p>event's metadata, beginning at 0. Useful for denoting if the shape of</p><p>metadata has changed in any way.</p> |
189
-
| legacy_metadata |[EventParameters.LegacyMetadataEntry](#telemetrygateway-v1-EventParameters-LegacyMetadataEntry)| repeated | <p>Legacy metadata format that only accepted int64 - use the new metadata</p><p>field instead, which accepts float values. Values sent through this proto</p><p>field will be merged into the new metadata attributes.</p><p>We don't use a [deprecated = true] tag because we use this field to handle</p><p>accepting exporters sending metadata in this format.</p> |
192
+
| legacy_metadata |[EventParameters.LegacyMetadataEntry](#telemetrygateway-v1-EventParameters-LegacyMetadataEntry)| repeated | <p>DEPRECATED, legacy metadata format that only accepted int64 - use the new</p><p>'metadata' field instead, which accepts float values. Values sent through</p><p>this proto field will be merged into the new metadata attributes.</p><p>We don't use a [deprecated = true] tag because we use this field to handle</p><p>accepting exporters sending metadata in this format.</p> |
190
193
| metadata |[EventParameters.MetadataEntry](#telemetrygateway-v1-EventParameters-MetadataEntry)| repeated | <p>Strictly typed metadata, restricted to integer values to avoid accidentally</p><p>exporting sensitive or private data.</p> |
191
194
| private_metadata |[google.protobuf.Struct](#google-protobuf-Struct)| optional | <p>Additional potentially sensitive metadata - i.e. not restricted to integer</p><p>values.</p><p>🚨 SECURITY: This metadata is NOT exported from instances by default, as it</p><p>can contain arbitrarily-shaped data that may accidentally contain sensitive</p><p>or private contents.</p><p>This metadata is only exported on an allowlist basis based on terms of</p><p>use agreements and combinations of event feature and action, alongside</p><p>careful audit of callsites.</p> |
| server |[EventSource.Server](#telemetrygateway-v1-EventSource-Server)|| <p>Information about the Sourcegraph instance that received the event.</p> |
242
+
| server |[EventSource.Server](#telemetrygateway-v1-EventSource-Server)|| <p>Information about the server that is publishing the event, based on</p><p>RecordEventsRequestMetadata.Identifier.</p> |
240
243
| client |[EventSource.Client](#telemetrygateway-v1-EventSource-Client)| optional | <p>Information about the client that generated the event.</p> |
241
244
242
245
@@ -253,7 +256,7 @@ Sourcegraph.com instance.
253
256
| Field | Type | Label | Description |
254
257
| ----- | ---- | ----- | ----------- |
255
258
| name |[string](#string)|| <p>Source client of the event.</p> |
256
-
| version |[string](#string)| optional | <p>Version of the cleint.</p> |
259
+
| version |[string](#string)| optional | <p>Version of the client.</p> |
257
260
258
261
259
262
@@ -268,7 +271,7 @@ Sourcegraph.com instance.
268
271
269
272
| Field | Type | Label | Description |
270
273
| ----- | ---- | ----- | ----------- |
271
-
| version |[string](#string)|| <p>Version of the Sourcegraph server.</p> |
274
+
| version |[string](#string)|| <p>Version of the server emitting the event, corresponding to</p><p>RecordEventsRequestMetadata.Identifier. For example, if the Identifier</p><p>indicates the publisher is a Sourcegraph instance, the version represents</p><p>the version of the Sourcegraph server.</p> |
272
275
273
276
274
277
@@ -283,8 +286,9 @@ Sourcegraph.com instance.
283
286
284
287
| Field | Type | Label | Description |
285
288
| ----- | ---- | ----- | ----------- |
286
-
| user_id |[int64](#int64)| optional | <p>Database user ID of signed in user. User IDs are specific to a Sourcegraph</p><p>instance, and not universal.</p><p>We use an int64 as an ID because in Sourcegraph, database user IDs are</p><p>always integers.</p> |
287
-
| anonymous_user_id |[string](#string)| optional | <p>Randomized unique identifier for an actor (e.g. stored in localstorage in</p><p>web client).</p> |
289
+
| user_id |[int64](#int64)| optional | <p>Sourcegraph instance database user ID of the user. User IDs are specific to</p><p>a Sourcegraph instance, and are not universal across Sourcegraph instances.</p><p>We use an int64 as an ID because in Sourcegraph, database user IDs are</p><p>always integers.</p> |
290
+
| anonymous_user_id |[string](#string)| optional | <p>Randomized unique identifier representing the user (typically stored in</p><p>localstorage in web clients, or similar mechanisms elsewhere). This is</p><p>often used for unauthenticated users, but can persist to authenticated</p><p>users as well.</p> |
291
+
| sams_external_id |[string](#string)| optional | <p>Sourcegraph Accounts Management System (SAMS) account associated with the</p><p>user, represented by a SAMS external user ID in a UUID format. This is only</p><p>valid for services leveraging SAMS as an identity provider - in other words,</p><p>traditional Sourcegraph instances will not provide this.</p><p>Learn more about SAMS: https://handbook.sourcegraph.com/departments/engineering/teams/core-services/sams</p> |
| managed_service |[Identifier.ManagedServiceIdentifier](#telemetrygateway-v1-Identifier-ManagedServiceIdentifier)|| <p>A service operated and managed by the Sourcegraph team, for example</p><p>a service deployed by https://handbook.sourcegraph.com/departments/engineering/teams/core-services/managed-services/platform/</p> |
| identifier |[Identifier](#telemetrygateway-v1-Identifier)|| <p>Telemetry publisher self-identification - for example, a Sourcegraph</p><p>instance of some other kind of service.</p> |
384
431
385
432
386
433
@@ -415,7 +462,8 @@ Sourcegraph.com instance.
415
462
416
463
| Method Name | Request Type | Response Type | Description |
| RecordEvents |[RecordEventsRequest](#telemetrygateway-v1-RecordEventsRequest) stream |[RecordEventsResponse](#telemetrygateway-v1-RecordEventsResponse) stream | <p>RecordEvents streams telemetry events in batches to the Telemetry Gateway</p><p>service. Events should only be considered delivered if recording is</p><p>acknowledged in RecordEventsResponse.</p><p>🚨 SECURITY: Callers should check the attributes of the Event type to ensure</p><p>that only the appropriate fields are exported, as some fields should only</p><p>be exported on an allowlist basis.</p> |
465
+
| RecordEvents |[RecordEventsRequest](#telemetrygateway-v1-RecordEventsRequest) stream |[RecordEventsResponse](#telemetrygateway-v1-RecordEventsResponse) stream | <p>RecordEvents streams telemetry events in batches to the Telemetry Gateway</p><p>service. Events should only be considered delivered if recording is</p><p>acknowledged in RecordEventsResponse.</p><p>This is the preferred mechanism for exporting large volumes of events in</p><p>bulk.</p><p>🚨 SECURITY: Callers exporting for single-tenant Sourcegraph should check</p><p>the attributes of the Event type to ensure that only the appropriate fields</p><p>are exported, as some fields should only be exported on an allowlist basis.</p> |
466
+
| RecordEvent |[RecordEventRequest](#telemetrygateway-v1-RecordEventRequest)|[RecordEventResponse](#telemetrygateway-v1-RecordEventResponse)| <p>RecordEvent records a single telemetry event to the Telemetry Gateway service.</p><p>If the RPC succeeds, then the event was successfully published.</p><p>This mechanism is intended for low-volume managed services. Higher-volume</p><p>use cases should implement a batching mechanism and use the RecordEvents</p><p>RPC instead.</p><p>🚨 SECURITY: Callers exporting for single-tenant Sourcegraph should check</p><p>the attributes of the Event type to ensure that only the appropriate fields</p><p>are exported, as some fields should only be exported on an allowlist basis.</p> |
0 commit comments