You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/deep-dives/experimenter/rollouts.mdx
+2-1Lines changed: 2 additions & 1 deletion
Original file line number
Diff line number
Diff line change
@@ -16,6 +16,7 @@ Rollouts are single-branch experiments that differ from a traditional experiment
16
16
- A client can be enrolled in both a single experiment AND rollout for a given feature.
17
17
- The experiment feature value takes precedence over the rollout feature value.
18
18
- Rollouts use a separate bucketing namespace from experiments so you don't need to worry about the populations colliding.
19
+
- Rollouts are not measurement tools. There is no comparison branch so no way to tell "did rollout change ____?"
19
20
20
21
## What is a rollout?
21
22
@@ -47,7 +48,7 @@ Before you do this, you should consider:
47
48
- If you have multiple stages, how will you know whether to advance or roll back?
48
49
- What signals will help you make your decision? Where will they come from?
49
50
- If you are relying on the experiment to guide you, make sure that the timelines are compatible.
50
-
-_Consult data science_ before relying on signals derived from the behavior of the rollout group, since rollouts are not measurement tools and lack a control.
51
+
-Rollouts are not measurement tools. There is NO CONTROL to compare to. There is no results page generated. Rollouts you are potentially manually watching for bugs or user sentiment issues or observing telemetry directly. There is no data science support for rollouts. There is an OpMon dashboard - which that provides raw data without comparison.
0 commit comments