Skip to content

Commit d5322f2

Browse files
authored
Update rollouts.mdx (#617)
1 parent 96e6ee0 commit d5322f2

File tree

1 file changed

+2
-1
lines changed

1 file changed

+2
-1
lines changed

docs/deep-dives/experimenter/rollouts.mdx

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,7 @@ Rollouts are single-branch experiments that differ from a traditional experiment
1616
- A client can be enrolled in both a single experiment AND rollout for a given feature.
1717
- The experiment feature value takes precedence over the rollout feature value.
1818
- Rollouts use a separate bucketing namespace from experiments so you don't need to worry about the populations colliding.
19+
- Rollouts are not measurement tools. There is no comparison branch so no way to tell "did rollout change ____?"
1920

2021
## What is a rollout?
2122

@@ -47,7 +48,7 @@ Before you do this, you should consider:
4748
- If you have multiple stages, how will you know whether to advance or roll back?
4849
- What signals will help you make your decision? Where will they come from?
4950
- If you are relying on the experiment to guide you, make sure that the timelines are compatible.
50-
- _Consult data science_ before relying on signals derived from the behavior of the rollout group, since rollouts are not measurement tools and lack a control.
51+
- Rollouts are not measurement tools. There is NO CONTROL to compare to. There is no results page generated. Rollouts you are potentially manually watching for bugs or user sentiment issues or observing telemetry directly. There is no data science support for rollouts. There is an OpMon dashboard - which that provides raw data without comparison.
5152

5253
You would need to:
5354

0 commit comments

Comments
 (0)