Skip to content

Commit e078efb

Browse files
freshstrangemusicfreshstrangemusic
authored
Add docs for the pref flipper (#632)
1 parent b057642 commit e078efb

File tree

2 files changed

+297
-0
lines changed

2 files changed

+297
-0
lines changed

‎docs/deep-dives/desktop/desktop-incident-response.md

Lines changed: 260 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,260 @@
1+
---
2+
id: desktop-incident-response
3+
title: Launching Incident Response Pref Flips
4+
slug: /desktop-incident-response
5+
---
6+
7+
As of Firefox 129 (and 128.2 ESR), Firefox supports flipping any pref via
8+
Nimbus. This is done with the `prefFlips` feature. The JSON Schema definition
9+
for the feature value can be found [here][jsonschema].
10+
11+
# Warning
12+
13+
This feature is *not intended for use in experimentation*. If you want to do an
14+
experiment on some number of prefs in Firefox, you **must** register these prefs
15+
with the [Feature Manifest][feature-manifest]. This ensures that prefs
16+
controlled via Nimbus are subject to code review. See the docs on [Desktop pref
17+
experiments][setPref] for more information.
18+
19+
Because the incident response feature can change *any pref*, it should be used
20+
with caution. Remember: with great power comes great responsibility.
21+
22+
Only the release management team can approve incident reponse pref flips.
23+
24+
# Behaviour
25+
26+
When a client enrolls into an experiment/rollout using this feature, Nimbus will
27+
set (or unset) all the prefs to the values specified. Prefs on the `user` branch
28+
will persist through startup and be available immediately. Prefs set on the
29+
`default` branch will not persist through startup and will not have their
30+
correct values until Nimbus finishes initialization.
31+
32+
When the client unenrolls, all prefs will be restored to their values before
33+
enrollment. If the pref did not exist before enrollment, it will be cleared if
34+
it was set on the user branch. The default branch does not support clearing, so
35+
the pref and its value from the enrollment will persist until the browser
36+
closes.
37+
38+
If the client is enrolled in both an experiment and a rollout then only the
39+
experiment will take effect. This behaviour is likely to change once multiple
40+
enrollment support is added to desktop.
41+
42+
## Multi-feature
43+
44+
The incident response feature *cannot* be used with other features.
45+
46+
47+
# Launching a Pref Flip
48+
49+
1. Create a new experiment in
50+
[Experimenter](https://experimenter.services.mozilla.com/nimbus/) with an
51+
appropriate public name (this will appear in `about:studies`) for Firefox
52+
Desktop.
53+
2. Fill in the "overview" page as appropriate.
54+
3. Fill in the "branches" page:
55+
1. Select the `prefFlips` feature.
56+
2. Check the "This is a rollout (single branch)" checkbox.
57+
58+
This is not strictly required, but rollout enrollment can be dialed up or
59+
down without requiring ending the experiment and re-launching. If you do
60+
not want to use a rollout so that you can set different prefs on different
61+
branches you likely want to run an experiment, in which case this feature
62+
is not appropriate for your use case.
63+
64+
3. Enter a description for the "Control" branch.
65+
4. Enter a value for the `prefFlips` (see the [example configuration](#example-configuration)).
66+
4. Fill in the "Audience" page:
67+
68+
<dl>
69+
<dt>channel</dt>
70+
<dd>
71+
You must specify a channel to use this feature. If you want to change a
72+
pref across multiple channels you must repeat this process for each
73+
channel.
74+
</dd>
75+
76+
<dt>Min Version</dt>
77+
<dd>
78+
You must specify a minimum version. The minimum versions supported are
79+
129 on the Nightly, Beta, Aurora (developer edition), and Release
80+
channel and 128.2 on the ESR channel.
81+
</dd>
82+
83+
<dt>Advanced Targeting</dt>
84+
<dd>
85+
<p>
86+
If you want to enroll the entire population, choose "No Targeting - All
87+
users". Otherwise, select an appropriate targeting criteria.
88+
</p>
89+
<p>
90+
If a targeting criteria does not exist, one will have to be added. See
91+
the <a href="/targeting/advanced-targeting#answer">advanced targeting
92+
FAQ</a> or ask for help in <code>#ask-experimenter</code> on Slack.
93+
</p>
94+
</dd>
95+
96+
<dt>Sticky Enrollment</dt>
97+
<dd>
98+
<p>
99+
Experiment targeting is re-evaluated every time the client checks for
100+
new recipes. If you want the selected advanced targeting to only be
101+
evaluated during enrollment, check this box.
102+
</p>
103+
<p>
104+
<strong>Note:</strong> Rollouts can re-enroll after unenrolling (e.g.,
105+
if enrollment percentage decreased and then increased). In these
106+
cases, rollouts will re-evaluate targeting, ignoring the sticky flag.
107+
</p>
108+
</dd>
109+
110+
<dt>Percent of Clients</dt>
111+
<dd>
112+
Enter the percent of clients you wish to enroll. This value can be
113+
changed after launch for rollouts.
114+
</dd>
115+
</dl>
116+
117+
5. On the Summary Page, request launch.
118+
119+
Experiments and rollouts using the incident response feature cannot be
120+
launched to preview. We highly reccomend you QA these recipes on the [staging
121+
instance of experimenter][stage] first.
122+
123+
**Note:** Experimenter may report errors with your recipe configuration. You
124+
must fix them before requesting launch. Ask in `#ask-experimenter` on Slack.
125+
126+
6. Ask release management to approve and launch the rollout.
127+
128+
**Note:** If you are a member of release management, you must get *another*
129+
member to to approve and launch. Experimenter requires a 2 person sign-off to
130+
launch experiments: the person who requests the launch cannot be the person
131+
who approves it.
132+
133+
134+
<a id="example-configuration"></a>
135+
136+
# Example Configuration
137+
138+
```json
139+
{
140+
"prefs": {
141+
"the.pref.to.set": {
142+
"branch": "user",
143+
"value": "hello world"
144+
},
145+
"another.pref.to.set": {
146+
"branch": "default",
147+
"value": true
148+
},
149+
"unset.this.pref": {
150+
"branch": "user",
151+
"value": null
152+
}
153+
}
154+
}
155+
```
156+
157+
# Causes of Unenrollment
158+
159+
## Mismatched Types
160+
161+
If you attempt to set a pref to a value of the wrong type, the pref will fail to
162+
set. This will cause the enrollment to end.
163+
This will result in a unenrollment event
164+
([glean][glean-telemetry], [legacy][legacy-telemetry]) being submitted with the
165+
following data:
166+
167+
<table>
168+
<thead>
169+
<tr>
170+
<th>Glean Field</th>
171+
<th>Legacy Field</th>
172+
<th>Description</th>
173+
</tr>
174+
</thead>
175+
<tbody>
176+
<tr>
177+
<td><code>reason</code></td>
178+
<td><code>reason</code></td>
179+
<td>The string <code>"prefFlips-failed"</code></td>
180+
</tr>
181+
<tr>
182+
<td><code>pref_name</code></td>
183+
<td><code>prefName</code></td>
184+
<td>The pref that that failed to set.</td>
185+
</tr>
186+
<tr>
187+
<td><code>pref_type</code></td>
188+
<td><code>prefType</code></td>
189+
<td>The type of the existing pref.</td>
190+
</tr>
191+
</tbody>
192+
</table>
193+
194+
## Conflicts with setPref Experiments
195+
196+
If a user is enrolled in a incident response pref flip and then enrolls in a
197+
[setPref] experiment that changes the same prefs, they will be unenrolled from
198+
the incident response rollout. This will result in a unenrollment event
199+
([glean][glean-telemetry], [legacy][legacy-telemetry]) being submitted with the
200+
following data:
201+
202+
<table>
203+
<thead>
204+
<tr>
205+
<th>Glean Field</th>
206+
<th>Legacy Field</th>
207+
<th>Description</th>
208+
</tr>
209+
</thead>
210+
<tbody>
211+
<tr>
212+
<td><code>reason</code></td>
213+
<td><code>reason</code></td>
214+
<td>The string <code>"prefFlips-conflict"</code></td>
215+
</tr>
216+
<tr>
217+
<td><code>conflicting_slug</code></td>
218+
<td><code>conflictingSlug</code></td>
219+
<td>The slug of the experiment that caused the unenrollment.</td>
220+
</tr>
221+
</tbody>
222+
</table>
223+
224+
## User Pref Changes
225+
226+
If a pref that is controlled by an incident response rollout is changed, either
227+
by the user or client code, the client will be unenrolled. This will result in an
228+
unenrollment event ([glean][glean-telemetry], [legacy][legacy-telemetry]) being
229+
submitted with the following data:
230+
231+
<table>
232+
<thead>
233+
<tr>
234+
<th>Glean Field</th>
235+
<th>Legacy Field</th>
236+
<th>Description</th>
237+
</tr>
238+
</thead>
239+
<tbody>
240+
<tr>
241+
<td><code>reason</code></td>
242+
<td><code>reason</code></td>
243+
<td>The string <code>"changed-pref"</code></td>
244+
</tr>
245+
<tr>
246+
<td><code>changed_pref</code></td>
247+
<td><code>changedPref</code></td>
248+
<td>The pref that triggered the unenrollment.</td>
249+
</tr>
250+
</tbody>
251+
</table>
252+
253+
254+
[feature-manifest]: https://searchfox.org/mozilla-central/source/toolkit/components/nimbus/FeatureManifest.yaml
255+
[jsonschema]: https://searchfox.org/mozilla-central/source/toolkit/components/nimbus/schemas/PrefFlipsFeature.schema.json
256+
[setPref]: /desktop-pref-experiments
257+
[advanced-targeting]: /targeting/advanced-targeting#answer
258+
[stage]: https://stage.experimenter.nonprod.dataops.mozgcp.net/nimbus/
259+
[glean-telemetry]: https://dictionary.telemetry.mozilla.org/apps/firefox_desktop/metrics/nimbus_events_unenrollment
260+
[legacy-telemetry]: https://probes.telemetry.mozilla.org/?search=unenroll&view=detail&probeId=event%2Fnormandy.unenroll%23unenroll

‎docs/deep-dives/desktop/desktop-pref-experiments.md

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,8 @@ title: Running Pref-setting Experiments on Desktop
44
slug: /desktop-pref-experiments
55
---
66

7+
[test][prefFlips]
8+
79
As of Firefox 107, Nimbus supports experiments that set preferences on Desktop.
810
Unlike Normandy, Nimbus cannot set arbitrary preferences; instead, the
911
preferences that may be set are determined by the feature manifest.
@@ -101,3 +103,38 @@ any feature specifies a pref as a fallback pref, no variable may set that
101103
variable as a set pref and vice versa.
102104

103105
These restrictions are enforced at build time.
106+
107+
## Conflicts with Incident Response Pref Flips
108+
109+
If a user is enrolled in a setPref experiment/rollout and then enrolls in an
110+
[incident response pref flip][prefFlips], they will be unenrolled from the
111+
setPref experiment/rollout. This will result in an unenrollment event
112+
([glean][glean-telemetry], [legacy][legacy-telemetry]) being submitted with the
113+
following data:
114+
115+
<table>
116+
<thead>
117+
<tr>
118+
<th>Glean Field</th>
119+
<th>Legacy Field</th>
120+
<th>Description</th>
121+
</tr>
122+
</thead>
123+
<tbody>
124+
<tr>
125+
<td><code>reason</code></td>
126+
<td><code>reason</code></td>
127+
<td>The string <code>"prefFlips-conflict"</code></td>
128+
</tr>
129+
<tr>
130+
<td><code>conflicting_slug</code></td>
131+
<td><code>conflictingSlug</code></td>
132+
<td>The slug of the experiment that caused the unenrollment.</td>
133+
</tr>
134+
</tbody>
135+
</table>
136+
137+
138+
[prefFlips]: /desktop-incident-response
139+
[glean-telemetry]: https://dictionary.telemetry.mozilla.org/apps/firefox_desktop/metrics/nimbus_events_unenrollment
140+
[legacy-telemetry]: https://probes.telemetry.mozilla.org/?search=unenroll&view=detail&probeId=event%2Fnormandy.unenroll%23unenroll

0 commit comments

Comments
 (0)