Skip to content
This repository was archived by the owner on Jul 2, 2024. It is now read-only.

Update usability-testing-how-to.md #8566

Closed
wants to merge 129 commits into from
Closed
Changes from 7 commits
Commits
Show all changes
129 commits
Select commit Hold shift + click to select a range
6878d89
Update usability-testing-how-to.md
rrhyne Feb 7, 2024
c10c195
squash! Prettier
rrhyne Feb 7, 2024
f5a4346
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Mar 25, 2024
3734084
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Mar 25, 2024
a448c95
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Mar 25, 2024
282a77a
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Mar 25, 2024
7a65a38
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Mar 26, 2024
341235d
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Mar 26, 2024
d08dcf9
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Mar 26, 2024
ed11c91
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Mar 26, 2024
1fd2144
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Mar 27, 2024
2694744
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Mar 27, 2024
40e8f28
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Mar 27, 2024
59cd857
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Mar 27, 2024
87872b9
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Mar 27, 2024
326644a
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Mar 27, 2024
ead2295
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Mar 27, 2024
1899c1d
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Mar 27, 2024
51b6613
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Mar 27, 2024
298df25
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Mar 27, 2024
53e5629
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Mar 27, 2024
4474d4a
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Mar 27, 2024
c9610e5
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Mar 28, 2024
ede0105
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Mar 28, 2024
4b896c5
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Mar 28, 2024
8c0ed5e
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Mar 29, 2024
2e96970
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Mar 29, 2024
03e30e0
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Mar 29, 2024
e09e8fe
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Mar 29, 2024
9dffd85
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Mar 29, 2024
50e5320
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Mar 29, 2024
a2182f4
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Mar 29, 2024
53dfe30
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Mar 29, 2024
6e4d1af
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Mar 29, 2024
38bbc90
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Mar 29, 2024
fd0812a
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Mar 29, 2024
c1a7dec
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Mar 29, 2024
7965639
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Mar 29, 2024
52a8e9b
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Mar 29, 2024
65de3f2
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Mar 29, 2024
55f10dc
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Apr 1, 2024
2618ed9
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Apr 1, 2024
23d4038
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Apr 2, 2024
f4b1b77
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Apr 2, 2024
dc66762
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Apr 2, 2024
8be0617
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Apr 2, 2024
a0baaf3
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Apr 2, 2024
91f7ac2
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Apr 2, 2024
16163bb
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Apr 2, 2024
af33072
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Apr 2, 2024
ee9f45e
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Apr 2, 2024
7524257
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Apr 3, 2024
c5a4be4
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Apr 3, 2024
e1ceb8a
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Apr 3, 2024
74ddd36
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Apr 3, 2024
eb35d6f
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Apr 4, 2024
13a0368
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Apr 4, 2024
631565f
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Apr 4, 2024
843fe51
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Apr 4, 2024
d8d757e
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Apr 5, 2024
46c1ef6
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Apr 5, 2024
b7812e5
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Apr 5, 2024
824c4df
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Apr 5, 2024
de5961c
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Apr 5, 2024
d309a11
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Apr 5, 2024
9e22663
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Apr 5, 2024
a2415c0
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Apr 5, 2024
b8d3877
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Apr 5, 2024
e88e7a0
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Apr 8, 2024
260e956
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Apr 8, 2024
a9e29af
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Apr 8, 2024
b136db9
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Apr 8, 2024
229cd47
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Apr 8, 2024
5bfb37f
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Apr 8, 2024
f0cc223
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Apr 8, 2024
09ed4ea
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Apr 8, 2024
64f45c6
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Apr 9, 2024
4be88e1
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Apr 9, 2024
b744745
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Apr 9, 2024
91a8371
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Apr 9, 2024
edceefd
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Apr 9, 2024
73d8afc
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Apr 9, 2024
d475062
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Apr 9, 2024
0ede9ad
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Apr 10, 2024
647d4d0
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Apr 10, 2024
d096bae
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Apr 10, 2024
8bd6cd4
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Apr 10, 2024
933fde9
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Apr 10, 2024
e02d4c0
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Apr 10, 2024
80a47a5
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Apr 10, 2024
077eee0
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Apr 10, 2024
c10bffb
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Apr 10, 2024
05a8563
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Apr 10, 2024
1f26733
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Apr 10, 2024
068f494
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Apr 10, 2024
c077348
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Apr 11, 2024
b11a068
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Apr 11, 2024
c877bd7
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Apr 11, 2024
b88e440
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Apr 11, 2024
6c416a0
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Apr 11, 2024
7266572
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Apr 11, 2024
c58a228
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Apr 15, 2024
1d9e6a7
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Apr 15, 2024
1064cf9
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Apr 17, 2024
a1a189f
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Apr 17, 2024
119081a
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Apr 18, 2024
fc51da1
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Apr 18, 2024
082def1
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Apr 18, 2024
42c215b
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Apr 19, 2024
8f2409f
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Apr 19, 2024
ba1816e
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Apr 19, 2024
57de545
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Apr 22, 2024
af6e132
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Apr 23, 2024
14b6169
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Apr 23, 2024
ccc2912
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Apr 24, 2024
b53408c
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Apr 26, 2024
5563ed6
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Apr 29, 2024
59b74c7
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot Apr 29, 2024
e835da7
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot May 1, 2024
6d395de
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot May 2, 2024
b2c544a
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot May 2, 2024
f8ea4bf
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot May 8, 2024
29f2cd0
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot May 8, 2024
6135e16
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot May 10, 2024
ee7a335
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot May 14, 2024
06b01db
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot May 16, 2024
7dc3138
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot May 17, 2024
759ee76
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot May 21, 2024
2aa63d1
Merge branch 'main' into rrhyne-patch-3
sourcegraph-bot May 24, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -1,5 +1,10 @@
# Usability testing

Quick links:

- [How to conduct an unmoderated user test](https://handbook.sourcegraph.com/departments/product/design/research/usability-testing-how-to/#how-to-conduct-an-unmoderated-user-test)
- [How to analyze a user test](https://handbook.sourcegraph.com/departments/product/design/research/usability-testing-how-to/#analyzing-a-usability-test)

At Sourcegraph, we use the following definition of usability when we conduct user research and design our product:

> To be usable, an interactive system should be effective, efficient, satisfying, and useful.
Expand Down Expand Up @@ -49,38 +54,15 @@ Feel free to add additional screening questions to these templates for your own

Generally, [the UX community recommends conducting evaluative usability tests with between 5 - 7 users to adequately capture most usability issues](https://www.nngroup.com/articles/why-you-only-need-to-test-with-5-users/).

## How to conduct a usability test

All usability testing is currently conducted in UserTesting.com by default. All usability testing is planned and analyzed in Dovetail, our research repository tool.

### Step 1: Write your usability test plan

We have created this [usability test project template](https://sourcegraph.dovetailapp.com/projects/3VnfTGMdL33acqZZ4zLwAM/readme) in Dovetail to help you write your research questions and define screening requirements outside of those already covered in our UserTesting screener templates. Key steps here are defining:

- your research questions
- your recruitment criteria
- specific tasks that you want the user to accomplish during your test
- how task success is defined

If there are any other metrics you want to track outside of built in usability metrics, make sure to note those in your plan as well.

Defining your research plan ahead of time will help you reduce scope creep and streamline analysis.

### Step 2: Choose and launch a usability test type

Based on your research questions, number of tasks, and time needed to complete this study, choose whether an unmoderated or moderated usability test best meets your research goal’s needs.
## How to conduct an unmoderated user test

#### Unmoderated tests

If you run an unmoderated usability test on UserTesting, create a test from the [Usability metrics template](https://app.usertesting.com/share/f5f2ba6e-6275-4db7-bbfe-5afa21f764fd), located in the “Account templates” section. Make sure to move this test to your team’s folder once created.

Choose an audience using one of our Audience templates:

- General Devs
- JetBrains Devs
- VSCode Devs

Update task language in the template to match the tasks in your usability test plan in Dovetail. Delete any parts of the template that you don’t need (for example, extra templated tasks).
1. Create new test in [usertesting.com from template](https://app.usertesting.com/workspaces/937482/choose-template)
1. Select account template
1. Test with usability metrics
1. Choose audience (typically General developers)
1. Edit test plan
1. Run test, making sure to validate task completion
1. [Analyze](https://handbook.sourcegraph.com/departments/product/design/research/usability-testing-how-to/#analyzing-a-usability-test)

If you run an unmoderated usability test that will run under 5 minutes, toggle the Short test option on when building your test plan.

Expand All @@ -98,7 +80,12 @@ Provide availability that best works with you and your team’s schedule.

## Analyzing a usability test

All tests should be analyzed using our research repository tool, Dovetail. The project you created in "Step 1: Write your usability test plan" includes analysis and tagging tools.
All tests should be analyzed using our research repository tool, Dovetail using this [usability test project template](https://sourcegraph.dovetailapp.com/projects/3VnfTGMdL33acqZZ4zLwAM/readme). Key steps here are defining:

- your research questions
- your recruitment criteria
- specific tasks that you want the user to accomplish during your test
- how task success is defined

Upload UserTesting videos to your usability test project in Dovetail. Do this by creating a copy of the “Analysis template” Note and adding the testing video to it.

Expand Down