Skip to content

Commit b083539

Browse files
KobzolKobzol
authored
Merge pull request #350 from Sakib25800/merge-queue-skeleton
Merge queue skeleton
2 parents 3d20c0e + 611c88f commit b083539

18 files changed

+688
-31
lines changed

‎.sqlx/query-b1fe02180d44f3cdc04ca0be76163606a84a5ed390079e4786e471b2e8850e49.json

Lines changed: 193 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎README.md

Lines changed: 15 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -23,14 +23,23 @@ required.
2323
| `--cmd-prefix` | `CMD_PREFIX` | @bors | Prefix used to invoke bors commands in PR comments. |
2424

2525
### Special branches
26-
The bot uses the following two branch names for its operations.
26+
The bot uses the following branch names for its operations.
27+
28+
#### Try builds
2729
- `automation/bors/try-merge`
2830
- Used to perform merges of a pull request commit with a parent commit.
2931
- Should not be configured for any CI workflows!
3032
- `automation/bors/try`
3133
- This branch should be configured for CI workflows corresponding to try runs.
3234

33-
The two branches are currently needed because we cannot set `try-merge` to parent and merge it with a PR commit
35+
#### Auto builds
36+
- `automation/bors/auto-merge`
37+
- Used to merge PR with the latest base branch commit.
38+
- Should not be configured for any CI workflows!
39+
- `automation/bors/auto`
40+
- This branch should be configured for CI workflows that need to run before merging to the base branch.
41+
42+
The merge and non-merge branches are needed because we cannot set branches to parent and merge them with a PR commit
3443
atomically using the GitHub API.
3544

3645
### GitHub app
@@ -42,5 +51,7 @@ Here is a guide on how to add a repository so that this bot can be used on it:
4251
describes the file can be found in `src/config.rs`. [Here](rust-bors.example.toml) is an example configuration file.
4352
2) Install the GitHub app corresponding to this bot to the corresponding repository. You can use the
4453
`https://github.com/settings/apps/<app-name>/installations` link (to be automated via `team` repo).
45-
3) Configure a CI workflow on push to the `automation/bors/try` branch.
46-
4) Give the bot permissions to push to `automation/bors/try` and `automation/bors/try-merge` (to be automated via `team` repo).
54+
3) Configure CI workflows on push to:
55+
- `automation/bors/try` branch (for try builds)
56+
- `automation/bors/auto` branch (for auto builds)
57+
4) Give the bot permissions to push to `automation/bors/try`, `automation/bors/try-merge`, `automation/bors/auto`, and `automation/bors/auto-merge` (to be automated via `team` repo).

‎docs/design.md

Lines changed: 84 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -113,6 +113,90 @@ the branch to parent, and then again after merging the PR commit).
113113

114114
Note that `automation/bors/try-merge` should not have any CI workflows configured! These should be configured for the `automation/bors/try` branch instead.
115115

116+
## Auto builds
117+
The merge queue is an automated system that processes approved pull requests and merges them into the base branch after
118+
ensuring they pass all CI checks. PRs are approved using the `@bors r+` command and then "queued" automatically.
119+
120+
Here is a sequence diagram that describes what happens when a PR is approved and enters the merge queue:
121+
122+
```text
123+
+-----+ +-------+ +-----+ +-----+ +---------+
124+
| PR | | bors | | GHA | | DB | | teamAPI |
125+
+-----+ +-------+ +-----+ +-----+ +---------+
126+
| | | | |
127+
| @bors r+ (via webhook) | | | |
128+
|---------------------------------->| | | |
129+
| | | | |
130+
| | check user permissions | | |
131+
| |------------------------------------------------------>|
132+
| | | | |
133+
| | store approval in DB | | |
134+
| |-------------------------------------------->| |
135+
| | | | |
136+
| comment: "Commit abc123 approved" | | | |
137+
|<----------------------------------| | | |
138+
| | | | |
139+
| | (every 30s: process merge queue) | | |
140+
| | | | |
141+
| | merge PR with base to `auto-merge` | | |
142+
| |------------------------------------>| | |
143+
| | | | |
144+
| | push merge commit to `auto` | | |
145+
| |------------------------------------>| | |
146+
| | | | |
147+
| | create check run on PR head | | |
148+
| |------------------------------------>| | |
149+
| | | | |
150+
| | store auto build | | |
151+
| |-------------------------------------------->| |
152+
| | | | |
153+
| comment: ":hourglass: Testing.." | | | |
154+
|<----------------------------------| | | |
155+
| | | | |
156+
| | workflow started | | |
157+
| |<------------------------------------| | |
158+
| | | | |
159+
| | store workflow in DB | | |
160+
| |-------------------------------------------->| |
161+
| | | | |
162+
| | workflow completed | | |
163+
| |<------------------------------------| | |
164+
| | | | |
165+
| | update workflow status | | |
166+
| |-------------------------------------------->| |
167+
| | | | |
168+
| | query check suites for commit | | |
169+
| |------------------------------------>| | |
170+
| | | | |
171+
| | (wait for all check suites) | | |
172+
| | | | |
173+
| | check suite completed | | |
174+
| |<------------------------------------| | |
175+
| | | | |
176+
| | update build status | | |
177+
| |-------------------------------------------->| |
178+
| | | | |
179+
| | update check run on PR head | | |
180+
| |------------------------------------>| | |
181+
| | | | |
182+
| | fast-forward base branch | | |
183+
| |------------------------------------>| | |
184+
| | | | |
185+
| comment: ":sunny: Test successful"| | | |
186+
|<----------------------------------| | | |
187+
| | | | |
188+
```
189+
190+
The merge queue first merges the PR with the latest base branch commit in `automation/bors/auto-merge`, then pushes
191+
this merged commit to `automation/bors/auto` where CI tests run. If all tests pass, the base branch is fast-forwarded
192+
to the merge commit.
193+
194+
Only one auto build runs at a time to ensure that each PR is tested against the same branch state it will be merged into,
195+
preventing the problem where two PRs pass tests independently but fail when combined.
196+
197+
Note that `automation/bors/auto-merge` should not have any CI workflows configured! These should be configured for the
198+
`automation/bors/auto` branch instead.
199+
116200
## Recognizing that CI has succeeded/failed
117201
With [homu](https://github.com/rust-lang/homu) (the old bors implementation), GitHub actions CI running repositories had
118202
to use a "fake" job that marked the whole CI workflow as succeeded or failed, to signal to bors if it should consider

‎rust-bors.example.toml

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,11 @@
22
# (Required)
33
timeout = 3600
44

5+
# Whether to enable the merge queue or not.
6+
# When enabled, approved PRs will be automatically merged.
7+
# (Optional, defaults to false)
8+
merge_queue_enabled = true
9+
510
# Labels that should be set on a PR after an event happens.
611
# "+<label>" adds the label, while "-<label>" removes the label after the event.
712
# Supported events:

‎src/bin/bors.rs

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -32,6 +32,9 @@ const MERGEABILITY_STATUS_INTERVAL: Duration = Duration::from_secs(60 * 10);
3232
/// How often should the bot synchronize PR state.
3333
const PR_STATE_PERIODIC_REFRESH: Duration = Duration::from_secs(60 * 10);
3434

35+
/// How often should the bot process the merge queue.
36+
const MERGE_QUEUE_INTERVAL: Duration = Duration::from_secs(30);
37+
3538
#[derive(clap::Parser)]
3639
struct Opts {
3740
/// Github App ID.
@@ -160,6 +163,7 @@ fn try_main(opts: Opts) -> anyhow::Result<()> {
160163
let mut cancel_builds_refresh = make_interval(CANCEL_TIMED_OUT_BUILDS_INTERVAL);
161164
let mut mergeability_status_refresh = make_interval(MERGEABILITY_STATUS_INTERVAL);
162165
let mut prs_interval = make_interval(PR_STATE_PERIODIC_REFRESH);
166+
let mut merge_queue_interval = make_interval(MERGE_QUEUE_INTERVAL);
163167
loop {
164168
tokio::select! {
165169
_ = config_refresh.tick() => {
@@ -177,6 +181,9 @@ fn try_main(opts: Opts) -> anyhow::Result<()> {
177181
_ = prs_interval.tick() => {
178182
refresh_tx.send(BorsGlobalEvent::RefreshPullRequestState).await?;
179183
}
184+
_ = merge_queue_interval.tick() => {
185+
refresh_tx.send(BorsGlobalEvent::ProcessMergeQueue).await?;
186+
}
180187
}
181188
}
182189
};

‎src/bors/event.rs

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -75,6 +75,8 @@ pub enum BorsGlobalEvent {
7575
RefreshPullRequestMergeability,
7676
/// Periodic event that serves for synchronizing PR state.
7777
RefreshPullRequestState,
78+
/// Process the merge queue.
79+
ProcessMergeQueue,
7880
}
7981

8082
#[derive(Debug)]

0 commit comments

Comments
 (0)