Bevy train release schedule

[alice-i-cecile]

In today's discussion in general-dev, the community raised the following concerns with the existing release strategy:

• important and interesting features that are not the main focus of the current release are hard to test, and blocked on the larger project
• users are left waiting on PR'ed or even merged bug fixes for months
• users cannot reasonably upgrade to main to get access to new fixes faster, as plugin interoperability is a nightmare unless you are willing to constantly patch every single dependency
• there is a large amount of uncertainty about when the next release will be finished
• large features are pushed out the door in order to ensure we can finally cut a release, solving the above pain points
• a huge review / release note burden is placed solely on @cart's shoulders at the end of the cycle

There were several proposed solutions, which are largely complementary:

1. release minor versions
   • reduces pain of broken bug fixes
   • increased administrative overhead; we may need to pay a contributor to take on this responsibility in a regular fashion
   • desynchronization between the latest minor version and main become a concern if the time between major releases is extremely long, making it harder to back-port bug fixes
   • value of bug fixes found in minor versions is reduced significantly with shorter, more predictable releases
2. swap to a train release schedule like Rust
   • we pick a regularly recurring calendar date, and on that day everything that is merged at that point is shipped
   • this will require work on large features to occur in a feature branch, as done with pipelined-rendering
   • Cart spends some regularly scheduled fraction of his time reviewing and merging work, perhaps at the start of each release cycle. The rest of the time can be focused on leading the current major feature.
   • Rust uses a 6 week cycle, but this is too fast for migration burdens, overhead and publicity reasons
   • 3 months is a conservative start
3. develop better processes for releases and delegate work to make the process less painful
   • patch notes and migration guides are currently a pain to assemble, and often error-prone
   • social media release is needed with every(?) release to promote Bevy
4. more merge / review authority is delegated to teams of subject matter experts in mature domains
   • ECS and rendering are the only current candidates for this
   • reduces the risk that mildly controversial improvements to these domains are blocked, and reduces workload on Cart
5. PRs are required to have a changelog entry before being merged
   • could modify a changelog file (merge conflicts might be painful)
   • could attach to the PR description and parse (only the author can edit PR description)
6. move migration guide to main Bevy repo
   • PRs that require migration guide must update it before being approved or merged
   • on release, take a snapshot of it, move to website and reset to template
7. use labelled commits as rallying points for plugin compatibility
   • avoids pain of semver
   • reduces compatibility issues caused by long release cycles
   • very opaque and somewhat convoluted
8. use labelled commits as release-candidates for plugin authors to get a head start on migration
   • select a commit a week before scheduled release
   • creates a small rallying point
   • ensures we can recover plugin interop before release
   • breaks up migration work (many plugins should be working on day 1)
   • more focused and serious testing by plugin authors and end users

[alice-i-cecile]

1. Release minor versions

[djeedai]

What does "value is reduced significantly with shorter, more predictable releases" mean? Value of what?

[colepoirier]

Follow on question, what does “descynchronization issues become a concern if the time between major releases is extremely long” mean?

[IceSentry]

It essentially means that the need for minor releases is less present when there's a consistent major release in a somewhat short period of time. If bevy released only once a year, having multiple minor release to fix things would be useful, but if bevy releases every month, then having a minor release to fix something that will be part of the bigger release 2 weeks later is not as important

[alice-i-cecile]

What does "value is reduced significantly with shorter, more predictable releases" mean? Value of what?

The value of doing minor versions is lower when we have shorter and more predictable releases. If we are releasing on a 2 or 3 month cadence already, getting a minor version every month matters a lot less than it would in the current state of things.

[alice-i-cecile]

descynchronization issues become a concern if the time between major releases is extremely long

If we wait too long between releases while releasing minor versions, there will be a large number of meaningfully differing commits. This makes it harder and harder to figure out which bug-fixes are safe to port.

Imagine the situation with derive(Component). Suddenly, every single new bug fix has to be checked and modified to ensure it works both with and without the new syntax.

[alice-i-cecile]

2. Train release

[mockersf]

Bevy is more subject to breaking changes than other projects like Rust or Chromium. Rust release train for breaking changes is 3 years. There is a worry that releasing new versions with breaking change more often will increase the burden of the community to keep up. Wether that's compensated by the smaller number of changes is hard to measure. 3 months release train seems a good start to check how that would work for Bevy

[colepoirier]

@mockersf IMO 3 months sounds like far too long, I was thinking every 2 weeks. I think every three months is hardly better than the current situation.

[IceSentry]

@colepoirier 2 weeks is definitely too short. Releases are still handled by cart for now and he still has to work on other things. Releasing every 2 weeks means he would pretty much only ever work on releasing and accepting PRs to be released.

I also think 3 months is on the longer side, but I would still prefer a guaranteed release every 3 months than what we have now. If it goes well, there's nothing stopping us from lowering that to 2 months, but 2 weeks is definitely too short.

[Nilirad]

I think we could directly try a 2 months period. Not too short, and not too long.

[djeedai]

Data point: due to some bugs in 0.5 ECS I've been trying now 3 times to consume Bevy main directly in my project. After hours of fighting with third-party support crates (like bevy_kira_audio which is quite unavoidable) dependency hell, cloning 5+ repos, adding Cargo [patch] sections and explicit dependency path=/git= in my Cargo.toml, so far I've been unable to find a config that works, even with help from Discord. I'm not a Rust expert, but I believe I have an average understanding of how to use Cargo. This means that I expect many users like me to be completely stuck if they hit a bug fixed on main but not released, so with 2-3 months between releases that means being stuck for 2-3 months, which is a deal breaker.

[alice-i-cecile]

3. Better release processes

[mockersf]

I don't believe in automated patch notes or migration guides, and I do think that having one person(or a few) in charge of that is a good thing. This could rotate for each release between members/volunteers
Social media posts - those could be replaced by a "this month in Bevy" that would go well with a 3 months release train

[inodentry]

As I replied under point 5, I don't think such things conflict with what you are saying about "having a cohesive presentation in the end". Before release, things can be edited and improved by a human.

[alice-i-cecile]

4. Delegate more review and merge authority

[djeedai]

Data point: as of today there are 50 PRs ready for final review. It feels a bit too high and hints at the fact the current organization probably doesn't work anymore with the popularity the project has gained.

[inodentry]

Right now is not a good time to judge this. Official dev time is currently focused 100% towards finalizing the new renderer and getting the 0.6 release out. Any PRs that are not part of that are temporarily being ignored. After 0.6, Bevy will start doing train releases. Let's wait and see how the project will manage then, before we can come to such conclusions. 🙂

[alice-i-cecile]

5. PRs must have a changelog entry

[ickk]

which, as jcornaz mentioned, is excellent because reviewers can change a pr title and so can enforce the convention on bors' commit message

[alice-i-cecile]

which, as jcornaz mentioned, is excellent because reviewers can change a pr title and so can enforce the convention on bors' commit message

Interesting, how can we do this? This is one of my big objections to moving to a enforced commit style, so it would be excellent to allow folks without merge rights the ability to tweak PR titles.

[jcornaz]

You can enforce the PR titles to follow the convention with https://github.com/zeke/semantic-pull-requests, or https://github.com/amannn/action-semantic-pull-request (for example, there are others similar tools available).

Of course it only make sure it adheres to the convention. A human should still make sure it is correct and understandable. People with write access to the repository can edit the PR titles (I don't know exactly how github permission works. But I frequently edit PR titles on my repos).

IMO, if opting for a convention, the "conventional-commit" convention has the advantage that many tools already support it. I personally like the semantic-pr-title check is one of them. I also personally like release-please

[jcornaz]

I think the github Triage role is enought to be able to edit PR title. We probably need to look more into github roles, and try it out. see: https://docs.github.com/en/enterprise-cloud@latest/organizations/managing-peoples-access-to-your-organization-with-roles/managing-custom-repository-roles-for-an-organization.

[jcornaz]

In the worse case, if it is really not possible or to complex to allow reviewers to tweak the PR title themselves, the reviewers may kindly ask the PR author to update the title. There no reason why we could ask for code change, but not for PR title change.

[alice-i-cecile]

6. Move migration guide to main Bevy repo

[mockersf]

as I mentioned in points 3 and 5, I don't think this would be a good idea

[inodentry]

I think this is a very good idea, because it can reduce the effort needed for humans to prepare the final version.
See: #2972 (reply in thread)

[alice-i-cecile]

7. Rallying point commits

[jcornaz]

I don't understand this proposition. When I tag a commit to make it a "rallying point" for consumers of my library, I call that a "release".

What's the point to define such "raylling point commits" instead of releasing?

[alice-i-cecile]

Right now, relying on third party plugins for anything other than the latest release is very hard, because each one selects a different commit to depend on when tracking main.

[jcornaz]

Right now, relying on third party plugins for anything other than the latest release is very hard, because each one selects a different commit to depend on when tracking main.

Yep definitely. Hence the need for more frequent releases.

I just don't get this particular point ("Rallying point commits"). For me a "version" is exactly that: "a rallying point" in the history of the software that every plugin author can use. So I don't understand how and why creating "rallying points commits" would be more helpful than releasing.

[alice-i-cecile]

It would be less helpful than frequent releases, but also lower overhead. Overall my opinion now is that a train release schedule is more useful, but a 2 week warning release candidate may be nice.

[alice-i-cecile]

8. Release candidate commits

[IceSentry]

1 week seems a bit short if we expect plugin authors to update their plugins in time.

[Nilirad]

Rather than a release candidate commit, I would prefer avoiding merging breaking changes n days before release day. That way, plugin maintainers can still sync to main and be fairly certain that the plugin will be compatible with the next release, so they can publish the new plugin version right after Bevy.

[alice-i-cecile]

1 week seems a bit short if we expect plugin authors to update their plugins in time.

Point taken. 2 weeks may be more sensible.

Rather than a release candidate commit, I would prefer avoiding merging breaking changes n days before release day. That way, plugin maintainers can still sync to main and be fairly certain that the plugin will be compatible with the next release, so they can publish the new plugin version right after Bevy.

This is also good, but because of how cargo works, won't let users / crates coordinate. This is particularly brutal for crates that have complex Bevy dependency trees, as they would need to manually patch all of their dependencies to get something that compiles.

[inodentry]

I think an important consideration here is: how long before the bevy release is bevy main going to be in an effectively "frozen" state for?

Because if we bring this further back (say, 2 weeks) to give plugin authors more time ... what if more breaking changes are accepted in that time? It kinda defeats the point, as plugin authors would have to continue to track that and update their stuff until the actual release anyway. So, it's a compromise ... because I don't think bevy should stop accepting contributions / breaking changes that far in advance.

That's why 1 week seems more sensible to me. It's much better than not having such a "release candidate" period at all, and that's also more-or-less the same time that Cart would be spending to prepare the release announcements and blogs anyway.

[NiklasEi]

I think we can do both, continue accepting breaking changes until a short period before the release and give plugin authors enough time to update. If we embrace release candidates there is nothing stopping us from releasing more than one. Maybe don't do 10, but two or three over two or three weeks would be fine in my opinion.

This would have the benefit that we could react to issues getting raised by plugin authors when using the first Bevy RC. We could merge a breaking change to fix a raised issue and then release Bevy RC nr. 2. This time it's probably a lot easier for plugin authors to update because the changeset should be minimal. Plugin authors could also make their own release candidates so that we can have sync points in the Bevy ecosystem and test the Bevy RCs in projects using multiple plugins.