Strategy for major version bumps in new collections

[azeey]

From @scpeters:

In previous release cycles, we would merge forward from release branches to main and then bump the major version of libraries that would have significant changes in the next release. I think it should be easy enough to merge forward from Jetty branches to main since there shouldn't be major conflicts, but we should think about our version bumping strategy.

Bumping versions is much easier now since we've removed version numbers from package names. The only significant place where we still have lots of version numbers is in our packaging metadata (debian release repositories / package names, homebrew formula names). So here are some options for version bumping:

1. Versions are cheap, just bump everything
2. Bump things whenever we make a breaking change

[azeey]

I don't have a strong opinion yet, but I wanted to write down what how I'm thinking so far:

I like the idea of using main branches across all our libraries so we can:

1. Have a consistent development guideline for core developers as well as community contributors: Work on main, backport to other branches. Contributors don't have to figure out which branch to target their PRs.
2. Utilize nightlies to speed up development on features that affect multiple libraries instead of making prereleases.
3. (From Michael C.) Working on main will allow us to implement features in the most optimal way without worrying about making breaking changes. Only backports will include the less optimal changes to make it work without breaking API/ABI.

When we're talking about bumping versions, I think there are two phases:

1. Increase the major version of a library without updating any downstream libraries.

   • The work here would still involve making a new -release repo, updating gazebodistro and homebrew. However, this version is not yet considered to be part of the next release
   • This step was done as a pre/post release step in the past and I believe we've done it every year, regardless of whether a library has breaking changes or not.
     • Examples:
       1. Garden feature freeze task: Bump all main branches to the next major version (X.0.0~pre1) in Release Garden #578
       • setup for harmonic
       1. For Ionic, we knew we were bumping everything in the release, so we didn't do this as part of the Harmonic post release: See Post-release section in: Release Harmonic #974
       2. Ionic post release task: Bump all main branches to the next major version in Release Ionic #1092

2. When there is a breaking change that is required by a downstream library, we update the dependency version.

   • Example: gz-msgs was bumped to 10 in Harmonic in the middle of the release cycle (March): Bump in Harmonic: gz-msgs and gz-transport #878

Costs of bumping all libraries now:

I think we should continue doing phase 1 of the two phases I mentioned above. This would ease the burden of a full major version bump when a breaking change is necessary. So, when thinking about what the cost of bumping all libraries now would be, I'm not going to count phase 1.

For phase 2, I will use #878 as an example. For each library, we'd need to do the following:

• Source code of all downstream libraries (example)
  • This step will not be necessary or should be very trivial once we have the alias packages. If the downstream package requires the new changes, it would have to add a VERSION argument to gz_find_package
• Release repositories of all downstream libraries (example)
  • This step will not be necessary or should be very trivial once we have the alias packages.
• Use main branch on gazebodistro (example)
• Build nightlies from the main branch (example)
• homebrew-simulation: create formula and update dependencies (example)
• docs (the collection’s page) (example)

Other than the costs of phase 2, other costs include:

• More libraries to make prereleases and releases of when making the final releases. This includes Gz/ros vendor packages.
• More CI resources since each new major will need it's own CI job
• Backports will need to be made to more branches.
• More releases to make throughout the year after backports.

[iche033]

+1 on doing phase 1 immediately following a release. We should bump the major version in the main branches, regardless of whether it'll go into Gazebo K or not. We can safely create PRs with breaking changes targeting main without later on being delayed or trying to figure out whether it's been bumped already or not.

As for selecting what goes into Gazebo K, I'm leaning towards bumping (switching to use the main branch of a particular gz package) only when a breaking change is needed. So as of now Gazebo K will have the exact same set of packages as Jetty, which is fine. It's very likely that we'll bump versions due to breaking changes in K, e.g. zenoh related work may require us to bump gz-transport.

[scpeters]

+1 on doing phase 1 immediately following a release. We should bump the major version in the main branches, regardless of whether it'll go into Gazebo K or not. We can safely create PRs with breaking changes targeting main without later on being delayed or trying to figure out whether it's been bumped already or not.

I think there is a downside to merging breaking changes into package branches that aren't being tested as part of a coherent collection. This bit me when I eventually bumped the gz-msgs major version after some deprecated functionality had been removed in gazebosim/gz-msgs#476 that was still being used in gz-transport. I had to revert the gz-msgs PR gazebosim/gz-msgs#512, fix gz-transport in gazebosim/gz-transport#647, and then restore the gz-msgs change in gazebosim/gz-msgs#520. If that main branch was part of a collection with nightlies being built, we would have seen failing CI jobs within 24 hours and asked the original PR author to fix the issue.

So I guess I'd like that any branches where we are targeting pull requests should have CI running regularly that tests that package along with its dependent packages. One way to accomplish this would be to bump all main branches together and create the needed -release repositories (a "rolling" Gazebo collection). This does frontload the work, but I think having coherent collection-level CI for all our branches is worth it.

Regarding reducing excess version bumps: if after a year there have been no breaking changes merged to a main branch, such that the only difference between main and the current stable branch are the major version and metadata in packages.apt, then it may be possible to change metadata to point to the stable release instead of main.