Update the release process section

Author: rvermeulen

docs/development_handbook.md

@@ -35,6 +35,7 @@
 | 0.26.0  | 2022-08-10 | Remco Vermeulen  | Address incorrect package file generation command. This was missing the required language argument.                                                                                                                                                                                       |
 | 0.27.0  | 2022-11-08 | Luke Cartey | Update the versions of C we intend to support to exclude C90, which reflects the intended scope at the outset of the project.                                                                                                                                                                                       |
 | 0.28.0  | 2023-08-14 | Luke Cartey | Remove references to LGTM which is now a legacy product.  |
+| 0.29.0 | 2023-10-11 | Remco Vermeulen | Update release process. |
 
 ## Scope of work
 
@@ -515,9 +516,29 @@ To upgrade the CodeQL external dependencies:
 
 ### Release process
 
-#### Version Numbering 
+The release process is a combination of release specific Action workflows and validation Action workflows executed on each PR.
+The flowchart below provides an overview of the release process and how the release specific Action workflows are related.
+
+```mermaid
+flowchart TD;
+  prepare-release["Prepare release (prepare-release.yml)"]
+  validate-release["Validate release (validate-release.yml)"]
+  compiler-validation["Compiler tests (release-engineering/release-compiler-validation.yml.)"]
+  performance-testing["Performance testing (release-engineering/release-performance-testing.yml)"]
+  existing-checks["Existing checks run on each PR"]
+  update-release["Update release (update-release.yml)"]
+  finalize-release["Finalize release (finalize-release.yml)"]
+
+  prepare-release-->validate-release
+  validate-release-->compiler-validation-->update-release
+  validate-release-->performance-testing-->update-release
+  prepare-release-->existing-checks-->update-release
+  update-release-->finalize-release
+```
+
+#### Version Numbering
 
-Version numbers follow semantic versioning and adhere to the following guidelines specific to Coding Standards. 
+Version numbers follow semantic versioning and adhere to the following guidelines specific to Coding Standards.
 
 Given the version `<MAJOR>.<MINOR>.<PATCH>`:
 
@@ -531,57 +552,46 @@ We use the "Releases" feature in GitHub to manage and track our releases. This p
 
 To simplify the process of generating the release information, the repository contains a number of scripts and Action workflows:
 
- - [`generate_release_notes.py`](../scripts/release/generate_release_notes.py) - a script for generating release notes based on the contents of the repository in comparison to the previous release.
- - [`create_draft_release.sh`](../scripts/release/create_draft_release.sh) - a script for creating a release by:
-    1. Downloading the appropriate artifacts
-    2. Generating the release notes by calling `generate_release_notes.py` with appropriate parameters
-    3. Generating the list of supported rules 
-    4. Creating a draft release on GitHub containing the artifacts from the previous steps
-    5. Triggering integration testing on the new release.
- - [`create-draft-release.yml`](../.github/workflows/create-draft-release.yml) - a GitHub Actions workflow for running the `create_draft_release.sh` on demand within the CI/CD environment.
+- [prepare-release.yml](./github/workflows/prepare-release.yml): The entry point for starting a new release. When provided with a version and a Git reference this workflow will
+  - Create a release branch.
+  - Create a release PR that will contain all the changes required for a release and will validate the release using checks.
+  - Create a draft release that will be updated during various stages of the release.
+- [update-release.yml](./github/workflows/update-release.yml): This workflow will update the draft release when all checks have passed successfully on the release PR. The draft release is updated to:
+  - Have the most recent release notes as generated by the [update-release-notes.py](scripts/release/update-release-notes.py) script.
+  - Have the most recent release assets as generated by the [update-release-assets.py](scripts/release/update-release-assets.py).
+- [finalize-release.yml](.github/workflows/finalize-release.yml): This will update the release tag and mark the release public when the release PR is merged to successfully conclude the release.
+- [update-release-status.yml](.github/workflows/update-release-status.yml): This workflow will update the status on the release by monitoring the status of individual validation steps. When all succeeded this will invoke the `update-release.yml` workflow.
+- [update-check-run.yml](.github/workflows/update-check-run.yml): Utility workflow that allow authorized external workflows (i.e., workflows in other repositories) to update the status of check runs in the coding standards repository.
+- [validate-release.yml](.github/workflows/validate-release.yml): Utility workflow that will start the performance and compiler compatibility testing that are orchestrated from the codeql-coding-standards-release-engineering repository.
 
 #### Branching workflow
 
-Each new major or minor release should have a dedicated release branch, with the name `rc/<major>.<minor>`. A new patch version should re-use the existing release branch for the release that is being patched.
+Each release should have a dedicated release branch, with the name `rc/<major>.<minor>.<patch>`. A new patch version should branch from the existing release branch for the release that is being patched.
 Ensure that the same release branch is created in the [codeql-coding-standards-help](https://github.com/github/codeql-coding-standards-help) repository.
 
-#### Artifact creation
+#### Release assets
 
-There is an automated CI/CD job ([Code Scanning Query Pack Generation](../.github/workflows/code-scanning-pack-gen.yml)) provided that generates the following release artifacts for Coding Standards:
+There is an automated CI/CD job ([Update Release](../.github/workflows/update-release.yml)) that will automatically generate the release assets according to the [release layout specification](scripts/release/release-layout.yml).
+Among the assets are:
 
-  - Code Scanning query pack - generates a zipped folder that can be used with the CodeQL CLI directly, or with GitHub Advanced Security.
+- Certification kit containing the proof obligations for ISO26262 certification.
+- Code Scanning query packs that can be used with the CodeQL CLI directly, or with GitHub Advanced Security.
 
 **Use of Code Scanning within GitHub Advanced Security is not in scope for ISO 26262 tool qualification. See [user_manual.md#github-advanced-security](user_manual.md#github-advanced-security) for more information**.
 
-These run on every push to `main` and `rc/*`, and on every pull request, and are releasable without modification, assuming all other status checks succeed on the same commit.
-
 #### Creating a release
 
 To create a new release:
- 1. Create an internal "release checklist" issue.
- 2. Determine the appropriate release version. Version numbers are generated
+
+ 1. Determine the appropriate release version. Version numbers are generated
     according to the guidelines in the section "Version Numbering."
- 3. If a new `MAJOR` version is necessary, create a new `rc/<MAJOR>.0` branch off of `main`. Otherwise, reuse the existing `rc` branch and merge work from `main` into the `rc` branch you have selected.
- 4. Ensure the same `rc` branch exists in the [codeql-coding-standards-help](https://github.com/github/codeql-coding-standards-help) repository. This branch will be used to include external help files.
- 5. Submit a PR to update the `qlpack.yml` version numbers on the `main` branch to the next anticipated release.
- 6. Submit a PR to update the `qlpack.yml` version numbers on the release branch to the new version.
- 7. Trigger a [workflow dispatch event](https://docs.github.com/en/actions/managing-workflow-runs/manually-running-a-workflow) for the [Create draft release](../.github/workflows/create-draft-release.yml) workflow, specifying the release branch. The output of this workflow should report a link to the draft release and a link to the integration testing workflow triggered for this release.
-    - In the event the workflow is unusable, the [`create_draft_release.sh`](../scripts/release/create_draft_release.sh) script can be run directly on a local machine.
- 8. Run the following workflows with the new version number, e.g., `v2.0.0`:
-    - [Test Linux/x86_64](https://github.com/github/codeql-coding-standards-release-engineering/actions/workflows/test-release-performance-linux-x86_64.yml)   
-    - [Test Windows/x86_64](https://github.com/github/codeql-coding-standards-release-engineering/actions/workflows/test-release-performance-windows-x86_64.yml)
-    - [Regenerate Performance Views](https://github.com/github/codeql-coding-standards-release-engineering/actions/workflows/regenerate-performance-views.yml)
- 9. Confirm the integration testing workflow completes successfully, and that the execution time is comparable to previous releases, taking into account that the execution time is expected to increase proportionally as more queries are added for each release. Results may be viewed on the release engineering repo: https://github.com/github/codeql-coding-standards-release-engineering
- 10. For release 1.0.0 and above, the integration testing results must be verified. For each "integration testing codebase":
-    - Download the SARIF result file
-    - Compare the results against the previously computed set of results for that integration testing codebase, and, for any new or changed results, spot check to confirm validity.
-    - For false positives and false negatives identified during this process issues should be opened on this repository to track the problems identified.
-    - For each issue opened, assess whether they are "significant" i.e. whether they are likely to cause problems in practice with customers. If so, consider Step 7. failed.
- 11. If the release fails steps 7. or 8. (if applicable), retain the draft release, and rename it to `vminor.major.patch-rc<count>`. Address the release blocking issues on the `rc/<major>.<minor>` branch, and restart the release process at Step 7.
- 12. If steps 7. and 8. (if applicable) succeeded, then the release can be marked as "published".
- 13. Release artifacts can now be distributed to customers.
- 14. Create an internal "release retrospective" issue, and document any pain points or other issues.
- 15. Create a PR that merges the release candidate branch into `main`.
+ 2. Determine the appropriate [Git reference](https://git-scm.com/book/en/v2/Git-Internals-Git-References) to base the new release on. For new major or minor releases, this will be `main`. For patch releases this will be the release branch that is patched.
+ 3. Trigger a [workflow dispatch event](https://docs.github.com/en/actions/managing-workflow-runs/manually-running-a-workflow) for the [Prepare CodeQL Coding Standards release](../.github/workflows/prepare-release.yml) workflow, specifying the release version for the input `version` and the Git reference for the input `ref`.
+ 4. Merge the PR that is created for the release, named `Release v<major>.<minor>.<patch>` where `<major>`, `<minor>`, and `<patch>` match with the input `version` of the workflow  [Prepare CodeQL Coding Standards release](../.github/workflows/prepare-release.yml) triggered in the previous step.
+
+The release automation consists of many test and validation steps that can fail. These can be addressed and the release can be restarted from step 3.
+A restart of a release **WILL RECREATE THE EXISTING RELEASE BRANCH AND RELEASE PR**. Any additional changes added to the PR **MUST** be reapplied.
+If a release has been marked public, the release can no longer be restarted or re-released without removing the release manually.
 
 ## False Positive Triage Rubric