Skip to content

Commit 8f41e07

Browse files
camilamacedo86vitorfloriano
camilamacedo86
and
vitorfloriano
committed
Proposal: Automating maintenance via tooling and actions
Co-authored-by: Vitor Floriano <vitorfloriano@users.noreply.github.com>
1 parent 06a047f commit 8f41e07

File tree

1 file changed

+382
-0
lines changed

1 file changed

+382
-0
lines changed

‎designs/update_action.md

Lines changed: 382 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,382 @@
1+
| Authors | Creation Date | Status | Extra |
2+
|-----------------|---------------|-------------|-------|
3+
| @camilamacedo86 | 2024-11-07 | Implementable | - |
4+
| @vitorfloriano | | Implementable | - |
5+
6+
# Proposal: Automating Operator Maintenance: Driving Better Results with Less Overhead
7+
8+
## Introduction
9+
10+
Code-generation tools like **Kubebuilder** and **Operator-SDK** have revolutionized cloud-native application development by providing scalable, community-driven frameworks. These tools simplify complexity, accelerate development, and enable developers to create tailored solutions while avoiding common pitfalls, establishing a strong foundation for innovation.
11+
12+
However, as these tools evolve to keep up with ecosystem changes and new features, projects risk becoming outdated. Manual updates are time-consuming, error-prone, and create challenges in maintaining security, adopting advancements, and staying aligned with modern standards.
13+
14+
This project proposes an **automated solution for Kubebuilder**, with potential applications for similar tools or those built on its foundation. By streamlining maintenance, projects remain modern, secure, and adaptable, fostering growth and innovation across the ecosystem. The automation lets developers focus on what matters most: **building great solutions**.
15+
16+
17+
## Problem Statement
18+
19+
Kubebuilder is widely used for developing Kubernetes operators, providing a standardized scaffold. However, as the ecosystem evolves, keeping projects up-to-date presents challenges due to:
20+
21+
- **Manual re-scaffolding processes**: These are time-intensive and error-prone.
22+
- **Increased risk of outdated configurations**: Leads to security vulnerabilities and incompatibility with modern practices.
23+
24+
## Proposed Solution
25+
26+
This proposal introduces a **workflow-based tool** (such as a GitHub Action) that automates updates for Kubebuilder projects. Whenever a new version of Kubebuilder is released, the tool initiates a workflow that:
27+
28+
1. **Detects the new release**.
29+
2. **Generates an updated scaffold**.
30+
3. **Performs a three-way merge to retain customizations**.
31+
4. **Creates a pull request (PR) summarizing the updates** for review and merging.
32+
33+
## Example Usage
34+
35+
### GitHub Actions Workflow:
36+
37+
1. A user creates a project with Kubebuilder `v4.4.3`.
38+
2. When Kubebuilder `v4.5.0` is released, a **pull request** is automatically created.
39+
3. The PR includes scaffold updates while preserving the user’s customizations, allowing easy review and merging.
40+
41+
### Local Tool Usage:
42+
43+
1. A user creates a project with Kubebuilder `v4.4.3`
44+
2. When Kubebuilder `v4.5.0` is released, they run `kubebuilder alpha update` which calls `kubebuilder alpha generate` behind the scenes
45+
3. The tool updates the scaffold and preserves customizations for review and application.
46+
4. In case of conflicts, the tool allows users to resolve them before push a pull request with the changes.
47+
48+
### Handling Merge Conflicts
49+
50+
**Local Tool Usage**:
51+
52+
If conflicts cannot be resolved automatically, developers can manually address
53+
them before completing the update.
54+
55+
**GitHub Actions Workflow**:
56+
57+
If conflicts arise during the merge, the action will create a pull request and
58+
the conflicst will be highlighted in the PR. Developers can then review and resolve
59+
them. The PR will contains the default markers:
60+
61+
**Example**
62+
63+
```go
64+
<<<<<<< HEAD
65+
_ = logf.FromContext(ctx)
66+
=======
67+
log := log.FromContext(ctx)
68+
>>>>>>> original
69+
```
70+
71+
## Open Questions
72+
73+
### 1. Do we need to create branches to perform the three-way merge,or can we use local temporary directories?
74+
75+
> While temporary directories are sufficient for simple three-way merges, branches are better suited for complex scenarios.
76+
> They provide history tracking, support collaboration, integrate with CI/CD workflows, and offer more advanced
77+
> conflict resolution through Git’s merge command. For these reasons, it seems more appropriate to use branches to ensure
78+
> flexibility and maintainability in the merging process.
79+
80+
> Furthermore, branches allows a better resolution strategy,
81+
> since allows us use kubebuilder alpha generate to-rescaffold the projects
82+
> using the same name directory and provide a better hsitory for the PRs
83+
> allowing users to see the changes and have better insights for conflicts
84+
> resolution.
85+
86+
### 2. What Git configuration options can facilitate the three-way merge?
87+
88+
Several Git configuration options can improve the three-way merge process:
89+
90+
```bash
91+
# Show all three versions (base, current, and updated) during conflicts
92+
git config --global merge.conflictStyle diff3
93+
94+
# Enable "reuse recorded resolution" to remember and reuse previous conflict resolutions
95+
git config --global rerere.enabled true
96+
97+
# Increase the rename detection limit to better handle renamed or moved files
98+
git config --global merge.renameLimit 999999
99+
```
100+
101+
These configurations enhance the merging process by improving conflict visibility,
102+
reusing resolutions, and providing better file handling, making three-way
103+
merges more efficient and developer-friendly.
104+
105+
### 3. If we change Git configurations, can we isolate these changes to avoid affecting the local developer environment when the tool runs locally?
106+
107+
It seems that changes can be made using the `-c` flag, which applies the
108+
configuration only for the duration of a specific Git command. This ensures
109+
that the local developer environment remains unaffected.
110+
111+
For example:
112+
113+
```
114+
git -c merge.conflictStyle=diff3 -c rerere.enabled=true merge
115+
```
116+
117+
### 4. How can we minimize and resolve conflicts effectively during merges?
118+
119+
- **Enable Git Features:**
120+
- Use `git config --global rerere.enabled true` to reuse previous conflict resolutions.
121+
- Configure custom merge drivers for specific file types (e.g., `git config --global merge.&lt;driver&gt;.name "Custom Merge Driver"`).
122+
123+
- **Encourage Standardization:**
124+
- Adopt a standardized scaffold layout to minimize divergence and reduce conflicts.
125+
126+
- **Apply Frequent Updates:**
127+
- Regularly update projects to avoid significant drift between the scaffold and customizations.
128+
129+
These strategies help minimize conflicts and simplify their resolution during merges.
130+
131+
### 5. How to create the PR with the changes for projects that are monorepos?
132+
That means the result of Kubebuilder is not defined in the root dir and might be in other paths.
133+
134+
We can define an `--output` directory and a configuration for the GitHub Action where users will define where in their repo the path for the Kubebuilder project is. However, this might be out of scope for the initial version.
135+
136+
### 6. How could AI help us solve conflicts? Are there any available solutions?
137+
138+
While AI tools like GitHub Copilot can assist in code generation and provide suggestions,
139+
however, it might be risky be 100% dependent on AI for conflict resolution, especially in complex scenarios.
140+
Therefore, we might want to use AI as a complementary tool rather than a primary solution.
141+
142+
AI can help by:
143+
- Providing suggestions for resolving conflicts based on context.
144+
- Analyzing code patterns to suggest potential resolutions.
145+
- Offering explanations for conflicts and suggesting best practices.
146+
- Assisting in summarizing changes.
147+
148+
## Summary
149+
150+
### Workflow Example:
151+
152+
1. A developer creates a project with Kubebuilder `v4.4`.
153+
2. The tooling uses the release of Kubebuilder `v4.5`.
154+
3. The tool:
155+
- Regenerates the original base source code for `v4.4` using the `clientVersion` in the `PROJECT` file.
156+
- Generates the base source code for `v4.5`
157+
4. A three-way merge integrates the changes into the developer’s project while retaining custom code.
158+
5. The changes now can be packaged into a pull request, summarizing updates and conflicts for the developer’s review.
159+
160+
### Steps:
161+
162+
The proposed implementation involves the following steps:
163+
164+
1. **Version Tracking**:
165+
- Record the `clientVersion` (initial Kubebuilder version) in the `PROJECT` file.
166+
- Use this version as a baseline for updates.
167+
- Available in the `PROJECT` file, from [v4.6.0](https://github.com/kubernetes-sigs/kubebuilder/releases/tag/v4.6.0) release onwards.
168+
169+
2. **Scaffold Generation**:
170+
- Generate the **original scaffold** using the recorded version.
171+
- Generate the **updated scaffold** using the latest Kubebuilder release.
172+
173+
3. **Three-Way Merge**:
174+
- Ensure git is configured to handle three-way merges.
175+
- Merge the original scaffold, updated scaffold, and the user’s customized project.
176+
- Preserve custom code during the merge.
177+
178+
4. **(For Actions) - Pull Request Creation**:
179+
- Open a pull request summarizing changes, including details on conflict resolution.
180+
- Schedule updates weekly or provide an on-demand option.
181+
182+
#### Example Workflow
183+
184+
The following example code illustrates the proposed idea but has not been evaluated.
185+
This is an early, incomplete draft intended to demonstrate the approach and basic concept.
186+
187+
We may want to develop a dedicated command-line tool, such as `kubebuilder alpha update`,
188+
to handle tasks like downloading binaries, merging, and updating the scaffold. In this approach,
189+
the GitHub Action would simply invoke this tool to manage the update process and open the
190+
Pull Request, rather than performing each step directly within the Action itself.
191+
192+
```yaml
193+
name: Update Kubebuilder Scaffold
194+
195+
on:
196+
workflow_dispatch:
197+
schedule:
198+
- cron: '0 0 * * 0'
199+
200+
jobs:
201+
update-scaffold:
202+
runs-on: ubuntu-latest
203+
204+
steps:
205+
- name: Check out the repository
206+
uses: actions/checkout@v4
207+
with:
208+
fetch-depth: 0
209+
210+
- name: Set up environment and dependencies
211+
run: |
212+
sudo apt-get update
213+
sudo apt-get install -y curl git jq make
214+
215+
- name: Install latest Kubebuilder CLI
216+
run: |
217+
LATEST_VERSION=$(curl -s https://api.github.com/repos/kubernetes-sigs/kubebuilder/releases/latest | jq -r .tag_name)
218+
curl -L https://github.com/kubernetes-sigs/kubebuilder/releases/download/${LATEST_VERSION}/kubebuilder_${LATEST_VERSION}_linux_amd64.tar.gz -o kubebuilder.tar.gz
219+
tar -zxvf kubebuilder.tar.gz
220+
sudo mv kubebuilder /usr/local/kubebuilder
221+
echo "/usr/local/kubebuilder/bin" >> $GITHUB_PATH
222+
223+
- name: Run Kubebuilder update if cliVersion is outdated
224+
run: |
225+
CURRENT_VERSION=$(grep "cliVersion" PROJECT | awk '{print $2}' | tr -d '"')
226+
LATEST_VERSION=$(curl -s https://api.github.com/repos/kubernetes-sigs/kubebuilder/releases/latest | jq -r .tag_name)
227+
228+
echo "Current Kubebuilder version: $CURRENT_VERSION"
229+
echo "Latest Kubebuilder release: $LATEST_VERSION"
230+
231+
if [ "$CURRENT_VERSION" != "$LATEST_VERSION" ]; then
232+
echo "cliVersion is outdated. Running kubebuilder alpha update..."
233+
kubebuilder alpha update
234+
else
235+
echo "cliVersion is already up-to-date. Skipping update."
236+
echo "SKIP_UPDATE=true" >> $GITHUB_ENV
237+
fi
238+
239+
- name: Check for skipped update
240+
if: env.SKIP_UPDATE == 'true'
241+
run: echo "Skipping PR creation because no update was needed."
242+
243+
- name: Set up Git user for commit
244+
if: env.SKIP_UPDATE != 'true'
245+
run: |
246+
git config user.name "github-actions"
247+
git config user.email "github-actions@github.com"
248+
249+
- name: Push merge branch
250+
if: env.SKIP_UPDATE != 'true'
251+
run: |
252+
git checkout merge
253+
git push -u origin merge
254+
255+
- name: Create Pull Request from merge branch
256+
if: env.SKIP_UPDATE != 'true'
257+
uses: peter-evans/create-pull-request@v6
258+
with:
259+
commit-message: "Update Kubebuilder scaffold"
260+
title: "Update Kubebuilder to ${LATEST_VERSION}"
261+
body: |
262+
This pull request updates the project scaffold to the latest
263+
Kubebuilder release, preserving customizations.
264+
265+
Note that this PR may contain merge conflicts that need to be resolved.
266+
branch: merge
267+
```
268+
269+
## Motivation
270+
271+
A significant challenge faced by Kubebuilder users is keeping their projects up-to-date with the latest
272+
scaffolds while preserving customizations. The manual processes required for updates are time-consuming,
273+
error-prone, and often discourage users from adopting new versions, leading to outdated and insecure projects.
274+
275+
The primary motivation for this proposal is to simplify and automate the process of maintaining Kubebuilder
276+
projects. By providing a streamlined workflow for updates, this solution ensures that users can keep
277+
their projects aligned with modern standards while retaining their customizations.
278+
279+
### Goals
280+
281+
- **Automate Updates**: Detect and apply scaffold updates while preserving customizations.
282+
- **Simplify Updates**: Generate pull requests for easy review and merging.
283+
- **Provide Local Tooling**: Allow developers to run updates locally with preserved customizations.
284+
- **Keep Projects Current**: Ensure alignment with the latest scaffold improvements.
285+
- **Minimize Disruptions**: Enable scheduled or on-demand updates.
286+
287+
### Non-Goals
288+
289+
- **Automating conflict resolution for heavily customized projects**.
290+
- **Automatically merging updates without developer review**.
291+
- **In Phase 1, supporting monorepo project layouts or handling repositories that contain more than just the Kubebuilder-generated code**.
292+
293+
## Proposal
294+
295+
### User Stories
296+
297+
- **As a Kubebuilder maintainer**, I want to help users keep their projects updated with minimal effort, ensuring they adhere to best practices and maintain alignment with project standards.
298+
- **As a user of Kubebuilder**, I want my project to stay up-to-date with the latest scaffold best practices while preserving customizations.
299+
- **As a user of Kubebuilder**, I want an easy way to apply updates across multiple repositories, saving time on manual updates.
300+
- **As a user of Kubebuilder**, I want to ensure my codebases remain secure and maintainable without excessive manual effort.
301+
302+
### Implementation Details/Notes/Constraints
303+
304+
- Introduce a new [Kubebuilder Plugin](https://book.kubebuilder.io/plugins/plugins) that scaffolds the
305+
**GitHub Action** based on the POC. This plugin will be released as an **alpha feature**,
306+
allowing users to opt-in for automated updates.
307+
308+
- The plugin should be added by default in the Golang projects build with Kubebuilder, so new
309+
projects can benefit from the automated updates without additional configuration. While it will not be escaffolded
310+
by default in tools which extend Kubebuilder such as the Operator-SDK, where the alpha generate and update
311+
features cannot be ported or extended.
312+
313+
- Documentation should be provided to guide users on how to enable and use the new plugin as the new alpha command
314+
315+
- The alpha command update should
316+
- provide help and examples of usage
317+
- allow users to specify the version of Kubebuilder they want to update to or from to
318+
- allow users to specify the path of the project they want to update
319+
- allow users to specify the output directory where the updated scaffold should be generated
320+
- re-use the existing `kubebuilder alpha generate` command to generate the updated scaffold
321+
322+
- The `kubebuilder alpha update` command should be covered with e2e tests to ensure it works as expected
323+
and that the generated scaffold is valid and can be built.
324+
325+
## Risks and Mitigations
326+
- **Risk**: Frequent conflicts may make the process cumbersome.
327+
- *Mitigation*: Provide clear conflict summaries and leverage GitHub preview tools.
328+
- **Risk**: High maintenance overhead.
329+
- *Mitigation*: Build a dedicated command-line tool (`kubebuilder alpha update`) to streamline updates and minimize complexity.
330+
331+
## Proof of Concept
332+
333+
The feasibility of re-scaffolding projects has been demonstrated by the
334+
`kubebuilder alpha generate` command.
335+
336+
**Command Example:**
337+
338+
```bash
339+
kubebuilder alpha generate
340+
```
341+
342+
For more details, refer to the [Alpha Generate Documentation](https://kubebuilder.io/reference/rescaffold).
343+
344+
This command allows users to manually re-scaffold a project, to allow users add their code on top.
345+
It confirms the technical capability of regenerating and updating scaffolds effectively.
346+
347+
This proposal builds upon this foundation by automating the process. The proposed tool would extend this functionality
348+
to automatically update projects with new scaffold versions, preserving customizations.
349+
350+
The three-way merge approach is a common strategy for integrating changes from multiple sources.
351+
It is widely used in version control systems to combine changes from a common ancestor with two sets of modifications.
352+
In the context of this proposal, the three-way merge would combine the original scaffold, the updated scaffold, and the user’s custom code
353+
seems to be very promising.
354+
355+
### POC Implementation using 3-way merge:
356+
357+
Initial implementation of the three-way merge has been successfully tested using a custom script:
358+
https://github.com/vitorfloriano/multiversion/pull/1
359+
360+
Some further examples of the three-way merge and the proposed solution above
361+
in action with a script to re-scaffold a project can be found here:
362+
363+
- Without conflicts:
364+
https://github.com/camilamacedo86/wordpress-operator/pull/1
365+
366+
- With conflicts:
367+
https://github.com/camilamacedo86/test-operator/pull/1
368+
369+
## Drawbacks
370+
371+
- **Frequent Conflicts:** Automated updates may often result in conflicts, making the process cumbersome for users.
372+
- **Complex Resolutions:** If conflicts are hard to review and resolve, users may find the solution impractical.
373+
- **Maintenance Overhead:** The implementation could become too complex for maintainers to develop and support effectively.
374+
375+
## Alternatives
376+
377+
- **Manual Update Workflow**: Continue with manual updates where users regenerate
378+
and merge changes independently, though this is time-consuming and error-prone.
379+
- **Use alpha generate command**: Continue with partially automated updates provided
380+
by the alpha generate command.
381+
- **Dependabot Integration**: Leverage Dependabot for dependency updates, though this
382+
doesn’t fully support scaffold updates and could lead to incomplete upgrades.

0 commit comments

Comments
 (0)