Upgrade CAPO version to v0.12.2 #152

noonedeadpunk · 2025-04-07T15:55:15Z

In CAPO version v0.11.2 there is a severe bug allowing to accomplish
Denial of Service by any tenant.

Manual removal of VM by tenant which is managed by CAPO results
in a pod crash in a loop. This has been fixed with [1] and is part
of the 0.12.2 release.

[1] kubernetes-sigs/cluster-api-provider-openstack#2477

noonedeadpunk · 2025-04-07T16:10:07Z

recheck

noonedeadpunk · 2025-04-07T16:10:23Z

OSError: [Errno 24] Too many open files - for linters seems unrelated

noonedeadpunk · 2025-04-07T16:56:31Z

So, in this CAPO version kind: Image is gone. So I'd guess that it also needs more modern CAPI or smth....

noonedeadpunk · 2025-04-07T18:45:03Z

Yeah, ok, it's not capi version, but missing ORC which was split into separate project. doh.

mnaser · 2025-04-07T19:07:55Z

@noonedeadpunk could we get away with bumping to latest 0.11.x which might have the fix?

noonedeadpunk · 2025-04-07T19:30:33Z

@mnaser this is the first I checked and unfortunately it's not there as of today. Probably could attempt backporting to 0.11, but I kinda not confident in stable policy in there :(

mnaser · 2025-04-07T19:32:01Z

Ah, the team is pretty flexible at backporting things especially if it's a crash. One moment.

noonedeadpunk · 2025-04-07T19:34:22Z

Oops, just realized I never added a fix, here it is: kubernetes-sigs/cluster-api-provider-openstack#2477

I'm also looking at what it would take to install ORC, as I'd guess sooner or later this needs to be done anyway.

mnaser · 2025-04-07T19:35:08Z

I pushed kubernetes-sigs/cluster-api-provider-openstack#2507

I'll ping folks for a review and hopefully we can get that landed, would still need a release :(

mnaser · 2025-04-07T19:35:56Z

I'm also looking at what it would take to install ORC, as I'd guess sooner or later this needs to be done anyway.

I think the best way to go about this is to go over the install instructions on a normal Kind cluster and then see how to "replicate" this into the playbook.

noonedeadpunk · 2025-04-07T19:43:04Z

fwiw, regarding rocky failures in molecule here: we've spotted same failures caused by apparmor blocking PAM inside of the docker with EL, when host is running Ubuntu 24.04. And become/or SSH.

With SSH workaround was to comment out UsePAM, but for become - we just dropped become from the role....

noonedeadpunk · 2025-04-07T20:04:56Z

Ok, so I was able to spawn a healthy cluster with this PR in:

~# openstack coe cluster show 1458e73e-2440-4aff-a57e-37d7acb46c2f -c created_at -c status -c health_status -c labels -c coe_version -c labels_added -c health_status_reason
+----------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Field                | Value                                                                                                                                                                                                             |
+----------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| status               | CREATE_COMPLETE                                                                                                                                                                                                   |
| health_status        | HEALTHY                                                                                                                                                                                                           |
| created_at           | 2025-04-07T19:46:37+00:00                                                                                                                                                                                         |
| coe_version          | v1.31.1                                                                                                                                                                                                           |
| labels               | {'cloud_provider_enabled': 'True', 'kube_tag': 'v1.31.1', 'calico_tag': 'v3.29.0', 'octavia_provider': 'amphorav2', 'octavia_lb_algorithm': 'SOURCE_IP_PORT', 'availability_zone': 'az1', 'auto_scaling_enabled': |
|                      | 'False', 'auto_healing_enabled': 'False', 'master_lb_floating_ip_enabled': 'True', 'kube_dashboard_enabled': 'True', 'ingress_controller': 'octavia'}                                                             |
| labels_added         | {'availability_zone': 'az1', 'auto_scaling_enabled': 'False', 'auto_healing_enabled': 'False', 'master_lb_floating_ip_enabled': 'True', 'kube_dashboard_enabled': 'True', 'ingress_controller': 'octavia'}        |
| health_status_reason | {'kube-pldql-default-worker-t4hhs-24zr6-59h7k.Ready': 'True', 'kube-pldql-default-worker-t4hhs-24zr6-7wqc2.Ready': 'True', 'kube-pldql-gx976-jgs24.Ready': 'True'}                                                |
+----------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

Though it adds new required variable: cluster_api_openstack_controller_version: 2.0.3

@mnaser With that I was wondering - how CHANGELOG.md is managed? Manually or automated from some fragments?

noonedeadpunk · 2025-04-07T20:19:28Z

The upgrade job seems validly broken :(

https://zuul.atmosphere.vexxhost.dev/build/5d0e70976ea648d9ab3dd9d548995378

roles/cluster_api/tasks/patch.yml

roles/cluster_api/tasks/main.yml

roles/cluster_api/files/controllers/openstack-resource-controller/v2.0.3/install.yaml

noonedeadpunk · 2025-04-14T09:40:49Z

Regarding ansible-test - gitlab does install requirements for /opt/hostedtoolcache/Python/3.10.16/x64 but then ansible-test units tries to execute through /usr/bin/python3.12

yaguangtang · 2025-04-23T01:34:19Z

@noonedeadpunk I have fixed the CI issue

noonedeadpunk · 2025-05-05T16:33:45Z

Would be really nice to get some reviews/progress on this one...

noonedeadpunk · 2025-07-10T08:15:52Z

Any updates?

In CAPO version v0.11.2 there is a severe bug allowing to accomplish Denial of Service by any tenant. Manual removal of VM by tenant which is managed by CAPO results in a pod crash in a loop. This has been fixed with [1] and is part of the 0.12.2 release. Signed-off-by: Dmitriy Rabotyagov <noonedeadpunk@gmail.com>

More modern CAPO also requires corresponding CAPI , otherwise VM creation fails with: `no matches for kind \"Image\" in version \"openstack.k-orc.cloud/v1alpha1\` Signed-off-by: Dmitriy Rabotyagov <noonedeadpunk@gmail.com>

CAPO 0.12.0 has removed ORC [1] and now it needs to be installed additionally. [1] https://github.com/kubernetes-sigs/cluster-api-provider-openstack/releases/tag/v0.12.0 Signed-off-by: Dmitriy Rabotyagov <noonedeadpunk@gmail.com>

Signed-off-by: Dmitriy Rabotyagov <noonedeadpunk@gmail.com>

* feat: allow set capo instance creation timeoput Signed-off-by: Tadas Sutkaitis <tadasas@gmail.com> * fix: license and rename variable Signed-off-by: Tadas Sutkaitis <tadasas@gmail.com> * fix: patch using native kubernetes module Signed-off-by: Tadas Sutkaitis <tadas.sutkaitis@vexxhost.com> --------- Signed-off-by: Tadas Sutkaitis <tadasas@gmail.com> Signed-off-by: Tadas Sutkaitis <tadas.sutkaitis@vexxhost.com> Signed-off-by: Dmitriy Rabotyagov <noonedeadpunk@gmail.com>

Signed-off-by: Dong Ma <dong.ma@vexxhost.com> Signed-off-by: Dmitriy Rabotyagov <noonedeadpunk@gmail.com>

noonedeadpunk · 2025-07-10T10:06:12Z

omfg... Adding DCO seemed to pull in quite some unrelated things with rebase... I have no idea how to resolve that in github tbh at this point...

noonedeadpunk · 2025-07-10T10:23:44Z

recheck - Error: etcdserver: request timed out

noonedeadpunk · 2025-07-10T10:24:11Z

recheck

noonedeadpunk · 2025-07-10T10:38:01Z

In favor of #165 due to DCO mess-up

noonedeadpunk marked this pull request as draft April 7, 2025 16:55

noonedeadpunk force-pushed the feature/capo_0.12.2 branch from 4cb46b0 to 489481c Compare April 7, 2025 20:09

mnaser requested changes Apr 8, 2025

View reviewed changes

noonedeadpunk force-pushed the feature/capo_0.12.2 branch from 489481c to 084a728 Compare April 14, 2025 09:34

noonedeadpunk force-pushed the feature/capo_0.12.2 branch from 084a728 to 3e1e1db Compare April 14, 2025 09:50

noonedeadpunk marked this pull request as ready for review April 14, 2025 09:51

noonedeadpunk requested a review from mnaser April 16, 2025 15:58

yaguangtang requested review from yaguangtang and okozachenko1203 May 6, 2025 02:15

noonedeadpunk and others added 4 commits July 10, 2025 12:02

Add manifests and clusterctl of 1.9.6

f0c495d

More modern CAPO also requires corresponding CAPI , otherwise VM creation fails with: `no matches for kind \"Image\" in version \"openstack.k-orc.cloud/v1alpha1\` Signed-off-by: Dmitriy Rabotyagov <noonedeadpunk@gmail.com>

Install ORC for CAPO >= 0.12.0

c476ea2

CAPO 0.12.0 has removed ORC [1] and now it needs to be installed additionally. [1] https://github.com/kubernetes-sigs/cluster-api-provider-openstack/releases/tag/v0.12.0 Signed-off-by: Dmitriy Rabotyagov <noonedeadpunk@gmail.com>

feat: allow disabling kube-vip

a5f158f

Signed-off-by: Dmitriy Rabotyagov <noonedeadpunk@gmail.com>

mnaser and others added 5 commits July 10, 2025 12:02

Add kube_vip_enabled

536adf2

Signed-off-by: Dmitriy Rabotyagov <noonedeadpunk@gmail.com>

Release 2.2.0

5d8c10c

Signed-off-by: Dmitriy Rabotyagov <noonedeadpunk@gmail.com>

fix ci job failure (vexxhost#154)

0cff7af

Signed-off-by: Dmitriy Rabotyagov <noonedeadpunk@gmail.com>

fix: Ensure dbus is installed on Debian to set hostname (vexxhost#163)

49b81cd

Signed-off-by: Dong Ma <dong.ma@vexxhost.com> Signed-off-by: Dmitriy Rabotyagov <noonedeadpunk@gmail.com>

noonedeadpunk force-pushed the feature/capo_0.12.2 branch from dddf49f to 49b81cd Compare July 10, 2025 10:02

Merge branch 'main' into feature/capo_0.12.2

e0a521f

noonedeadpunk closed this Jul 10, 2025

noonedeadpunk mentioned this pull request Jul 10, 2025

Upgrade CAPO version to v0.12.2 (v2) #165

Open

Upgrade CAPO version to v0.12.2 #152

Upgrade CAPO version to v0.12.2 #152

Uh oh!

Conversation

noonedeadpunk commented Apr 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

noonedeadpunk commented Apr 7, 2025

Uh oh!

noonedeadpunk commented Apr 7, 2025

Uh oh!

noonedeadpunk commented Apr 7, 2025

Uh oh!

noonedeadpunk commented Apr 7, 2025

Uh oh!

mnaser commented Apr 7, 2025

Uh oh!

noonedeadpunk commented Apr 7, 2025

Uh oh!

mnaser commented Apr 7, 2025

Uh oh!

noonedeadpunk commented Apr 7, 2025

Uh oh!

mnaser commented Apr 7, 2025

Uh oh!

mnaser commented Apr 7, 2025

Uh oh!

noonedeadpunk commented Apr 7, 2025

Uh oh!

noonedeadpunk commented Apr 7, 2025

Uh oh!

noonedeadpunk commented Apr 7, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

noonedeadpunk commented Apr 14, 2025

Uh oh!

yaguangtang commented Apr 23, 2025

Uh oh!

noonedeadpunk commented May 5, 2025

Uh oh!

noonedeadpunk commented Jul 10, 2025

Uh oh!

noonedeadpunk commented Jul 10, 2025

Uh oh!

noonedeadpunk commented Jul 10, 2025

Uh oh!

noonedeadpunk commented Jul 10, 2025

Uh oh!

noonedeadpunk commented Jul 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

noonedeadpunk commented Apr 7, 2025 •

edited

Loading

noonedeadpunk commented Jul 10, 2025 •

edited

Loading