Skip to content

[Feature] scadctl genscenario: CLI Command to generate scaling scenarios #49

@elankath

Description

@elankath

How to categorize this issue?

/area auto-scaling|
/kind enhancement

What would you like to be added:
We need the ability to generate Scaling Scenario's from real-world cluster. For now, we propose suggesting only gardener clusters, but tool must be generic enough to support scaling scenario generation for any cluster manager (cluster-api, etc), not just gardener.

CLI Tool scadctl genscenario

NOTE: scadctl genscenario is an auxiliary command for the scaling-advisor ctl client scadctl that generates scaling advisor scenarios from gardener shoot clusters.

Code Placement
  1. Introduce cobra command genscenario inside tools/cmd/scenario/genscenario.go
  2. Register cobra command into root.
  3. gardener cluster manager scenario generation code should go into tools/cmd/scenario/gardener. scenario generation for other cluster managers in the future (cluster-api) can go into subpackages like: client/cmd/scenario/<cluster-manager>. Ex: client/cmd/scenario/clusterapi
Command Overview
  1. scadctl genscenario <cluster-manager> <cluster-manager-specific-options>
    1. For gardener clusters: scadctl genscenario gardener -l <landscape> -p <project> -t <shoot-name> -d <scenario-dir>. # capture cluster state and generates scenarios from live cluster
scadctl genscenario -m gardener -l <landscape> -p <project> -t <shoot-name> -d <scenario-dir>
  1. generates viewer kubeconfig for shoot data planea and control plane into OS tmp/full-qualified-clustername directory if not already generated.
    1. Delegate to gardenctl CLI tool for this or use HTTP request to generate kubeconfigs. Do not introduce gardener code dependency.
    2. Example of full-qualified-clustername -> dev:i034796:aw
  2. Connect to cluster and create a svcapi.ClusterSnapshot. (See Scaling Advisor Service API)
  3. Connect to control cluster, get the Worker object and then create the sacorev1alpha1.aClusterScalingConstraints (See Scaling Advisor Common Core API)
  4. (optional) It should obfuscate the node names and pod names and pool names. Use generated human names component.
  5. Create the scenario-dir if it does not exist.
  6. Persist the the sacorev1alpha1.ClusterScalingConstraints into scenario-dir/<full-qualified-cluster-name>/cluster-scaling-constraints.json.
  7. It should now create cluster snapshot variants:
    1. scenario-dir/clusterName/snapshot-ddMMYY-latest-incremental.json
      1. From the captured cluster snapshot. remove the latest created node , unassign the pods from this node. Rewrite the cluster snapshot
    2. scenario-dir/clusterName/snapshot-ddMMYY-latest-5.json
      1. From the captured cluster snapshot. remove the latest 5 created node , unassign the pods from this node. Rewrite the cluster snapshot
    3. (Redo the above for 5-10-20 nodes)
  8. The scenario-dir can be checked into github.tools.sap/kubernetes/gardener-scaling-tools/scenarios to allow other gardener members to reply and reproduce issues in scenarios via the scalebench tool described later. (If scenarios are too large, we can either compress and. check them in or we can attach them as github release artifacts instead of direct source-checkin)
/scenario-dir
   /<full-qualified-cluster-name>
	   /cluster-scaling-constraints.json
	   /snapshot-ddMMYY-latest-incremental.json
	   /snapshot-ddMMYY-latest-5.json
	   ...

Why is this needed:

We need to test scaling advisor (and other scalers) against actual scaling scenarios. For the gardener cluster-manager, we need the ability to adapt the gardener shoot worker object and the state of the shoot cluster into a svcapi.ScalingAdviceRequest that the scaling-advisor can be invoked with.

Metadata

Metadata

Labels

kind/enhancementEnhancement, improvement, extension

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions