TELCODOCS-1690 - support node deletion via gitops ztp

Author: slovern

edge_computing/ztp-advanced-install-ztp.adoc

@@ -11,3 +11,5 @@ You can use `SiteConfig` custom resources (CRs) to deploy custom functionality a
 include::modules/ztp-customizing-the-install-extra-manifests.adoc[leveloffset=+1]
 
 include::modules/ztp-filtering-ai-crs-using-siteconfig.adoc[leveloffset=+1]
+
+include::modules/ztp-deleting-node-using-siteconfig.adoc[leveloffset=+1]

modules/ztp-deleting-node-using-siteconfig.adoc

@@ -0,0 +1,94 @@
+// Module included in the following assemblies:
+//
+// * edge_computing/ztp-advanced-install-ztp.adoc
+
+:_mod-docs-content-type: PROCEDURE
+[id="ztp-deleting-node-siteconfig_{context}"]
+= Deleting a node by using the SiteConfig CR
+
+By using a `SiteConfig` custom resource (CR), you can delete and reprovision a node. 
+This method is more efficient than manually deleting the node.
+
+.Prerequisites
+
+* You have configured the hub cluster to generate the required installation and policy CRs.
+
+* You have created a Git repository in which you can manage your custom site configuration data. The repository must be accessible from the hub cluster and be defined as the source repository for the Argo CD application.
+
+
+.Procedure
+
+. Update the  `SiteConfig` CR to include the `bmac.agent-install.openshift.io/remove-agent-and-node-on-delete=true` annotation:
++
+[source,yaml]
+----
+apiVersion: ran.openshift.io/v1
+kind: SiteConfig
+metadata:
+  name: "cnfdf20"
+  namespace: "cnfdf20"
+spec:
+  Clusters:    
+    nodes:
+    - hostname: node6
+      role: "worker"
+      crAnnotations:
+        add:
+          BareMetalHost:
+            bmac.agent-install.openshift.io/remove-agent-and-node-on-delete: true
+# ...
+----
+
+. Suppress the generation of the `BareMetalHost` CR by updating the `SiteConfig` CR to include the `crSuppression.BareMetalHost` annotation:
++
+[source,yaml]
+----
+apiVersion: ran.openshift.io/v1
+kind: SiteConfig
+metadata:
+  name: "cnfdf20"
+  namespace: "cnfdf20"
+spec:
+  clusters:
+  - nodes:
+    - hostName: node6
+      role: "worker"
+      crSuppression:
+      - BareMetalHost
+# ...
+----
+
+. Push the changes to the Git repository and wait for deprovisioning to start. 
+The status of the `BareMetalHost` CR should change to `deprovisioning`. Wait for the `BareMetalHost` to finish deprovisioning, and be fully deleted.
+
+.Verification
+
+. Verify that the `BareMetalHost` and `Agent` CRs for the worker node have been deleted from the hub cluster by running the following commands:
++
+[source,terminal]
+----
+$ oc get bmh -n <cluster-ns>
+----
++
+[source,terminal]
+----
+$ oc get agent -n <cluster-ns>
+----
+
+. Verify that the node record has been deleted from the spoke cluster by running the following command:
++
+[source,terminal]
+----
+$ oc get nodes
+----
++
+[NOTE]
+====
+If you are working with secrets, deleting a secret too early can cause an issue because ArgoCD needs the secret to complete resynchronization after deletion. 
+Delete the secret only after the node cleanup, when the current ArgoCD synchronization is complete.
+====
+
+.Next Steps
+
+To reprovision a node, delete the changes previously added to the `SiteConfig`, push the changes to the Git repository, and wait for the synchronization to complete. 
+This regenerates the `BareMetalHost` CR of the worker node and triggers the re-install of the node.