Updates to EC disaster recovery + related KOTS snapshots docs updates #2916

paigecalvert · 2024-12-20T21:06:56Z

Updated the Embedded Cluster DR topic to explain the new steps to add the Backup and Restore resources: https://deploy-preview-2916--replicated-docs.netlify.app/vendor/embedded-disaster-recovery
Updated the table in the Velero Backup resource topic for Snapshots to better show how the fields are nested (per Ethan's request): https://deploy-preview-2916--replicated-docs.netlify.app/reference/custom-resource-backup
Also reorged the Snapshots docs so all the vendor and end user info for snapshots is grouped together in a single section under KOTS:

^ As part of this reorg, I also did the following:
* Made sure that the snapshots topics all included "Snapshots" in their titles to avoid confusion with the Embedded Cluster disaster recovery stuff if you are searching the docs
* Condensed the vendor snapshots overview and enterprise user snapshots overview topics into a single topic (we no longer need two separate topics for this now that it's all grouped in the same sidebar section)
Did some rewriting in the existing Configuring Snapshots topic for clarity. Mostly sentence-level changes as this topic hadn't been touched in awhile and the language was a little wordy/confusing in places (more details in the comments of the Files changes tab): https://deploy-preview-2916--replicated-docs.netlify.app/vendor/snapshots-configuring-backups

netlify · 2024-12-20T21:07:15Z

✅ Deploy Preview for replicated-docs ready!

Name	Link
🔨 Latest commit	`57060b8`
🔍 Latest deploy log	https://app.netlify.com/sites/replicated-docs/deploys/6798eec27100a100084d2662
😎 Deploy Preview	https://deploy-preview-2916--replicated-docs.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

netlify · 2024-12-20T21:07:16Z

✅ Deploy Preview for replicated-docs-upgrade ready!

Name	Link
🔨 Latest commit	`57060b8`
🔍 Latest deploy log	https://app.netlify.com/sites/replicated-docs-upgrade/deploys/6798eec216951200084e9b5c
😎 Deploy Preview	https://deploy-preview-2916--replicated-docs-upgrade.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

paigecalvert · 2025-01-03T17:52:09Z

docs/enterprise/snapshots-troubleshooting-backup-restore.md

@@ -1,6 +1,6 @@
 import NodeAgentMemLimit from "../partials/snapshots/_node-agent-mem-limit.mdx"

-# Troubleshooting Backup and Restore
+# Troubleshooting Snapshots


^ Updated various topic titles so that it's easier to see at a glance which ones apply to snapshots and which ones apply to EC DR

paigecalvert · 2025-01-03T17:53:39Z

docs/reference/custom-resource-backup.md

@@ -98,74 +93,76 @@ The following Velero fields are supported for full backups, as shown in the prev
    <td>(Optional) Specifies the actions to perform at different times during a backup. The only supported hook is executing a command in a container in a pod (uses the pod exec API). Supports <code>pre</code> and <code>post</code> hooks.</td>
  </tr>
  <tr>
-    <td><code>resources</code></td>
+    <td><code>hooks.resources</code></td>


^ updated the table per Ethan's feedback that it was hard to see how the given fields are nested

paigecalvert · 2025-01-03T17:56:02Z

docs/vendor/embedded-disaster-recovery.mdx


-1. You must specify which Pod volumes you want backed up. This is done with the `backup.velero.io/backup-volumes` annotation. For more information, see [File System Backup](https://velero.io/docs/v1.14/file-system-backup/) in the Velero documentation.
+1. In a new release containing your application files, add a Velero Backup resource. In the Backup resource, use namespace-based or label-based selection to indicate the application resources that you want to be included in the backup. For more information, see [Backup API Type](https://velero.io/docs/latest/api-types/backup/) in the Velero documentation.


^ clarify that both namespaces and labels can be used

paigecalvert · 2025-01-03T17:57:46Z

docs/vendor/embedded-disaster-recovery.mdx

+       # the name of the Backup resource that you added
+       backupName: backup
+       includedNamespaces:
+       - '*'


^ I also pulled this Restore resource from what Ethan had provided in the story.

Wasn't sure if the includedNamespaces field could be removed (according to the velero docs: "If unspecified, all namespaces are included.")

paigecalvert · 2025-01-03T18:00:22Z

docs/vendor/embedded-disaster-recovery.mdx

@@ -96,11 +124,17 @@ To enable disaster recovery for a customer:

   When your customer installs with Embedded Cluster, Velero will be deployed if the **Allow Disaster Recovery** license field is enabled.

-## Configure Backup Storage and Take Backups in the Admin Console
+## Take Backups and Restore


^ added this new h2 to nest the procedures for setting up backup storage and restoring from a backup. Felt nice to break up the topic into separate subsections that amount to: "configuring and enabling the feature" and "using the feature"

paigecalvert · 2025-01-03T18:01:29Z

sidebars.js

@@ -328,16 +328,6 @@ const sidebars = {
                'vendor/packaging-air-gap-excluding-minio',
              ],
            },
-            {


Edited the snapshots info in the sidebar so that both the vendor info about end user info are in the same section to make it all easier to find

paigecalvert · 2025-01-06T16:21:10Z

docs/vendor/embedded-disaster-recovery.mdx


 * The disaster recovery feature flag must be enabled for your account. To get access to disaster recovery, reach out to Alex Parker at [alexp@replicated.com](mailto:alexp@replicated.com).
-* Embedded Cluster version 1.4.1 or later
+* Embedded Cluster version **X.X.X** or later


^ is there a new version that we should put here now that this topic talks about the new "add a Restore resource" method?

paigecalvert · 2025-01-06T16:22:25Z

docs/vendor/embedded-disaster-recovery.mdx


 * If the `--admin-console-port` flag was used during install to change the port for the Admin Console, note that during a restore the Admin Console port will be used from the backup and cannot be changed. For more information, see [Embedded Cluster Install Command Options](/reference/embedded-cluster-install).

-## Configure Disaster Recovery for Your Application 
+## Configure Disaster Recovery


^ split this section into 2 subsections: configure the velero resources and then enable dr for customers

paigecalvert · 2025-01-06T18:18:28Z

docs/vendor/embedded-disaster-recovery.mdx

@@ -34,44 +36,68 @@ Embedded Cluster disaster recovery has the following limitations and known issue

   [View a larger version of this image](/images/ec-version-command.png)

-* You can only restore from the most recent backup.
+* Any Helm extensions included in the `extensions` field of the Embedded Cluster Config are _not_ included in backups. Helm extensions are reinstalled as part of the restore process. To include Helm extensions in backups, configure the Velero Backup resource to include the extensions using namespace-based or label-based selection. For more information, see [Configure the Velero Custom Resources](#config-velero-resources) below.


^ add limitation that extensions aren't included

Not sure if we want to include this part:

To include Helm extensions in backups, configure the Velero Backup resource to include the extensions using namespace-based or label-based selection.

paigecalvert · 2025-01-06T18:18:53Z

docs/vendor/embedded-disaster-recovery.mdx

+       orLabelSelectors:
+       - matchExpressions:
+         # Exclude Replicated resources from the backup
+         - { key: kots.io/kotsadm, operator: NotIn, values: ["true"] }


^ this is the example that Ethan had provided in the related eng story

paigecalvert · 2025-01-06T18:19:49Z

docs/vendor/embedded-disaster-recovery.mdx

-1. You must specify which Pod volumes you want backed up. This is done with the `backup.velero.io/backup-volumes` annotation. For more information, see [File System Backup](https://velero.io/docs/v1.14/file-system-backup/) in the Velero documentation.
+     :::important
+     If you use namespace-based selection to include all of your application resources deployed in the `kotsadm` namespace, ensure that you exclude the Replicated resources that are also deployed in the `kotsadm` namespace. Because the Embedded Cluster infrastructure components are always included in backups automatically, this avoids duplication.
+     ::: 


^ give people a heads up that they need to exclude replicated resource if they use namespace-based selection.

I put "duplication" as the reasoning here, but there might be a better way to explain it

paigecalvert · 2025-01-06T18:29:02Z

docs/vendor/embedded-disaster-recovery.mdx

-Disaster recovery for Embedded Cluster installations is implemented with Velero. For more information about Velero, see the [Velero](https://velero.io/docs/v1.14/) documentation.
+The backups that your customers take from the Admin Console will include both the Embedded Cluster infrastructure and the application resources that you specify.
+
+The Embedded Cluster infrastructure that is backed up includes components such as the KOTS Admin Console and the built-in registry that is deployed for air gap installations. No configuration is required to include Embedded Cluster infrastructure in backups. Vendors specify the application resources to include in backups by configuring a Velero Backup resource in the application release.


^ updated the overview to explain what's included in the backups

paigecalvert · 2025-01-06T18:31:57Z

docs/vendor/snapshots-overview.mdx


-# About Backing Up and Restoring with Snapshots
+# About Backup and Restore with Snapshots


^ note: previously, we had a vendor snapshots overview and an enterprise user snapshot overview as two different topics. I grouped that content together under this single overview topic:

docs/enterprise/snapshots-understanding.mdx + docs/vendor/snapshots-overview.md → docs/vendor/snapshots-overview.mdx

Everything here was copy and pasted from that existing content

paigecalvert · 2025-01-06T18:34:50Z

netlify.toml

+
+[[redirects]]
+  from="https://docs.replicated.com/enterprise/snapshots-understanding"
+  to="https://docs.replicated.com/vendor/snapshots-overview"    


^ redirect the deleted topic

paigecalvert · 2025-01-07T17:10:55Z

docs/vendor/snapshots-configuring-backups.md


-1. Configure backups for each volume that requires a backup. By default, no volumes are included in the backup. If any pods mount a volume that should be backed up, you must configure the backup with an annotation listing the specific volumes to include in the backup.


Configure backups for each volume that requires a backup.

In the rewrite, I removed this sentence because it doesn't seem to have meaning. Instead, I jumped right to saying how you need to add the annotation to volumes that must be backed up

paigecalvert · 2025-01-07T17:13:08Z

docs/vendor/snapshots-configuring-backups.md


-        ```
-    1. (Optional) Configure the resources annotation in the manifest so that it can be dynamically enabled based on a license field or a config option. For more information, see [Including Optional and Conditional Resources](packaging-include-resources/).


^ This might be something that vendor want to do, but as is, this step fell flat to me. It introduces a totally separate piece of information that doesn't really strictly have to do with the stated task at hand, which is configuring snapshots. I removed this step and figure vendors could go to the section about conditionally deploying resources when they feel like they want to do that

I think that's a good point

paigecalvert · 2025-01-07T17:14:43Z

docs/vendor/snapshots-configuring-backups.md

+   <details>
+    <summary>Why do I need to use the backup annotation?</summary>
+    <p>By default, no volumes are included in the backup. If any pods mount a volume that should be backed up, you must configure the backup with an annotation listing the specific volumes to include in the backup.</p>
+   </details>


^ moved the Why? info to a dropdown

paigecalvert · 2025-01-07T17:15:11Z

docs/vendor/snapshots-configuring-backups.md

^ changed this step from "Optional" to "kURL Only"

paigecalvert · 2025-01-07T17:15:33Z

docs/vendor/snapshots-configuring-backups.md


-:::note
-If you are using multiple applications, repeat this procedure for each application. Every application must have its own Backup resource to be included in a full backup.
-:::


^ in the rewrite, I moved this note down to the last step of the procedure

paigecalvert · 2025-01-07T17:16:11Z

docs/vendor/snapshots-configuring-backups.md

@@ -1,34 +1,41 @@
-# Configuring Backups
+# Configuring Snapshots


^ some rewrite to this topic for clarity

docs/reference/custom-resource-backup.md

ajp-io · 2025-01-27T20:10:22Z

docs/vendor/embedded-disaster-recovery.mdx


 * The disaster recovery feature flag must be enabled for your account. To get access to disaster recovery, reach out to Alex Parker at [alexp@replicated.com](mailto:alexp@replicated.com).
-* Embedded Cluster version 1.4.1 or later
+* Embedded Cluster version **X.X.X** or later


ajp-io · 2025-01-27T20:24:42Z

docs/vendor/snapshots-configuring-backups.md


-        ```
-    1. (Optional) Configure the resources annotation in the manifest so that it can be dynamically enabled based on a license field or a config option. For more information, see [Including Optional and Conditional Resources](packaging-include-resources/).


I think that's a good point

ajp-io · 2025-01-28T14:29:18Z

docs/vendor/embedded-disaster-recovery.mdx


-To configure disaster recovery for your application:
+### Configure the Velero Custom Resources {#config-velero-resources}


I noticed the KOTS snapshots docs mention a few things like adding annotations to volumes you want backed up, etc. I believe a lot of that is still applicable here, and while our advice here is more general ("just create the needed backup and restore resources"), we might to consider adding in those points to help guide people too.

We can merge as is, but we could follow up with Diamon to see if he agrees.

Since he onboarded Progress to DR, it might be good for him to read this stuff to see if he would add any more advice or steps in

Good call, I'll do that. (Merge then ask him to read through the DR steps. Might be easier to review the procedure that way compared to seeing all these inline changes)

docs/vendor/embedded-disaster-recovery.mdx

Co-authored-by: Alex Parker <7272359+ajp-io@users.noreply.github.com>

paigecalvert added 5 commits December 18, 2024 13:45

editing snapshots velero info

3f3f1ab

edits

09cce5f

edits

4a6c4e8

edits

b1133db

update overview content

5b541bd

replicated-ci added type::docs Improvements or additions to documentation type::feature labels Dec 20, 2024

paigecalvert added 3 commits December 24, 2024 10:42

update backup resource fields table

78fe558

edits

ec8769a

update Helm extension limitation

78db832

paigecalvert marked this pull request as ready for review January 2, 2025 15:42

paigecalvert requested a review from a team as a code owner January 2, 2025 15:42

paigecalvert added 5 commits January 2, 2025 11:26

edits

7e3aad2

edits

298f217

edits

48a0f0d

edit sidebar

e5bb214

edits

25dab66

paigecalvert commented Jan 3, 2025

View reviewed changes

paigecalvert added 3 commits January 3, 2025 11:04

remove enterprise snapshots understanding topic

7de769e

update xrefs

00835be

add anchor link to limitation

c7783d1

paigecalvert commented Jan 6, 2025

View reviewed changes

paigecalvert added 3 commits January 6, 2025 09:23

edits

a3b1066

fix links

5e2d9ef

copy edits

4bc84ad

paigecalvert commented Jan 6, 2025

View reviewed changes

edits and xref fixes

e4ae8c7

paigecalvert commented Jan 6, 2025

View reviewed changes

paigecalvert changed the title ~~Updates to EC disaster recovery~~ Updates to EC disaster recovery + related KOTS snapshots docs updates Jan 6, 2025

edits

a41395a

paigecalvert commented Jan 7, 2025

View reviewed changes

remove unneeded first step

76cbaa2

paigecalvert commented Jan 7, 2025

View reviewed changes

paigecalvert requested a review from ajp-io January 21, 2025 16:46

ajp-io reviewed Jan 28, 2025

View reviewed changes

paigecalvert commented Jan 28, 2025

View reviewed changes

docs/vendor/embedded-disaster-recovery.mdx Outdated Show resolved Hide resolved

Apply suggestions from code review

57060b8

Co-authored-by: Alex Parker <7272359+ajp-io@users.noreply.github.com>

paigecalvert merged commit fd6a135 into main Jan 28, 2025
5 checks passed

paigecalvert deleted the 117608 branch January 28, 2025 14:53


		1. You must specify which Pod volumes you want backed up. This is done with the `backup.velero.io/backup-volumes` annotation. For more information, see [File System Backup](https://velero.io/docs/v1.14/file-system-backup/) in the Velero documentation.
		1. In a new release containing your application files, add a Velero Backup resource. In the Backup resource, use namespace-based or label-based selection to indicate the application resources that you want to be included in the backup. For more information, see [Backup API Type](https://velero.io/docs/latest/api-types/backup/) in the Velero documentation.


		# About Backing Up and Restoring with Snapshots
		# About Backup and Restore with Snapshots


		1. Configure backups for each volume that requires a backup. By default, no volumes are included in the backup. If any pods mount a volume that should be backed up, you must configure the backup with an annotation listing the specific volumes to include in the backup.


		```
		1. (Optional) Configure the resources annotation in the manifest so that it can be dynamically enabled based on a license field or a config option. For more information, see [Including Optional and Conditional Resources](packaging-include-resources/).

		@@ -1,34 +1,41 @@
		# Configuring Backups
		# Configuring Snapshots


		To configure disaster recovery for your application:
		### Configure the Velero Custom Resources {#config-velero-resources}

Updates to EC disaster recovery + related KOTS snapshots docs updates #2916

Updates to EC disaster recovery + related KOTS snapshots docs updates #2916

Uh oh!

Conversation

paigecalvert commented Dec 20, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

netlify bot commented Dec 20, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Deploy Preview for replicated-docs ready!

Uh oh!

netlify bot commented Dec 20, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Deploy Preview for replicated-docs-upgrade ready!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

paigecalvert Jan 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

paigecalvert Jan 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

paigecalvert Jan 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

paigecalvert commented Dec 20, 2024 •

edited

Loading

netlify bot commented Dec 20, 2024 •

edited

Loading

netlify bot commented Dec 20, 2024 •

edited

Loading

paigecalvert Jan 3, 2025 •

edited

Loading

paigecalvert Jan 6, 2025 •

edited

Loading

paigecalvert Jan 6, 2025 •

edited

Loading