Skip to content

Migrate backing up Plone docs #1919

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 22 commits into from
Apr 29, 2025
Merged
Changes from 1 commit
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
fea8c1e
migrate backup plone docs
Manas-Kenge Mar 19, 2025
68df52e
Merge branch '6.0' into migrate-backing-up-plone
Manas-Kenge Mar 20, 2025
7d5c32b
rephrase sentences
Manas-Kenge Mar 20, 2025
d728aa1
Merge branch '6.0' into migrate-backing-up-plone
Manas-Kenge Mar 28, 2025
68a4db8
Merge branch '6.0' into migrate-backing-up-plone
stevepiercy Apr 1, 2025
cc94877
Merge branch '6.0' into migrate-backing-up-plone
stevepiercy Apr 2, 2025
5669806
Merge branch '6.0' into migrate-backing-up-plone
stevepiercy Apr 2, 2025
5b49675
Merge branch '6.0' into migrate-backing-up-plone
rohnsha0 Apr 5, 2025
be2b03d
use one sentence per line
Manas-Kenge Apr 5, 2025
abc6ce5
Merge branch '6.0' into migrate-backing-up-plone
Manas-Kenge Apr 6, 2025
030180a
Merge branch '6.0' into migrate-backing-up-plone
Manas-Kenge Apr 9, 2025
fb9494c
Merge branch '6.0' into migrate-backing-up-plone
stevepiercy Apr 13, 2025
82e39a6
Remove todo and note
stevepiercy Mar 25, 2025
8dababc
Merge branch '6.0' into migrate-backing-up-plone
stevepiercy Apr 13, 2025
93da5f3
Merge remote-tracking branch 'Manas-Kenge/documentation/migrate-backi…
stevepiercy Apr 13, 2025
5d8fe94
- Complete review
stevepiercy Apr 13, 2025
6d6ba51
Merge branch '6.0' into migrate-backing-up-plone
Manas-Kenge Apr 24, 2025
51e59d5
Merge branch '6.0' into migrate-backing-up-plone
stevepiercy Apr 28, 2025
e85a07a
move content
Manas-Kenge Apr 28, 2025
47cc2f5
Merge branch '6.0' into migrate-backing-up-plone
Manas-Kenge Apr 28, 2025
efe2d4e
Update preparations.md
stevepiercy Apr 29, 2025
54e2735
Update docs/glossary.md
stevepiercy Apr 29, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
85 changes: 85 additions & 0 deletions docs/backend/upgrading/preparations.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,91 @@ See Plone 5.2 documentation, [Backing up your Plone deployment](https://5.docs.p
Migrate the Plone 5.2 docs for Backing up your Plone deployment into Plone 6 docs.
```

### Introduction

The key rules of backing up a working system are probably:

- Back up everything
- Maintain multiple generations of backup
- Test restoring your backups

```{note}
This guide assumes that you are already doing this for your system as a whole, and will only cover the considerations specific to Plone. When we say we are assuming you’re already doing this for the system as a whole, what we mean is that your system backup mechanisms - rsync, bacula, whatever - are already backing up the directories into which you’ve installed Plone.
```

Your buildout and buildout caches are already backed up, and you’ve tested the restore process.
Your remaining consideration is making sure that Plone’s database files are adequately backed up and recoverable.

### Objects in motion

Objects in motion tend to remain in motion. Objects that are in motion are difficult or impossible to back up accurately.

Translation: Plone is a long-lived process that is constantly changing its content database. The largest of these files, the Data.fs filestorage which contains everything except Binary Large OBjects (BLOBs), is always open for writing. The BLOB storage, a potentially complex file hierarchy, is constantly changing and must be referentially synchronized to the filestorage.

This means that most system backup schemes are incapable of making useful backups of the content database while it’s in use. We assume you don’t want to stop your Plone site to backup, so you need to add procedures to make sure you have useful backups of Plone’s data. (We assume that you know that the same thing is true of your relational database storage.)

### Where’s my data?

Your Plone instance installation will contain a `./var` directory (in the same directory as buildout.cfg) that contains the frequently changing data files for the instance. Much of what’s in `./var`, though, is not your actual content database. Rather, it’s log, process id, and socket files.

The directories that actually contain content data are:

#### Filestorage
`./var/filestorage`:

This is where Zope Object Database filestorage is maintained. Unless you’ve multiple storages or have changed the name, the key file is Data.fs. It’s typically a large file and contains everything except BLOBS.

The other files in filestorage, with extensions like .index, .lock, .old, .tmp are ephemeral, and will be recreated by Zope if they’re absent.

#### Blobstorage
`./var/blobstorage`:

This directory contains a deeply nested directory hierarchy that, in turn, contains the BLOBs of your database: PDFs, image files, office automation files and such.

The key thing to know about filestorage and blobstorage is that they are maintained synchronously. The filestorage has references to BLOBs in the blobstorage. If the two storages are not synchronized, you’ll get errors.

### collective.recipe.backup

[collective.recipe.backup](https://pypi.python.org/pypi/collective.recipe.backup) is a well-maintained and well-supported recipe for solving the “objects in motion” problem for a live Plone database. It makes it easy to both back up and restore the object database. The recipe is basically a sophisticated wrapper around `repozo`, a Zope database backup tool, and `rsync`, the common file synchronization tool.

If you’re using any of Plone’s installation kits, `collective.recipe.backup` is included in your install. If not, you may add it to your buildout by adding a `backup` part:

```
[buildout]
parts =
...
backup
...
[backup]
recipe = collective.recipe.backup
```

There are several useful option settings for the recipe, all set by adding configuration information. All are documented on [the PyPI page](https://pypi.python.org/pypi/collective.recipe.backup). Perhaps the most useful is the `location` option, which sets the destination for backup files:

```
[backup]
recipe = collective.recipe.backup
location = /path/to/reliably/attached/storage/filestorage
blobbackuplocation = /path/to/reliably/attached/storage/blobstorage
```

If this is unspecified, the backup destination is the buildout var directory. The backup destination, though, may be any reliably attached location - including another partition, drive or network storage.

### Operation

Once you’ve run buildout, you’ll have `bin/backup` and `bin/restore` scripts in your buildout. Since all options are set via buildout, there are few command-line options, and operation is generally as simple as using the bare commands. `bin/restore` will accept a date-time argument if you’re keeping multiple backups. See the docs for details.

Backup operations may be run without stopping Plone. Restore operations require that you stop Plone, then restart after the restore is complete.

`bin/backup` is commonly included in a cron table for regular operation. Make sure you test backup/restore before relying on it.

### Incremental backups

`collective.recipe.backup` offers both incremental and full backup and will maintain multiple generations of backups. Tune these to meet your needs.

When incremental backup is enabled, doing a database packing operation will automatically cause the next backup to be a full backup.

If your backup continuity needs are extreme, your incremental backup may be equally extreme. There are Plone installations where incremental backups are run every few minutes.

(upgrade-setup-a-test-environment-to-rehearse-the-upgrade-label)=

Expand Down