mirror:support version

[dormanze]

Community Contribution License

All community contributions in this pull request are licensed to the project maintainers
under the terms of the Apache 2 license.
By creating this pull request I represent that I have the right to license the
contributions to the project maintainers under the Apache 2 license.

Description

1.Supports version during bucket-to-bucket mirroring.
2.Check whether objects of other versions exist before remove.

Motivation and Context

fix #5139

How to test this PR?

case1:
1.Creating Buckets and Enabling Versioning Control

mc mb myminio/bucket --with-versioning
mc mb other/bucket --with-versioning

2.Start the mirror process.

mc mirror --overwrite --remove --watch myminio/bucket other/bucket

3.Upload the object to the bucket twice.

mc cp test.txt myminio/bucket/
mc cp test.txt myminio/bucket/

4.Check whether the versions of test.txt at the local and peer ends are consistent.

mc ls myminio/bucket/test.txt --versions
mc ls other/bucket/test.txt --versions

5.Delete an object based on the version number.

mc rm myminio/bucket/test.txt --version-id xxxxxx

Check whether the corresponding version is deleted from the peer site.

case2:
1.Creating a Bucket and Enabling Versioning Control

mc mb myminio/bucket --with-versioning

2.Start the mirror process.

mc mirror --overwrite --remove --watch myminio/bucket /local/

3.Upload the object to the bucket twice.

mc cp test.txt myminio/bucket/
mc cp test.txt myminio/bucket/

4.Delete an object based on the version number.

mc rm myminio/bucket/test.txt --version-id xxxxxx

Check whether local files are deleted.

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Optimization (provides speedup with no functional changes)
• Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

• Fixes a regression (If yes, please add commit-id or PR # here)
• Unit tests added/updated
• Internal documentation updated
• Create a documentation update request here

[klauspost]

It seems to me that this would be a problem if mirroring between 2 versioned S3 sites, preventing non-current versions to be deleted.

I can see the idea with this, but I'm not sure if this is the solution.

[dormanze]

It seems to me that this would be a problem if mirroring between 2 versioned S3 sites, preventing non-current versions to be deleted.

I can see the idea with this, but I'm not sure if this is the solution.

Yes, I am considering this point, I am working on the version control capability between two s3 sites, the scope of the modification is relatively large, still in verification.

[dormanze]

@klauspost I've finished my pr. Please take time to review it.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed after 21 days if no further activity occurs. Thank you for your contributions.

[oh2fih]

I think the documentation for this change needs some revision to clearly explain the difference between operations with and without the --with-versioning flag.

[oh2fih]

I haven't tested these changes since I don't have a testing environment for both of the use cases this supports, but it looks good and it seems dormanze has already tested this in February.