mirror:not overwrit target object when mirror watch #5193

dormanze · 2025-04-23T02:52:45Z

Community Contribution License

All community contributions in this pull request are licensed to the project maintainers
under the terms of the Apache 2 license.
By creating this pull request I represent that I have the right to license the
contributions to the project maintainers under the Apache 2 license.

Description

Motivation and Context

make sure mirror command overwrit same when the watch is used and when the watch command is not used.

How to test this PR?

mc mb local/bucket
mc mb peer/bucket
echo "1" > txt
mc cp txt peer/bucket
mc mirror -w local/bucket peer/bucket
echo "2" > txt
mc cp txt local/bucket
mc cat peer/bucket/txt

mc mb local/bucket
mc mb peer/bucket
echo "3" > txt
mc cp txt peer/bucket
mc mirror -w --overwrite local/bucket peer/bucket
echo "4" > txt
mc cp txt local/bucket
mc cat peer/bucket/txt

Types of changes

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Optimization (provides speedup with no functional changes)
Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

Fixes a regression (If yes, please add commit-id or PR # here)
Unit tests added/updated
Internal documentation updated
Create a documentation update request here

not overwrit target object when mirror watch

sheiy · 2025-04-25T05:50:28Z

@jiuker pls check this pr thanks

jiuker · 2025-07-18T08:08:04Z

Let me test first. Will get back here

harshavardhana · 2025-07-23T20:37:01Z

Let me test first. Will get back here

@jiuker ^^?

jiuker · 2025-07-23T23:44:41Z

Sorry. I can't reprdouce it.

mc mb local/bucket
mc mb peer/bucket
echo "1" > txt
mc cp txt peer/bucket
mc mirror -w local/bucket peer/bucket
echo "2" > txt
mc cp txt local/bucket
mc cat peer/bucket/txt

And your pr is change around mj.opts.activeActive, is not related to --overwrite

restart mc mirror

mc: <ERROR> Failed to perform mirroring, with error condition (mm-source-mtime) Overwrite not allowed for `http://172.25.244.240:9001/bucket/txt`. Use `--overwrite` to override this behavior.

when mc running
do this shell

echo "5" > txt
mc cp txt minio9000/bucket
mc cat minio9001/bucket/txt
echo "6" > txt
mc cp txt minio9000/bucket
mc cat minio9001/bucket/txt
echo "7" > txt
mc cp txt minio9000/bucket
mc cat minio9001/bucket/txt

echo "8" > txt
# a new object to source
mc cp txt minio9000/bucket/txt2
mc cat minio9001/bucket/txt2


root@jiuker:/mnt/d/workspace/go/src/eos/stage# mc cat minio9001/bucket/txt
3
root@jiuker:/mnt/d/workspace/go/src/eos/stage# sh issue_mc_5193.sh 
/mnt/d/workspace/go/src/eos/stage/txt:  2 B / 2 B ┃▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓┃ 26 B/s 0s
3
/mnt/d/workspace/go/src/eos/stage/txt:  2 B / 2 B ┃▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓┃ 27 B/s 0s
3
/mnt/d/workspace/go/src/eos/stage/txt:  2 B / 2 B ┃▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓┃ 27 B/s 0s
3
/mnt/d/workspace/go/src/eos/stage/txt:  2 B / 2 B ┃▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓┃ 29 B/s 0s
root@jiuker:/mnt/d/workspace/go/src/eos/stage# mc cat minio9001/bucket/txt2
8
root@jiuker:/mnt/d/workspace/go/src/eos/stage#

Couldn't found any issues. Please share more info.

jiuker · 2025-07-23T23:57:52Z

cmd/mirror-main.go

@@ -418,21 +418,23 @@ func (mj *mirrorJob) doRemove(ctx context.Context, sURLs URLs, event EventInfo)
 // doMirror - Mirror an object to multiple destination. URLs status contains a copy of sURLs and error if any.
 func (mj *mirrorJob) doMirrorWatch(ctx context.Context, targetPath string, tgtSSE encrypt.ServerSide, sURLs URLs, event EventInfo) URLs {
 	shouldQueue := false
-	if !mj.opts.isOverwrite && !mj.opts.activeActive {
+	if !mj.opts.isOverwrite {


not related to --overwrite

jiuker · 2025-07-23T23:58:02Z

cmd/mirror-main.go

 	}
-	if shouldQueue || mj.opts.isOverwrite || mj.opts.activeActive {
+	if shouldQueue || mj.opts.isOverwrite {


not related to --overwrite

jiuker · 2025-07-23T23:58:06Z

cmd/mirror-main.go

 		targetClient, err := newClient(targetPath)
 		if err != nil {
 			// cannot create targetclient
 			return sURLs.WithError(err)
 		}
 		_, err = targetClient.Stat(ctx, StatOptions{sse: tgtSSE})
 		if err == nil {
-			if !sURLs.SourceContent.RetentionEnabled && !sURLs.SourceContent.LegalHoldEnabled {
+			if !mj.opts.activeActive && !sURLs.SourceContent.RetentionEnabled && !sURLs.SourceContent.LegalHoldEnabled {


not related to --overwrite

dormanze · 2025-07-24T03:14:52Z

Sorry. I can't reprdouce it.

mc mb local/bucket
mc mb peer/bucket
echo "1" > txt
mc cp txt peer/bucket
mc mirror -w local/bucket peer/bucket
echo "2" > txt
mc cp txt local/bucket
mc cat peer/bucket/txt

And your pr is change around mj.opts.activeActive, is not related to --overwrite

restart mc mirror

mc: <ERROR> Failed to perform mirroring, with error condition (mm-source-mtime) Overwrite not allowed for `http://172.25.244.240:9001/bucket/txt`. Use `--overwrite` to override this behavior.

when mc running do this shell

echo "5" > txt
mc cp txt minio9000/bucket
mc cat minio9001/bucket/txt
echo "6" > txt
mc cp txt minio9000/bucket
mc cat minio9001/bucket/txt
echo "7" > txt
mc cp txt minio9000/bucket
mc cat minio9001/bucket/txt

echo "8" > txt
# a new object to source
mc cp txt minio9000/bucket/txt2
mc cat minio9001/bucket/txt2


root@jiuker:/mnt/d/workspace/go/src/eos/stage# mc cat minio9001/bucket/txt
3
root@jiuker:/mnt/d/workspace/go/src/eos/stage# sh issue_mc_5193.sh 
/mnt/d/workspace/go/src/eos/stage/txt:  2 B / 2 B ┃▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓┃ 26 B/s 0s
3
/mnt/d/workspace/go/src/eos/stage/txt:  2 B / 2 B ┃▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓┃ 27 B/s 0s
3
/mnt/d/workspace/go/src/eos/stage/txt:  2 B / 2 B ┃▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓┃ 27 B/s 0s
3
/mnt/d/workspace/go/src/eos/stage/txt:  2 B / 2 B ┃▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓┃ 29 B/s 0s
root@jiuker:/mnt/d/workspace/go/src/eos/stage# mc cat minio9001/bucket/txt2
8
root@jiuker:/mnt/d/workspace/go/src/eos/stage#

Couldn't found any issues. Please share more info.

this #5192 my fixed by #5197, I will close my PR.

jiuker · 2025-07-24T03:32:27Z

Thank you for your contribution.

not overwrit target object when mirror watch

29f0d74

not overwrit target object when mirror watch

Merge branch 'master' into watch-overwrite

badc2e2

harshavardhana requested a review from jiuker July 13, 2025 18:53

Merge branch 'master' into watch-overwrite

f675f29

Merge branch 'master' into watch-overwrite

6d58890

jiuker requested changes Jul 23, 2025

View reviewed changes

dormanze closed this Jul 24, 2025

dormanze deleted the watch-overwrite branch July 24, 2025 03:36

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

mirror:not overwrit target object when mirror watch #5193

mirror:not overwrit target object when mirror watch #5193

Uh oh!

dormanze commented Apr 23, 2025 •

edited

Loading

Uh oh!

sheiy commented Apr 25, 2025 •

edited

Loading

Uh oh!

jiuker commented Jul 18, 2025

Uh oh!

harshavardhana commented Jul 23, 2025

Uh oh!

jiuker commented Jul 23, 2025 •

edited

Loading

Uh oh!

jiuker Jul 23, 2025

Uh oh!

jiuker Jul 23, 2025

Uh oh!

jiuker Jul 23, 2025

Uh oh!

dormanze commented Jul 24, 2025

Uh oh!

jiuker commented Jul 24, 2025

Uh oh!

Uh oh!

mirror:not overwrit target object when mirror watch #5193

mirror:not overwrit target object when mirror watch #5193

Uh oh!

Conversation

dormanze commented Apr 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Community Contribution License

Description

Motivation and Context

How to test this PR?

Types of changes

Checklist:

Uh oh!

sheiy commented Apr 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jiuker commented Jul 18, 2025

Uh oh!

harshavardhana commented Jul 23, 2025

Uh oh!

jiuker commented Jul 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jiuker Jul 23, 2025

Choose a reason for hiding this comment

Uh oh!

jiuker Jul 23, 2025

Choose a reason for hiding this comment

Uh oh!

jiuker Jul 23, 2025

Choose a reason for hiding this comment

Uh oh!

dormanze commented Jul 24, 2025

Uh oh!

jiuker commented Jul 24, 2025

Uh oh!

Uh oh!

dormanze commented Apr 23, 2025 •

edited

Loading

sheiy commented Apr 25, 2025 •

edited

Loading

jiuker commented Jul 23, 2025 •

edited

Loading