Going from 3*(2+1) to 3*(3+1) glusterFS

[akanugan]

I have a gluster volume like:

Volume Name: ovirt
Type: Distributed-Replicate
Volume ID: xyzzy
Status: Started
Snapshot Count: 0
Number of Bricks: 3 x (2 + 1) = 9
Transport-type: tcp
Bricks:
Brick1: ovirt1:/data/brick1/gv0
Brick2: ovirt2:/data/brick1/gv0
Brick3: ovirt4:/data/brick1/gv0 (arbiter)
Brick4: ovirt4:/data/brick2/gv0
Brick5: ovirt1:/data/brick2/gv0
Brick6: ovirt2:/data/brick2/gv0 (arbiter)
Brick7: ovirt2:/data/brick3/gv0
Brick8: ovirt4:/data/brick3/gv0
Brick9: ovirt1:/data/brick3/gv0 (arbiter)

If I need to add a new node (ovirt3 with 3 bricks), I read that in some old docs, I can't have more than 2 data replicas? To add extra redundancy and to be able to use the bricks from ovirt3, I am looking for suggestions. The options that came to my mind was:

1. Probably I can remove the arbiter (promoting that to full replica) and add new bricks from ovirt3? [3 x (2 + 1) --> 3x 4]. Not useful I guess? No quorum rules can be established
2. I think I can do 4 x (2 + 1). In this case, the 4th replicated set would be (2 + 1) with all bricks on ovirt3 --> No HA
3. Ideally I would like to have the option of 3 x (3 + 1). Like just have all the new bricks to be data replica with the rest of the other bricks as it is now.

Any suggestions on option 3 would be greatly appreciated. Thank you!

[pranithk]

@akanugan Are you thinking of adding extra capacity or moving replica-3 for additional redundancy

[akanugan]

@pranithk Ideally looking to expand the dist-repl by adding an additional replica set with 3 bricks, but not sure if this can be done? Also open to see if I can have these extra node with 3 nodes as redundancy. Just wanted to see on how I can make use of this new node

[pranithk]

You need to add 2 extra data bricks and 1 arbiter brick for this purpose, i.e. 2+1

[edrock200]

Add your new node to the cluster. Make a new folder on the bricks for the new volume. Create your new volume. Copy or move everything from old volume to new volume. Migrate clients to new volume. Delete old volume. As far as I know if you simply increase your replica set, you can do so on the fly. But changing ec requires new volume and migration.