Merge `feature/vm-anti-affinity` to `master` #5765

BengangY · 2024-07-02T02:19:25Z

Merge feature/vm-anti-affinity to master

CP-47655: [Toolstack] - Associate/disassociate VM to/from anti-affinity group Signed-off-by: Bengang Yuan <bengang.yuan@cloud.com>

…ffinity group UT - CP-47304: [Toolstack] - Add data model for anti-affinity group Signed-off-by: Bengang Yuan <bengang.yuan@cloud.com>

…P-47304 CP-47304: Add anti-affinity groups to the datamodel

In VM anti-affinity datamodel PR (xapi-project#5573), there is many-many relation between VMs and VM groups. When changing the VM.groups manually, the VM_group.VMs will be updated automatically. But when removing a VM group, the VM.groups will not be updated automatically. This commit aims to update VM.groups for all VMs in a VM group when it is being removed. Signed-off-by: Bengang Yuan <bengang.yuan@cloud.com>

…A-391880 CA-391880: Update related field 'groups' of VM when destroying VM group.

VM in an anti-affinity group will choose a host based on the following steps: 1. If the VM is associated with an anti-affinity group, compute the number of VMs associated with this anti-affinity group for each host. 2. Group these hosts based on the VM's count. The hosts in one group have the same number of VMs in one anti-affinity group. Sort all the host groups ascending based on the VM's count. 3. If a host is the affinity host of this VM, it will be treated as a single host group and be put on the head of the group list. 4. Choose a host randomly which the VM can run on it from each group. Signed-off-by: Bengang Yuan <bengang.yuan@cloud.com>

Signed-off-by: Bengang Yuan <bengang.yuan@cloud.com>

…P-47302 CP-47302: VM start with anti-affinity

Sync master into feature/vm-anti-affinity

When reverting to a snapshot, "groups" needs to be in `do_not_copy` to reserve the latest value of the VM. Signed-off-by: Bengang Yuan <bengang.yuan@cloud.com>

…A-392177 CA-392177: Keep current group after reverting from snapshot

Sync master into feature/vm-anti-affinity

Resolve dependency cycling issue if we call Xapi_alerts inside pool_features. Error: dependency cycle between modules in _build/default/ocaml/xapi: Xha_statefile -> Sm -> Sm_exec -> Xapi_session -> Pool_features -> Xapi_vm_group_helpers -> Xapi_alert -> Xapi_http -> Xapi_session Signed-off-by: Lunfan Zhang <Lunfan.Zhang@cloud.com>

The generation of breach alerts during operations on VMs or groups, triggered when all running VMs within a group are breached (i.e., placed on the same host). The following may generate or dismiss alerts: Starting and resuming a VM. Stopping and suspending a VM. Pausing and unpausing a VM. Migrating a VM. Moving a VM from one group to another. Destroying a VM group. Changing the license from Premium to Standard dismisses all breach alerts, while changing from Standard to Premium generates them. In other scenarios, such as when a VM changes state from paused to shutdown, rebooting the VM will not trigger or dismiss alerts. Signed-off-by: Lunfan Zhang <Lunfan.Zhang@cloud.com>

…official/CP-47656 CP-47656 VM anti-affinity generate breach alert

The recommendations are read from files '/etc/xapi.pool-recommendations.d/*' and exposed to clients. They are read-only and only for clients, hence they don't impact the behavior of Xapi directly but only indirectly as they inform a client's behavior. Signed-off-by: Bengang Yuan <bengang.yuan@cloud.com>

The recommendations are read from '/etc/xapi.pool-recommendations.d/*.conf'. When Xapi starts, it will read each file and update the recommendations into the database. If we need to add recommendations to other type objects, e.g. host, VM, etc in the future, just add a similar 'xapi.*-recommendations.d' directory and define the path in xapi_globs. Signed-off-by: Bengang Yuan <bengang.yuan@cloud.com>

…P-48570 CP-48570: Load recommendations from config file when Xapi starts

Check feature flag in these places: 1. VM start. 2. Host evacuation. When this PR is raised, the host evacuation PR is still in review. So this PR doesn't include the checking for host evacuation. It will be included in another new PR. 3. Create VM group. 4. VM.set_groups. Adding VMs to a group and removing VMs from a group are all forbidden. If customers need to remove VMs from a group, just destroy the group. 5. Send VM anti-affinity alerts. Also, based on our discussion, the name of feature is changed from `VM_anti_affinity` to `VM_group`. Signed-off-by: Bengang Yuan <bengang.yuan@cloud.com>

…P-48011-review CP-48011: XAPI Support anti-affinity feature check

Control domains, templates, and snapshots cannot be added to VM groups. Signed-off-by: Bengang Yuan <bengang.yuan@cloud.com>

…rom_master Update feature/vm-anti-affinity by merging from master

Add func host_to_vm_count_map to be used, rename RefMap to HostMap Signed-off-by: Gang Ji <gang.ji@citrix.com>

fixup review comments Signed-off-by: Gang Ji <gang.ji@citrix.com>

Signed-off-by: Gang Ji <gang.ji@citrix.com>

…A-393421 CA-393421: Special VMs cannot be added to VM groups

Host evacuation plan with anti-affinity support will be carried out in 3 phases: 1. Try to get a "spread evenly" plan for anti-affinity VMs, done if all the rest VMs got planned using binpack, otherwise continue. 2. Try to get a "no breach" plan for anti-affinity VMs, done if all the rest VMs got planned using binpack, otherwise continue. 3. Carry out a binpack plan ignoring VM anti-affinity. Signed-off-by: Gang Ji <gang.ji@citrix.com>

Add "groups" "power_state" for VM, vm set_resident_on host Signed-off-by: Gang Ji <gang.ji@citrix.com>

Signed-off-by: Gang Ji <gang.ji@citrix.com>

…5.Psq CP-49665: VM anti-affinity support for host evacuation

1. Rename feature flag VM_group to VM_groups. 2. Add feature check in host evacuation Signed-off-by: Bengang Yuan <bengang.yuan@cloud.com>

…P-48011 CP-48011: Xapi Support anti-affinity feature flag

…finity

coveralls · 2024-07-02T02:27:19Z

coverage: 44.887%. remained the same
when pulling 769ffba on BengangY:private/bengangy/vm-anti-affinity-to-master
into 2008661 on xapi-project:master.

coveralls · 2024-07-02T03:17:14Z

coverage: 44.887%. remained the same
when pulling daa687c on BengangY:private/bengangy/vm-anti-affinity-to-master
into 2008661 on xapi-project:master.

gangj · 2024-07-02T08:53:19Z

The last commit in the source branch: BengangY@daa687c is not in the feature branch yet, I think you should first have a PR to merge master into the feature branch.

coveralls · 2024-07-02T10:15:21Z

coverage: 44.887%. remained the same
when pulling c4e9238 on BengangY:private/bengangy/vm-anti-affinity-to-master
into f11657e on xapi-project:master.

edwintorok · 2024-07-02T10:26:54Z

The schema hash needs to be updated to fix the CI failure.

Signed-off-by: Bengang Yuan <bengang.yuan@cloud.com>

coveralls · 2024-07-02T13:58:51Z

coverage: 44.887%. remained the same
when pulling 813b3b5 on BengangY:private/bengangy/vm-anti-affinity-to-master
into ee8e800 on xapi-project:master.

gangj · 2024-07-03T02:11:01Z

`
$ git show 813b3b5
commit 813b3b5 (HEAD -> private/bengangy/vm-anti-affinity-to-master, bengang_github/private/bengangy/vm-anti-affinity-to-master)
Merge: 6cd2d82 f11657e
Author: Bengang Yuan 118869129+BengangY@users.noreply.github.com
Date: Tue Jul 2 18:11:42 2024 +0800

Merge branch 'master' into feature/vm-anti-affinity

Signed-off-by: Bengang Yuan <bengang.yuan@cloud.com>

diff --cc ocaml/idl/schematest.ml
index 1d48ba780,8ec116452..f2ee8fe4b
--- a/ocaml/idl/schematest.ml
+++ b/ocaml/idl/schematest.ml
@@@ -3,7 -3,7 +3,7 @@@ let hash x = Digest.string x |> Digest.
(* BEWARE: if this changes, check that schema has been bumped accordingly in
ocaml/idl/datamodel_common.ml, usually schema_minor_vsn *)

- let last_known_schema_hash = "92e75e20c5d0e319b0df73145be12b8e"
-let last_known_schema_hash = "e34cd0d32cdcec7805c2d3ed4e4a0c25"
++let last_known_schema_hash = "efdb1c7e536362523741ccdb7f33f797"

let current_schema_hash : string =
let open Datamodel_types in
diff --cc quality-gate.sh
index 09cd3ef06,7bc8f35cf..b504ed69d
--- a/quality-gate.sh
+++ b/quality-gate.sh
@@@ -25,7 -25,7 +25,7 @@@ verify-cert ()
}

mli-files () {
- N=516
- N=514
++ N=515
# do not count ml files from the tests in ocaml/{tests/perftest/quicktest}
MLIS=$(git ls-files -- '/*.mli' | grep -vE "ocaml/tests|ocaml/perftest|ocaml/quicktest|ocaml/message-switch/core_test" | xargs -I {} sh -c "echo {} | cut -f 1 -d '.'" ;)
MLS=$(git ls-files -- '/*.ml' | grep -vE "ocaml/tests|ocaml/perftest|ocaml/quicktest|ocaml/message-switch/core_test" | xargs -I {} sh -c "echo {} | cut -f 1 -d '.'" ;)
`

@BengangY I see ml files without mli increased from 514 to 515, there is a new file from feature/vm-anti-affinity added without a mli?

BengangY · 2024-07-03T02:27:35Z

` $ git show 813b3b5 commit 813b3b5 (HEAD -> private/bengangy/vm-anti-affinity-to-master, bengang_github/private/bengangy/vm-anti-affinity-to-master) Merge: 6cd2d82 f11657e Author: Bengang Yuan 118869129+BengangY@users.noreply.github.com Date: Tue Jul 2 18:11:42 2024 +0800
Merge branch 'master' into feature/vm-anti-affinity

Signed-off-by: Bengang Yuan <bengang.yuan@cloud.com>
diff --cc ocaml/idl/schematest.ml index 1d48ba780,8ec116452..f2ee8fe4b --- a/ocaml/idl/schematest.ml +++ b/ocaml/idl/schematest.ml @@@ -3,7 -3,7 +3,7 @@@ let hash x = Digest.string x |> Digest. (* BEWARE: if this changes, check that schema has been bumped accordingly in ocaml/idl/datamodel_common.ml, usually schema_minor_vsn *)

let last_known_schema_hash = "92e75e20c5d0e319b0df73145be12b8e" -let last_known_schema_hash = "e34cd0d32cdcec7805c2d3ed4e4a0c25" ++let last_known_schema_hash = "efdb1c7e536362523741ccdb7f33f797"

let current_schema_hash : string = let open Datamodel_types in diff --cc quality-gate.sh index 09cd3ef06,7bc8f35cf..b504ed69d --- a/quality-gate.sh +++ b/quality-gate.sh @@@ -25,7 -25,7 +25,7 @@@ verify-cert () }

mli-files () { - N=516 - N=514 ++ N=515 # do not count ml files from the tests in ocaml/{tests/perftest/quicktest} MLIS=$(git ls-files -- '/*.mli' | grep -vE "ocaml/tests|ocaml/perftest|ocaml/quicktest|ocaml/message-switch/core_test" | xargs -I {} sh -c "echo {} | cut -f 1 -d '.'" ;) MLS=$(git ls-files -- '/*.ml' | grep -vE "ocaml/tests|ocaml/perftest|ocaml/quicktest|ocaml/message-switch/core_test" | xargs -I {} sh -c "echo {} | cut -f 1 -d '.'" ;) `

@BengangY I see ml files without mli increased from 514 to 515, there is a new file from feature/vm-anti-affinity added without a mli?

This feature added datamodel_vm_group.ml without an mli file. But seems there is no mil file for the most of other datamodel files.

BengangY and others added 30 commits April 18, 2024 15:02

CP-47304: [Toolstack] - Add data model for anti-affinity group

c0aedf9

CP-47655: [Toolstack] - Associate/disassociate VM to/from anti-affinity group Signed-off-by: Bengang Yuan <bengang.yuan@cloud.com>

UT - CP-47655: [Toolstack] - Associate/disassociate VM to/from anti-a…

92d51d8

…ffinity group UT - CP-47304: [Toolstack] - Add data model for anti-affinity group Signed-off-by: Bengang Yuan <bengang.yuan@cloud.com>

Merge pull request xapi-project#5546 from BengangY/private/bengangy/C…

55a5916

…P-47304 CP-47304: Add anti-affinity groups to the datamodel

Merge pull request xapi-project#5573 from BengangY/private/bengangy/C…

81a05a0

…A-391880 CA-391880: Update related field 'groups' of VM when destroying VM group.

CP-47302: UT for VM start with anti-affinity

0de00ba

Signed-off-by: Bengang Yuan <bengang.yuan@cloud.com>

Merge pull request xapi-project#5577 from BengangY/private/bengangy/C…

aeac4cc

…P-47302 CP-47302: VM start with anti-affinity

Merge branch 'master' into feature/vm-anti-affinity

a77d149

Merge pull request xapi-project#5594 from gangj/feature/vm-anti-affinity

8a6c4c8

Sync master into feature/vm-anti-affinity

CA-392177: Keep current group after reverting from snapshot

506fb31

When reverting to a snapshot, "groups" needs to be in `do_not_copy` to reserve the latest value of the VM. Signed-off-by: Bengang Yuan <bengang.yuan@cloud.com>

Merge pull request xapi-project#5598 from BengangY/private/bengangy/C…

3bf62b7

…A-392177 CA-392177: Keep current group after reverting from snapshot

Merge branch 'master' into feature/vm-anti-affinity

b9ae796

Merge pull request xapi-project#5607 from gangj/feature/vm-anti-affinity

8505ed1

Sync master into feature/vm-anti-affinity

Merge pull request xapi-project#5605 from LunfanZhang/private/luzhan_…

da8ad7f

…official/CP-47656 CP-47656 VM anti-affinity generate breach alert

Merge pull request xapi-project#5637 from BengangY/private/bengangy/C…

d03febe

…P-48570 CP-48570: Load recommendations from config file when Xapi starts

Merge pull request xapi-project#5631 from BengangY/private/bengangy/C…

6529c0b

…P-48011-review CP-48011: XAPI Support anti-affinity feature check

CA-393421: Special VMs cannot be added to VM groups

7fcef06

Control domains, templates, and snapshots cannot be added to VM groups. Signed-off-by: Bengang Yuan <bengang.yuan@cloud.com>

Merge branch 'master' into feature/vm-anti-affinity

5a8614f

Merge pull request xapi-project#5658 from gangj/private/gangj/merge_f…

3dc2b9e

…rom_master Update feature/vm-anti-affinity by merging from master

CP-48625: Code refactoring

1efaf1f

Add func host_to_vm_count_map to be used, rename RefMap to HostMap Signed-off-by: Gang Ji <gang.ji@citrix.com>

CP-48625: Code refactoring

d1316f1

fixup review comments Signed-off-by: Gang Ji <gang.ji@citrix.com>

opam: add psq to xapi dependencies

eaf7945

Signed-off-by: Gang Ji <gang.ji@citrix.com>

Merge pull request xapi-project#5653 from BengangY/private/bengangy/C…

6640b38

…A-393421 CA-393421: Special VMs cannot be added to VM groups

Gang Ji and others added 7 commits May 31, 2024 17:07

CP-48752: Update UT

fcbab2b

Add "groups" "power_state" for VM, vm set_resident_on host Signed-off-by: Gang Ji <gang.ji@citrix.com>

CP-48752: Add UT for host evacuation with anti-affinity support

c626cdb

Signed-off-by: Gang Ji <gang.ji@citrix.com>

Merge pull request xapi-project#5652 from gangj/private/gangj/CP-4862…

d9c6535

…5.Psq CP-49665: VM anti-affinity support for host evacuation

CP-48011: Xapi Support anti-affinity feature flag

8a9074f

1. Rename feature flag VM_group to VM_groups. 2. Add feature check in host evacuation Signed-off-by: Bengang Yuan <bengang.yuan@cloud.com>

Merge pull request xapi-project#5666 from BengangY/private/bengangy/C…

de580e5

…P-48011 CP-48011: Xapi Support anti-affinity feature flag

Merge master into feature/vm-anti-affinity

6078991

Merge pull request xapi-project#5708 from BengangY/feature/vm-anti-af…

6cd2d82

…finity

BengangY force-pushed the private/bengangy/vm-anti-affinity-to-master branch from 769ffba to daa687c Compare July 2, 2024 03:13

BengangY marked this pull request as ready for review July 2, 2024 08:26

BengangY force-pushed the private/bengangy/vm-anti-affinity-to-master branch from daa687c to 6cd2d82 Compare July 2, 2024 10:05

Merge branch 'master' into feature/vm-anti-affinity

813b3b5

Signed-off-by: Bengang Yuan <bengang.yuan@cloud.com>

BengangY force-pushed the private/bengangy/vm-anti-affinity-to-master branch from c4e9238 to 813b3b5 Compare July 2, 2024 13:55

gangj approved these changes Jul 3, 2024

View reviewed changes

minglumlu approved these changes Jul 4, 2024

View reviewed changes

minglumlu merged commit e61e0ac into xapi-project:master Jul 4, 2024
15 checks passed

BengangY deleted the private/bengangy/vm-anti-affinity-to-master branch February 5, 2025 01:59

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Merge `feature/vm-anti-affinity` to `master` #5765

Merge `feature/vm-anti-affinity` to `master` #5765

Uh oh!

BengangY commented Jul 2, 2024

Uh oh!

coveralls commented Jul 2, 2024

Uh oh!

coveralls commented Jul 2, 2024

Uh oh!

gangj commented Jul 2, 2024 •

edited

Loading

Uh oh!

coveralls commented Jul 2, 2024

Uh oh!

edwintorok commented Jul 2, 2024

Uh oh!

coveralls commented Jul 2, 2024

Uh oh!

gangj commented Jul 3, 2024 •

edited

Loading

Uh oh!

BengangY commented Jul 3, 2024

Uh oh!

Uh oh!

Uh oh!

Merge feature/vm-anti-affinity to master #5765

Merge feature/vm-anti-affinity to master #5765

Uh oh!

Conversation

BengangY commented Jul 2, 2024

Uh oh!

coveralls commented Jul 2, 2024

Uh oh!

coveralls commented Jul 2, 2024

Uh oh!

gangj commented Jul 2, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

coveralls commented Jul 2, 2024

Uh oh!

edwintorok commented Jul 2, 2024

Uh oh!

coveralls commented Jul 2, 2024

Uh oh!

gangj commented Jul 3, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

BengangY commented Jul 3, 2024

Uh oh!

Uh oh!

Uh oh!

Merge `feature/vm-anti-affinity` to `master` #5765

Merge `feature/vm-anti-affinity` to `master` #5765

gangj commented Jul 2, 2024 •

edited

Loading

gangj commented Jul 3, 2024 •

edited

Loading