CP-48625, CP-48752: VM anti-affinity support for host evacuation and related UT

[gangj]

Host evacuation plan with anti-affinity support will be carried out in 3 steps:

1. Try to get a "spread evenly" plan for anti-affinity VMs, done if
   all the rest VMs got planned using binpack, otherwise continue.
2. Try to get a "no breach" plan for anti-affinity VMs, done if all the rest VMs
   got planned using binpack, otherwise continue.
3. Carry out a binpack plan ignoring VM anti-affinity.

[minglumlu]

I think one of the goals of this change is to keep the time complexity as
O ( M * log H), where the M is the number of VMs to be migrated and the H is the number of hosts.
E.g. the following data structures are used for this purpose in planning for spreading evenly:

1. A map (host -> free memory): int64 HostMap.t
2. A two-layer map (vm_group -> host -> vm_cnt): int HostMap.t VMGroupMap.t
3. A map (vm_group -> set of (vm_cnt, free_mem, h)): HostsSet.t VMGroupMap.t

For each VM placement schedule on one host:
The data 1 is updated in O(log H)
The data 2 is updated in O(G * log H) where the G is the number of VM groups
The data 3 is updated in O(log H), as the set can keep the order.

Currently, we assume the G is a constant number. So overall it is still O(M * log H).

==== update ====
Based on above, the psq (suggested by Edwin in another communicate channel) looks the right data structure. Alternatively, a home-brewed module would make the code more compact as well. E.g. (just some snippets)

module type S = sig
  type t
  type k
  type v

  val add -> k -> p -> t -> t
  val update -> k -> (v option -> v option) -> t -> t
  val pop -> t -> (k * v option)
end

module SearchQueue (K) (V) :
  S with type k = K.t and type v = V.t =
struct
  type k = K.t
  type v = V.t

  type t = {m: Map.make (k); s: Set.make (v * k)}

  let update k f t =
    match HostMap.find_opt k t.m with
    | Some v ->
        let s = HostSet.remove (v, k) t.s in
        let v' = f v in
        let m' = HostMap.update k (fun _ -> v') t.m in
        let s' = HostSet.add (v', k) t.s in
        {t with m=m'; s= s'}
    | None ->
        t
end

module HostKey = struct
  type t = string
  let compare = compare
end

module HostPlanState = struct
  type t = int * int
  let compare = ...
end

let plan_state = SearchQueue.make (HostKey) (HostPlanState) in

let update_all_plan_states h vm_size all_states =
  let update' =
    Option.map (fun (cnt, free) -> (cnt + 1, free - vm_size))
  in
  let update'' =
    Option.map (fun (cnt, free) -> (cnt, free - vm_size))
  in
  VMGroup.mapi (fun group sq ->
    match group = group' with
    | true ->
        Sq.update h update' sq
    | false ->
        Sq.update h update'' sq
  ) all_states

Not necessary to use data 1 and data 2 above. Instead, for each VM group, they are wrapped into SearchQueue.

[gangj]

The data 2 is updated in O(G * log H) where the G is the number of VM groups

I think one of the goals of this change is to keep the time complexity as O ( M * log H), where the M is the number of VMs to be migrated and the H is the number of hosts. E.g. the following data structures are used for this purpose in planning for spreading evenly:

1. A map (host -> free memory): int64 HostMap.t
2. A two-layer map (vm_group -> host -> vm_cnt): int HostMap.t VMGroupMap.t
3. A map (vm_group -> set of (vm_cnt, free_mem, h)): HostsSet.t VMGroupMap.t

For each VM placement schedule on one host: The data 1 is updated in O(log H) The data 2 is updated in O(G * log H) where the G is the number of VM groups The data 3 is updated in O(log H), as the set can keep the order.

Currently, we assume the G is a constant number. So overall it is still O(M * log H).

Thanks for the suggestion, discussion and review!
One small update:
The data 2 is updated in O(log G * log H)
The data 3 is also updated in O(log G * log H)
as both of them are inside a VMGroupMap.

[duobei]

It took me a while to figure out that below is a function:

  match grp = group with
      | true ->
          HostsSet.add (vm_cnt + 1, updated_host_size, host)
      | false ->
          HostsSet.add (vm_cnt, updated_host_size, host)

I think we can refactor it to better for explicit:

      let hosts = HostsSet.remove (vm_cnt, host_size, host) hosts in
      match grp = group with
      | true ->
          HostsSet.add (vm_cnt + 1, updated_host_size, host) hosts
      | false ->
          HostsSet.add (vm_cnt, updated_host_size, host) hosts

[duobei]

Can host_size be None?

[gangj]

No, the Option.get is to make sure it should always exist in the map.

[robhoes]

It will be None if host is not in the host_size_map, and then Option.get will raise and exception. I think we are relying on init_host_size_map to have added the host here? It would be better to not risk raising exceptions here, as they won't be caught.

[gangj]

==== update ==== Based on above, the psq (suggested by Edwin in another communicate channel) looks the right data structure. Alternatively, a home-brewed module would make the code more compact as well. E.g. (just some snippets)

module type S = sig
  type t
  type k
  type v

  val add -> k -> p -> t -> t
  val update -> k -> (v option -> v option) -> t -> t
  val pop -> t -> (k * v option)
end

module SearchQueue (K) (V) :
  S with type k = K.t and type v = V.t =
struct
  type k = K.t
  type v = V.t

  type t = {m: Map.make (k); s: Set.make (v * k)}

  let update k f t =
    match HostMap.find_opt k t.m with
    | Some v ->
        let s = HostSet.remove (v, k) t.s in
        let v' = f v in
        let m' = HostMap.update k (fun _ -> v') t.m in
        let s' = HostSet.add (v', k) t.s in
        {t with m=m'; s= s'}
    | None ->
        t
end

module HostKey = struct
  type t = string
  let compare = compare
end

module HostPlanState = struct
  type t = int * int
  let compare = ...
end

let plan_state = SearchQueue.make (HostKey) (HostPlanState) in

let update_all_plan_states h vm_size all_states =
  let update' =
    Option.map (fun (cnt, free) -> (cnt + 1, free - vm_size))
  in
  let update'' =
    Option.map (fun (cnt, free) -> (cnt, free - vm_size))
  in
  VMGroup.mapi (fun group sq ->
    match group = group' with
    | true ->
        Sq.update h update' sq
    | false ->
        Sq.update h update'' sq
  ) all_states

Not necessary to use data 1 and data 2 above. Instead, for each VM group, they are wrapped into SearchQueue.

Changed to the Psq version, please help to review again, thanks!

[robhoes]

It will be None if host is not in the host_size_map, and then Option.get will raise and exception. I think we are relying on init_host_size_map to have added the host here? It would be better to not risk raising exceptions here, as they won't be caught.

[robhoes]

The Option.get will raise an exception if the host_vm_cnt_map = None, i.e. the group is not in the map.

[duobei]

How about using guard here:

Suggested change

	| Some (host, (_vm_cnt, h_size)) -> (
	match vm_size <= h_size with
	| true ->
	Some host
	| _ ->
	select_host_for_spread_evenly_plan vm vm_size
	(HostsSet.remove (vm_cnt, host_size, host) hosts_set)
	hosts_queue
	|> AntiAffinityEvacuationPlanHostQueue.remove host
	|> select_host_for_spread_evenly_plan vm vm_size
	)
	| Some (host, (_vm_cnt, h_size)) when vm_size <= h_size ->
	Some host
	| Some (host, _) ->
	hosts_queue
	|> AntiAffinityEvacuationPlanHostQueue.remove host
	|> select_host_for_spread_evenly_plan vm vm_size

[gangj]

Merge the one and below select_host_for_no_breach_plan

[gangj]

change to List.fold_left

[gangj]

change to List.fold_left

[gangj]

check total_hosts before the plan

[gangj]

change to monad

[duobei]

_vm_cnt should be vm_cnt.

[duobei]

Why is num_failures 1 , or is it just from 1?

[gangj]

I think it is used by binpack for HA plan only, not used for "host evacuation".

[gangj]

Continues here: #5652