Merge master into pool licensing feature branch #6276
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
No functional change. Signed-off-by: Edwin Török <edwin.torok@cloud.com>
No functional change. Signed-off-by: Edwin Török <edwin.torok@cloud.com>
…queue Wake up the scheduler immediately when there are more tasks. Otherwise timeouts <10s may not work correctly, and it is difficult to test the periodic scheduler if you need to wait 10s for it to start working. If there are no tasks, then it will still sleep efficiently, but as soon as more tasks are added (with [~signal:true], which is the default) it will immediately wake up and calculate the next sleep time. In practice it is probably quite rare for XAPI's queue to be empty (there are usually periodic tasks), but we cannot rely on this. Signed-off-by: Edwin Török <edwin.torok@cloud.com>
…internal communication Feature flag: use-xmlrpc Signed-off-by: Edwin Török <edwin.torok@cloud.com>
…owercase Tasks are lowercase Signed-off-by: Edwin Török <edwin.torok@cloud.com>
Subscription.object_matches already does it Signed-off-by: Edwin Török <edwin.torok@cloud.com>
Signed-off-by: Edwin Török <edwin.torok@cloud.com>
Creates a new library `Tgroup`, that abstracts and manages groups of execution threads in xapi. When xapi is under load, all the threads need to share a single cpu in dom0 because of ocaml runtime single-cpu restrictions. This library is meant to orchestrate the threads in different priority groups. Signed-off-by: Gabriel Buica <danutgabriel.buica@cloud.com>
`set_cgroup` adds the functionality of adding the current thread in a cgroup based on its creator. Signed-off-by: Gabriel Buica <danutgabriel.buica@cloud.com>
Add functionality of setting the tgroup based on a http header named `originator`. Signed-off-by: Gabriel Buica <danutgabriel.buica@cloud.com>
Signed-off-by: Gabriel Buica <danutgabriel.buica@cloud.com>
Clears the `extra_headers` of `UDSTransport` instance when making a connection. Previously, this was done only when tracing was enabled inside the `with_tracecontext` method to avoid the header duplication when `make_connection` was used multiple times. Currently, there is not other use of `add_extra_headers` or other update to the `_extra_headers`, making it safe to clear it when we make a new connection. (`xmlrpclib.Transport` updates the `_extra_headers` attribute only inside `make_connection` method but we override this method with our own for `UDSTransport`.) Signed-off-by: Gabriel Buica <danutgabriel.buica@cloud.com>
XenAPI.py now passes an additional originator header when making requests to xapi, if the "ORIGINATOR" env var is present. Sm_exec now passes an env var, "ORIGINATOR=SM". To classify the threads correctly, we first need to determine the requests originators. This commit makes it possibly to explicitly state the originators as headers. Signed-off-by: Gabriel Buica <danutgabriel.buica@cloud.com>
For now, the thread executing `Xapi.server_init` and it's children are classified as External. The only excception are http requests that come through the smapi internally. If those contain the originator header with the value set as "sm", the thread executing the request will be classified as internal. This represents the first phase of classifing xapi threads as internal vs external. Signed-off-by: Gabriel Buica <danutgabriel.buica@cloud.com>
Adds a configurable variable in `xapi_globs`, `tgroups_enabled` that is meant to ask a guard for tgroup classification of the threads. If the guard is `false` all Tgroups functionality should act as a no op. For instance, adding the line: tgroups-enabled = false will result in the thread classification being skipped. Signed-off-by: Gabriel Buica <danutgabriel.buica@cloud.com>
…n flag This is more efficient: we can watch a single task, instead of everything in the DB. Feature-flag: use-event-next No functional change. Signed-off-by: Edwin Török <edwin.torok@cloud.com>
int32_of_rpc doesn't accept Int32 as input, just Int because none of the current deserializers actually produce an Int32. (Int32 is only used by serializers to emit something different). This is an upstream ocaml-rpc bug that should be fixed, meanwhile convert Rpc.Int32 to Rpc.Int, so that the 'fake_rpc' inside XAPI can use Event.from. Otherwise you get this error: ``` Expected int32, got 'I32(0) ``` Signed-off-by: Edwin Török <edwin.torok@cloud.com>
…vent.next This is more efficient: we can watch a single task, instead of everything in the DB. Feature-flag: use-event-next Signed-off-by: Edwin Török <edwin.torok@cloud.com>
…cution threads. (xapi-project#6076) This is the follow up to xapi-project#6020 The initial phase is to classify the threads between Internal and External. - External is the default (for now), - Internal are threads that process internal requests coming from smapi. BVT + BST: 207007 (Dev Run)
… of Event.next Feature flag: use-event-next Signed-off-by: Edwin Török <edwin.torok@cloud.com>
…pi-project#6122) When XAPI's queue is empty the periodic scheduler would sleep for 10s, during which time newly added jobs would get ignored. This is mostly noticeable during unit tests, but we cannot rely on the queue not being empty in production either. It is better to wake up the scheduler on the soonest of these 2 events: * after 10s * when a new task gets added to the queue
Previously it'd only run when we added or removed entries, but on an idle system we'd keep a large number of connections open that we don't need, and this then exceeded the connection limits on the coordinator. Signed-off-by: Edwin Török <edwin.torok@cloud.com>
Introduce separate coordinator_max_stunnel_cache and member_max_stunnel_cache settings, and set these on XAPI startup based on host role. No functional change, the defaults for both match the previous hardcoded value (70). Signed-off-by: Edwin Török <edwin.torok@cloud.com>
To avoid labeled argument with ' in name Signed-off-by: Edwin Török <edwin.torok@cloud.com>
This parameter was passed through unchanged. However VDI.epoch_begin/epoch_end doesn't actually need it, so drop it. We intend to change how we compute the 'domain' parameter, and VDI.epoch_begin/end wouldn't have sufficient information to compute it anymore. No functional change. Signed-off-by: Edwin Török <edwin.torok@cloud.com>
We are going to change how we compute it, so factor it out into a function. No functional change. Signed-off-by: Edwin Török <edwin.torok@cloud.com>
This allows to test independent modules faster more easily Signed-off-by: Pau Ruiz Safont <pau.ruizsafont@cloud.com>
…nce matrices Instead disable NUMA for the host Fixes xapi-project#6240 Signed-off-by: Pau Ruiz Safont <pau.ruizsafont@cloud.com>
Now the tests use the actual data from the test specifications instead of being hardcoded, and the distance matrices used for testing are in its own module for better clarity. Signed-off-by: Pau Ruiz Safont <pau.ruizsafont@cloud.com>
…able Unreachable nodes do not contain any CPUs, and therefore VCPUs cannot be scheduled on them. They marked with a value of (2ˆ32) - 1. Instead of failing to produce a NUMA object that allows for scheduling, create an object that contains only schedulable NUMA nodes. This means changing how the datastructures node_cpus and candidates are created to ignore the unreachable ones. Signed-off-by: Pau Ruiz Safont <pau.ruizsafont@cloud.com>
These could be created accidentally by dividing by 0. Signed-off-by: Pau Ruiz Safont <pau.ruizsafont@cloud.com>
It's unclear why the candidates with single nodes where always added, since the algorithm that generates all the subsets already includes these. Signed-off-by: Pau Ruiz Safont <pau.ruizsafont@cloud.com>
It's already printed by xenopsd, and now that development has stabilised, unit-test can print this useful unformation. Signed-off-by: Pau Ruiz Safont <pau.ruizsafont@cloud.com>
Signed-off-by: Pau Ruiz Safont <pau.ruizsafont@cloud.com>
…culation Now unreachable nodes are not considered when calculating all the subsets for the NUMA nodes combinations for scheduling a domain. Signed-off-by: Pau Ruiz Safont <pau.ruizsafont@cloud.com>
…nce matrices (NUMA) (xapi-project#6249) Unreachable nodes do not contain any CPUs, and therefore VCPUs cannot be scheduled on them. They marked with a value of (2ˆ32) - 1. Instead of failing to produce a NUMA object that allows for scheduling, create an object that contains only schedulable NUMA nodes. This means changing how the datastructures node_cpus and candidates are created to ignore the unreachable ones. Fixes two minor potential issues (distances being NaN, and adding duplicates to candidates, which adds to running time); and separates the xenopsd unit tests into 3 binaries for ease of testing. Fixes xapi-project#6240
Signed-off-by: Bernhard Kaindl <bernhard.kaindl@cloud.com>
Signed-off-by: Bernhard Kaindl <bernhard.kaindl@cloud.com>
…roject#6270) doc, comment: Fix all spellings of omitted(from omit) from incorrect: "ommitted" - Even if it may sound like "mm", English dictionaries and spelling sites say "omitted".
Signed-off-by: Bernhard Kaindl <bernhard.kaindl@cloud.com>
Signed-off-by: Bernhard Kaindl <bernhard.kaindl@cloud.com>
Signed-off-by: Bernhard Kaindl <bernhard.kaindl@cloud.com>
…e. (xapi-project#6269) Apply a few trivial spelling fixes here and there for: - Accidental typos - Word repetitions - Uppercase of "Markdown" and "JSON-RPC"
…c) (xapi-project#6271) Fix a few minor spelling issues (missing spaces, words, etc) here and there.
eBPF exitprobes do not work on OCaml functions that are tailcalls. Introduce some noop hooks that are forced to be non-tailcalls, by disabling inlining on them. This enables `bpftrace` to be able to hook these functions for debugging and performance measurement purposes, and otherwise have negligible runtime impact (just a function call and a return).
…hread.yield while we hold it We'd still yield when the tick thread yields, or on syscalls. Signed-off-by: Edwin Török <edwin.torok@cloud.com>
Currently all of these are strings, but it may change: * table * db_ref * field_name * field * uuid This enables changing the `field` type from `string` and avoiding costly serialization/deserialization when not needed. No functional change. Signed-off-by: Edwin Török <edwin.torok@cloud.com>
sexp_of is used when saving the schema, but of_sexp is unused. This enables a future commit to store the deserialized value as a pair value, and deserialization (unmarshaling) requires knowing the schema, whereas serialization (marshaling) does not. Thus we couldn't implement a generic `t_of_sexp` function, but with this change we won't have to. No functional change. Signed-off-by: Edwin Török <edwin.torok@cloud.com>
Builds on top of xapi-project#6177 and xapi-project#6179, the commit new in this PR is CP-49141.
This is to resolve the conflict between master and feture branches. The schema_minor_vsn in ocaml/idl/datamodel_common.ml has been bumped up for the feature. Signed-off-by: Ming Lu <ming.lu@cloud.com>
Signed-off-by: Ming Lu <ming.lu@cloud.com>
changlei-li
approved these changes
Feb 5, 2025
Vincent-lau
approved these changes
Feb 5, 2025
minglumlu
merged commit 10bb483
into
xapi-project:feature/pool-licensing
15 checks passed
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The following two commits are new ones:
677996e: Bump up last_known_schema_hash
962abf1: Update datamodel_lifecycle for pool.license_server