Epoll: the rest of the changes #5706
Closed
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Lin Liu <lin.liu@citrix.com>
CA-389241: import-update-key compat with xs8 and xs9
* CP-46944: Update yum plugins to dnf plugins Signed-off-by: Lin Liu <lin.liu@citrix.com> * CP-46944: Update unittest to make github CI happy github CI has issues with - ubuntu-22.04 container - python11 - python3-dnf Instead of patching the upstreams, we just provide a stub for the fix Signed-off-by: Lin Liu <lin.liu@citrix.com> --------- Signed-off-by: Lin Liu <lin.liu@citrix.com>
* Feature branch sync
* CP-45921: Use dnf as package manager for XS9 Given XS9 has updated to dnf and no yum is available, xapi will choose package manager basing on following - If dnf exists, use dnf - otherwise, fallback to yum xapi just presume dnf or yum is available in the system. Because xapi decides to use dnf or yum according to the running environment, this commit is compatible with yum/xs8 and dnf/xs9 Signed-off-by: Lin Liu <lin.liu@citrix.com> * CP-45921: Code refine for - Move test code to its own suite - Replace `active` with `manager`, which is static - Add cmd_line record type to abstract cmd line type - Refine doc comment and license --------- Signed-off-by: Lin Liu <lin.liu@citrix.com>
The default gpg public keyring database updated - pubring.kbx for new gpg on XS9 - pubring.gpg for old gpg on XS8 Detect whether pubring.kbx exists and fallback to old one for XS8 Signed-off-by: Lin Liu <lin.liu@citrix.com>
…project#5564) * CP-48221: Update plugin detect package manager dynamically Given XS9 has deplicated yum with dnf, to be compatible with XS8, - Use dnf if dnf is detected - Fallback to yum otherwise Signed-off-by: Lin Liu <lin.liu@citrix.com> * CP-48221: Make CI happy with various scans --------- Signed-off-by: Lin Liu <lin.liu@citrix.com> Co-authored-by: Luca Zhang <feiya.zhang@cloud.com>
Routine feature branch sync
edwintorok
force-pushed
the
private/edvint/epoll3
branch
from
7bc67d5 to
136636e
Compare
edwintorok
force-pushed
the
private/edvint/epoll3
branch
3 times, most recently
from
048e876 to
24abe38
Compare
Signed-off-by: Vincent Liu <shuntian.liu2@cloud.com>
Use the Atomic module to track whether a watcher has been created. Signed-off-by: Vincent Liu <shuntian.liu2@cloud.com>
Signed-off-by: Vincent Liu <shuntian.liu2@cloud.com>
This allows one to force sync the state of xapi db with the cluster stack, useful for cluster API methods change the state of the cluster. Signed-off-by: Vincent Liu <shuntian.liu2@cloud.com>
Previously there were two ways an alert for a cluster host join/leave can be raised: 1. through the cluster change watcher; 2. through the api call. These two can generate duplicate alerts as an API call can cause the cluster change watcher to notice the change as well. The idea of the fix here is still to let API and watcher raise alerts separately, but now add synchronous API calls to allow API call (cluster-host-join, etc) to call the cluster change update code at the right time so that the cluster change watcher won't see the change again, hence not generating duplicate alerts. Signed-off-by: Vincent Liu <shuntian.liu2@cloud.com>
Signed-off-by: Vincent Liu <shuntian.liu2@cloud.com>
…ess-alert CA-394109: Reduce number of alerts
Merge feature/perf to master
The new fingerprint_sha256 and fingerprint_sha1 fields will be empty when upgrading from a version without the fields. This commit checks for this and fills them in, stopping the certificate from being needlessly reinstalled. Signed-off-by: Steven Woods <steven.woods@citrix.com>
edwintorok
force-pushed
the
private/edvint/epoll3
branch
from
24abe38 to
0031253
Compare
edwintorok
force-pushed
the
private/edvint/epoll3
branch
2 times, most recently
from
691882f to
d2888f9
Compare
The float argument gets converted to microseconds when making the system call. If it is <1us then it gets converted to 0, which has special meaning * for select/epoll it means timeout immediately * for setsockopt SO_RCVTIMEO it means never timeout So do not generate tests with such small timeout, instead replace them with no delays. We cannot call `QCheck2.assume` here (it can only be called from the body of a test), otherwise sometimes we might get a failure like this: ``` Test Dune__exe__Unixext_test.test_proxy failed: ERROR: uncaught exception in generator for test Dune__exe__Unixext_test.test_proxy after 100 steps: Exception: QCheck2.Failed_precondition Backtrace: Raised at QCheck2.assume in file "src/core/QCheck2.ml" (inlined), line 57, characters 29-54 Called from Xapi_fd_test__Generate.delay_of_size in file "ocaml/libs/xapi-stdext/lib/xapi-fd-test/generate.ml", line 66, characters 2-34 Called from QCheck2.Tree.bind in file "src/core/QCheck2.ml", line 207, characters 28-31 Called from QCheck2.Gen.bind in file "src/core/QCheck2.ml", line 271, characters 72-80 Called from QCheck2.Tree.bind in file "src/core/QCheck2.ml", line 207, characters 28-31 Called from QCheck2.Test.check_state in file "src/core/QCheck2.ml", line 1749, characters 12-32 ``` Signed-off-by: Edwin Török <edwin.torok@cloud.com>
… must be at least 1 Reject <1us timeouts, they are likely the result of a rounding down bug. If we try to use a timeout that is >0, but < 1e-6 it'd be truncated to 0. 0 has a special meaning of 'no timeout', rather than immediately timeout. The generator got already changed to avoid generating such short timeouts. Signed-off-by: Edwin Török <edwin.torok@cloud.com>
…-49811 CP-49811: Remove redundant method object from span name
…n call really_input takes a timeout parameter, however this is the timeout for each individual read call, not the entire function. So if we have a sender that sends a 64KiB buffer 1 byte at a time, and we set a timeout of 100ms then the operation can take up to 109 minutes. The timeout generated by quick-check is meant to be the total timeout, so divide it by the number of read/write calls that we are expected to make, to ensure that the overall timeout is not excessive. If the timeout ends up being too small then we tell quickcheck to generate another configuration with `assume`. Signed-off-by: Edwin Török <edwin.torok@cloud.com>
…ix-timeouts Fix @StressTest failure
Use SO_RCVTIMEO socket option instead. This will cause EAGAIN or EWOULDBLOCK to be returned according to `socket(7)`. Need to be careful about the distinction between select 0 and SO_RCVTIMEO 0. select 0 would timeout immediately, but SO_RCVTIMEO would timeout never. Signed-off-by: Steven Woods <steven.woods@citrix.com> Signed-off-by: Edwin Török <edwin.torok@cloud.com>
TODO: but then how can we upstream this again to mirage? Signed-off-by: Edwin Török <edwin.torok@cloud.com>
…Unix/Lwt It was a mix of Lwt and Unix code, which means that if the Unix call blocks the entire Lwt code blocks too. This was only installed by the specfile in a -devel package. `message-cli tail --follow` can be used to debug the IDL protocol instead. Signed-off-by: Edwin Török <edwin.torok@cloud.com>
Its implementation was identical, except for the use of time_limited_read in Threadext, but the semantics is identical. Use one well tested implementation instead of duplicating code. One less function to convert to epoll. Signed-off-by: Edwin Török <edwin.torok@cloud.com>
Absolute deadlines can be represented using `Mtime.t`, or `Mtime_clock.counter` and comparing against a deadline. We hide this as an implementation detail in `Timer.t` and use counters (although an implementation using `Mtime.t` would work just as well). Relative deadlines are represented using `Mtime.Span.t`. We wrap it in `Timeout.t` to avoid accidentally mistaking it for a time duration calculated elsewhere. These helper modules correctly convert from time represented as strings (e.g. in configuration files), and calculate the remaining time until the deadline. `Mtime.Span.t` expects `float`s as nanoseconds, which can be quite error prone when converting existing code if the conversion factor is forgotten. The various `time_limited*` functions will be changed to take a `Timer.t` as argument in a followup commit. Signed-off-by: Edwin Török <edwin.torok@cloud.com>
Signed-off-by: Edwin Török <edwin.torok@cloud.com>
Instead of Unix.gettimeofday. This is required to start using these functions in Jsonrpclient, which already uses Mtime. Mtime should also be more robust against clock changes on the host. TODO: needs some extra testing of block_device_io, which doesn't many suitable tests. Signed-off-by: Edwin Török <edwin.torok@cloud.com>
Signed-off-by: Steven Woods <steven.woods@citrix.com>
Note: Unixext.proxy is more efficient and doesn't recreate the epoll instance many times. But xsh is not performance critical. Signed-off-by: Steven Woods <steven.woods@citrix.com>
Signed-off-by: Edwin Török <edwin.torok@cloud.com>
…t_timed_read Signed-off-by: Edwin Török <edwin.torok@cloud.com>
…Unix.select Signed-off-by: Edwin Török <edwin.torok@cloud.com>
If we want to reproduce a failure we need to know the exact commandline that was used. Longer than 80 chars is not a problem, this is a logfile, and a truncated line is worse than a long line. Signed-off-by: Edwin Török <edwin.torok@cloud.com>
TODO: double check that the semantics is the same, especially wrt to handling of Unix exceptions, and the kind of exceptions that are raised. Signed-off-by: Edwin Török <edwin.torok@cloud.com>
…edundant while loop as its contents has been removed in a previous commit. TODO: this needs to be switched to use Timer.t too Signed-off-by: Steven Woods <steven.woods@citrix.com>
TODO: needs tests Signed-off-by: Edwin Török <edwin.torok@cloud.com>
…API and xenopsd This is somewhat of a hack, as it parses the disassembled binary of xapi and xenopsd, and might break with future compiler versions that invoke functions differently. There could also be false positives if a dependency calls Unix.select for just sleeping, i.e. with all 3 arguments as empty lists, but so far we don't have any. We still have the runtime test if that happens. If the commits are reordered this does find the 'Thread.wait_timed_read' call in 'Master_database_connection', and with the correct order of the commits it passes, offering some assurance that we don't actually call 'select' anymore, not even indirectly via one of our dependencies. Signed-off-by: Edwin Török <edwin.torok@cloud.com>
Signed-off-by: Edwin Török <edwin.torok@cloud.com>
Signed-off-by: Edwin Török <edwin.torok@cloud.com>
edwintorok
force-pushed
the
private/edvint/epoll3
branch
from
447ec0d to
01ebbab
Compare
|
Mostly obsoleted by #5864 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Builds on top of #5705, but has some TODO comments.
It works, but marking it as draft for now, until those TODOs are addressed.