Skip to content

Merge master into feature/py3 #5624

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

bernhardkaindl
Copy link
Collaborator

@bernhardkaindl bernhardkaindl commented May 10, 2024

Merge master into feature/py3

@stephenchengCloud - of course this has to be approved by you

Motivation

@psafont said that master should be merged into feature/py3 before applying #5622 (#5622 (comment)):

I would prefer if master was merged to feature/py3, then add the necessary patches on top (like the pytype and pyright fixes)

@stephenchengCloud, @psafont wrote in #5622 (comment):

My preference would be the responsibility to merge these changes to be transferred over to stephen and other working on the python3 feature so they can prioritise this to and use these contributions easier as they see fit.

I gladly accept:

  • I have no further tasks to submit.
  • My work on it is done.
  • I had some time left, so I thought I could submit the missing merge of master to feature/py3.

Manually resolved merge-conflicts

There were only a few areas that needed manual work to resolve the merge:

  • pyproject.toml:
    • feature/py3 updated the pylint configuration (e.g. suppressed pylint messages) etc
    • master as well, and this caused also a duplicate config section, largely equivalent
    • resolved using changes from master which were more nicely formatted and documented for those.
    • master removed issuing python warnings from the scripts directory to reduce noise in PRs.
    • for those, keep the warnings to keep the remaining Python2 work visible.
  • .github/workflows/main.yml -> * .github/workflows/other.yml
    • master moved the non-OCaml parts to other.yml, while some changes were made in py3.
    • Merged the needed changes into the new .github/workflows/other.yml
  • In feature/py3, a few tests and files were moved and created
    • With the merge of the configuration updates in pyproject.toml, as

List of all resolved merge-conflict and suppressions of warnings

@stephenchengCloud:
The files that needed manual merging and suppression of warnings are:

  • pyproject.toml
  • .github/workflows/main.yml
  • .github/workflows/other.yml
  • python3/tests/import_helper.py
  • python3/libexec/usb_scan.py
  • scripts/mail-alarm
  • scripts/examples/python/provision.py

I added these suppressions in this PR to fix CI with the merge of pyproject.toml with master.

ydirson and others added 30 commits March 6, 2024 16:56
This test did not exercised the SR type expected by user, and notably
failed when no default SR was defined.

Adds the option of installing a VM in a non-default SR using
VM.with_new.

Signed-off-by: Yann Dirson <yann.dirson@vates.fr>
Also update `pom.xml` to remove `org.apache.xmlrpc` packages and add `jackson-databind` and `org.apache.httpcomponents.client5`

Signed-off-by: Danilo Del Busso <danilo.delbusso@cloud.com>
Signed-off-by: Danilo Del Busso <danilo.delbusso@cloud.com>
Signed-off-by: Danilo Del Busso <danilo.delbusso@cloud.com>
Signed-off-by: Danilo Del Busso <danilo.delbusso@cloud.com>
Signed-off-by: Danilo Del Busso <danilo.delbusso@cloud.com>
Signed-off-by: Danilo Del Busso <danilo.delbusso@cloud.com>
Signed-off-by: Danilo Del Busso <danilo.delbusso@cloud.com>
As reported by `make check`

Signed-off-by: Danilo Del Busso <danilo.delbusso@cloud.com>
Signed-off-by: Danilo Del Busso <danilo.delbusso@cloud.com>
Signed-off-by: Danilo Del Busso <danilo.delbusso@cloud.com>
Instead use `Fun.protect`

Signed-off-by: Danilo Del Busso <danilo.delbusso@cloud.com>
Signed-off-by: Danilo Del Busso <danilo.delbusso@cloud.com>
Signed-off-by: Danilo Del Busso <danilo.delbusso@cloud.com>
Exception is already subclass of stdlib exception `IOException` and there's no need to expose it explicitly

Signed-off-by: Danilo Del Busso <danilo.delbusso@cloud.com>
Also remove a redundant `List.map` in `java/main.ml`

Signed-off-by: Danilo Del Busso <danilo.delbusso@cloud.com>
Signed-off-by: Danilo Del Busso <danilo.delbusso@cloud.com>
`jackson-databind` to `2.16.1` and `httpclient5` to `5.4`

Signed-off-by: Danilo Del Busso <danilo.delbusso@cloud.com>
- Shorten docs for Java's `IOException`
- Use `String.concat` instead of multiple `^` calls in Java's `main.ml`
- Rename `pascal_case_` to `pascal_case_rec` to avoid confusion for the fuinction's consumers
- Use `{| |}` to replace escaped quotes when possible in Java's `main.ml`

Signed-off-by: Danilo Del Busso <danilo.delbusso@cloud.com>
- Deprecates `Connection` constructors that take in `requestTimeout` or `connectionTimeout`
- Add docs to tell user to use `Connection.client#setXYZ` instead
- Add setters to `JsonRpcClient`
- Replace `protected` with `private` in `JsonRpcClient` when possible, since API consumers will now be able to access `Connection.client`
- Make `Connection.client` `public`
- Fix misc formatting in `Connection`

Signed-off-by: Danilo Del Busso <danilo.delbusso@cloud.com>
…rsion

This version replicates what we do in the C# SDK, and uses fewer calls to fetch the API version

Signed-off-by: Danilo Del Busso <danilo.delbusso@cloud.com>
Signed-off-by: Danilo Del Busso <danilo.delbusso@cloud.com>
- Restore use of `Set` instead of `HashSet` to avoid consumer implementation breaking
- Remove use of raw types to avoid runtime exceptions
- Deprecate `toX(Object)` methods in `Types` since they should be internal
- Restore `toX(Object, Connection)` methods
- Supress unchecked cast warnings with annotation to remove them during compilation.

Signed-off-by: Danilo Del Busso <danilo.delbusso@cloud.com>
Adds "@deprecated (since = x )" annotations, and populates it for methods and fields.
Also adds "@deprecated xyz" in method JavaDocs.

Signed-off-by: Danilo Del Busso <danilo.delbusso@cloud.com>
xapi returns strings of the form "0.0" for `event.timestamp` fields.
This commit adds a new custom JSON deserializer to handle such values.

Signed-off-by: Danilo Del Busso <danilo.delbusso@cloud.com>
…-java_CP-45888

CP-45888: Replace XML-RPC client in Java SDK with JSON-RPC
These were errors, even if they were accidentally working

Signed-off-by: Pau Ruiz Safont <pau.ruizsafont@cloud.com>
…es any more.

Signed-off-by: Konstantina Chremmou <Konstantina.Chremmou@cloud.com>
…tes.

Signed-off-by: Konstantina Chremmou <Konstantina.Chremmou@cloud.com>
Signed-off-by: Bernhard Kaindl <bernhard.kaindl@cloud.com>
Christian Lindig and others added 17 commits April 30, 2024 15:58
Signed-off-by: Christian Lindig <christian.lindig@cloud.com>
…/CP-48195-instrument-forkexecd-client

CP-48195: Instrument client side of `forkexecd`
- Use `Types.checkError` instead of throwing a generic `XenAPIException`. This ensures the `Types.XYZ` family of exceptions are being used
- Use `@JsonValue` to ensure base class objects are deserialised as a simple opaque_ref string, as opposed to a mapping of each field. This ensures the API's behaviour is unchanged.
- Parse the results of `task.getResult` calls. The jsonrpc method returns value payloads of the form `"value" : "<value>OpaqueRef:XYZ</value>"` so we need to strip the surrounding XML

Signed-off-by: Danilo Del Busso <danilo.delbusso@cloud.com>
…e build @check

Bytecode builds for `http_lib` are disabled due to '(modes best)',
and that means that anything that depends on it must have it disabled too to avoid this warning.

Avoids these kinds of warnings:
```
File "_none_", line 1:
Error: Module `Buf_io' is unavailable (required by `Http_svr')
```

Signed-off-by: Edwin Török <edwin.torok@cloud.com>
Signed-off-by: Pau Ruiz Safont <pau.ruizsafont@cloud.com>
…ries

There were 3 modules with conflicting names with compiler libraries: Watch,
Debuginfo and Stats. Debuginfo was renamed, the others's libraries were changed
to be wrapped.

Signed-off-by: Pau Ruiz Safont <pau.ruizsafont@cloud.com>
Pinning the libraries runs dune subst, which needs a project name, define it

Signed-off-by: Pau Ruiz Safont <pau.ruizsafont@cloud.com>
Do not output loglines that are part of the normal operation. Use debug for
them, they are not usually logged, but can be enabled if need be by changing
the loglevel

Signed-off-by: Pau Ruiz Safont <pau.ruizsafont@cloud.com>
Signed-off-by: Pau Ruiz Safont <pau.ruizsafont@cloud.com>
New version got released

Signed-off-by: Pau Ruiz Safont <pau.ruizsafont@cloud.com>
The query on HTTP endpoint /updates will return the available updates in
JSON format. Prior to the changes in this commit, if a query arrives
when another query is being handled, a "GET_UPDATES_IN_PROGRESS" error
will be returned immediately. This behaviour is not friendly to GUI
client XenCenter.

In this commit, the behaviour is changed to wait and retry in handling
the query in xapi since the "*_IN_PROGGRESS" error is a transient
failure. Tolerating it in xapi (server) side avoids error handling in
client side.

With the change, the "GET_UPDATES_IN_PROGRESS" will not be an error
exposed to users any more. Therefore it is removed.

Signed-off-by: Ming Lu <ming.lu@cloud.com>
…389319

CA-389319: Wait and retry for GET_UPDATES_IN_PROGRESS
As part of a start, resources are allocated for a VM in "scheduled_to.."
fields. These need to be cleared if the start fails. It turned out that
this was incomplete for PCI slots and those were leaking. This patch
tries to be more systematical about it.

Signed-off-by: Christian Lindig <christian.lindig@cloud.com>
Signed-off-by: Bernhard Kaindl <bernhard.kaindl@cloud.com>
@bernhardkaindl bernhardkaindl requested a review from psafont May 10, 2024 13:33
@stephenchengCloud
Copy link
Contributor

Because the Python CI config was moved to other.yml from main.yml in the master branch recently, git merge cannot detect the conflicts in this newly added other.yml file. Need to manually move the changes in main.yml made by the feature/py3 branch to other.yml.
Check my comments above. Not sure if I missed anything. Really thank @bernhardkaindl for this, it's time-consuming work.

Copy link
Collaborator Author

@bernhardkaindl bernhardkaindl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because the Python CI config was moved to other.yml from main.yml in the master branch recently, git merge cannot detect the conflicts in this newly added other.yml file. Need to manually move the changes in main.yml made by the feature/py3 branch to other.yml.
Check my comments above. Not sure if I missed anything. Really thank @bernhardkaindl for this, it's time-consuming work.

Hi @stephenchengCloud, I reviewed and answered your comments above:

The locations you commented on are indeed correct as they are now.

This PR is correct to merge as it is now.

@liulinC
Copy link
Collaborator

liulinC commented May 21, 2024

Merge this PR for now, and fix the comments/issues in future branch.

@liulinC liulinC merged commit 2a2e962 into xapi-project:feature/py3 May 21, 2024
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.