CP-46060/feature/py3: Support running tests locally before pushing to GitHub

[bernhardkaindl]

Hi,
this is the 2nd PR in this series for CP-46060 (Update Xapi GitHub CI & test to use py3 only):

• With CA-390883, I was asked to rebase Some ideas for updating CI #5504 to feature/py3:

Summary:

Move the setup for running pytest from .github/workflows/main.yml to pyproject.toml and .pre-commit-config.yaml, to make it easy to run all tests locally using pre-commit (see below).

Details:

• Move the configuration of how to run pytest from a .github/workflows to pyproject.toml.
• Enable the support for running all other checks using pip install pre-commit;pre-commit run -av.
  • This was welcomed as well by @liulinC and @stephenchengCloud in Some ideas for updating CI #5504
• Fix the remaining pylint and pyright warnings in python3/.
  • Some of those have not already been in Some ideas for updating CI #5504, these areas:
    • just a few simple changes
    • maybe more than half of them are just comments that suppress warnings
    • The changes that are done are only a couple one-line and short updates.
    • Therefore, I think they are easy to review.
    • Add a few type annotations for mypy and static analysis (only a few)

Minor points also addressed:

• Clean-up the configuration for pytype_reporter:
  • Empty-out expected_to_fail as we are now focusing on the python3/ tree in feature/py3 branch.

[stephenchengCloud]

Thanks a lot for Bernhard's work. I was doing similar work in my private repo trying to sync CI changes in the master to the feature branch.
As most of the changes (#5575) are well-reviewed and already in the master branch, no further comments from me.

[psafont]

I would prefer if master was merged to feature/py3, then add the necessary patches on top (like the pytype and pyright fixes)

[bernhardkaindl]

On a personal note:

I would prefer if master was merged to feature/py3,

I was hoping that the Python3 branch would be a branch that is rebased on top of master using git rebase master for updating it, not using git merge.

But as branches that start with feature/ are prevented from being rebased on master, that would require a new Python3 branch.

[psafont]

I was hoping that the Python3 branch would be a branch

Unfortunately this won't work. Having merges is the only way we know all the commits in the feature branch have been double-reviewed when merging the feature branch into master. This is because github doesn't have an easy way to add the reviewed-by metadata to commits

Otherwise maintainers are forced to re-review all the commit in a long branch, again, wasting quite a bit of time.

[bernhardkaindl]

I would prefer if master was merged to feature/py3, then add the necessary patches on top (like the pytype and pyright fixes)

I got out of sync, and did not know that master had many of those updates and that these weren't merged into feature/py3.

I assumed that @stephenchengCloud would do regular merges of master into feature/py3.

I could wait for that to be done, but I'd like to complete my assignment of finishing CA-390883, where I was asked to rebase this to feature/py3.

[psafont]

I could wait for that to be done, but I'd like to complete my assignment of finishing CA-390883, where I was asked to rebase this to feature/py3.

My preference would be the responsibility to merge these changes to be transferred over to stephen and other working on the python3 feature so they can prioritise this to make it contributions easier as they see fit. From the maintenance point of view I merged the aspects of the CI that I thought were more pressing. I don't think it makes sense for you to wait on others and see when you can contribute this, I recommend you to close the ticket as done, as most of you contributions are already benefitting xen-api.

[bernhardkaindl]

My preference would be the responsibility to merge these changes to be transferred over to stephen and other working on the python3 feature so they can prioritise this to and use these contributions easier as they see fit.

Thanks, I gladly accept the recommended transfer of the responsibility to merge these changes to @stephenchengCloud so they can prioritise this to make it contributions easier as they see fit as preferred!

I don't think it makes sense for you to wait on others and see when you can contribute this, I recommend you to close the ticket as done, as most of you contributions are already benefitting xen-api.

Thanks, I'm glad! I'll close CA-390883 that was about updating the old PRs as done as recommended!

[stephenchengCloud]

Thanks @bernhardkaindl and @psafont for the discussion. I will do the syncing and include changes of this PR.

Updated:
I just saw Bernhard had raised another PR (#5624) for syncing master branch. Thanks.

[bernhardkaindl]

Thanks, merged with #5641.