Skip to content

Commit 72f7800

Browse files
p-wysockialmilosz
andauthored
[PyOV] Migrate from pre-commit git hook to automatic PR for stubs generation (#30744)
~~Blocked: currently waiting for a formal approval from security team~~ ### Details: - Example PR generated by the GHA job to my fork: p-wysocki#182 - The PR is automatically generated daily on weekdays - The PR can also be generated manually - The new GHA job is working correctly on my fork, there may be some corrections needed to get it working on OpenVINO master (for example, in repository settings the ability for GHA to make PRs has to be checked, also I can't test openvino_provider tool on fork since it doesn't have artifacts mounted) - This approach is miles better than unstable per-dev-machine generation - The hook install step has been replaced with hook uninstall step, which will be removed altogether in the future - Removed the CI check for keeping the stubs updated ### Tickets: - CVS-167922 --------- Signed-off-by: p-wysocki <przemyslaw.wysocki@intel.com> Co-authored-by: Alicja Miloszewska <alicja.miloszewska@intel.com>
1 parent 4a4b0d3 commit 72f7800

File tree

7 files changed

+77
-195
lines changed

7 files changed

+77
-195
lines changed

.github/actions/validate_pyi_files/action.yml

Lines changed: 0 additions & 14 deletions
This file was deleted.

.github/actions/validate_pyi_files/compare_pyi_files.py

Lines changed: 0 additions & 122 deletions
This file was deleted.

.github/workflows/job_python_api_tests.yml

Lines changed: 0 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -94,10 +94,6 @@ jobs:
9494
- name: Install Python API tests dependencies
9595
run: python3 -m pip install -r ${INSTALL_TEST_DIR}/bindings/python/requirements_test.txt
9696

97-
- name: Verify Python stub .pyi files
98-
uses: ./.github/actions/validate_pyi_files
99-
continue-on-error: true
100-
10197
#
10298
# Tests
10399
#
Lines changed: 63 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,63 @@
1+
name: Update Python API Stubs
2+
3+
permissions:
4+
contents: write
5+
pull-requests: write
6+
7+
on:
8+
schedule:
9+
- cron: "0 2 * * 1-5"
10+
workflow_dispatch: # Allow manual triggering
11+
12+
jobs:
13+
update-stubs:
14+
runs-on: ubuntu-latest
15+
steps:
16+
- name: Checkout repository
17+
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
18+
19+
- name: Set up Python
20+
uses: actions/setup-python@0ae58361cdfd39e2950bed97a1e26aa20c3d8955 # v4
21+
with:
22+
python-version: '3.10'
23+
24+
- name: Get latest OpenVINO artifacts
25+
id: openvino_download
26+
uses: openvinotoolkit/openvino/.github/actions/openvino_provider@master
27+
with:
28+
platform: 'ubuntu22'
29+
revision: latest_available_commit
30+
31+
- name: Install OpenVINO and stubgen
32+
run: |
33+
python -m pip install openvino==${{ steps.openvino_download.outputs.ov_version }} ${{ steps.openvino_download.outputs.ov_wheel_source }}
34+
STUBGEN_VERSION=$(grep pybind11-stubgen src/bindings/python/requirements_test.txt)
35+
python -m pip install "$STUBGEN_VERSION"
36+
37+
- name: Regenerate Python API stubs
38+
run: |
39+
find ./ -name "*.pyi" -delete
40+
python src/bindings/python/scripts/generate_pyapi_stubs.py
41+
42+
- name: Add and commit .pyi files
43+
run: |
44+
git config --local user.email "action@github.com"
45+
git config --local user.name "GitHub Action"
46+
find src/bindings/python -name "*.pyi" -exec git add {} \;
47+
git diff --quiet && git diff --staged --quiet || git commit -m "Update Python API stub files"
48+
49+
- name: Create Pull Request
50+
uses: peter-evans/create-pull-request@271a8d0340265f705b14b6d32b9829c1cb33d45e # v7.0.8
51+
with:
52+
token: ${{ secrets.GITHUB_TOKEN }}
53+
commit-message: Update Python API stub files
54+
title: "[PyOV] Update Python API stub files"
55+
signoff: true
56+
sign-commits: true
57+
reviewers: openvinotoolkit/openvino-ie-python-api-maintainers
58+
body: |
59+
- Updated Python API stub (.pyi) files from the latest available artifacts
60+
61+
Auto-generated by GitHub Actions
62+
branch: update-pyapi-stubs
63+
base: master

src/bindings/python/CMakeLists.txt

Lines changed: 8 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -416,6 +416,7 @@ if(OpenVINODeveloperPackage_FOUND)
416416
endif()
417417

418418
# only for developers, skip on CI
419+
# to be removed once developers remove hook from their .git: CVS-168334
419420
if(NOT (DEFINED ENV{CI} OR DEFINED ENV{TF_BUILD} OR DEFINED ENV{JENKINS_URL}))
420421

421422
# check pybind11-stubgen package
@@ -436,15 +437,15 @@ if(NOT (DEFINED ENV{CI} OR DEFINED ENV{TF_BUILD} OR DEFINED ENV{JENKINS_URL}))
436437
message(WARNING "pre-commit package is not installed or version is not detected. This package is necessary for developing Python API.")
437438
endif()
438439

439-
# attach the pre-commit hook for Python stub generation
440-
execute_process(COMMAND pre-commit install -c "${CMAKE_CURRENT_SOURCE_DIR}/scripts/stub_generation_precommit.yaml"
441-
RESULT_VARIABLE precommit_install_result
442-
ERROR_VARIABLE precommit_install_error
440+
# uninstall the pre-commit hook for Python stub generation (now handled by GitHub Actions)
441+
execute_process(COMMAND pre-commit uninstall
442+
RESULT_VARIABLE precommit_uninstall_result
443+
ERROR_VARIABLE precommit_uninstall_error
443444
OUTPUT_STRIP_TRAILING_WHITESPACE
444445
ERROR_STRIP_TRAILING_WHITESPACE)
445-
if(NOT precommit_install_result EQUAL "0")
446-
message(WARNING "Python stub generation pre-commit hook install failed with error: ${precommit_install_error}. It is necessary for developing Python API.")
446+
if(NOT precommit_uninstall_result EQUAL "0")
447+
message(WARNING "Failed to uninstall Python stub generation pre-commit hook: ${precommit_uninstall_error}")
447448
else()
448-
message(STATUS "pre-commit hook for Python API stub generation has been installed")
449+
message(STATUS "Pre-commit hook for Python API stub generation has been uninstalled. Stubs are now generated by GitHub Actions workflow.")
449450
endif()
450451
endif()

src/bindings/python/docs/stubs.md

Lines changed: 6 additions & 30 deletions
Original file line numberDiff line numberDiff line change
@@ -6,20 +6,9 @@ Stub files (`.pyi`) are used to provide type hints for Python code. They describ
66

77
## Automation of stub file generation in OpenVINO
88

9-
In OpenVINO, the generation of stub files is automated as part of the development workflow. When building the Python API for the first time, a Git pre-commit hook is installed into the OpenVINO repository's `.git` directory. The related Python dependencies are `pybind11-stubgen` for stub generation and `pre-commit` for automating git hooks.
9+
In OpenVINO, the generation of stub files is automated as part of the development workflow. This is now handled by a GitHub Actions workflow that runs on a schedule (weekdays at 2:00 AM UTC) and can be triggered manually when needed. The workflow is defined in [.github/workflows/update_pyapi_stubs.yml](../../../../.github/workflows/update_pyapi_stubs.yml). The related Python dependency is `pybind11-stubgen` for stub generation.
1010

11-
### What is a git pre-commit hook?
12-
13-
A Git pre-commit hook is a script that runs automatically before a commit is finalized. It allows developers to enforce specific checks or perform automated tasks, such as code formatting, linting, or file generation, ensuring that the repository maintains a consistent and high-quality state.
14-
15-
### What happens during the pre-commit hook?
16-
17-
When the pre-commit hook is triggered, the following steps occur:
18-
19-
1. **Stub file generation**: The `pybind11-stubgen` tool is executed to generate new `.pyi` stub files for the OpenVINO Python API. This tool uses the Python package `openvino` to extract type information and create the stub files, so it's important that currently installed OpenVINO version contains all changes related to your Pull Request.
20-
2. **Automatic addition to commit**: The newly generated `.pyi` files are automatically staged and added to the current commit. This ensures that the stub files are always up-to-date with the latest changes in the Python API.
21-
22-
### Ensuring changes are installed
11+
### Running stub generation manually
2312

2413
To ensure that `pybind11-stubgen` works correctly, the `openvino` Python package must include your latest changes. This can be achieved by:
2514

@@ -31,23 +20,10 @@ To ensure that `pybind11-stubgen` works correctly, the `openvino` Python package
3120

3221
- Installing an up-to-date wheel containing your changes using `pip install`.
3322

34-
### Skipping the pre-commit hook
35-
36-
If you need to skip the pre-commit hook for any reason, you can do so by setting the `SKIP` environment variable before running the `git commit` command:
37-
38-
```bash
39-
SKIP=generate_stubs git commit -m "Your commit message"
40-
```
41-
42-
This bypasses the stub file generation process, allowing you to commit without triggering the hook.
43-
44-
### Uninstalling the pre-commit hook
45-
46-
If you wish to remove the hook, you can run:
47-
48-
```bash
49-
pre-commit uninstall
50-
```
23+
Then, to run stub generation manually, ensure your changes are installed and run:
24+
```bash
25+
python openvino/src/bindings/python/scripts/generate_pyapi_stubs.py
26+
```
5127

5228
## See also
5329

src/bindings/python/scripts/stub_generation_precommit.yaml

Lines changed: 0 additions & 18 deletions
This file was deleted.

0 commit comments

Comments
 (0)