Generate api docs for EESSI test suite #319

xinan1911 · 2024-10-10T11:43:38Z

A dependency of this PR is on EESSI/test-suite#192 as PR 192 resolves the build error for API docs

xinan1911 · 2024-10-11T15:21:53Z

@casparvl Adjusted the deploy and schedule. This is ready for test

warning message: The following actions use a deprecated Node.js version and will be forced to run on node20: actions/checkout@93ea575, actions/setup-python@13ae5bb. For more info: https://github.blog/changelog/2024-03-07-github-actions-all-actions-will-run-on-node20-instead-of-node16-by-default/

casparvl · 2024-11-13T14:33:33Z

Just to log for myself: to test this, one needs to

Git clone this feature branch
Clone the test suite in an src subdir: mkdir src && cd src && git clone https://github.com/EESSI/test-suite.git
Run mkdocs serve -a localhost:<someport>
Create a reverse tunnel to that node using ssh -L <someport>:localhost:<someport> <host_running_mkdocs_serve>
Go to localhost:<someport> in your browser.

mkdocs.yml

casparvl · 2024-11-13T14:37:36Z

.github/workflows/test.yml

+      uses: actions/checkout@v4
+      with:
+        repository: eessi/test-suite
+        path: src


Why don't we just do this without path, i.e. in the current workdir? It will still clone into a subidr test-suite by default right? No need to nest that further inside a src I'd think.

casparvl · 2024-11-13T14:38:08Z

docs/gen_ref_pages.py

+# need to adjust to the test suite hook
+#root = Path(__file__).parent.parent
+
+TEST_SUITE = "src/test-suite"


Same here, this would then just be `"test-suite"

casparvl · 2024-11-13T14:39:06Z

docs/gen_ref_pages.py

+
+import mkdocs_gen_files
+
+# need to adjust to the test suite hook


There seem to be quite some comments that are no longer relevant in this gen_ref_pages.py. Would be good to clean those up.

casparvl · 2024-11-13T14:39:55Z

mkdocs.yml

@@ -43,7 +43,6 @@ nav:
    - Set up environment: using_eessi/setting_up_environment.md
    - Basic commands: using_eessi/basic_commands.md
    - Demos: using_eessi/eessi_demos.md
-    - EESSI in CI: using_eessi/eessi_in_ci.md


Not sure why this was removed here, but this has a conflict with upstream that needs to be resolved anyway. I guess that'll sort this out too.

casparvl · 2024-11-13T14:41:24Z

mkdocs.yml

+      default_handler: python
+      handlers:
+        python:
+          paths: [src/test-suite] 


This could also be shortened to paths: [test-suite], see the above changes.

casparvl · 2024-11-13T14:46:30Z

.github/workflows/deploy.yml

+      uses: actions/checkout@v4
+      with:
+        repository: eessi/test-suite
+        path: src


Why don't we just do this without path, i.e. in the current workdir? It will still clone into a subidr test-suite by default right? No need to nest that further inside a src I'd think.

casparvl · 2024-11-13T14:51:53Z

.github/workflows/deploy.yml

@@ -1,6 +1,9 @@
 # documentation: https://help.github.com/en/articles/workflow-syntax-for-github-actions
 name: deploy documentation (only on push to main branch)
 on:
+  schedule: 


Actually, I don't think we want this in the regular deploy.yml. Having a cronjob that directly deploys the API docs means we have no human in the loop to check the changes to the API docs, and whether they make sense. I think the way this was implemented for the software overview in https://github.com/EESSI/docs/blob/main/.github/workflows/update_available_software.yml makes more sense for now: do a build of the docs, then create a PR based on that. It means the auto-generated docs will actually be part of this repo, i.e. the physical *.md files will be files within the EESSI/docs repo. That'll allow us to do a check on them before they get deployed.

Long term, we might want to do this fully automatically, without human in the loop. But to start with, I think the human in the loop would be a good thing (even if someone needs to keep an eye on those PRs and merge them).

casparvl

I think the main change I'd suggest here is to not do the API docs generation as part of the deploy.yml, but in a way similar to how the update_available_software.yml workflow works (i.e. copy that workflow, and adapt it to generate the API docs instead of the available software list). This workflow creates a PR to EESSI/docs every X time (on a cronjob), which allows us to have a human in the loop to check what was generated and if that makes sense.

In the future, we could even see if the workflow could be triggered on push to the main branch of EESSI/test-suite. It seems something like that should be possible https://medium.com/hostspaceng/triggering-workflows-in-another-repository-with-github-actions-4f581f8e0ceb but that's no priority right now.

…hat's fine

…ure branch

…st.yml

… update_testsuite_apidocs

laraPPr · 2025-05-06T10:19:52Z

docs/generate_eessi_testsuite_api_docs.py

@@ -0,0 +1,50 @@
+"""


This file should be added to ´scripts/generate_eessi_test-suite_api_docs´ with a README.md and a seperate requiremnets.txt file to stay consistent with what we did for the software_overview page.

You also mentioned this on slack

Yes, but you'll need to create a subdir test-suite, then clone the test suite in there (yes, that gives you a test-suite/test-suite dir...)

Can you also add that to the README.md

And if we need to create a subdir with test-suite we should also add test-suite to .gitignore

Hm, I think there's some misunderstanding here. I'm not doing the same as the software overview page. I was planning to do that, but it turns out that is fundamentally impossible, because the way mkdocs autogenerates API docs is not by creating *.md files in the source directory (i.e. docs/docs), but by directly creating files in the site directory. I.e. those are not (and should not be) part of the sources - they are what you get after you 'build' the docs.

So, integrating the requirements for this into the regular requirements file is completely valid: the API docs are now an integral part of the docs. As soon as you generate the documentation with an mkdocs serve or similar command, it will create the API docs.

Can you also add that to the README.md

That's actually not a bad idea, but should probably be in the README.md of the main docs repo.

And if we need to create a subdir with test-suite we should also add test-suite to .gitignore

We could. I wouldn't consider it super essential, since you'd only checkout the test-suite when building the docs - and all of that happens in a CI workflow anyway - but sure, if you'd build it locally and don't want to see the test-suite if you doe git status... why not :)

laraPPr · 2025-05-06T10:34:00Z

docs/generate_eessi_testsuite_api_docs.py

+nav = mkdocs_gen_files.Nav()
+
+# Loop through all python files in test-suite/eessi
+for path in sorted(Path(f"{TEST_SUITE}/eessi/").rglob("*.py")):


I would like some logging and error reporting because I had not added the subdir test-suite and the script was just ran doing nothing. But I believe it should error out when it cannot find the test-suite subdir.

Good point, I'll look at that.

I would like some logging and error reporting because I had not added the subdir test-suite and the script was just ran doing nothing. But I believe it should error out when it cannot find the test-suite subdir.

@casparvl will you still return to this comment? Or should we just merge and leave this for later?

laraPPr · 2025-05-06T11:05:28Z

I tried but I'm getting this error

INFO    -  DeprecationWarning: The `get_logger` function is deprecated.

             File

           "/home/lara/.local/lib/python3.9/site-packages/_griffe/agents/nodes/runtime.py",

           line 17, in <module>

               _logger = get_logger("griffe.agents.nodes._runtime")

             File

           "/home/lara/.local/lib/python3.9/site-packages/_griffe/logger.py",

           line 117, in get_logger

               warnings.warn("The `get_logger` function is deprecated.",

           DeprecationWarning, stacklevel=1)

INFO    -  DeprecationWarning: The `name` parameter is deprecated.

             File

           "/home/lara/.local/lib/python3.9/site-packages/_griffe/docstrings/google.py",

           line 49, in <module>

               _warn = docstring_warning("griffe.docstrings.google")

             File

           "/home/lara/.local/lib/python3.9/site-packages/_griffe/docstrings/utils.py",

           line 83, in docstring_warning

               warnings.warn("The `name` parameter is deprecated.",

           DeprecationWarning, stacklevel=1)

INFO    -  DeprecationWarning: invalid escape sequence \S

             File

           "/home/lara/.local/lib/python3.9/site-packages/_griffe/agents/visitor.py",

           line 203, in get_module

               top_node = compile(self.code, mode="exec",

           filename=str(self.filepath), flags=ast.PyCF_ONLY_AST, optimize=1)

             File

           "/home/lara/Documents/EESSI/docs/test-suite/test-suite/eessi/testsuite/tests/apps/tensorflow/tensorflow.py",

           line 55, in

               accuracy = sn.extractsingle('^Final accuracy: (?P<accuracy>\S+)',

           self.stdout, 'accuracy', float)  # noqa: W605

WARNING -  griffe: test-suite/test-suite/eessi/testsuite/common_config.py:76:

           Failed to parse field directive from ':param: prefix: prefix for the

           report_file'

I was running with python 3.9.21

laraPPr · 2025-05-06T12:24:48Z

I tried but I'm getting this error

INFO    -  DeprecationWarning: The `get_logger` function is deprecated.

             File

           "/home/lara/.local/lib/python3.9/site-packages/_griffe/agents/nodes/runtime.py",

           line 17, in <module>

               _logger = get_logger("griffe.agents.nodes._runtime")

             File

           "/home/lara/.local/lib/python3.9/site-packages/_griffe/logger.py",

           line 117, in get_logger

               warnings.warn("The `get_logger` function is deprecated.",

           DeprecationWarning, stacklevel=1)

INFO    -  DeprecationWarning: The `name` parameter is deprecated.

             File

           "/home/lara/.local/lib/python3.9/site-packages/_griffe/docstrings/google.py",

           line 49, in <module>

               _warn = docstring_warning("griffe.docstrings.google")

             File

           "/home/lara/.local/lib/python3.9/site-packages/_griffe/docstrings/utils.py",

           line 83, in docstring_warning

               warnings.warn("The `name` parameter is deprecated.",

           DeprecationWarning, stacklevel=1)

INFO    -  DeprecationWarning: invalid escape sequence \S

             File

           "/home/lara/.local/lib/python3.9/site-packages/_griffe/agents/visitor.py",

           line 203, in get_module

               top_node = compile(self.code, mode="exec",

           filename=str(self.filepath), flags=ast.PyCF_ONLY_AST, optimize=1)

             File

           "/home/lara/Documents/EESSI/docs/test-suite/test-suite/eessi/testsuite/tests/apps/tensorflow/tensorflow.py",

           line 55, in

               accuracy = sn.extractsingle('^Final accuracy: (?P<accuracy>\S+)',

           self.stdout, 'accuracy', float)  # noqa: W605

WARNING -  griffe: test-suite/test-suite/eessi/testsuite/common_config.py:76:

           Failed to parse field directive from ':param: prefix: prefix for the

           report_file'

I was running with python 3.9.21

My mistake it was working it is just not very intuitive how to navigate the pages right now. Because you have so many empty pages might it be possible to add urls to the empty pages to the pages that are underneat it?

casparvl · 2025-05-07T14:18:40Z

Ah yeah,

WARNING -  griffe: test-suite/test-suite/eessi/testsuite/common_config.py:76:

           Failed to parse field directive from ':param: prefix: prefix for the

           report_file'

Is actually a bug we had in the test suite (incorrect docstring, because we have the :param: prefix:, which is not correct.

Fix EESSI/docs#319 (comment)

Signed-off-by: laraPPr <lara.peeters@ugent.be>

laraPPr · 2025-06-25T11:04:56Z

Ok I wanted to get this one over the finish line so made the updates that I wanted.
but now it is failing with

ERROR   -  mkdocstrings: eessi.testsuite.tests.apps.openfoam.openfoam could

           not be found

ERROR   -  Error reading page

           'testsuite_api/eessi/testsuite/tests/apps/openfoam/openfoam.md':

ERROR   -  Could not collect 'eessi.testsuite.tests.apps.openfoam.openfoam'

laraPPr · 2025-06-25T11:07:02Z

Ok I wanted to get this one over the finish line so made the updates that I wanted. but now it is failing with

ERROR   -  mkdocstrings: eessi.testsuite.tests.apps.openfoam.openfoam could

           not be found

ERROR   -  Error reading page

           'testsuite_api/eessi/testsuite/tests/apps/openfoam/openfoam.md':

ERROR   -  Could not collect 'eessi.testsuite.tests.apps.openfoam.openfoam'

@satishskamath Can you look into those missing docstrings?

laraPPr · 2025-06-25T11:07:23Z

I'll also try to resolve the merge conflicts

satishskamath · 2025-06-26T11:36:05Z

Ok I wanted to get this one over the finish line so made the updates that I wanted. but now it is failing with
ERROR   -  mkdocstrings: eessi.testsuite.tests.apps.openfoam.openfoam could

           not be found

ERROR   -  Error reading page

           'testsuite_api/eessi/testsuite/tests/apps/openfoam/openfoam.md':

ERROR   -  Could not collect 'eessi.testsuite.tests.apps.openfoam.openfoam'
@satishskamath Can you look into those missing docstrings?

Will take a look into it.

Signed-off-by: laraPPr <lara.peeters@ugent.be>

This file added for generation of docs. Refer to the following PR: EESSI/docs#319

satishskamath · 2025-06-26T12:40:18Z

@laraPPr Created EESSI/test-suite#271 to fix the openfoam ERRORs here. Can you merge it?

satishskamath · 2025-06-26T12:43:25Z

docs/generate_eessi_testsuite_api_docs.py

+
+    # If something is an __init__, use the directory name as the name of the python module instead of the filename
+    parts = list(module_path.parts)
+    if parts[-1] == "__init__":


I think the OpenFOAM PATH was not created properly because of this if condition, it does not have an else.

Signed-off-by: laraPPr <lara.peeters@ugent.be>

.github/workflows/test.yml

Signed-off-by: laraPPr <lara.peeters@ugent.be>

…test-suite Signed-off-by: laraPPr <lara.peeters@ugent.be>

…ain branch Signed-off-by: laraPPr <lara.peeters@ugent.be>

laraPPr · 2025-06-26T14:14:43Z

@casparvl we are gonna need a new release of the test-suite but than this pr should be ready to merge

generate api docs

30b717a

xinan1911 requested a review from casparvl October 10, 2024 11:43

xinan1911 mentioned this pull request Oct 10, 2024

Resolve error for generating api docs EESSI/test-suite#192

Merged

xinan1911 added 3 commits October 11, 2024 17:18

Update deploy.yml and schedule

3fdb896

Update deploy.yml syntax

80f2c3a

Update test.yml

5c7ef65

xinan1911 added 5 commits October 14, 2024 10:52

Update deploy.yml for action version

65131b8

Update test.yml

1994da4

Update test.yml resolve failed build

0004eb3

Update test.yml actions/setup-python@v5

12dc637

casparvl reviewed Nov 13, 2024

View reviewed changes

mkdocs.yml Outdated Show resolved Hide resolved

casparvl reviewed Nov 13, 2024

View reviewed changes

casparvl requested changes Nov 13, 2024

View reviewed changes

boegel changed the title ~~Generate api docs~~ Generate api docs for EESSI test suite Nov 13, 2024

Caspar van Leeuwen and others added 7 commits April 28, 2025 12:38

Moved location of testsuite API docs, as well as subdir

9ca30ae

Resolved conflicts with upstream

170da31

Don't clone into src subdir, the subdir will already be test-suite, t…

9a027b2

…hat's fine

Put back EESSI in CI section, it was unintendely removed in this feat…

422a20e

…ure branch

Also remove the subdir for the checkout of EESSI test-suite in the te…

dd28578

…st.yml

Expand update_generated_time functionality so that we may reuse it in…

b870f15

… update_testsuite_apidocs

Make this a manual action

0d242ef

laraPPr requested changes May 6, 2025

View reviewed changes

casparvl pushed a commit to casparvl/test-suite that referenced this pull request May 15, 2025

Fix EESSI/docs#319 (comment)

e995a84

casparvl mentioned this pull request May 15, 2025

Fixes syntax issue in docstring that blocks automatic API doc generation EESSI/test-suite#265

Merged

satishskamath added a commit to EESSI/test-suite that referenced this pull request May 15, 2025

Merge pull request #265 from casparvl/fix_api_docstring

b18cdfd

Fix EESSI/docs#319 (comment)

add error for when test-suite repo is not available

3364006

Signed-off-by: laraPPr <lara.peeters@ugent.be>

resolve merge conflicts

25e5c37

Signed-off-by: laraPPr <lara.peeters@ugent.be>

satishskamath added a commit to EESSI/test-suite that referenced this pull request Jun 26, 2025

Create __init__.py

fb92676

This file added for generation of docs. Refer to the following PR: EESSI/docs#319

satishskamath mentioned this pull request Jun 26, 2025

Create __init__.py for directory OpenFOAM EESSI/test-suite#271

Merged

satishskamath reviewed Jun 26, 2025

View reviewed changes

laraPPr added 5 commits June 26, 2025 15:08

updated the versions of the actions in the pr

4d9deda

Signed-off-by: laraPPr <lara.peeters@ugent.be>

update github actions

3c5ee20

Signed-off-by: laraPPr <lara.peeters@ugent.be>

fix workflow. hopefully?

29dbb15

Signed-off-by: laraPPr <lara.peeters@ugent.be>

fix workflow. With a little help

b92db86

Signed-off-by: laraPPr <lara.peeters@ugent.be>

fix now test.yml as well

8a3aa5d

Signed-off-by: laraPPr <lara.peeters@ugent.be>

ocaisa reviewed Jun 26, 2025

View reviewed changes

.github/workflows/test.yml Outdated Show resolved Hide resolved

laraPPr added 6 commits June 26, 2025 15:28

another fix of the CI

fb59aab

Signed-off-by: laraPPr <lara.peeters@ugent.be>

fix typo in workflow

25c86d5

make sure that the test-suite is available in the right location

f213810

Signed-off-by: laraPPr <lara.peeters@ugent.be>

don't use fetch-true

6c593b3

Signed-off-by: laraPPr <lara.peeters@ugent.be>

check if the CI is now building the docs with the main branch of the …

d831e05

…test-suite Signed-off-by: laraPPr <lara.peeters@ugent.be>

make sure that the latest release of test-suite is used and not the m…

2597103

…ain branch Signed-off-by: laraPPr <lara.peeters@ugent.be>


		import mkdocs_gen_files

		# need to adjust to the test suite hook

Generate api docs for EESSI test suite #319

Are you sure you want to change the base?

Generate api docs for EESSI test suite #319

Uh oh!

Conversation

xinan1911 commented Oct 10, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

xinan1911 commented Oct 11, 2024

Uh oh!

casparvl commented Nov 13, 2024

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

casparvl Nov 13, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

casparvl left a comment

Choose a reason for hiding this comment

Uh oh!

laraPPr May 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

laraPPr commented May 6, 2025

Uh oh!

laraPPr commented May 6, 2025

Uh oh!

casparvl commented May 7, 2025

Uh oh!

laraPPr commented Jun 25, 2025

Uh oh!

laraPPr commented Jun 25, 2025

Uh oh!

laraPPr commented Jun 25, 2025

Uh oh!

satishskamath commented Jun 26, 2025

Uh oh!

satishskamath commented Jun 26, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

laraPPr commented Jun 26, 2025

Uh oh!

Uh oh!

xinan1911 commented Oct 10, 2024 •

edited

Loading

casparvl Nov 13, 2024 •

edited

Loading

laraPPr May 6, 2025 •

edited

Loading