Skip to content

Speed up CLI tests #560

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 5 commits into from
Mar 21, 2025
Merged

Speed up CLI tests #560

merged 5 commits into from
Mar 21, 2025

Conversation

yousefmoazzam
Copy link
Collaborator

Fixes #558

Checklist

  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes
  • I have made corresponding changes to the documentation

@yousefmoazzam
Replace subprocess-launching in CLI version test
c8a5103
@yousefmoazzam
Replace subprocess-launching in CLI help test
83271bc
@yousefmoazzam
Replace CLI no-args test to check correct behaviour
2d99fea 
Prior to this change the test was checking that, when running the CLI
with no args, if an error was raised, to ensure a specific exit code was
returned.

However, when the CLI is given no args, an error is not expected to be
raised; instead, the help message is expected to be displayed.
Therefore, the test was checking for behaviour that the CLI never
exhibited.

Also, this change replaces the subprocess-launching in the test with
testing functionality provided by `click` to run the CLI.
@yousefmoazzam
Replace subprocess-launching in CLI check command test
d3fcf86
@yousefmoazzam
Replace subprocess-launching in CLI GPU ID flag test
d93ec95
@yousefmoazzam
Copy link
Collaborator Author

The following numbers are for running the tests in test_cli.,py

Before: ~10s

(/dls/science/users/twi18192/conda-envs/httomo-new) [twi18192@pc0074 httomo (main)]$ pytest --durations=0 tests/test_cli.py
========================================== test session starts ===========================================
platform linux -- Python 3.12.9, pytest-8.3.4, pluggy-1.5.0 -- /dls/science/users/twi18192/conda-envs/httomo-new/bin/python3.12
cachedir: .pytest_cache
rootdir: /dls/science/users/twi18192/httomo
configfile: pyproject.toml
plugins: xdist-3.6.1, mock-3.14.0, cov-6.0.0, mpi-0.6
collected 17 items

tests/test_cli.py::test_cli_version_shows_version PASSED                                           [  5%]
tests/test_cli.py::test_cli_help_shows_help PASSED                                                 [ 11%]
tests/test_cli.py::test_cli_noargs_raises_error PASSED                                             [ 17%]
tests/test_cli.py::test_cli_check_pass_data_file PASSED                                            [ 23%]
tests/test_cli.py::test_cli_pass_gpu_id PASSED                                                     [ 29%]
tests/test_cli.py::test_cli_transforms_memory_limits[0-0] PASSED                                   [ 35%]
tests/test_cli.py::test_cli_transforms_memory_limits[500-500] PASSED                               [ 41%]
tests/test_cli.py::test_cli_transforms_memory_limits[500k-512000] PASSED                           [ 47%]
tests/test_cli.py::test_cli_transforms_memory_limits[1M-1048576] PASSED                            [ 52%]
tests/test_cli.py::test_cli_transforms_memory_limits[1m-1048576] PASSED                            [ 58%]
tests/test_cli.py::test_cli_transforms_memory_limits[3g-3221225472] PASSED                         [ 64%]
tests/test_cli.py::test_cli_transforms_memory_limits[3.2g-3435973836] PASSED                       [ 70%]
tests/test_cli.py::test_cli_fails_transforming_memory_limits[abcd] PASSED                          [ 76%]
tests/test_cli.py::test_cli_fails_transforming_memory_limits[nolimit] PASSED                       [ 82%]
tests/test_cli.py::test_cli_fails_transforming_memory_limits[124A] PASSED                          [ 88%]
tests/test_cli.py::test_cli_fails_transforming_memory_limits[23ki] PASSED                          [ 94%]
tests/test_cli.py::test_output_folder_name_correctly_sets_run_out_dir_global_constant PASSED       [100%]

=========================================== slowest durations ============================================
2.37s call     tests/test_cli.py::test_cli_version_shows_version
1.96s call     tests/test_cli.py::test_cli_check_pass_data_file
1.91s call     tests/test_cli.py::test_cli_pass_gpu_id
1.90s call     tests/test_cli.py::test_cli_noargs_raises_error
1.85s call     tests/test_cli.py::test_cli_help_shows_help

(46 durations < 0.005s hidden.  Use -vv to show these durations.)
========================================== 17 passed in 10.61s ===========================================

After: <1s

(/dls/science/users/twi18192/conda-envs/httomo-new) [twi18192@pc0074 httomo (speed-up-cli-tests)]$ pytest --durations=0 tests/test_cli.py
========================================== test session starts ===========================================
platform linux -- Python 3.12.9, pytest-8.3.4, pluggy-1.5.0 -- /dls/science/users/twi18192/conda-envs/httomo-new/bin/python3.12
cachedir: .pytest_cache
rootdir: /dls/science/users/twi18192/httomo
configfile: pyproject.toml
plugins: xdist-3.6.1, mock-3.14.0, cov-6.0.0, mpi-0.6
collected 17 items

tests/test_cli.py::test_cli_version_shows_version PASSED                                           [  5%]
tests/test_cli.py::test_cli_help_shows_help PASSED                                                 [ 11%]
tests/test_cli.py::test_cli_noargs_shows_help PASSED                                               [ 17%]
tests/test_cli.py::test_cli_check_pass_data_file PASSED                                            [ 23%]
tests/test_cli.py::test_cli_pass_gpu_id PASSED                                                     [ 29%]
tests/test_cli.py::test_cli_transforms_memory_limits[0-0] PASSED                                   [ 35%]
tests/test_cli.py::test_cli_transforms_memory_limits[500-500] PASSED                               [ 41%]
tests/test_cli.py::test_cli_transforms_memory_limits[500k-512000] PASSED                           [ 47%]
tests/test_cli.py::test_cli_transforms_memory_limits[1M-1048576] PASSED                            [ 52%]
tests/test_cli.py::test_cli_transforms_memory_limits[1m-1048576] PASSED                            [ 58%]
tests/test_cli.py::test_cli_transforms_memory_limits[3g-3221225472] PASSED                         [ 64%]
tests/test_cli.py::test_cli_transforms_memory_limits[3.2g-3435973836] PASSED                       [ 70%]
tests/test_cli.py::test_cli_fails_transforming_memory_limits[abcd] PASSED                          [ 76%]
tests/test_cli.py::test_cli_fails_transforming_memory_limits[nolimit] PASSED                       [ 82%]
tests/test_cli.py::test_cli_fails_transforming_memory_limits[124A] PASSED                          [ 88%]
tests/test_cli.py::test_cli_fails_transforming_memory_limits[23ki] PASSED                          [ 94%]
tests/test_cli.py::test_output_folder_name_correctly_sets_run_out_dir_global_constant PASSED       [100%]

=========================================== slowest durations ============================================
0.07s call     tests/test_cli.py::test_cli_pass_gpu_id
0.02s call     tests/test_cli.py::test_cli_check_pass_data_file

(49 durations < 0.005s hidden.  Use -vv to show these durations.)
=========================================== 17 passed in 0.68s ==========================================

@yousefmoazzam yousefmoazzam merged commit 3111f23 into main Mar 21, 2025
5 of 8 checks passed
@yousefmoazzam yousefmoazzam deleted the speed-up-cli-tests branch March 21, 2025 12:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

CLI tests launching a subprocess each take >1s to run
1 participant
@yousefmoazzam