Skip to content

Updating tests for stability #2261

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
May 27, 2025
Merged

Updating tests for stability #2261

merged 2 commits into from
May 27, 2025

Conversation

bistline
Copy link
Contributor

@bistline bistline commented May 20, 2025

This fixes two tests that have become unstable in recent weeks.

  1. StudyValidationTest#test_should_fail_all_ingest_pipeline_parse_jobs: a race condition is happened during cleanup after failed parse jobs. The files are correctly marked as 'failed', but underlying source data doesn't always clean up in time before an assertion is checked. A 5 second sleep has been added to help avoid this
  2. SummaryStatsUtilsTest#test_should_get_data_retention_policy_report: depending on what other test suites have been run before this test, the number of expected studies can change. Now, a query is run to get the correct number and all mocks are assembled to avoid errors.

Lastly, the --disable-frozen-string-literal cli option has been set for the Terra Orchestration smoke test to avoid polluting the log with /usr/local/rvm/rubies/ruby-3.4.2/lib/ruby/3.4.0/open-uri.rb:455: warning: literal string will be frozen in the future (run with --debug-frozen-string-literal for more information) repeatedly.

Copy link

codecov bot commented May 20, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 71.03%. Comparing base (0389ba3) to head (4659b1b).
Report is 11 commits behind head on development.

Additional details and impacted files

Impacted file tree graph

@@               Coverage Diff               @@
##           development    #2261      +/-   ##
===============================================
+ Coverage        71.01%   71.03%   +0.01%     
===============================================
  Files              335      335              
  Lines            28820    28820              
  Branches          2527     2527              
===============================================
+ Hits             20466    20471       +5     
+ Misses            8213     8208       -5     
  Partials           141      141              

see 3 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@bistline bistline marked this pull request as ready for review May 27, 2025 14:17
@bistline bistline requested a review from eweitz May 27, 2025 14:17
@bistline bistline merged commit 2b2f1ee into development May 27, 2025
6 checks passed
@github-actions github-actions bot deleted the jb-test-stability-fix branch May 27, 2025 19:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants