Skip to content

Conversation

jbocce
Copy link
Collaborator

@jbocce jbocce commented Jul 3, 2025

Context

The playwright tests were lacking archived results in github.
The playwright tests were taking a bit long to run.
We wanted to be sure that the playwright tests were not giving us false negatives (i.e. a negative is a passing test).

Changes & Results

Lower tolerances for Playwright screenshots comparisons.
Added packaging of Playwright test results to the github workflow. Removed unnecessary waits in tests to improve performance.

Testing

Run the tests on github.

Checklist

PR

  • My Pull Request title is descriptive, accurate and follows the
    semantic-release format and guidelines.

Code

  • My code has been well-documented (function documentation, inline comments,
    etc.)

Public Documentation Updates

  • The documentation page has been updated as necessary for any public API
    additions or removals.

Tested Environment

  • "OS: Windows 11
  • "Node version: 20.9.0
  • "Browser: Chrome 138.0.7204.97

Added packaging of Playwright test results to the github workflow.
Removed unnecessary waits in tests to improve performance.
@jbocce jbocce requested a review from sedghi July 4, 2025 00:04
@wayfarer3130
Copy link
Collaborator

Where do the screenshots get stored? It seems like there are failing tests, but I'm not quite sure where to look for the results.

@wayfarer3130
Copy link
Collaborator

Found the results - it looks like there are video files for all the tests, not just the failing ones. Is there a way to only included failed test results, and only include all test results if a flag is set?

@jbocce
Copy link
Collaborator Author

jbocce commented Jul 7, 2025

Found the results - it looks like there are video files for all the tests, not just the failing ones. Is there a way to only included failed test results, and only include all test results if a flag is set?

@wayfarer3130 that is a great suggestion. I will try to see if there is something I can do for that.

Also, I removed a bunch of sleeps which is fine if there are existing screen shots, but when new ones are required I am afraid that we will need the sleeps again.

@wayfarer3130 will you be looking at the failed tests?

@jbocce
Copy link
Collaborator Author

jbocce commented Jul 8, 2025

Ok @wayfarer3130 , I added a flag to only capture video for the first retry. This should eliminate noise and reduce the size of the packaged test results. We can add a separate config to capture more stuff in the future should we see fit. Otherwise, I think this is good for now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants