Skip to content

[enhancement] Improve programming error messages #3489

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Jun 18, 2025

Conversation

vkarak
Copy link
Contributor

@vkarak vkarak commented May 23, 2025

This PR improves substantially the programming error messages in ReFrame in an attempt to help users when writing tests. More specifically, the reframe syntax errors are treated, which are those directly related to the special test syntax and were the most difficult to decipher.

The key challenge of such messages is that they are thrown raised class construction and not at the exact point where they are sourced. Although we could get precise line information for every variable, parameter and fixture definition and even every assignment in a test class, this would make loading of the tests prohibitively slow, as it requires stack inspection. Thus, for variables, parameters and fixtures we only store the name of the owner class, so that we can provide precise guidance to the user when error occurs. We also provide source code information for every reframe syntax error, but this is limited to the line where the class is defined.

Also all the syntax error messages where rephrased carefully taking the test developer's perspective. Some of them also contain suggestions on how to properly fix the code.

Closes #2308.

Todos

  • Remove precise code annotation for Python < 3.11
  • Check if we need to update the docs (listings etc.)

@vkarak vkarak added this to the ReFrame 4.8.2 milestone May 23, 2025
@vkarak vkarak self-assigned this May 23, 2025
@vkarak vkarak force-pushed the feat/improve-prog-error-messages branch from 26c98c1 to 69c39a1 Compare May 25, 2025 21:33
@vkarak vkarak changed the title Improve programming error messages [enhancement] Improve programming error messages May 25, 2025
Copy link
Contributor

@teojgo teojgo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have commented mostly about string formatting

@github-project-automation github-project-automation bot moved this from Todo to In Progress in ReFrame Backlog Jun 17, 2025
@vkarak vkarak force-pushed the feat/improve-prog-error-messages branch from 898867f to 238d94c Compare June 17, 2025 16:20
@vkarak vkarak requested a review from teojgo June 17, 2025 16:21
Copy link
Contributor

@victorusu victorusu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@vkarak vkarak merged commit 6a33b94 into reframe-hpc:master Jun 18, 2025
37 checks passed
@github-project-automation github-project-automation bot moved this from In Progress to Done in ReFrame Backlog Jun 18, 2025
@vkarak vkarak deleted the feat/improve-prog-error-messages branch June 18, 2025 17:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

3 participants