Skip to content

verrou into Workflow #937

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 7 commits into
base: master
Choose a base branch
from
Draft

Conversation

Malmahrouqi3
Copy link
Collaborator

@Malmahrouqi3 Malmahrouqi3 commented Jul 12, 2025

User description

Description

Concerning (#650),
Intended as standalone floating-point error checker.

Debugging info,
The current Valgrind setup ought to be sufficient, so debugging locally (nektos/act extension) or on the CI are all viable.

To-dos,

  • Proper Valgrind commands to generate the most amount of errors.
  • Save and process all reports such that non-empty error summaries are displayed.

Ref verrou docs


PR Type

Tests, Enhancement


Description

  • Add GitHub Actions workflow for floating-point compliance checking

  • Integrate Valgrind's verrou tool for numerical stability testing

  • Set up automated CI pipeline to detect sketchy float operations


Changes diagram

flowchart LR
  A["GitHub Actions Trigger"] --> B["Setup Valgrind with verrou"]
  B --> C["Run MFC tests with verrou"]
  C --> D["Check floating-point compliance"]
Loading

Changes walkthrough 📝

Relevant files
Tests
verrou.yml
Add verrou floating-point compliance workflow                       

.github/workflows/verrou.yml

  • Create new GitHub Actions workflow for floating-point compliance
  • Install Valgrind with verrou tool from latest GitHub release
  • Configure workflow to run on pull requests and pushes
  • Add test execution with verrou tool integration
  • +31/-0   

    Need help?
  • Type /help how to ... in the comments thread for any questions about Qodo Merge usage.
  • Check out the documentation for more information.
  • @Copilot Copilot AI review requested due to automatic review settings July 12, 2025 15:29
    Copy link

    PR Reviewer Guide 🔍

    Here are some key observations to aid the review process:

    🎫 Ticket compliance analysis 🔶

    650 - Partially compliant

    Compliant requirements:

    • Use Valgrind's verrou tool to check for poorly conditioned float operations
    • Set up automated detection of floating-point issues

    Non-compliant requirements:

    • Implement checking mechanism to identify sketchy float operations
    • Address source of randomly failing tests when compiled differently

    Requires further human verification:

    • Verify that the workflow actually detects floating-point issues in practice
    • Confirm that the verrou tool configuration generates meaningful error reports
    • Test that the workflow helps identify causes of randomly failing tests

    ⏱️ Estimated effort to review: 2 🔵🔵⚪⚪⚪
    🧪 No relevant tests
    🔒 No security concerns identified
    ⚡ Recommended focus areas for review

    Incomplete Implementation

    The workflow runs tests with --dry-run flag which means no actual execution occurs, and the compliance check step only prints a message without processing any verrou output or reports.

      run: valgrind --tool=verrou ./mfc.sh test -a --dry-run
    
    - name: Check floating point compliance
      run: |
        echo "Checking floating point compliance..."
    Missing Error Handling

    The Valgrind installation process lacks error handling and verification steps, and there's no mechanism to capture or process verrou error reports as mentioned in the PR description.

    - name: valgrind Setup
      run: |
        # get recent release of valgrind from GitHub API
        VALGRIND_VERSION=$(curl -s https://api.github.com/repos/LouisBrunner/valgrind/releases/latest | grep -oP '"tag_name": "\K(.*)(?=")')
        # download and extract it
        wget https://github.com/LouisBrunner/valgrind/releases/download/${VALGRIND_VERSION}/valgrind-${VALGRIND_VERSION}.tar.bz2
        tar -xjf valgrind-${VALGRIND_VERSION}.tar.bz2
        cd valgrind-${VALGRIND_VERSION}
        ./autogen.sh
        ./configure --enable-only64bit --prefix=/usr/local
        make
        sudo make install
        source /usr/local/env.sh

    Copy link
    Contributor

    @Copilot Copilot AI left a comment

    Choose a reason for hiding this comment

    The reason will be displayed to describe this comment to others. Learn more.

    Pull Request Overview

    Adds a new GitHub Actions workflow to run floating-point compliance checks with Valgrind’s verrou tool.

    • Introduces .github/workflows/verrou.yml for triggering on PRs and pushes
    • Downloads, builds, and installs the latest Valgrind release
    • Executes tests under verrou and provides a placeholder for compliance reporting
    Comments suppressed due to low confidence (4)

    .github/workflows/verrou.yml:24

    • The step sources /usr/local/env.sh which is not generated by the Valgrind install; this will likely fail. Instead, export the path (e.g., export PATH=/usr/local/bin:$PATH) or source the correct environment file to ensure Valgrind is on the PATH.
              source /usr/local/env.sh
    

    .github/workflows/verrou.yml:27

    • Using --dry-run prevents actual test execution under Valgrind, so no floating-point errors will be detected. Remove --dry-run to execute the tests for real.
            run: valgrind --tool=verrou ./mfc.sh test -a --dry-run
    

    .github/workflows/verrou.yml:6

    • [nitpick] The job identifier self is ambiguous; rename it to something descriptive like floating_point_compliance or verrou_check for clarity.
      self:
    

    .github/workflows/verrou.yml:31

    • This step only echoes a message and does not actually parse or summarize the verrou output. Consider adding commands to collect, report, and fail the workflow when errors are found.
              echo "Checking floating point compliance..."
    

    Copy link

    qodo-merge-pro bot commented Jul 12, 2025

    PR Code Suggestions ✨

    Explore these optional code suggestions:

    CategorySuggestion                                                                                                                                    Impact
    Possible issue
    Fix environment variable persistence
    Suggestion Impact:The suggestion was implemented by removing the 'source /usr/local/env.sh' line entirely, which addresses the persistence issue mentioned in the suggestion

    code diff:

    -          source /usr/local/env.sh

    The source command only affects the current shell session and won't persist to
    subsequent workflow steps. Use echo to add environment variables to $GITHUB_ENV
    for persistence across steps.

    .github/workflows/verrou.yml [24]

    -source /usr/local/env.sh
    +echo "PATH=/usr/local/bin:$PATH" >> $GITHUB_ENV
    +echo "LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH" >> $GITHUB_ENV
    Suggestion importance[1-10]: 9

    __

    Why: This is a critical fix, as source only affects the current shell step, and environment variables will not persist to subsequent steps, likely causing the valgrind command to fail.

    High
    Add error handling for API calls

    Add error handling for the curl command to prevent workflow failure if the
    GitHub API is unavailable or returns unexpected data. Consider adding a fallback
    version or retry mechanism.

    .github/workflows/verrou.yml [15]

    -VALGRIND_VERSION=$(curl -s https://api.github.com/repos/LouisBrunner/valgrind/releases/latest | grep -oP '"tag_name": "\K(.*)(?=")')
    +VALGRIND_VERSION=$(curl -s https://api.github.com/repos/LouisBrunner/valgrind/releases/latest | grep -oP '"tag_name": "\K(.*)(?=")') || { echo "Failed to fetch latest version"; exit 1; }
    Suggestion importance[1-10]: 7

    __

    Why: This suggestion correctly points out that the curl and grep pipeline could fail, making the workflow more robust by adding error handling and preventing cryptic failures in later steps.

    Medium
    General
    Implement actual compliance checking

    This step only prints a message without performing any actual compliance
    checking. Either implement the actual verification logic or remove this
    placeholder step.

    .github/workflows/verrou.yml [30-31]

     run: |
    -  echo "Checking floating point compliance..."
    +  echo "Floating point compliance check completed"
    +  # Add actual verification commands here
    Suggestion importance[1-10]: 3

    __

    Why: The suggestion correctly identifies that the step is a placeholder, which is likely intentional. While valid, it points out incompleteness rather than a flaw in the existing code.

    Low
    • Update

    @Malmahrouqi3
    Copy link
    Collaborator Author

    Status Update: Valgrind builds nicely and PR is awaiting (#938) to merge in to replicate the results of (#650).

    @sbryngelson
    Copy link
    Member

    I assume you know this but the use of verrou in our case is very specific (we are looking for quite specific problems involving dangerous floating point operations that could cause round-off error problems). This is what this 'linter' of sorts should be looking for.

    @sbryngelson sbryngelson marked this pull request as draft July 14, 2025 07:02
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Development

    Successfully merging this pull request may close these issues.

    2 participants