Skip to content

feat: add named exercise groups to web-evals #6083

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

roomote[bot]
Copy link

@roomote roomote bot commented Jul 22, 2025

This PR adds support for predefined named groups of eval exercises that can be quickly selected in the web-evals app.

Changes

  • Added exercise-groups.json file to store predefined exercise groups
  • Created a "Hard" group containing 26 challenging exercises across Go, Java, JavaScript, Python, and Rust
  • Added server action getExerciseGroups() to load exercise groups
  • Updated the NewRun component to display exercise group buttons when "Some" exercises are selected
  • Clicking a group button automatically selects all exercises in that group

UI Enhancement

When users select "Some" exercises instead of "All", they now see predefined group buttons above the exercise selector. Clicking "Hard (26)" will instantly select all 26 exercises in the Hard group.

This feature makes it easier to run specific sets of exercises without manually selecting each one.


Important

Adds predefined exercise groups to web-evals, allowing quick selection of exercise sets in the NewRun component.

  • Behavior:
    • Adds exercise-groups.json to define predefined exercise groups.
    • Implements getExerciseGroups() in exercise-groups.ts to fetch exercise groups.
    • Updates NewRun component to show group buttons when "Some" exercises are selected.
    • Clicking a group button selects all exercises in that group.
  • UI Enhancement:
    • Displays predefined group buttons in NewRun when "Some" exercises are selected.
    • "Hard (26)" button selects 26 challenging exercises across multiple languages.

This description was created by Ellipsis for 9861ca6. You can customize this summary. It will automatically update as commits are pushed.

- Added exercise-groups.json with "Hard" group containing 26 exercises
- Created server action to load exercise groups
- Updated NewRun component to display predefined exercise groups
- Added UI buttons to quickly select exercise groups
@roomote roomote bot requested review from mrubens, cte and jr as code owners July 22, 2025 22:09
@dosubot dosubot bot added size:L This PR changes 100-499 lines, ignoring generated files. enhancement New feature or request labels Jul 22, 2025
if (group) {
setValue("suite", "partial")
setValue("exercises", group.exercises)
toast.success(`Selected "${groupName}" exercise group with ${group.exercises.length} exercises`)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use the i18n function (e.g., t('...')) for the success toast message instead of inline English strings.

Suggested change
toast.success(`Selected "${groupName}" exercise group with ${group.exercises.length} exercises`)
toast.success(t('Selected "{{groupName}}" exercise group with {{count}} exercises', { groupName, count: group.exercises.length }))

This comment was generated because it violated a code review rule: irule_C0ez7Rji6ANcGkkX.

@hannesrudolph hannesrudolph added the Issue/PR - Triage New issue. Needs quick review to confirm validity and assign labels. label Jul 22, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request Issue/PR - Triage New issue. Needs quick review to confirm validity and assign labels. size:L This PR changes 100-499 lines, ignoring generated files.
Projects
Status: Triage
Development

Successfully merging this pull request may close these issues.

2 participants