Skip to content

Allow non testcase folders under testcases #269

@roybeny

Description

@roybeny

Steps
Create an empty folder under "testcases"
run the summarize report

Describe the bug

Summarizing evaluation results in user-actions
Found 3 testcases: ['gemini2flash', 'gemini2flashlight', 'frames']
An unexpected error occurred: Eval results file not found at user-actions/testcases/frames/eval_results.jsonl

Expected behavior
the empty folder should be ignored, with a warning

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions