Skip to content

bug: reverse translation outputs in report.jsonl when no translation used #1201

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
leondz opened this issue May 8, 2025 · 1 comment
Open
Labels
bug Something isn't working

Comments

@leondz
Copy link
Collaborator

leondz commented May 8, 2025

steps

  1. do a run (e.g. garak -m test -p test.Test -g 1) with no translation enabled
  2. look in the output report.jsonl

expected behavior

report.jsonl doesn't mention translation

actual behavior

report.jsonl does mention translation

{"entry_type": "attempt", "uuid": "4d5f402e-ec17-47b8-8b31-c19396827ccf", "seq": 6, "status": 1, "probe_classname": "test.Test", "probe_params": {}, "targets": [], "prompt": "T9btvwOX'EV/)q2+o$kF7#", "outputs": ["Sed quiquia dolor quaerat quiquia magnam."], "detector_results": {}, "notes": {"pre_translation_prompt": "T9btvwOX'EV/)q2+o$kF7#"}, "goal": "send some test strings, ignore the response", "messages": [[{"role": "user", "content": "T9btvwOX'EV/)q2+o$kF7#"}, {"role": "assistant", "content": "Sed quiquia dolor quaerat quiquia magnam."}]], "lang": "en", "reverse_translation_outputs": ["Sed quiquia dolor quaerat quiquia magnam."]}
@leondz leondz added the bug Something isn't working label May 8, 2025
@leondz leondz added this to the release 0.11.0 milestone May 8, 2025
@leondz
Copy link
Collaborator Author

leondz commented May 8, 2025

requires #1089

@leondz leondz removed this from the release 0.11.0 milestone May 8, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant