Skip to content

[BUG]: experimental_eval.py cannot be successfully run with 0.0.11 or 0.1.0-rcx #200

@lwaekfjlk

Description

@lwaekfjlk

Description of the bug

class name including EvaluationForTwoAgents is not supported

Steps To Reproduce

run experimental_eval.py with sotopia==0.0.11

independently install sotopia

python experimental_eval.py
--gin_file sotopia_conf/generation_utils_conf/generate.gin
--gin_file sotopia_conf/server_conf/server.gin
--gin_file sotopia_conf/run_async_server_in_batch.gin
'--gin.ENV_IDS=["01H7VFHNV13MHN97GAH73E3KM8", "01H7VFHN5WVC5HKKVBHZBA553R", "01H7VFHNN7XTR99319DS8KZCQM", "01H7VFHN9W0WAFZCBT09PKJJNK", "01H7VFHPDZVVCDZR3AARA547CY", "01H7VFHPQQQY6H4DNC6NBQ8XTG", "01H7VFHPQQQY6H4DNC6NBQ8XTG", "01H7VFHN7WJK7VWVRZZTQ6DX9T", "01H7VFHN7A1ZX5KSMT2YN9RXC4", "01H7VFHPS5WJW2694R1MNC8JFY", "01H7VFHPS5WJW2694R1MNC8JFY", "01H7VFHNN7XTR99319DS8KZCQM", "01H7VFHQ11NAMZS4A2RDGDB01V", "01H7VFHQ11NAMZS4A2RDGDB01V", "01H7VFHPSWGDGEYRP63H2DJKV0", "01H7VFHPSWGDGEYRP63H2DJKV0", "01H7VFHNF4G18PC9JHGRC8A1R6", "01H7VFHNNYH3W0VRWVY178K2TK", "01H7VFHP8AN5643B0NR0NP00VE", "01H7VFHN7A1ZX5KSMT2YN9RXC4"]'
'--gin.AGENT1_MODEL="gpt-3.5-turbo"'
'--gin.AGENT2_MODEL="gpt-3.5-turbo"'
'--gin.BATCH_SIZE=1'
'--gin.PUSH_TO_DB=True'
'--gin.TAG="gpt-3.5-turbo_vs_gpt-3.5-turbo-0927"'

Additional Information

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions