Skip to content

How to add params in api_url #891

@moluzhui

Description

@moluzhui

Self-Check List

Before submitting an issue, please ensure you have completed the following steps:

Problem Description

Please briefly describe the problem you encountered.

The third-party VLM API carries params, which can be accessed via curl. However, params cannot be configured through evalscope code, making evaluation impossible

EvalScope Version (Required)

v1.0.2

Tools Used

  • Native / Native framework
  • Opencompass backend
  • VLMEvalKit backend
  • RAGEval backend
  • Perf / Model inference stress testing tool
  • Arena / Arena mode

Executed Code or Instructions

Please provide the main code or instructions you executed.

from evalscope import TaskConfig, run_task
from evalscope.constants import EvalType

task_cfg = TaskConfig(
    work_dir='outputs',
    model='gpt-5',
    api_url='https://xxx.xxx.xxx.xxx/openai/deployments/gpt-5?api-version=xxxx-preview',
    api_key='apikey',
    eval_type=EvalType.SERVICE,
    datasets=[
        'real_world_qa',
    ],
    eval_batch_size=5,
    generation_config={
        'max_completion_tokens': 10000, 
        'n': 1,  
    },
)

run_task(task_cfg=task_cfg)

Error Log

Please paste the complete error log or console output.

} prediction failed: due to Error code: 404 - {'error': {'code': '404', 'message': 'Resource not found'}}
Traceback:
Traceback (most recent call last):
  File "/data/zhangxinfeng3/evalscope/evalscope/evalscope/evaluator/evaluator.py", line 182, in get_answers
    task_state = future.result()
                 ^^^^^^^^^^^^^^^
  File "/usr/python/lib/python3.11/concurrent/futures/_base.py", line 449, in result

Running Environment

  • Operating System: Centos7.9
  • Python Version: python3.11

Additional Information

If there is any other relevant information, please provide it here.

I need to add an extra params at the end of the url and try to get the same error through generation_config["extra_body"]

Access the normal curl command

curl https://xxx/openai/deployments/gpt-5/chat/completions?api-version=xxxx-preview \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer xxxx" \
  -d $'{
    "model": "gpt-5",
    "messages": [
        {
            "content": [
                {
                    "image_url": {
                        "url": "images_url"
                    },
                    "type": "image_url"
                },
                {
                    "text": "What does the picture mainly talk about?",
                    "type": "text"
                }
            ],
            "role": "user"
        }
    ]
}'

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions