Skip to content

[Apibench] Evaluation Metrics Confusion #1139

@Dipto084

Description

@Dipto084

In the paper it says 'For all models except Gorilla, we only check if they provide the correct domain names", isn't it the same as functional accuracy which is calculated by the provided eval scripts? In that case, the accuracy metric for Gorilla is functional accuracy too, right? If not, how is the calculation of accuracy different for Gorilla and baselines?

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions