Skip to content

Upload new model performance #21

@pulkitmehtawork

Description

@pulkitmehtawork

I thought larger model would do better so simply ran eval on below and found that it is doing better(overall avg then even openai ada) {
"Model": "BAAI/bge-large-en-v1.5",
"Model Size (Million Parameters)": 335,
"URL": "https://huggingface.co/BAAI/bge-large-en-v1.5",
"Apps": 5.076,
"CosQA": 36.608,
"Synthetic Text2sql": 44.875,
"CodeSearchNet": 73.92,
"CodeSearchNet-CCR": 50.708,
"CodeTrans-Contest": 49.155,
"CodeTrans-DL": 19.833,
"StackOverFlow QA": 80.747,
"CodeFeedBack-ST": 68.588,
"CodeFeedBack-MT": 36.496,
"Avg": 46.60
} .. also , attaching all the eval results

apps.json
codefeedback-mt.json
codefeedback-st.json
CodeSearchNet-ccr-go.json
CodeSearchNet-ccr-java.json
CodeSearchNet-ccr-javascript.json
CodeSearchNet-ccr-php.json
CodeSearchNet-ccr-python.json
CodeSearchNet-ccr-ruby.json
CodeSearchNet-go.json
CodeSearchNet-java.json
CodeSearchNet-javascript.json
CodeSearchNet-php.json
CodeSearchNet-python.json
CodeSearchNet-ruby.json
codetrans-contest.json
codetrans-dl.json
cosqa.json
stackoverflow-qa.json
synthetic-text2sql.json

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions