-
Notifications
You must be signed in to change notification settings - Fork 9
Description
I thought larger model would do better so simply ran eval on below and found that it is doing better(overall avg then even openai ada) {
"Model": "BAAI/bge-large-en-v1.5",
"Model Size (Million Parameters)": 335,
"URL": "https://huggingface.co/BAAI/bge-large-en-v1.5",
"Apps": 5.076,
"CosQA": 36.608,
"Synthetic Text2sql": 44.875,
"CodeSearchNet": 73.92,
"CodeSearchNet-CCR": 50.708,
"CodeTrans-Contest": 49.155,
"CodeTrans-DL": 19.833,
"StackOverFlow QA": 80.747,
"CodeFeedBack-ST": 68.588,
"CodeFeedBack-MT": 36.496,
"Avg": 46.60
} .. also , attaching all the eval results
apps.json
codefeedback-mt.json
codefeedback-st.json
CodeSearchNet-ccr-go.json
CodeSearchNet-ccr-java.json
CodeSearchNet-ccr-javascript.json
CodeSearchNet-ccr-php.json
CodeSearchNet-ccr-python.json
CodeSearchNet-ccr-ruby.json
CodeSearchNet-go.json
CodeSearchNet-java.json
CodeSearchNet-javascript.json
CodeSearchNet-php.json
CodeSearchNet-python.json
CodeSearchNet-ruby.json
codetrans-contest.json
codetrans-dl.json
cosqa.json
stackoverflow-qa.json
synthetic-text2sql.json