Is there a way to verify model correctness in a new backend?

[luo-yicong]

I support a new backend, but running llama2 7b with llama-cli generates incorrect results. The result of test_llama in test-backend-ops is correct. Is there another way to verify model correctness in a new backend?

[slaren]

There are some tests in test-backend-ops (test_llama, test_falcon) that are disabled by default that test an entire layer of a model. You can try enabling them to see where it fails.

[luo-yicong]

Thanks for your reply! Do you mean just change the n_layer in llama_hparams to 2 or more and test test_llama?

[slaren]

I mean that you have to add the test cases in make_test_cases_eval because by default they are not used.