code-eval

You're looking for "HumanEval" tests. Not saying this is the best way to test it, but it's the only standard test I know of that code models are compared with and are commonly benchmarked for The current best models you'd want to try that I'm aware of is WizardCoder(15B), Starcoder(15B), and replit's code model(3B). Replit's instruct model is interesting because of it's competitive performance while only being a 3B model so it's the easiest/fastest to use. Perhaps interestingly none of these are based on LLama https://github.com/abacaj/code-eval - This is a large mostly up to date list of benchmarks https://huggingface.co/WizardLM/WizardCoder-15B-V1.0 - has a chart with a mostly up to date comparison EDIT: License-wise I think you might be able to commercially use Replit's model and Starcoder, I don't think you're allowed to use WizardCod