codealpaca

This is great! Addressing the model limitations a bit: in the demonstration data that is provided to the base model, we should prevent computed or "looked up" answers. I've seen some of the demonstration data that people are using to train instruction-tuned models and are being taught to respond by making up answers to solutions it shouldn't try to compute. Btw, the output is wrong. { "instruction": "What would be the output of the following JavaScript snippet?", "input": "let area = 6 * 5;\nlet radius = area / 3.14;", "output": "The output of the JavaScript snippet is the radius, which is 1.91." }, [1] The UI note for now would get us very far but by filtering out demonstrations that retrieve or compute information we can go even farther. Symbol tuning [2] is addressing the quality of demonstrations but we can take it further by removing

One argument is that it can contaminate training data from output of itself or other models. We have already documented evidence of the effect of this. In the GPT-4 technical report [1], they reported contamination of humaneval data in the training data. They did measure against a "non-contaminated" training set but no idea if that can still be trusted. Why would this matter? We can have seemingly strong benchmarks for containments but measures poorly against new and quarantined information. Classic over fitting. Another argument is that data being put out there could very much be wrong and the amounts of it amplified by other models. Take a look at this sample of demonstration data for codealpaca [2]. Not only is its output wrong but bad practices like,making up a random computation without it having access to a place to run a calculatio

I think these are two very separate concepts. What we are mostly seeing when it comes to fine-tuning is making a model promptable. Models like LLaMA or the original GPT3 weren't promptable. They were fine-tuned with demonstration data that looks like a prompt input, prompt output. See below: { "instruction": "What would be the output of the following JavaScript snippet?", "input": "let area = 6 * 5;\nlet radius = area / 3.14;", "output": "The output of the JavaScript snippet is the radius, which is 1.91." }, [1] Prompt engineering is really just carefully designing what inputs and outputs on a prompt-ready model work best. I highly recommend skimming this RLHF article and looking for the parts where it talks about demonstration data [2] 1: https://github.com/sahil280114/codealpaca/blob/master/data

Coding with LLMs was easier after understanding their limitations. 1. They don't know what they are saying until they have said it. 2. Your inputs and its outputs help make the next message. 3. LLMs are not suited for information retrieval like databases and search engines. LLMs excel at reasoning and predicting subsequent text based on given context. Their strength lies in their ability to generate relevant and cohesive responses. To optimize results, outline clear rules, strategies, or ideas for the LLM to follow. This helps the model craft, revise, or build upon the established context. Starting with a precise query and introducing rules or constraints incrementally can help steer the model's output in the desired direction. Avoid zero-shot queries as these can lead to the model generating unexpected or unrelated responses. Be cautious

Sorry for the late reply, as I said Flan-UL2 (or Flan-T5 if you want lighter models) fine-tuned against a dataset like CodeAlpaca's[0] is probably the best solution if it's intended for commercial use (otherwise LLaMa should perform better). [0]: https://github.com/sahil280114/codealpaca