llm4regression

Initially I experimented with house prediction. But the models have probably seen this data, so I ended up experimenting with random functions. I tried with linear regression with irrelevant features (e.g., NI 1/2 -> 1 informative variable, 2 total variables). The models still perform reasonably well. All results can be seen in this heatmap: https://github.com/robertvacareanu/llm4regression/blob/main/...

If you're interested in exploring other metrics, the output of the models and code examples for how to read them are available at: https://github.com/robertvacareanu/llm4regression/tree/main/... For example, slightly adapting the code from the `README.md` for re-creating a barplot similar to Figure 1: ``` print(cdf.groupby(by=['dataset', 'model']).apply(lambda x: np.sqrt(((x['pred'] - x['gold']) ** 2).mean()))) ``` The code above prints (the dataset is Original #3): ``` dataset model original3 Claude 3 Opus 11.862656 DBRX 32.186706 GPT-4 31.672733 Gradient Boosting 21.985396 KNN 55.043003 Linear Regression 29.987077 Random Forest 31.688074 ```

Code is available at: https://github.com/robertvacareanu/llm4regression I also added google colab examples: A small example with GPT-4: https://colab.research.google.com/drive/1Bk9uBCBvzuX00Rex-t1... A small scale eval with GPT-4: https://colab.research.google.com/drive/1_-uHvW2oLtcCXz0c-G_... Also, feel free to DM or email me if you’d like to chat more about it!

Chat examples with GPT-4: https://github.com/robertvacareanu/llm4regression/tree/main/... (the experiments used the API though) For example, for Friedman #1, GPT-4 predicts 12.89 while the true value is 11.69 (https://chat.openai.com/share/177571ad-3845-46a1-952f-963647...) For Original #1, GPT-4 predicts 83.63 while the true value is 80.39 (https://chat.openai.com/share/808da995-99e6-444a-94da-fc7cd5...)