chatgpt-refusals

I distinctly remember someone making an experiment by asking ChatGPT to write jokes (?) about different groups and calculating the likelihood of it refusing, to produce a ranking. I think it was a medium article, but now I cannot find it anymore. Does anyone have a link? EDIT: At least here is a paper aiming to predict ChatGPT prompt refusal https://arxiv.org/pdf/2306.03423 with an associated dataset https://github.com/maxwellreuter/chatgpt-refusals EDIT2: Aha, found it! https://davidrozado.substack.com/p/openaicms An interesting graph is about 3/4 down the page, showing what ChatGPT moderation considers to be hateful.

Data for that paper: https://github.com/maxwellreuter/chatgpt-refusals This looks like the most interesting file from their data: https://lite.datasette.io/?json=https://github.com/maxwellre...

".... The small manually-labeled dataset is used to train a refusal classifier, which achieves an accuracy of 92%. Second, we use this refusal classifier to bootstrap a larger (n=10,000) dataset adapted from the Quora Insincere Questions dataset. With this machine-labeled data, we train a prompt classifier to predict whether ChatGPT will refuse a given question, without seeing ChatGPT’s response. This prompt classifier achieves 76% accuracy on a test set of manually labeled questions (n=1,009). We examine our classifiers and the prompt 𝑛-grams that are most predictive of either compliance or refusal. Datasets and code are available at https://github.com/maxwellreuter/chatgpt-refusals..."