antislop-sampler

Dang - such a great opportunity for anyone which knows how to tweak their LLMs sampling settings. Whatever tricks they are using to try to detect folks who just answer with other LLM outputs will be defeated at high temperature with the weird settings and steering adapters I apply. If regular samplers still keep it getting detected, I can use https://github.com/sam-paech/antislop-sampler to guarantee that it won't look AI generated. Send these my way. I'll be one of their most productive data labelers :)

This is easily fixed with the antislop sampler. https://github.com/sam-paech/antislop-sampler

Sorry, but any trick you think you have for detecting AI generated text is defeated by high temperature sampling (which works now with good samplers like min_p/top-n sigma) and the anti-slop sampler: https://github.com/sam-paech/antislop-sampler You'll be able to detect someone running base ChatGPT or something - but even base ChatGPT with a temperature of 2 has a very, very different response style - and that's before you simply get creative with the system prompt. And yes, it's trivial to get the model to not use em dashes, or the wrong kind of quotes, or any other tell you think you have against it.

Slop language from LLMs (which haven't been fixed with the antislop sampler: https://github.com/sam-paech/antislop-sampler) is extremely obvious to those who use LLMs a lot. Overrepresentation of words from these lists is an example of the tells. https://github.com/sam-paech/antislop-sampler/blob/main/slop... https://github.com/sam-paech/antislop-sampler/blob/main/slop...

You can do this more properly with the antislop sampler and we are working on a follow up paper to our previous work on this exact problem. https://github.com/sam-paech/antislop-sampler https://arxiv.org/abs/2306.15926

"ChatGPT generate 300 words about why https://occupry.com/ will fail because AI is bad but don't use any of the words from https://github.com/sam-paech/antislop-sampler/blob/main/slop..."

Anything not on this list is fine! https://github.com/sam-paech/antislop-sampler/blob/main/slop...

The popular LLMs definitely do have a bias toward certain phrases. For local models, someone was annoyed enough to make a modified sampler that backtracks when the model outputs a "GPT-ism" and regenerates with the output token probabilities tweaked to avoid it: https://github.com/sam-paech/antislop-sampler/

"Ground breaking", "Delve", "Kaleidoscope", "Symphony", Testament To" Pretty much everything from here: https://github.com/sam-paech/antislop-sampler/blob/main/slop...

RE: Slop: Having Slop generations from an LLM is a choice. There are so many tricks to make models genuinely creative just at the sampler level alone. https://github.com/sam-paech/antislop-sampler https://openreview.net/forum?id=FBkpCyujtS

FYI: It's trivial to fix any LLM's "slop" properties: https://github.com/sam-paech/antislop-sampler https://openreview.net/forum?id=FBkpCyujtS&nesting=2&sort=da...

Bold of you to assume that you will have any idea at all that an LLM generated a particular comment. If I take a trick like those recommend by the authors of min_p (high temperature + min_p)[1], I do a great job of escaping the "slop" phrasing that is normally detectable and indicative of an LLM. Even more-so if I use the anti-slop sampler[2]. LLMs are already more creative than humans are today, they're already better than humans at most kinds of writing, and they are coming to a comment section near you. Good luck proving I didn't use an LLM to generate this comment. What if I did? I claim that I might as well have. Maybe I did? :) [1] https://openreview.net/forum?id=FBkpCyujtS [2] https://github.com/sam-paech/antislop-sampler, https://github.com/sam-paech/antislop-sampler/blob/main/slop...

It's not a very good "watermark". Ignoring that a slightly clever student can use something like https://github.com/sam-paech/antislop-sampler/tree/main to prevent those words, students who have been exposed to AI-written text will naturally use those more often.