open_nsfw

Nobody's distributing a free dataset of child abuse, animal torture and terror beheading images, for obvious reasons. There are some open-weights NSFW detectors [1] but even if your detector is 99.9% accurate, you still need an appeals/review mechanism. And someone's got to look at the appeals. [1] https://github.com/yahoo/open_nsfw

AFAIK Discord's NSFW filter is not a perceptual hash nor uses the NCMEC database (although that might indeed be in the pipeline elsewhere) but instead uses a ML classifier (I'm certain it doesn't use perceptual hashes as Discord doesn't have a catalogue of NSFW image hashes to compare against). I've guessed it's either open_nsfw[0] or Google's Cloud Vision since the rest of Discord's infrastructure uses Google Cloud VMs. There's a web demo available of this api[1], Discord probably pulls the safe search classifications for determining NSFW. 0: https://github.com/yahoo/open_nsfw 1: https://cloud.google.com/vision#section-2

The Yahoo NSFW classifier that is mentioned is available here: https://github.com/yahoo/open_nsfw I wonder if there's a way to ship this model and run it in an end-user app, in a way that does not require additional setup. I could use that.

I've found promising results trying this one out: https://github.com/yahoo/open_nsfw

It would be nice if someone could compare these commercially available APIs with Yahoo's open_nsfw model in terms of accuracy: https://github.com/yahoo/open_nsfw I'm currently building an API wrapper around it and running it on a Hetzner server with a GTX 1080 - prediction takes about 0.25 seconds and while I haven't optimised it for parallel execution, I think it should be able to handle at least +10 images/sec comfortably. I'm also testing video moderation by using ffmpeg to slice the video into screenshots and predicting the min/avg/high scores. Moderating 25 million explicit images using Google Cloud Vision would cost around $19,500/mo vs €99/mo on Hetzner.

There's a table with runtime comparisons for a convnet here: https://github.com/ryanjay0/miles-deep/ (GPU CuDNN: 15s, GPU: 19s, CPU: 159s) (Also written w/ Caffe: https://github.com/yahoo/open_nsfw)

By malicious, do you mean inappropriate, or containing a malicious metadata inside the image? The images are probably stripped and/or compressed, removing any malicious metadata. You can use machine learning to classify images of various categories to detect inappropriate images. For example, this is what Yahoo open sourced: https://github.com/yahoo/open_nsfw Additionally, user-generated content sites will almost always have a way for users to flag inappropriate content. If something's flagged disproportionately, it will be removed.

So this is to counter stuff like GAN's (generative adversarial networks) [1] to reverse engineer data out of a black box systems? Like Yahoo's NSFW [2] classifier for example. [1] https://en.wikipedia.org/wiki/Generative_adversarial_network... [1] https://arxiv.org/abs/1406.2661 [2] https://github.com/yahoo/open_nsfw

There's this by Yahoo: https://github.com/yahoo/open_nsfw

Related: open_nsfw [0] (code for running Model and code for Not Suitable for Work (NSFW) classification using deep neural network Caffe models). [0] https://github.com/yahoo/open_nsfw

Direct link to Github: https://github.com/yahoo/open_nsfw