compas-analysis

It's the same underlying analysis. Their github repo contains the same script I linked to: https://github.com/propublica/compas-analysis https://github.com/propublica/compas-analysis/blob/master/Co... It even contains the same weasel words "almost statistically significant" for one analysis.

In block [37] of the ipython notebook, are racial main effects missing? I only see interactions. https://github.com/propublica/compas-analysis/blob/master/Co...

It's far easier to hide an agenda behind verbiage and anecdotes. Go read the author's actual statistical analysis: https://github.com/propublica/compas-analysis/blob/master/Co... In the statistical analysis (unlike the verbiage) she is completely unable to hide the lack of bias and the accuracy of the algorithm, all of which are clearly on display in line [36]. In contrast, her verbiage somehow conveys the exact opposite impression.

Go read the description of the statistical analysis or just view their R notebook: https://github.com/propublica/compas-analysis/blob/master/Co... Their own analysis shows that (p ~= 0) that high and medium risk factors are predictive. They also showed that the racial bias terms (race_factorAfrican-American:score_factorHigh, etc) are probably not predictive (p > 0.05). Your quotes are not evidence of bias, though I see how they might confuse an innumerate reader. It's interesting how good a job this article is doing confusing the innumerate - it's almost as if it was written to mislead without technically lying. For example, black defendants being pegged as being more likely to commit crimes can be caused by one of two things: bias or perhaps black defends actually are more likely to commit crimes. According to ProPublica's own analysis (

The validity of the algorithm can be - and apparently has been - reliably tested and been found to be useful and mostly unbiased. This analysis has been performed by both the algorithm's creators and highly adversarial third parties, such as the author of this article. Both found that whatever bias there is is small, and cannot be distinguished from random chance. For example, the author of this very article has done such an analysis. Here's her R notebook: https://github.com/propublica/compas-analysis/blob/master/Co... Her analysis shows (within the limitations of the frequentist paradigm) that: a) the predictor is useful - score_factorHigh and score_factorMedium both have p-values that are essentially zero. b) The predictor is not racially biased that much - race_factorAfrican-American:score_factorHigh and the other bias terms have p-va

According to propublicas own analysis, the claim of bias cannot be shown to be statistically significant. https://www.propublica.org/article/how-we-analyzed-the-compa... This article is terrible data journalism and probably deliberately misleading. Step 1: write down conclusion. Step 2: do analysis. Step 3: if analysis doesn't support conclusion, write down a bunch of anecdotes. Really, here's her R script: https://github.com/propublica/compas-analysis/blob/master/Co... Just read that. It's vastly better than this nonsensical article.