In Wisconsin, a Backlash Against Using Data to Foretell Defendants’ Futures
nytimes.com
nytimes.com
That's the part that is most Orwellian. "Our magic box says you are going to commit another crime if we let you out. Sorry, no, you aren't allowed to know anything about the magic box."
And just like with Theranos, the stakes are too high for people to just assume these people know what they're doing. If someone is going to be spending years of their life in jail because of an algorithm, it better be accurate.
Wisconsin courts use an algorithm called Compas for sentencing (though not exclusively AFAICT). Compas is an algorithm that calculates the likelihood of someone committing another crime and suggests what kind of supervision a defendant should receive in prison.
Compas' algorithm is secret:
Company officials say the algorithm’s results are backed by research, but they are tight-lipped about its details. They do acknowledge that men and women receive different assessments, as do juveniles, but the factors considered and the weight given to each are kept secret.
“The key to our product is the algorithms, and they’re proprietary,” said Jeffrey Harmon, Northpointe’s general manager.
I'd like to know the size of the list. If they made predictions regarding 100 people, that's more credible than a list of 1000. Context is missing for a thorough evaluation of the results.
Or, you know, the predictor in the random forest that is weighted very low.
_another_ ? Why not optimize further and just determine the likelihood of committing of the original crime and lock up based on that even before it happens?
When you realize that an algorithm is just a process to be performed, what is new here other than using a computer and a large amount of data? Judges always have been using information to predict the harm that a defendant might do in the future. I dont have the data, but I'm sure that past convictions, antisocial behavior, friends that have pro criminal beliefs, etc do in fact correlate with future criminal behavior.
Now I know that correlation is not causation, but still. Is this just because the algorithm is closed source? A judges thought process is closed source also, no? To me this seems like an obvious and good thing.
edit: paragraphs
As another example, government can't stop you from saying whatever you like (freedom of speech), but your employer can restrict what you say at work.
So, to be clear, a higher standard means the government is not allowed to use data-driven algorithms to make decisions? Keep in mind that judges already had and continue to have latitude to independently conclude the same thing the algorithm does and act appropriately.
Or is your disagreement with the closed nature of the algorithms? Which is an argument, but one that should also be applied to insurance companies, especially those brokering mandatory products like car and health.
EDIT: To copy from my other reply:
If the problem is in the openness of the decision, then let's open up everything. If the problem is in making data-driven, algorithmic decisions in the first place... I'm not sure that's something I can get behind.
No, I generally agree with you. However, I'm very concerned that judges will rely too heavily on the computer, either through laziness or through misunderstanding the limitations of an algorithm.
> is your disagreement with the closed nature of the algorithms?
Mostly, yes. IMO:
* A spec, which the code meets, must be published. The spec should be comprehensible to non-technical readers - at least judges and lawyers. (EDIT: The spec governs, not the code that implements it.)
* The code also must be open and written to some standards of coding comprehensibility and documentation, so others can verify that it meets the spec.
If they just open source some code, it's useless. Even experts such as software developers probably aren't going to quickly grasp someone else's code.
> Which is an argument, but one that should also be applied to insurance companies, especially those brokering mandatory products like car and health.
I tend to agree, but I think it's still a matter of degree.
Alternatively, If an algorithm for recidivism is questioned, why should the algorithm for insurance also not be questioned?
Two wrongs do not make a right.
You can, of course, question them. In many cases they're open source, they're usually government-audited, and in some countries, even government mandated.
Edit: hackuser beat me to the point - https://news.ycombinator.com/item?id=11957351
> Keep in mind that judges already had and continue to have latitude to independently conclude the same thing the algorithm does and act appropriately.
Standardizing logic does not equate to elimination of bias. From some bias (due to humans), we may move to standardized bias (due to algorithms). I would not call that a win.
If the system is open, then the bias can be found through statistical analysis with experimental data. If the system is closed -- as it must be in human logic -- then the bias can only be discovered through actual outputs after enough runs have accumulated. In this case, that means people's lives are being affected by something that we can't necessarily tell is biased until its too late to prevent it.
So yes, moving to standardized logic is a win against bias, because standardized logic can be opened and analyzed and modified.
The algorithm in the original article is not open and generally, that is the problem.
> If the problem is in the openness of the decision, then let's open up everything. If the problem is in making data-driven, algorithmic decisions in the first place... I'm not sure that's something I can get behind.
No, a judge's thought process is determined by open law, open precedent, and the judge's reasoning is explained openly in the published legal opinion. That reasoning can be challenged in an appeal.
Openness in courts is a fundamental of democracy, liberty and human rights. Otherwise, you can't defend yourself and the government doesn't have to explain its actions to the public.
Sure, but they are put out in the open, and the judge does justify it in some written account, so that it can be referenced when challenged in an appeals court.
What answer should be given to someone who is given a sentence based on this algorithm and challenges it?
"Well, the algorithm said so, and studies say that it's pretty accurate, and we don't know how it came to this particular rating of you, so you're shit out of luck?"
edit: since i cant reply to hackuser below, here is a quote from the article: "Company officials say the algorithm’s results are backed by research, but they are tight-lipped about its details. "
EDIT:
ahartman00: I'm not sure why you don't see the reply link, but another option is the click the time ('x minutes/hours ago'), and you can reply there.
Regarding that quote, I thought it meant they were tight-lipped about the algorithm, not the research, but I see that it's ambiguous.
http://criminology.fsu.edu/wp-content/uploads/Validation-of-...
I'm pretty sure that's not true
* Judges often complain about sentencing requirements in the law, and those requirements are a or the major factor in the high incarceration rates in the 'War on Drugs' and for 'Three Strikes' offenders - so certainly they are prevalent.
* I'm pretty sure judges look carefully at other sentences for similar crimes and criminals (precedent), and I know many courts use formulas that consider past convictions and other factors in determining a sentence.
* Sentences can be appealed, so the judge has to follow certain standards to avoid having their sentence overturned by a higher court.
The laws, e.g. the U.S. sentencing guidelines, give you a pretty wide range for any given scenario.
> I'm pretty sure judges look carefully at other sentences for similar crimes and criminals (precedent)
Not really, at least not formally. Within the guideline range, it's mostly about extenuating or aggravating circumstances unique to the case, not a comparison with other similar cases.
> and I know many courts use formulas that consider past convictions and other factors in determining a sentence.
The sentencing guidelines do use formulas (add X points for prior criminal history, add Y points for amount of money stolen). That doesn't mean they're not totally ad-hoc and unscientific.
Here are the federal sentencing guidelines for larceny, theft, etc: http://www.ussc.gov/guidelines/2015-guidelines-manual/archiv.... E.g. if the amount lost was between $200,000 and $400,000, you add 12 points to the score. If it involved more than 250 victims, you add 6 points.
There is no science or rigor here. Just the drafters of the guidelines picking round numbers of dollars for one column and increasing the other column two levels at a time. What purpose does it serve to sentence someone differently based on whether they stole $350,000 or $450,000? And what impact does the difference in sentences have on things we care about: deterrence, recidivism, etc? It doesn't: It's quackery. It adopts the pretense of science without being in any way scientific.
I'm not an attorney so I don't have much more to say. I do happen to know a lawyer with expertise in criminal appeals, but probably an HN discussion isn't worth interrupting them for.
Yes, they are.
> Within a wide range, sentences are heavily dependent on judges' discretion and ad-hoc and unscientific consideration of various factors.
Whether that's true in any particular jurisdiction for any particular offense is, itself, a matter of law (including case law or precedent.) Under the federal system, while there are a possibility of departures -- and which departures are allowed is also a matter of law and precedent -- sentences tend to tightly follow the federal sentencing guidelines (which are a law) as well as being bound by the specific criminal statute under which a sentence is issued, so sentences are very much prescribed by law and precedent.
That's a subjective description, but I don't think its true for what most people would consider "pretty wide". Once you've added up the scoring points to get to an entry on the sentencing table, the ranges for any given cell on the table on the low end are 6 months wide (centered on 3 months for the lowest categories), those near the high end (where there is a definite range) get up to just under 7 years wide centered around 30 years. (After that, there are 30 years to life bands, which may be fairly wide, and bands which have only a life option, which leave no wiggle room.)
And the actual ranges for a particular sentence may be narrower than the cell, since they are also constrained by any minimums or maximums specified for the particular offense.
> On top of that, they're arbitrary.
No more, or less, than the definitions of crimes themselves are arbitrary.
"Arbitrary" to me means "not making rational distinctions based on criteria we care about." Murder versus arson isn't arbitrary. NY's 5 degrees of larceny probably are. Sentencing someone differently for stealing $250k versus $350k is.
I would suggest that the US Sentencing Commission guidelines do not meet that description, generally, especially if "we" includes the very large share of the population that -- whether you or I do or not -- care about retribution within the criminal justice system.
> Sentencing someone differently for stealing $250k versus $350k is.
Taking into account scale of theft probably is not. Obviously, any system that simplifies and uses breakpoints vs. a continuous function of value is going to seem to exaggerate the differences near the breakpoints, and breakpoints are inherently arbitrary in the usual definition (though perhaps less so in yours.)
Published opinions are exceptional, rather than normal, for trial courts.
Yet, it strikes people as "unfair" to play the statistics like that, because peoples' sense of justice requires some sort of individualized due process. That is why, for example, you generally can't introduce evidence of past crimes in order to prove guilt for a different crime.
I haven't seen any stats about the assets seized; do you know of any?
I don't think we should have civil asset forfeiture. But I think it's improbable that more than a small minority of those assets are legitimate (especially in the $10,000 in the glove box scenarios).
Im not sure about this: "this exploitative practice still stacks the deck against property owners, essentially requiring them to prove their innocence to gain back their belongings. For people whose property has been seized through civil asset forfeiture, legally regaining such property is notoriously difficult and expensive, with costs sometimes exceeding the value of the property" - http://www.aclu-tn.org/civil-asset-forfeiture.html
From what I've read, that's often because contesting it costs more than it's worth. You need lawyers, etc. For $5K, for example, it's not worth it.
> I think it's improbable that more than a small minority of those assets are legitimate (especially in the $10,000 in the glove box scenarios).
I don't know. However, it seems like your reasoning is based on certain scenarios, such as $10K in the glovebox; I don't think that's valid unless we know how common those scenarios are. We need some data.
Also, while I may not have a reason to carry $10K, there could be many people who live very different lives than me who have perfectly legal reasons to do so. Maybe small retail stores take their money to the bank, or people without access to banking services, or goodness knows who and why ...
That's not true. To contest the forfeiture, you have to file an administrative claim, which is basically a sworn statement identifying your interest in the property. If the government still wants to go through with it, it needs to file a civil case in court, where they bear the burden of proof. If they do that, then you may retain a lawyer, but it's a more streamline process than an ordinary civil case so you don't really need one.
The 80-85% statistic isn't about people who are unsuccessful in getting their property back; it's people who don't even take the first step of filing that sworn statement.
Your version is a little more heartening, but I also know the rules of the criminal legal system and its practice often are much different. For example, most people have little understanding of their rights and few resources to assert them.
Note before going further: I strongly support innocence projects, public defenders, etc. But if you read the news you might think most convictions are faulty, or most people are coerced into pleading guilty to crimes they didn't commit. But if you ask someone who works at an innocence project or as a public defender, most will tell you: the overwhelming majority of people who are convicted or accused are actually guilty. I saw this directly as a judge's clerk: tons of sentencing appeals where the conviction was based on literally a pile of hard evidence.
So yeah, people want to get rid of civil asset forfeiture (I do too!). So they tell you about the hard-working guy who saved $10,000 in cash and had it seized on the way to the dealership. They don't tell you about the 12 drug dealers who had $10,000 seized from their glove compartment the same day.
Yes, and only few people that are arrested that aren't guilty and only a few people the cops shoot don't deserve it.
However, each of these rare instances constitutes its own crime perpetuated against an innocent victim. If the justice system doesn't actually strive for justice, it's just another gang of thugs.
But yes I think people concerned about these issues would do well to understand that they're the minority of cases. If they were that common, things would easily change through the democratic process. The job is then convincing the majority, who will likely never have a problem, that such injustice could happen to them.
When a jury decides that someone committed a crime "beyond reasonable doubt" they are reasoning off probabilities.
Doesnt using the individuals data create an individualized due process? And this is for sentencing length, not conviction. Its not, did they do it, its how likely will they do it again. Not sure how I feel now...
If I sue you saying that you kicked me out of my house and are now living in it, I only have to prove that with 51% certainty.
Basically, the simply act of owning $10k in cash should not be considered sufficient evidence that you shouldn't be allowed to own that $10k. Regardless of the bar you're setting, that suspicion simply isn't proof of anything. The reason why I said it's arguably unconstitutional is because the way asset seizure works, the victim is put in the position of having to prove that they do legally own the asset, rather than having the police be put in the position of proving that they don't legally own it. Our justice system is based on innocent until proven guilty, so why does asset seizure work any differently? This is yet another bad consequence of the war on drugs, I just don't understand why our judicial system continues to allow for this perversion of justice.
This is precisely the problem. If the algorithm is dependent on a certain process, this process should be vetted for potential biases - big data or not.