As another example, government can't stop you from saying whatever you like (freedom of speech), but your employer can restrict what you say at work.
So, to be clear, a higher standard means the government is not allowed to use data-driven algorithms to make decisions? Keep in mind that judges already had and continue to have latitude to independently conclude the same thing the algorithm does and act appropriately.
Or is your disagreement with the closed nature of the algorithms? Which is an argument, but one that should also be applied to insurance companies, especially those brokering mandatory products like car and health.
EDIT: To copy from my other reply:
If the problem is in the openness of the decision, then let's open up everything. If the problem is in making data-driven, algorithmic decisions in the first place... I'm not sure that's something I can get behind.
No, I generally agree with you. However, I'm very concerned that judges will rely too heavily on the computer, either through laziness or through misunderstanding the limitations of an algorithm.
> is your disagreement with the closed nature of the algorithms?
Mostly, yes. IMO:
* A spec, which the code meets, must be published. The spec should be comprehensible to non-technical readers - at least judges and lawyers. (EDIT: The spec governs, not the code that implements it.)
* The code also must be open and written to some standards of coding comprehensibility and documentation, so others can verify that it meets the spec.
If they just open source some code, it's useless. Even experts such as software developers probably aren't going to quickly grasp someone else's code.
> Which is an argument, but one that should also be applied to insurance companies, especially those brokering mandatory products like car and health.
I tend to agree, but I think it's still a matter of degree.
Alternatively, If an algorithm for recidivism is questioned, why should the algorithm for insurance also not be questioned?
Two wrongs do not make a right.
You can, of course, question them. In many cases they're open source, they're usually government-audited, and in some countries, even government mandated.
Edit: hackuser beat me to the point - https://news.ycombinator.com/item?id=11957351
> Keep in mind that judges already had and continue to have latitude to independently conclude the same thing the algorithm does and act appropriately.
Standardizing logic does not equate to elimination of bias. From some bias (due to humans), we may move to standardized bias (due to algorithms). I would not call that a win.
If the system is open, then the bias can be found through statistical analysis with experimental data. If the system is closed -- as it must be in human logic -- then the bias can only be discovered through actual outputs after enough runs have accumulated. In this case, that means people's lives are being affected by something that we can't necessarily tell is biased until its too late to prevent it.
So yes, moving to standardized logic is a win against bias, because standardized logic can be opened and analyzed and modified.
The algorithm in the original article is not open and generally, that is the problem.
> If the problem is in the openness of the decision, then let's open up everything. If the problem is in making data-driven, algorithmic decisions in the first place... I'm not sure that's something I can get behind.