If you cannot explain why you have made such decisions, then the state will look at disproportionate impact. EG: You hired/lent to/rented to black people 25% less often than you did to white people. "Because the AI said so." doesn't cut it.
If you cannot explain why you have made such decisions, then the state will look at disproportionate impact. EG: You hired/lent to/rented to black people 25% less often than you did to white people. "Because the AI said so." doesn't cut it.
I don't like when laws are worded such that we told we shouldn't care what the words say, we all know what we meant, this is a law for getting the bad guys, so don't worry about the actual words we use. You shouldn't need a law degree to know what OR means.
“Adverse or disproportionate impact” is a well litigated phrase. It has a specific meaning in law, which is not immediately obvious from a layman’s definition.
For instance, in Texas motor vehicle code regarding driving under the influence, there is a definition for "Motor Vehicle." While colloquially a person would assume that such a vehicle should have a motor, the definition actually states:
> "Motor vehicle" means a device in, on, or by which a person or property is or may be transported or drawn on a highway, except a device used exclusively on stationary rails or tracks.
When it comes to laws, you have to read the entire law (including definitions) and not just rely on your own understanding of the terms within. Then it gets even more complicated when it comes to so called "Case Law." This is why companies have entire sections of lawyers to inform their managers on compliance.
A large portion of the interview touched statistical inference as it related to ML, specifically how it related to simple neural nets up to deep learning vs classical modeling. The answer I gave aligned with their expectations, which was that models used in lending should not be black box and should be able to quantify which features led to the prediction/output and how much weight they contributed. This was specifically done to address potential discrimination lawsuits.
I have a hard time believing any company that rents, lends, etc. would employ a black box for decisioning. Both private and public lawyers would sue them into oblivion immediately.
https://www.occ.treas.gov/publications-and-resources/publica...
https://www.theguardian.com/us-news/2025/jan/25/health-insur...
See also: "It's not a crime if you do it with an app" [/s] - https://pluralistic.net/2025/01/25/potatotrac/