What the Boston School Bus Schedule Can Teach Us about AI
wired.com
wired.com
I think I'd be less cynical if they had done less to bury the lede deep in the midst of the article, and not once mention "due diligence," let alone as the crux or conclusion that I took as the real Root Cause here.
To me this isn't a story about AI. This is "just another" story of how readily public decision making can be swayed by biased special interests and an uninformed media/electorate.
The author should be praised for revisiting the topic, being upfront about their change in stance and trying to close the loop on it; but I think I am seeing the problem space from a very different angle than they are.
Author of the article is Joi Ito, director of the MIT Media Lab. I wish he had spoken to the original creators of the algorithm (down the street) before co-authoring the article.
But I give him credit for this kind-of a mea culpa w/ additional context.
Its also not quite the same problem for those of us who ride the bus, parents aren't trapped schleping kids on the bus. I'm surprised Boston Schools don't use the local mass transit system for middle & high schoolers, we've been pushing for that change in Seattle for years (all highschool students get ORCA cards finally). Why needlessly duplicate service?
Even in the well to do neighborhoods of Seattle, most of the students from upper income families are riding the bus to school after elementary. Driving your kid to school is very odd, even for the most wealthy. That being said, the private Catholic schools in Seattle are a whole different ballgame, they're singlehandedly bringing back long gone diseases as the archdiocese up here is very permissive of anti-vaxxers.
I don't think driving your kids to school in Boston is unreasonable. Some school buses were chronically late, delivering kids to school up to an hour after the start of the first class.
It may surprise you to learn that Boston is not Seattle. For example, the public schools here are the ones with antivaxxers and diseases you learned about playing Oregon Trail.
Do you think they were not able to identify what is in their best interest?
Having appropriate attention to the story you are about to spread matters, especially for bureaucrats and scientists. These job groups are less inclined to think marketing matters than, say, business people.
I see this a lot in local bureaucrats. Even the most well willing are usually not thinking in terms of customers, focus groups and targeted information packs. They should. Think about what your goals were and communicate them. If you suspect opposition, create targeted narratives. Heck, call, mail or email those groups most negatively impacted. It's what good businesses do well, but most organizations fail at.
A sidestep would be that all those goals met, including costs savings while getting some kids home at 1pm and others at 4pm has another problem: a too low budget. In my part of the world schools just end at 12.30 or 3pm and parents just need to figure out the rest. Of course that rest is subsidized childcare.
I'm not sure what this means, and I wish the article had gone into more detail. If it means "the preferences of families in higher-income neighborhoods were weighed the same as those of those in lower-income housing", that seems great, and uncontroversial. Those families probably had an outsized voice in the old scheduling process and so will do relatively worse in the new one, but oh well, that's life.
If instead it means "the preferences of families in high-income neighborhoods were intentionally given less weight than those of low-income families [to discount their privilege]," that's rather more ethically questionable.
To those with privilege, this will feel like bias against them (which is how I read the scare quotes above - feels like but is not actual bias.) They are losing an advantage they once held.
Being white is highly correlated with being privileged due to historical circumstances, but it is only a racist reading of the word that thinks that being privileged is being white.
When pro-white racists exist and are more common than anti-white racists in positions of influence, being white is, on its own, an advantage other people don't have.
It may also correlate with wealth, parental education, and other sources of privilege, but it is a source of privilege itself.
> but it is only a racist reading of the word that thinks that being privileged is being white.
Being white is, ceteris paribus, being privileged, not the other way around.
And, sure, that's a product of the existence of racism, but it's not racist thinking to recognize the effects of racism.
>One of the first things they showed us was a photo of the parents who had protested against the schedules devised by the algorithm. Nearly all of them were white. The majority of families in the Boston school system are not white. White families represent only about 15 percent the public school population in the city.
followed by:
>Optimizing the algorithm for greater “equity" also meant many of the planned changes were "biased" against families with privilege.
Since the author didn't clarify what they meant, or differentiate between them I am assuming they used the two terms interchangeably.
One funny thing I noticed. You could argue that since white families only constitute 15% of the public school population these changes disproportionately negatively affect a minority group.
When we speak of wanting to assist minorities this is when they are disadvantaged minorities. i.e. they are the opposite of privileged.
Please don't just play word games.
Don't disproportionate negative consequences constitute a disadvantage? Especially when faced with the existing issues of being a minority in a school system? Who really is disadvantaged here?
You can compound that with the new racism definition and I'm starting to wonder where all this supposed privilege is.
Similar aspects appear in many automation scenarios in many industries. In general, if you want a system to be adopted, it needs to provide at least some benefit also to those who are in power; if the current status quo is better for them, then the planned changes can be (and usually are) sabotaged. There are many cases when you can get a win-win where the changes benefit everyone; but if they put the privileged groups at a disadvantage (e.g. hospital systems that significantly reduce work for nurses but add some inconvenience to administrators or top doctors) then it's an understatement to say that you'll have an uphill battle. It's unrealistic to consider the possibility of some proposed changes purely on technical or ethical merit - you can ignore politics but politics won't ignore you.
A key step in implementing any changes is to identify all stakeholders (especially if they may have conflicting interests) and identify which of these stakeholders have critical influence and (possibly) a de facto veto vote. If you can't sell the proposal to these stakeholders, then it doesn't matter how good your proposal is and what the others think (unless they can influence the opposition), it's not worth the effort to proceed further because you won't succeed without having them on board.
Perhaps it's worth looking at it as a Chesterton's fence situation. If you come from outside, unaware of the local situation, and find that the system is biased to provide a nontrivial advantage to group X, then before action you should ask, how come? Why is the system set up this way? It may be that this is accidental, but more likely than not it's an indicator that actually (no matter what the official, documented, publicly disseminated policies might say) group X has the ability to decide, or at least influence, what the policies are going to be. And that's valuable information, especially if you want to change that. Let's not put the cart ahead of the horse - you won't reduce the influence of group X by changing that policy, rather you'll need to reduce the influence of group X before you'll be able to change that policy.
We can speculate all day about how people may have felt, but we just don't know. The district should have done polling to see what the overall reaction was before they pulled the plug. And if the counter-arguments amount to "I don't like change" then I think the district has a prerogative to optimize for health, safety, and costs.
This type of work has been put into practice for longer than the author has been alive. I don't think we need to reflect upon the changes it will bring to society since it's already old hat. I will admit that "What the Boston School Bus Schedule Can Teach us About Junior year Undergraduate Mathematics" doesn't have the same ring to it.
You are right about the popular (and, increasingly, technical also) mis-use of the term "AI" but that ship sailed a long time ago.
There are simplified problems that can be presented to undergrads, sure - just like most areas. Intro courses often use a toy version of this to show integer programming and the like. But there is as reason these are "toy" version - some of the general variants are NP-hard and have connections to interesting and open problems. Many real world variants are not trivial to undertake.
On a general note - if you hear someone who is even slightly plausibly qualified say "Problem X is hard" and you think "Huh, I solved problem X in undergrad" - the overwhelmingly most likely reason is that you aren't, in fact, talking about the same "problem X". This can of course be due to the method of communication (e.g. science journalism)
In practice you can often get away with improving an existing system instead of finding the "best" solution to the problem. This is what pragmatic people do, and it is often a lot easier than we think it is.
If you frame the problem as "finding the best possible solution to the bus routing problem factoring in all the needs of the populace", I would agree that's a very hard problem. If you frame the problem as "can we improve the existing Boston bus routing system", it will likely be a lot easier. Always remember that perfect is the enemy of done.
As you say, it's what pragmatic people do (and it's usually still more complicated than the intro textbook versions)
Now I have no specific knowledge of this particular problem, but the article does suggest it is something that has been tried before (rescheduling) and failed for various reasons. I'll give them the benefit of the doubt, why won't you?
Really I'm objecting to the idea that "if only we could have found a math undergrad, this would have been improved years ago". That's a particularly naive view.
the thing that makes these sorts of problems not quite so simple is that there are fairness constraints, not just constraints on routes and schedules. there may also be a requirement that the solution should be chosen in such a way that a rational agent has no incentive to lie about their true preferences. the best way to achieve these many conflicting goals is definitely not settled.
The school has figured out that starting HS late-ish and letting all student out before dark provides the best situation for students.
But if all school levels start at the same time, the district has would have to have a bunch of school buses and drivers that wouldn't be used other times.
To avoid all this "waste", the district has to decide which schools to shove into some other, worse schedule. They thus came up with a algorithm to minimize the amount of bad-scheduling but which maintains the existence of bad-schedules. And when it sent an entirely different group than before into the bad-scheduled situation, naturally that group complained - it didn't help these were wealthier, whiter groups with influence.
Sure, you thus have more affluent people saying they didn't want to be the one sacrificed here. But you also ask, why sacrifice anyone? Why not bite the bullet and have a good, standard schedule for all schools? You'd offending the modern god of efficiency but you'd actually be aiming for a greater good.
The article talks about Trolley Car Problems a bit and how people relate to algorithms "solving" these. The thing is instead of saying "that there are sacrifices is given, now we must decide who", which guarantees divisiveness, one could say "let us make choices as a society which mean that no person a priori need be sacrificed."
Your first quote describes reality, your second is nonsense. If there were an outcome where everyone's needs are met and nothing need be sacrificed, there would be no opposition to the changes. Every decision comes at a cost, we don't have unlimited resources so we should reach for tools that attempt to maximize what we are willing to spend. If you believe that no one needs to sacrifice in order to solve society's problems, you are living in a fantasy.
Basic income is system that would approximately guarantee all people's basic needs are met yet there is certainly opposition to it.
Please explain (or cite a source that does) how you roll out a basic income in a way that doesn't create new schemes that leave more money on the table to be hoovered up by the oligarchic corporations. I strongly biased to believe that if you add resources arbitrarily to a human society, then a segment of the population will, will, find a way to capture them.
The point is that in policy today, "everyone's needs will be met" isn't an automatically a winning argument.
I put different amount of faith in different states, but not very much in any state. Not in the long run. The problem is, the state is going to dictate more and more conditions for receiving that income. Couple it with oligarchic interests and no good can come of it.
Problem is, I don't see much good coming from our current systems either, subverted as we speak.
There is an Excel sheet for solving such problems, and if one wishes to go further, the basic heuristic and metaheuristic algorithms are not that hard to implement either. http://people.bath.ac.uk/ge277/index.php/vrp-spreadsheet-sol...
With these keywords you are welcome to fall into this rabbit hole as deep as you wish. I promise to greet you on my way up (currently finishing my PhD on the topic).
I gather that the new schedule was not presented with a score card, comparing it with the old schedule, or that the score card was maybe buried in a report.
Numbers like:
- expected decrease in number of elementary students getting home after dark
- expected decrease in number of early pickup of high school students
- expected decrease in cost
- expected capacity for special needs, compared to old system
- numbers directly supporting the claim of increased equity
can be understood by everybody, and would go a long way.
* if you make a fair choice, it'll appear to screw over previously privileged people.
* just because it's an optimal solution doesn't mean it's a reasonable one
* actually talk to people about their needs
Nobody likes the scruffy operations researcher who awkwardly tells decision makers what to do in the form of an optimal solution, because this takes the power away from them. However, if one presents several good, preferably nearly optimal but different to each other, solutions between which the decision can be made freely, you have a better chance to make a change to better.
1) Don't use approaches that are black-box in nature if they will negatively impact people who have a say in its adoption. Stay simple or create tools that let users play with the parameters to understand how the consideration of factors creates a different outcomes. Expose them to how it works don't just ask for trust.
2) The narrative that takes hold around an algorithm supersedes the reality of what the algorithm does and how. Manage the soft side of the rollout just as strictly as the technical side. Just like in startups, sales is equally as important (if not more so) than having a working product.
Shouldn't that "people who are going to be harmed by it." I mean, this situation makes it clear that no one likes being ridden-over by a black box. We shouldn't say "well, it's OK if they can't complain." (and the racist-AI parole system example shows this is a real risk [1]).
[1] https://www.wired.com/2017/04/courts-using-ai-sentence-crimi...
I've found that this works only if the tools can actually be used reliably to arrive at an outcome without supervision. If there are intermediate results that have to be checked against descriptive knowledge that isn't explicitly included in the model, then people will be disappointed when they choose a configuration and learn that it can't work for some reason or other.
Such as, "Oh, the calibration standard doesn't work in that range, I didn't put it in the model because I didn't expect anybody to try it, and it wasn't in the requirements at the time."
Politics, on the other hand, is all-or-nothing. They didn't go back and reconsider the algorithm or the weights or discuss why certain factors were assigned certain weights. It was either implement the algorithm or don't. They balanced progress with political reputation and when "too many" voters voiced displeasure they abandoned ship for fear of a day or two of bad PR. That's not a recipe for success when it comes to technological change, it's a recipe for disaster.
I’m surprised the school board even proposed it after seeing it. It sounds like they completely ignored their own knowledge of political realities and tried to rely on saying “the machine told us so” would insulate them from criticism.
My mom spent a little time on this issue (bus schedules) in high school. Our district officials were extremely aware of the pushback they would get to almost any significant change in busing and school start times, no matter what direction. People don’t like their cheese moved.
https://apps.bostonglobe.com/ideas/graphics/2018/09/equity-m...
My only complaint is that requiring someone to type something halfway through an article isn't great UX. I still think the overall goal--educating people about policy tradeoffs and the role of technology--was phenomenal for a column mainly tackling local issues.
(I wonder what would have happened if they set the algorithms' preference as an end goal, but only shifted start times towards that by five or ten minutes per year, so no one's family schedule was radically affected in any given period, but they still would have realized the benefits eventually.)
https://news.ycombinator.com/item?id=18059535
> "start times is something the district had wanted to do for a long time" So, was the real motivation here one of efficiency and cost-savings then?
and the response was:
"No, as far as I know having later school start times for high schoolers was the main objective, and that was the strongest one in the solution that was chosen by Boston when they used our algorithm."
"It clearly stated that cost-savings was one of the outcomes to be balanced, but not the motivation."
https://news.ycombinator.com/item?id=18059535
However, I'm still not entirely convinced this was purely done "for the students".
even in this article it claims it "started as a cost-calculation algorithm".
So please, no, you don't get to change the motivation from cost-calculation/savings to "later times for students" by trying to re-purpose such an algorithm, even if it does benefit students in such a way (albiet with other negatives that the community felt outweighed the benefit to students).
Although, what i didn't understand was that the opposition was "15 percent the public school population" and only from wealthy families, which changes the perspective somewhat.
Here's an article with a bunch of examples from elevator door close buttons to fake thermostats:
https://www.nytimes.com/2016/10/28/us/placebo-buttons-elevat...
So much of the pushback seems to have come down to the framing of the "algorithm" as this emotionless black box devoid of input instead of what it really is: a model based upon inputs from all sides.