In an extreme case, imagine that your typical, currently congested highway gained another 6 lanes in each direction, and we raise speed limits to 100 mph. First, the right of way is now way bigger, so we need to wreck a lot of buildings for this infrastructure. It makes being able to travel near that highway really nice for a while: Minimal congestion! A straight out windfall by people owning land near it. But that also means that the area redevelops, as farmland that was in the middle of nowhere is now a reasonable commute downtown! But as the development continues, the highway will fill up. And since downtown streets were not changed, it's the area downtown near the highway that is now a congested mess, as the limitation is getting from there to buildings.
And again, a subsidy for you can be a harm for me: If every street is 8 lanes wide, it's very hard to cross. Every pedestrian crossing added is also infrastructure, but it slows down traffic. Every bit of lawn you use is another bit of distance I have to wade through to get to my destination, vs an area with the same number of houses, but without lawns.
One can like one tradeoff more than the other, but the fact that infrastructure means subsidy, and that urbanism choices are picking some people's interests over others is just factual.