My wife and I live in a townhome and we hate it. The noise through the walls, especially if you have inconsiderate neighbors, is a huge pain and you have essentially zero control over it.
Ofc, occasional loud situations, but that is no different from monsters that have parties outside their house.