I lived in a suburban single-family home with about 20-25 other people for a year when I was in college (each of us had our own bedroom, except for families where children slept with parents). For the most part everyone there was quiet, either going to the college nearby or else working while studying for a professional license or graduate school. The landlord also lived there with an extended family of about 10 people.
We had a couple of rotten apples, but we self-regulated and kicked out people who caused problems in our house or for our neighbors. Yet, we still had people in the neighborhood calling the zoning inspector to "shut us down." Zoning laws make it hard to even live with "6 other girls" these days. Boarding houses are mostly illegal, and people don't really build SROs anymore.