If you are using "lifestyle business" to mean "a business has priorities other than profit at all costs", you are using it incorrectly. A lifestyle business is one where the owners have set a target business size and profit level, and do not try to grow past this; some owners even actively inhibit growth to keep the business from getting too large. The purpose of the business is to give its owners a fixed cash flow.

A company whose owners are working to grow the business consistently and organically is not a lifestyle business; it is also not pursuing "profit at all costs". Most privately-held businesses fall into this category.

As to "lifestyle business" being a slur, it is unfortunate that members of the growth-obsessed communities use it as such. They don't see a business with a fixed target size as serious. Incidentally, many also think the same about the organic growth case I mentioned. I ignore them, and most other things from VC startup culture.