This is a common thing to hear from parents but I feel it misses the mark for some couples that have rationally decided not to have children for precisely the reasons you stated. I think the difficulty you mentioned is actually a large reason as to why less couples are having kids in the first place.
Edit: completely reworded.
Everyone knows that kids are a pain in the ass. Everyone who has kids says "I knew they were a pain in the ass but they are way more of a pain than I even anticipated."
So no, you don't know. You think you know but you don't. Continue not having kids. I will also continue not having kids. But don't be the "well I haven't seen it because I don't have a tv" guy because nobody gives a shit.
You can understand that well enough to decide that you don't want any part of it, without understanding it to the depth of a person who has experienced it.
For me, having a kid has certainly been an (expected) money drain. But beyond that, my life is pretty similar to pre-kid. Wife and I still work our day jobs. Outside of work we still enjoy the same sorts of things we enjoyed before having a kid and continue to do those things. Now we just have an extra source of entertainment and responsibility in the kid as well.