Yeah, IMO the correct combination is spandex shorts and cheap cotton for everything else. Having nice gear doesn't mean you aren't having fun.
I like my cycling tunic/jersey thing because it gets really good airflow, but I've gone on long rides in just a plain old t-shirt, too.
I've heard a lot of stories of people going through quite a few different saddles until they finally find one that "works", but in the meantime they've also been spending a lot of time riding and have toughened up their posterior. So was it the saddle or the adaptation that solved the issue?
I'll wear padded shorts and lycra for a looong ride (2-6 hours) as it's just more comfortable, but plain tights under short and a cotton tshirt is fine for the 30min commute.