that is generally what the features module is used for http://drupal.org/project/features
> which makes upgrading incredibly painful
yup, this is a real problem with small community sites that drupal is other wiser awesome for. the work involved can be rather arduous (and impossible if you are using a module that lacks an upgrade path).
that said, the drupal community is huge and in my 8+ years of building and running sites i have always been able to work out an upgrade solution, finding the time to implement it is the bigger problem.