You can compile, but you can't link with the standard GNU toolchain to get a usable library (at least last I heard), because binutils simply doesn't support the Mach-O format (see http://gcc.gnu.org/ml/gcc/2009-09/msg00121.html).
If this turns into a "real issue", such that XCode 3 was rescinded and XCode 4 remained paid only - as opposed to some stupid accounting problem which might mean it would be bundled free with Lion, with XC3 hanging around for (Snow) Leopard - then I suspect we'll soon see MacPorts or their kin come up with a downloadable toolchain based on this existing Apple open source work that would allow them to continue virtually unchanged.