Knowing computer science stuff like regular languages is really not required to understand the incompleteness theorems; grasping first order logic, a little arithmetic and some elementary recursion theory is sufficient.
I repeat my objection to the Goldstein book raised elsewhere in this discussion.