Try the recursive options of wget. It should be pretty much what you are talking about.
Nice, that means the tool does exist already.
What I am thinking of would really go beyond what you could achieve with wget. No doubt offline browsing and personal crawlers have been around for a while. Think of a 10TB+ tivo-like device that comes pre loaded with everything from wikipedia to musicBrainz. People would't need to figure out what to download, it would already be there, and they could augment it with anything.
Just a thought - I am going to keep looking into it and will create a public repo for it if it makes sense.