There are two differences. One, gold is impossible to counterfeit. The Fed can print more money, companies can dilute their stocks, Topps can introduce new lines of baseball cards, but short of building a particle collider, you can't make more gold. The chance of a new discovery affecting the price of gold is rather slim. The Earth is pretty well explored at this point. The second difference is that gold is a natural Schelling Point (
http://en.wikipedia.org/wiki/Schelling_point). Society needs one object to serve as a store of value. We call this object money. Throughout the ages, this object has consistently been gold. Currently, we are in a historically abnormal period where the store of value is a U.S. Federal Federal Reserve Note. If it looks like the U.S. government is abusing this privilege by diluting the currency, people start hedging back towards gold. On the other hand, if someone like Paul Volcker starts running the show, faith in the dollar is restored, and the price of gold collapses.
So yeah, gold is like the ultimate baseball card. But the emphasis is on the world "ultimate", and that makes all the difference.