Yeah the greater fool theory doesn't sound too appealing to me.
Fiat currency is the same way. Rather than point a giant network of contracts together in order to create some monster transaction that satisfies a few hundred people evenly, we just agree to respect the unique-but-valueless papers produced by the crazy guy who will imprison anybody who tries to produce those same papers. It didn't have to be these papers; it didn't have to be bitcoin. It just had to be something that had the necessary properties: unique, scarce, lightweight, secure, and, most importantly, used by as many other people as possible.