Why would a competitor build this Riak benchmark? How are they supposed to know which Riak configuration performs best?

That's the point of YCSB. Each vendor can submit the optimal configuration for their system, and they all run the same benchmark.

At the end of the day, each vendor is going to publish benchmarks that show their system performing better than all others.

It's your job, not theirs, to verify those benchmarks for your particular work load.

Edit: Also, while Mongo wouldn't classify as a distributed store, Cassandra definitely would. In fact, it's more "distributed" than Riak, since you have to pay Basho for multiple DC support (unless that's in the open source version?), whereas Cassandra is completely free.