My thoughts are generally in line with yours - big problem from the consumer's perspective is tendency for most titles to end up in the 3.8 to 4.2 range. Not very helpful when I'm browsing titles.
The stars are also just a snapshot of feeling, and don't account for any context like expectations, identity, mood etc. Maybe I'd give Black Swan five stars because it was impressive and won awards, but it doesn't mean I necessarily enjoyed it or would recommend it to others. Perhaps a system based loosely on Net Promoter Score ideas could be more helpful