> What we did is we had them predict on a zero to 10 scale how members of the opposite sex would rate them in terms of their attractiveness

> accuracy was not significantly better than chance

I think this experiment conflates people having different opinions on attractiveness with the inherent difficulty on mapping the attractiveness spectrum onto the scale of 1 to 10.

I'd be interested in an experiment that instead had participants order the attractiveness of many photos from most to least without assigning numbers to them. The photos should range from grotesque to average to supermodels. I think almost everyone would agree on the broad ordering, with some variation in exact ordering.

Then, to capture the personal aspect of the original experiment, subjects could insert themselves into the ranking and see if the opposite sex agrees. My hypothesis is that performance would be far better than chance in this scenario.