I don't think that this is just an attitude for less technical users, I have the same attitude myself whether I am configuring an HTTP server or playing a video I want a low friction way to do this. I don't think that power users exactly seek out the hardest interfaces.
The problem is though that the highest friction way to do something is to not be able to do it at all and have to work around that. I think this is why there is much resistance to things like Unity on Ubuntu.
It's also worth bearing in mind that consistency is also important, so doing something like releasing an "easier" Window Manager that is a huge departure from what was there before will not necessarily be welcomed by users as much as you think.
Having said that , I don't think that usability is the #1 thing holding back FOSS desktop adoption. The biggest things are MS inertia and also compatibility, for example without Silverlight4 + DRM you can't use things like netflix which is going to really hurt adoption (although I'm not sure what could be easily done about this case).