No, expertise is not the difference. If you're a private person with 100 years experience in trading, you still can't do HFT. You need to be an instutition, have lots of capital to invest in servers, software development maintainance etc. As a private person I think you don't even get access to the API.
"Of course whoever has more capital has advantages in the market"? Of course they do, but I don't think "of course they should".
For this discussion, that funds don't beat the market average over a long term is irrelevant. Why not say "who cares you get more latency than the other bank, if you want money just invest in S&P500 and long term you'll beat them". But you don't apply that to banks against banks, only to banks against Joe. Why?