As you say, capital gains are not "only" due to speculation, but just sometimes. You also assumed that your risk capital was originally "earned" implying that you paid full tax rates on it. Unless your original earnings were from investment or, ahem, luck of birth (I mean, inheritance).

Do you consider being lucky enough to be born into a wealthy family to be doing something productive?