I completely agree, but I feel it should be mandatory. (I've proposed this same "explain why" mechanism for downvotes a few times in the past.)

Currently downvoting has unbalanced incentives. You need a fairly high level of karma to use the feature, which suggests that it's considered a privilege. Yet, once you have it, there's no responsibility attached to the privilege.

This can make downvoting kind of sophomoric: "I was unreasonably downvoted when I was a newbie, now it's payback time!"