I have heard managers respond that if you can get your responsibilities done in half the time, you should be getting more responsibilities.
At least I hope I would.
Normally I don't think of witty things like that to say until about a week after the event.