I like having both, my M12 impact has a speed/torque setting which comes in handy for lighter uses, but the adjustable clutch of drills is also useful for driving screws.
I think a lot of the frustration with using a drill to drive screws is just that the adjustable chuck is annoying to use with a hex shank driver bit, which is solved by drills with interchangeable chuck heads such as Festool's drills with their FastFix chuck system, the Bosch 12v small driver, or the Milwaukee installation driver.