By that logic, you must also permit students another extremely well-established labor-saving technique: Hiring someone else do the homework/exams on their behalf!
After all, for students from certain families, forcing them to use the calculator personally would likewise be "robbing time from them", denying them valuable experience honing their skills in managing and subcontracting. They'll never be in a situation where they don't have a calc^H^H^H^H lackey.
I trust we can agree that (A) permitting outsourcing is absurd, (B) that cheating is not "ontologically good" even when reduces the total human labor, and (C) prohibiting the practice should not put a teacher into a state of stinky-fruit damnation.
I submit that we do care about how students do it and which skills they use, even if we disagree on what those are. It is not as simple as saying that good education means spurring them to supply Valid Answers By Any Contemporary Efficient Means.