You can theoretically turn off your iPhone easily as well. Unless your dumbphone has a removable battery it would be just as difficult to trust it's actually off than a modern phone. But sure, removable battery would be an advantage on that dimension, I agree.