If some people paid by other people want me to sit and listen to them all day and to think about what they want me to think about then it isn't "giving up on me" to let me go and do something else if I would prefer to do so.
It's simply acknowledging my status as a human being.