you spend about 50% of waking hours with co-workers, it doesn't make sense to me that you would want to ignore them. The majority of my friends are former/current co-workers.
And how do you know they don't want the same thing? I know tons of people that would rather spent time with their loved ones than their coworkers. You might want to rethink your definition of ignore, because it sounds very much like unhealthy peer pressure.