There's a political element to this to though. Remember, human beings, including your boss and coworkers, are social creatures. What you consider time spent "bullshitting with coworkers" might actually be an investment into office culture and politics.