To be frank, it sounds like your colleagues may be holding you back.