Then you'll just be labeled as "not passionate enough." BTW, even if you worked all weekend, you're still expected to stick around at work late on Thursday for some whiskey tasting or some other bullshit, because if not, you're not a team player or not a good cultural fit :/