The hard part of the job isn't writing code to a detailed spec that's assigned to you. It's knowing how to ask for clarification on requirements; how to balance speed of delivery with nebulous ideas of quality (how readable is my code, how effective are my tests, how extensible); how to decide which tools are right for the job.

When I look for cultural fit, I mean: will you have productive conversations with your colleagues; have you shown yourself to be inquisitive about new things beyond what you're assigned so that you have a good understanding of what a solved problem looks like; will you force yourself to find a way around hard problems or will you quietly give up and call the problem impossible or blame someone else; are you interested in solving customer problems and iterating rapidly, or in building the most "well-engineered" solution; do you think your role in the organization is to build products or direct them.

These aren't easy questions to answer, and they do open themselves up to bias in how you evaluate candidates. I like asking about folks' CV - even though there's already a written summary - to see what parts of a problem you considered most exciting and hard; I think that's revealing. But may not work for everyone.

Granted it's a bit vague. See my response here for a more detailed explanation: https://news.ycombinator.com/item?id=6417834