Hacker News Reader

Comment by mbil | Hacker News Reader

I’ve been conducting AI Coding interviews at my job. Candidate will screen share and use AI-assisted coding environment and tools of their choice.

I ask them to implement xyz thing. What I’m looking for is how they interact with the AI agent. Do they ask the agent to plan first? Do they review the plan? Etc. It’s pretty typical stuff that you might expect an experienced engineer to do if they effectively use such tools daily.

There are a series of follow-up questions about how to productionize the toy system, which gives some additional signal about how well they understand what they’re making. I sprinkle these in when there’s dead air waiting for the bot to think.

I think we’ll need to evolve and refine this problem and process as the models continue to improve.

View on HN

>It’s pretty typical stuff that you might expect an experienced engineer to do if they effectively use such tools daily.

It's also the kind of stuff that somebody can learn in a week, so not hiring the right person who just didn't spend this week of time yet for whatever reason is a loss.

Yeah it tells you nothing of value. It's the Recency Bias taken to its ultimate conclusion isn't it. Anything I learned about AI I learned in the last few months but somehow that's as important as all the experience of my decades long software career? Come on.

It signals you don't have enough time or will to chase the latest stuff, otherwise you would have learned it with other cool kids half a year earlier. It carries some signal, but I would not choose based on this alone. I would even consider selecting against this to a certain point, but I also work in a place that COBOL on a mainframe. YMMV.

> I would even consider selecting against this to a certain point

I agree. A technology professional who views themselves as "one of the cool kids" (read: easily manipulated by social media) is a legitimate security threat, as are many of the popularly promoted approaches to "LLM-assisted software developement".

I haven't considered looking at it from the security thread angle. Can you expand on this?

I used to think this was true until my company adopted a similar process, and the amount of very senior candidates who used it very ineffectively was highly surprising (eg staff candidate only using the default Google search AI), whereas the more junior candidates figured out you could one shot the problem but were unable to explain their code

Reply on news.ycombinator.com