Hacker News Reader

Comment by fernandotakai | Hacker News Reader

so, i've gone through a bunch of iterations on how to interview in days of LLMs and i came up with a process that is quite good:

first, a coding problem that is solved live (right now, online) -- it's not a hard one and most LLMs can easily one shot it.

BUT: if you are not a senior developer, you need to do it without LLM, and if you are a senior+ and do it via LLM, the bar is super raised and now you need to explain in details everything the LLM wrote as if you wrote it yourself.

the main component here is to know if the candidate understands coding in general, understands problem solving and how to dive into an existing (although quite small) codebase. solving the problem is not really the goal -- in fact, most candidates we hired did not solve it 100%, but were able to understand and explain everything that they put it on code. this usually takes 1h30.

(we also ask the candidate to open a PR against the repo just to see if they know the basics of git -- you would be surprise how many people cannot do it).

the second is an architecture interview, where we build a system together BUT instead of doing the whole spec, miro board, whatever, we do it in more of a free way: we start with a really simple, MVP level, spec and we ask how the candidate would build it. then, we add another feature, and we ask how they would go for it. during this whole thing, i ask random questions like "why would you use x instead of y?" or "why do you prefer this approach?".

one important thing: we allow the candidate to say "i don't know" and we tell the candidate that there's no gotcha questions (which is true, i hate these kind of questions). this lets the candidate be super comfortable (feedback we had from real people).

in general, this flow helped us a ton and i really really like running it :)

(edit: spelling)

View on HN

I feel like git operations are one of the few things that I truly don't care at all about whether someone uses an LLM for and doesn't tell me. PR descriptions and commit messages should be written by humans for humans, but like, if someone doesn't know how to manually configure their git remotes and what exact command to use to amend/squash their commits into the shape needed and instead has an agent prepare all that and use the GitHub CLI to open the PR, why should I care? Beyond the code itself, I care about the things that will continue existing in the long run fitting my team's conventions (the number of commits, the commit messages, the PR description and comments, and then whether there's a merge commit at the end), but the process used to create those things doesn't matter to me.

Wasn't this a think before_ai? Enough people are using IDEs which handle this for them. I still have to check a few commands now and then.

Though ideally interviewers should appreciate this - unless its a hard requirement you do it by hand day to day - the rest of us can look this up quick enough.

At previous jobs, I was the one coworkers would come to when they had messy git problems and needed help cleaning them up. I've put in the hours in the past to learn how to do all of that, and then later forgotten it because it's just not worth it anymore. I switched to jj a few years ago and had the majority of the annoying rebase issues go away, and whatever's left goes into my LLM to spit out a potential solution that I can then manually tweak if the result isn't perfect.

My OS professor in college once told us about what he considered the difference between "knowledge" and "information", with "knowledge" being the underlying facts about how something works and "information" being the terminology/labeling/bucketing/etc. of how to refer to it. The point he made was that knowledge was more important than information, and that he would try to test us on knowledge rather than information. To me, knowing how the underlying version control model works and what "shapes" you might want to make with it is useful knowledge. The exact set of commands, interactions with an IDE, or specific prompts you can use to ask an LLM to produce them is information. Arguably the entire benefit of LLMs is that you can ask them to do things even if you don't have the exact information memorized; if you can describe it in a rigorous way, and recognize whether what it gives you as an output fits that or not, you don't need to know any of the actual information of how to do it manually.

Problem for me is that unless I see someone being proficient at using git commands to rebase, merge etc. … I have hard time believing they understand underlying model. Even if someone knows all the right words.

There is no way to verify it just by talking.

i guess your difficulty is that half the commands i can write off hand and the other half i cant, and i only really care to understand the parts of the data model that i actually touch.

like, i dont really need to know all the stash details when i only stash one thing at a time, very temporarily

i also dont really care to know all the origin and remote definitions stuff when i only touch it once per project or so.

i could certainly draw graphs of what different git repos look like, but what makes it salient to know the rest of the details unless im actually building git itself? if youre using every git feature regularly you're probably doing something wrong

Your comment hits the nail on what I wrote.

I couldn’t care less about stashing commands.

I do care about … if someone understands origins and not by definition because developer needs to know what is his local version and what is on the server and how those two affect each other. Bonus points if they understand server is not special in Git and their colleagues copy on their laptop can be origin as well.

Drawing graphs of what different repos look like? That’s irrelevant.

Irrelevant if one understands that Git uses DAG and is content addressable store.

You see someone might know those hard words but when working daily with Git reasoning about rebase or merge, cleaning up their own work with interactive rebase, making pull/merge requests or resolving conflicts it is much easier if one knows what is local what is remote and how Git stores data.

You also learn it once. In my 15 years of using Git I basically didn’t learn much new commands but having mental model of how it works was my tool to fix any problems other people had. Knowing commands or definitions is secondary.

> like, i dont really need to know all the stash details when i only stash one thing at a time, very temporarily

If I were hiring, I would care about whether you can `git help stash`, skim over it (bonus points for actually using the pager competently) and figure out what you need to do.

I used to go to stackoverflow to resolve issues with git. The trouble was, I'd get 15 answers, all different, all saying "this might work", "try this", "try that".

Now I just ask google, and I get a straightforward "do this", and it works.

That's largely because the OP couldn't clearly describe a specific situation to ask about (the question underlying the problem, but also a clearly recognizable, reproducible example); and solving your problem might depend on things that are different in your setup vs. the OP's.

That's the thing, though. Stack Overflow is not and never was intended to solve your problem, nor to solve the OP's problem. It was intended to answer questions. It was your responsibility, both as someone posting a question and as someone reading the Q&A later, to adapt knowledge to your circumstances. That's why they're called questions and answers, and are not called problems and solutions.

People who use a web site to get a solution to a problem are looking for a help desk, or a forum. Stack Overflow was specifically created to be not that, and that choice was specifically motivated by the frustration people had with trying to search those kinds of sites for information, and the frustration that experts had with trying to help people (for free) in that kind of environment.

>I switched to jj a few years ago and had the majority of the annoying rebase issues go away

jj? what's that?

I wish better names would be chosen by the tech industry, that would make search engine discoverability easier.

I hadn't heard of this either, but they have an interesting reason for the name:

> The command-line tool is called jj for now because it's easy to type and easy to replace (rare in English). The project is called "Jujutsu" because it matches "jj".

https://github.com/jj-vcs/jj

Jujutsu, another VCS. It is popular enough to not require introduction on HN, or so the parent must have thought.

>It is popular enough

Define popular. This is the first time I'm hearing about it.

>not require introduction on HN

It's not necessary to explain acronyms, but gawd damn, I wish these products had names that were easily googleable.

Based on "or so the parent must have thought", I don't think they were being fully serious when they said it was "popular enough". I've seen it brought up pretty much in every conversation around git on Hacker News for the past year or so, and I did make an assumption that most people reading several comments deep into a debate about how important it is to memorize the exact incantations of git commands would be familiar with it. It's largely not relevant to my overall point though; you could replace it with "a different git tool" and it would not really change what I was saying.

https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que...

I'm not deep in the VCS discourse but I've seen jj come up many times on HN. I don't think it need an introduction, but it's also think it's reasonable to request clarification because I agree the name is confusing.

Tangent, but when I was younger, my main programming language was Processing, and I bet you can imagine how difficult it was to google issues with it :)

>I wish these products had names that were easily googleable.

They... are?

https://www.google.com/search?aep=11&udm=50&q=should+you+swi...

You can't google for "X" like it's early stack overflow days, but adding the minimum context "X that has something to do with Y" works 99% of the time.

Simpler: https://duckduckgo.com/?q=git+vs+jj

> but like, if someone doesn't know how to manually configure their git remotes and what exact command to use to amend/squash their commits into the shape needed and instead has an agent prepare all that and use the GitHub CLI to open the PR, why should I care?

Because LLMs aren't a reason to not learn your tools. A determinist CLI command vs using a statistical machine for it. It's just a waste of tokens at the very least.

No, the reason not to learn your tools is if they're not your tools anymore, and something else is. I don't spend time trying to manually invoke filesystem APIs to create files in my work projects and instead will often just have my LSP do it when I declare a module and the file isn't found. I could memorize the exact sequence of system calls needed, but my employer would prefer I spend time on things that are more worthwhile to them.

I don't disagree that having a deterministic CLI is useful, but I'd argue that's not the question here, because the CLI is what the statistical machine will be using. The question is whether the time spent doing it manually is worthwhile compared to the tokens spent on it. I'd further argue that if you instead learn how to pick the right model for the right job so that you don't use something way more expensive than needed for the task at hand, the token overhead will be extremely minimal for stuff like git operations; knowing that you can use something cheaper than like, Astra or Opus if all you want to do is a quick git cleanup and push is going to save your employer a lot more money in the long run than if you insist on doing every messy rebase by hand.

The point of knowing your tools is to stop escalating to other people every time something doesn't work with them.

Even if an LLM can hold your hand through everything, the disruption I've had because people don't understand what the LLM tool is doing, and ultimately does damage (we've had repos corrupted, systems taken offline, systems brought their knees, numerous data security incidents... it's been a fucking nightmare) is immense.

Repos corrupted? From arbitrary usage of a git client? I feel like it's been generally a bad idea to allow arbitrary force pushes for anything shared for a while before LLMs entered the picture.

Automation bias!

Even worse, it gave a developer a command to run that basically crushed all history across all branches, it was a mess to clean up and lost a significant amount of work. It wasn't Claude running it on its own, but was entirely driven by a mix of a developer's lack of understanding their tools, and trusting Claude's output too much.

This was also the dev trying to "remove a key he had accidentally committed" again, just letting Claude take the wheel.

Of course, the new team _also_ really loved leaning on AI so they were all behind his plan to "clean up the repo" and not just reset the key and move on.

It was a nightmare.

While using LLMs is a great idea, it has become crucial to put safeguards in place to ensure that their use doesn't deviate from team practices or compromise security.

I'd argue that these are not really specific to git (or even LLMs honestly); in terms of security, don't put company data into anything that your company hasn't given you explicit permission to put it into.

In terms of team practices, that basically seems like a code review thing. If my team has conventions around git, and a PR doesn't fit them, that's something I would point out regardless of how the PR was generated. If a team member is repeatedly not meeting the expectations we've all agreed upon, I would deal with it in the same way I always have; reach out and offer guidance for how to do things the way they're supposed to, notify my manager if they continue having trouble with it, and eventually just not bother looking at their PRs if they absolutely refuse to do things the way everyone else on the team has agreed they should be done.

The idea of saying the LLM is "my tool" as opposed to all the actual tools it calls, is about as appealing to me as saying the same about, say, Remote Desktop. "The statistical machine" is there to generate candidate actions, not to implement them.

In the before time, this is the sort of thing people would just google.

One time setup commands are the sort of thing people do once when setting up the repo, but dont use day to day. its understandable that people dont have that memorized and will reach for a man page, google or an AI.

I happen to have memorised how to add new remotes a long time ago, but I would 100% not care if someone I worked with googled it every time. That's how it was for me for several years. I don't think that has anything to do with "learning the tools". And getting claude to do it is not that much different from copy-pasting from google.

TBH I usually just search my shell history for the last time I added a remote and edit it.

It's also not particularly hard to write a shell function to avoid needing to care about the syntax, e.g.:

    add_github_remote() {
        git remote add $1 git@github.com:$2/$3.git
    }

Better yet, assign the positional arguments to named variables for documentation purposes; you can see them when you `type` the function. (Comments get stripped, unless you make an actual script.)

thats just the same problem though, ig you arent using it regularly you are going to forget that part of your shell functions

1. Develop the instinct of "I probably have a shell function for that, if it isn't already obvious; and if I don't, I need to fix that"

2. Give your shell functions a common prefix, and/or big-endian naming convention (e.g. things that help you with git start with `@git-`)

3. Tab completion is your friend.

I wholeheartedly agree that it's the same, but I fundamentally disagree about it being a "problem", or at least, one worth caring about. I don't honestly believe that typing out the extra 35 characters that are filled in by the helper function I gave here would solve anything that comes close to the convenience of not needing to memorize them.

> It's just a waste of tokens at the very least.

Some 6-figure per year developer spending a quarter’s worth of tokens to do something they do once a month or so is not something I’ll ever care to optimize away…

I misread this as a "quarter" as in "three month period of the calendar year" and was very confused until I remembered that was also the name of a coin

Tokens are cheaper than brain power. Why waste my time trying to recall the details of git's byzantine, ill-begotten command line interface when I can delegate that boring task to a robot?

what happens when there's a massive merge conflict that requires human review?

Git is an essential tool; not knowing how a key tool works is dangerous IMO

I think that actual direct knowledge about how git merges things and what ends up surfacing as "conflicts" as a result of that is not particularly well understood by quite a large number of people who in practice do not have any trouble using git every day. Moreover, there's nothing specific to git in terms of reviewing code, which is fundamentally what you get from the result of fixing conflicts; once the conflicts are all resolved, there's a diff from the old state (pre-merge) to the new state, and you already should be reviewing it to see if it's what you want even if there were never any conflicts.

LLMs solves them great. So who cares? At most it will ask for a few guidelines on what takes priority, but no one is using git directly anymore.

I have not run a single git command in like the entire 2026.

Yeah, I feel like knowing how to use tools is one of the key smells for finding good engineers.

Nothing.

Should you resolve the conflicts in a bad way we're going to see the result in the MR, will we not?

I don't think git knowledge is essential anymore.

Back in the days of the crappy git GUIs this was a problem, because people would regularly misunderstand even the basics of what they were doing and managed to lose their local changes.

That shouldn't happen with LLMs.

just ask the agent "no mistakes" /s

Omzsh user checking in, I always stumble a little in interviews about this because they usually have a default git configured and commands like gco -b and gp don’t work, and I have to think about it for a second.

Because someone who's familiar with those sorts of things can get the commands typed and running before someone who isn't has even figured out the prompt.

(Well, not so much if there's a need to identify the base commit for a squash, do some cherry-picking, etc. But in those cases I'd rather see direct human judgment of what to do than hoping the LLM understands a vague description.)

That's pretty much where I'm at, so long as they're not filling the code with "///CLAUDE ADDITION" type guards (ie noise).

Personally I ask the LLM to summarise what the commit does and explain it to me, make sure I understand it, then review the PR. I'm getting half the speed the High Speed Sloperators are getting but I was the only person submitting PRs the other day when the LLM service was down.

For nasty multi-way git merges and rebases, it's been a game changer. And of course that's reviewable by doing a delta on the final rebase branch vs. the original branch.

With all that said... having to use LLMs at work is making me want to use them less on my own projects.

> Personally I ask the LLM to summarise what the commit does and explain it to me, make sure I understand it, then review the PR

For the more important changes, I find it easier to stay on my toes by taking a pass at the codebase before comparing notes with the LLM. Peeking at the answer key first makes it too easy to nod uncritically.

Oh, I was not even attempting to discuss using them for the code itself in my above comment. Code that's checked into a repo has a much more complex set of tradeoffs that I'm much less convinced have a one-size-fits-all answer. My point was more that even if reasonable people can disagree on how much you need to care about the code itself being LLM generated, I can't wrap my head around why anyone should care whether an LLM happened to execute the `git commit` and `git push` shell commands!

I had a coworker a few years ago who was very stuck up and the first place I saw it was his opinions how people should use git. I use the terminal for some stuff and the gui for others, but he was weirdly judgemental about doing anything other than full terminal.

> (we also ask the candidate to open a PR against the repo just to see if they know the basics of git -- you would be surprise how many people cannot do it).

This doesn't really say anything about someone's familiarity with git, considering pull requests are not a feature of git.

When I worked at a FAANG we used git but had our own internal code forge that was nothing like GitHub.

If you're on the repos git page, it's a total of three clicks. It doesn't really require prior knowledge.

Also, AI can do that well now too. I haven’t had to deal with merge conflicts for months.

  we also ask the candidate to open a PR against the repo just to see if they know the basics of git

That's the basics of Github, not git.

A candidate may be able to open a PR and not be able to make a commit without an IDE.

Another candidate might know plenty of git, but haven't worked with Github.

But, like... wouldn't you want them to at least know the steps of `git add` -> `git commit` -> `git push` and then click the button on Github? And Gitlab has basically the same web UI.

I have no problem with letting an IDE or LLM handle those details in practice... but I don't think I'd want to work with someone who doesn't understand the very basics of what's going on under the hood.

> wouldn't you want them to at least know the steps of `git add` -> `git commit` -> `git push`

The GP said "open a PR", not "make a commit and push it to a remote".

Well, I can't read the GP's mind, but I'm fairly certain they didn't literally mean "candidate knows how to click one button in the web UI for a specific proprietary developer platform and type some stuff into a text box".

When I tell my LLM to "open a PR for those changes" (which is how I do it nowadays), my LLM understands that what I mean is "make a commit, push it to a remote and raise a PR".

Does it mean we've reached AGI when machines understand what we mean better than some humans?

> When I tell my LLM to "open a PR for those changes" (which is how I do it nowadays), my LLM understands that what I mean is "make a commit, push it to a remote and raise a PR".

This makes it seem like all of those things are part of a single operation to you. That doesn't make sense to me.

On the occasions where I've submitted a PR to a project (anywhere, not just on Github), making the commit was not part of the same operation. I'd already made the commit and satisfied myself that it captured everything I wanted, before making the PR. I don't see those things as part of a single operation. I don't think the poster you were responding to does either.

  > That doesn't make sense to me.

So you think the OP is asking candidates a question, where the solution is:

1. Navigate your browser to GitHub.mycompany.example.com

2. Click the repo name

3. Click pull requests

4. Click the source branch[1] and destination branch

5. Click "create pull request"

???

I started out by assuming this was some kind of good-faith disagreement, but I guess you're just trying to be funny.

[1] Presumably already extant, because they never mentioned anything about committing code, nor pushing it to github (and that's not the same operation!).

> So you think

The OP said:

"we also ask the candidate to open a PR"

So I think the OP meant to have the candidate, um, open a PR. Which is not the same thing as making a commit and pushing it to a remote. Maybe you always conflate those operations, but that doesn't make them the same operation. If the OP actually meant that they ask the candidate to make a commit, push it to the remote, and then open a PR, I would have expected the OP to say, um, all of those things, instead of just the last one.

I couldn't say how the OP expected the candidate to open a PR, since I don't know what their standard workflow is. There are ways of doing it from the CLI, as well as through the web interface, as well as through various IDEs.

> I started out by assuming this was some kind of good-faith disagreement, but I guess you're just trying to be funny.

Please save your patronizing for another time. And stop trying to put words in my mouth.

Years ago I was the “git expert” on my team, the person you went to when you got into trouble. But I have long since moved to jj, and I basically don’t remember git commands anymore. I also don’t use GitHub much, so when I need to open a PR I prompt an LLM to do it for me.

Why should anyone care? This is something that can be taught and learned in 2 minutes.

> might know plenty of git, but haven't worked with Github

That seems very unlikely. Github is the entire reason git is so popular, vs. any of the other distributed VC systems.

Edit: I'll refine that, after a moment of thought. Git is popular because of Github. But, I guess that now maybe there are people who are using git because it's the popular/default version control choice, but they might not be using Github. They might be using another forge, something homegrown, or just manual builds.

You got dinged but it's a learning opportunity:

Github was like Yahoo's directory, a way to share content in a searchable place and as a backup/bounce point. In the early days you could also upload to 'downloads' so it was a quick share host. Before that we had email, exposed ssh and cgit/gitweb but Github gave us `PR` to promote social interactions with forking. So it could be argued that Github tainted the spirit of git.

Having personally evolved from making date stamped zip files, to subversion and then to git, it earned its place quite quickly for myself and my roaming nature, didn't hurt who hacked it together either. When Linux adopted it from Bitkeeper, it became the way for many.

It seems rather likely for people who use git at work only on GitLab or (god beware) Bitbucket.

Personally I prefer GitLab over GitHub. The CI is better, and it doesn't go down every other week unless you self host on Azure.

Git is a simple tool. I expect any engineer can learn to use it. While git is common, I would not expect an engineer who worked at a company using a different version control system to know how to use git (which is a really awful - I still miss mercurial). I expect you to know the general concept of version control, but the specific on is a tool I expect you can learn quickly.

There is nothing simple about git. 'ls' is a simple tool. git is probably the most complicated console command (in widespread use) I have ever used, although I'm open to other examples

Agreed, the amount of stuff checkout can do is crazy. And the concepts of hard and soft resets mixed with garbage collection make deletion non trivial to understand. Or using interactive rebase to variously remove, squash, and rewrite commits.

Fundamentally git is conceptually complex in a way that almost no other tool is. Others might have scarier syntax to newcomers but rarely expose the same level of underlying complexity.

Version control is simple for the typical user. Git is a terrible tool though and doesn't properly abstract the complexity that should be hidden away from users. Even still, the typical user only needs to know how to clone, stage, commit, push, and merge (I'd like to add branch, but git doesn't have real branches, so I guess tag which should be an advanced topic nobody needs)

not for 95% of what people will use it for IMO

initializing a repo, staging files, adding them to the changeset, commiting then pushing those changes are easy to do. solving merge conflicts is a little complicated but IDEs make that much easier to do.

It is not simple like a hammer, but imo the bar for being a software engineer is being intelligent enough to be able to understand it well enough to do things with it, since it is a tool of the trade.

jj is far simpler and more powerful at the same time. There is absolutely no reason anyone should waste time learning git in 2026

reason: not use a tool whose name collides with million things

Not a reason to use git. Git has been a common word since long before the git version control system was created: https://en.wiktionary.org/wiki/git

I run a live coding challenge on a rather benign coding problem with the explicit expectation that candidates will use an LLM.

Interestingly, an LLM involved feels like it’s made it easier to assess candidates. Senior candidates know exactly what they’re doing and strongly driving the LLM. They pre-communicating design decisions, gotchas, and requirements. They also ask a lot of clarifying questions and dial in the scope (after all not building something is better than building it).

Less experienced candidates tend to jump right in, build way too much, and stumble because they delegated all of the critical thinking to the agent. They often end up in a place where the agent doesn’t align with their vision and they can’t really reign it back in.

> we also ask the candidate to open a PR against the repo just to see if they know the basics of git

I've heard this multiple times now and it still boggles my mind that git is something you have to test for.

At one job I had, we used git extensively, but not GitHub. We had internal systems for review and branch management. I could tell you all the gory details about various workflows and landmines in git. Is the test here really about git, or git+GitHub?

I rarely use anything beyond checkout, clone, pull, commit, cherry-pick, revert, and push in a job. If I have to, I'm probably asking the AI what to do by giving it my situation. And hopefully the AI can read about whatever git standards the company would like to be enforced so the engineer doesn't have to interrupt someone for that.

This is more about opportunity cost because instead of asking about git, you could ask about other valuable skills in your limited time with the interviewee, but I guess if git is really that important, then it's justified.

Well, I've been using git at work for 6 or 7 years, but I can't out of hand open a PR; at work we very rarely use different branches and the last time I touched Github I was a student.

Do you mean specifically Github, or any code forge at all? When you say "we rarely use different branches", what do you mean - you're all pushing to master, or you have branches for each team or something? I would regard "code forge" and "separate branch per reviewable unit of work" as extremely normal, nearly universal.

Yes, we're all pushing in the same branch (not called master, we have one branch per version (usually we have one branch that's the current version deployed in production and the development branch)). To group up commits for review, we add in the commit message the Jira issue number and Jira will keep a list of all commits through Bitbucket. Bitbucket supports PRs, but I don't think it's ever been used here.

I can gather that the way we're using is not the most usual ;).

it's incredibly sad but it's real -- so many senior candidates have problems with commit messages, opening PRs.

not even joking: i've seen people with like 10y of experience trying to push to main :(

Dave Farley (of the 2010 "Continuous Delivery" book fame) explicitly recommends committing and pushing directly to main (or master), and he actively argues against using feature branches and blocking pull requests.

https://www.davefarley.net/?p=247

What's incredible is how set some people are in their ways that they can't conceive of alternatives.

PRs are not only not a basic feature of git - it's not a git feature at all. Though button clicking on GitHub is anyway a crazy thing to test

> (we also ask the candidate to open a PR against the repo just to see if they know the basics of git -- you would be surprise how many people cannot do it).

How does that work? Git has no concept of pull requests.

Extremely funny for OP to advertise this gotcha style question that's built on a misunderstanding of the tools

Doesn't surprise me at all that many developers can't do this, most professional software development isn't happening on Gitub.

If you are asking any human to write code, you are effectively asking them to write byte code. Humans should not be writing code in this era, and only in rare circumstances should humans be reading code.

What I though of was to give candidates a not-so-smart LLM, one that is almost guaranteed to make mistakes.

The idea is to see how the candidates deals with it. Maybe he will just say "fuck it" and do it by hand, or patiently point out every mistake so that the LLM gets it right. Both indicate competence, what you don't want is someone who doesn't see the mistakes and hand you slop.

> (we also ask the candidate to open a PR against the repo just to see if they know the basics of git -- you would be surprise how many people cannot do it).

Worked with a "super senior", lead architect. Management was raving about the guy before he joined. Insane resume. Skipped the regular interview process because of this.

Had to show him how to open a PR and setup his git config so his commits are linked to his account. And then his PR's failed the automated ci checks for descriptions and titles, and he came to me to ask how to fix it. The failing gh action literally says what was broken.

This is all with Claude available btw, so one could literally just ask.

I guess it's a bit of a modern fizzbuzz. Where it's so simple but still manages to stump a lot of candidates.

    > we also ask the candidate to open a PR against the repo just to see if they know the basics of git -- you would be surprise how many people cannot do it

This part seems weird. "[B]asics of git"? You can Google for that and find a fossilized answer on StackOverflow. Do you really have candidates who cannot do git commit/push? Also, if you allow LLMs, then any idiot LLM will also know git basics.

Funny that about 80% of the replies to you comment are about your git sidebar. Bikeshedding indeed

How would you prepare for the coding problem as a senior? What are the assumptions that underpin this?

Not GP.

The trick is to get a good sense of the algorithms associated with basic data structures (list, stack, queue, graph, hashmap,...). They are primitives and almost any standard problem can be solved with those.

What you need to train most is the capability to break down problems: Like I have this input (and associated metadata), I have this context (may be null), I have those constraints (values, ranges, memory space, time, both, or neither), and I need these outputs. Then it's just a puzzle.

I no longer prepare for coding problems other than a quick refresh of the syntax of the main language. And if it's a whiteboard type of things, I go with pseudocode.

honestly, for a senior, the coding challenge is just a checkbox -- because they can use llms, the main thing is making sure whatever code they accept, they can explain from top to bottom.

the coding challenge itself is just a way to make sure that people can understand a request within an existing codebase. as an example, i pay attention to the questions the candidate asks -- questions asked always mean something.

my general assumption with seniors are "they can think high level", which means i "mark them" with that assumption, so for me, the architecture talk is much more important.

at the end of the day, you need to understand what your org needs and adapt.

how do they grade a right answer, suggested by the agent itself??

my suggestion is one should ask the agent to go through series of research, analyse its own proposed solutions/fixes, then a human verify the plan, refine it further, instead of relying blindly on the agent. That's how I do coding with agents.

Something similar for senior devs: code reviews. Reading and understanding code has always been important. I find offering simple code reviews very telling. In a way, your LLM approach matches that. It's a way to do a code review. It's a test of understanding, and not just memorization.

> we also ask the candidate to open a PR against the repo just to see if they know the basics of git -- you would be surprise how many people cannot do it

Will you allow jj instead. I switched to jj 1.5 years ago (as many others have), and my git is now rusty.

I mean sure, I can do add, commit, basic diff, etc. But rebase/merge - oh dear, I forget which one is source/target (jj made the infinite design choice that the ordering doesn't matter - you have two parents making a child - it's symmetric).

>>> the bar is super raised and now you need to explain in details everything the LLM wrote as if you wrote it yourself.

You have missed the point entirely.

Reply on news.ycombinator.com