garyjlin

This is what we saw too when we were hiring for our own engineers 2 months ago. A surprisingly good filter is "can you even get started?". Like, the fact that our interviews required users to navigate Github, setup a dev env (even if it's just the make command), understand the existing code and prompt alone filtered out a non-negligible percentage of the candidate pool.

Completely agree! This is indeed a big problem, and we have so many CTOs, VPs and founders willing to jump into 6 figure contracts if it works. The biggest risk here is execution, not market size.

Right, biggest pain point we're trying to solve for is engineers getting pulled away from their work to build, maintain, conduct, and evaluate interviews. I recently spoke to Stripe engineers that spend 2 hours a day, every day, on this. A Brex EM also every engineer on the team conducts 1 - 2 onsite interviews every week. Eng time is the biggest expense in the recruiting funnel, and it's particularly difficult to solve for that problem. We're hoping that if Litebulb can take care of some of the more tedious code analysis work, then we can help teams reduce their eng spend, or use the newly available time to build deeper connections with candidates as humans.

When it comes to changing engineering culture, I completely agree that a single tool like this isn't going to cut it, but we're seeing a big shift in new startups focusing a lot more on real work relevant take homes and/or work trials. For example, Gumroad (disclaimer: a client), does a Litebulb interview as the entrypoint, then it's straight to a 4-8 week paid trial on flexible hours with a choice of which project to work on. We're hoping that we can be a part of, and accelerate, this culture shift.

Thank you, looking forward to chatting!

Appreciate it, thank you!

Honestly I feel you, I recall interviewing before and the number one thing I kept asking for was feedback, but rarely ever got any. Currently, when a candidate does a Litebulb interview and submits it, we email the same report PDF to the candidate that the employer gets. As in, feedback is built into the flow of the product.

The next step would be to open up that candidate-facing side, and hearing that you'd pay for just that alone is great feedback!

2 thoughts here:

1) This is actually why I was going back and forth on offering a trial version, and decided to use a form so that I can qualify employers who legitimately need this vs candidates just trying to game the system

2) It might not actually matter if candidates get preliminary access to Litebulb interview code bases. Our interviews aren't brainteasers that have a specific optimal solution, they're just regular feature requests, ie build a CRUD API for this new resource model, so it almost doesn't matter how long you have. From what we've seen with submissions so far, having extra time doesn't necessarily make your code quality better. Potential idea: open source all Litebulb interview codebases?

Thank you! And yeah we've had that request a lot, later this year we'd like to open a candidate-facing platform to hone their skills on Litebulb interviews. Conceptually, getting better at Litebulb interviews should just make you a better dev, not better at interviewing.

Potential feature: get high enough scores on a specific interview, get recommended to companies currently hiring that use this stack and get inserted mid-way through the funnel?

I've thought about this a lot too, which is why we heavily recommend using Litebulb for take-homes. Sit-down onsites on Litebulb are only good if the candidate knows the existing frameworks, if they don't it's pretty tough to figure it out on the spot.

Take homes are great because the candidate either knows the specific frameworks, or doesn't but can learn them quick enough. Either way, if you complete it within the allocated time and get relatively high enough scores, we'd recommend moving forward.

Actually @xiphias2 what type of interviews have you seen that were really good? Like even the in-person ones, what made them good?

Appreciate it tons! And yes, making this experience feel "native" and familiar is one of our primary focal points. In the context of an interview, any slight deviation of an expected form factor can contribute to nervousness or loss of candidate performance, which is why we're trying to make sure that dev env spin up is quick (eventually <60s), coding environment is familiar (VSCode or JetBrains), and submissions are familiar (probably just Git). For employers, we still have lots of work to do to ensure the signals collected integrate well with popular ATS's like Greenhouse or Lever.