Rethinking Triplebyte
triplebyte.com
triplebyte.com
this post is marketing with commenters associated with the company
That said, I think a hiring process that puts engineers in control (no ghosting, get data on when a recruiter looks at your application, search ranking by whether companies lie to candidates) is something that should exist. I want to try to build it!
The trick here is you need to figure out a business model where you don't take money from employers, because once you do its just the long slide to becoming a shittier LinkedIn.
I personally like this proposal and am probably going to at least draft a hypothetical spec of it and see how it fleshes out. Thanks for the suggestion!
Pretty obvious value-add for job searchers. Not to mention that having access to this data would enable tons of other product features. Shows that most services/sites don't care that much about the applicant experience.
Colorado now requires job postings to include something more or less like a "reasonable salary range". It hasn't been perfect, for example there are some companies who now just restrict their online job postings to say "except Colorado" instead of adding a salary.
But as someone living in Colorado it's been nice to know the salary range up-front more often and to have confidence that I can ask them for a salary range without getting the question turned back on me: "Well, what are you looking for?"
(c) An employer, upon reasonable request, shall provide the pay scale for a position to an applicant applying for employment.
[1] https://leginfo.legislature.ca.gov/faces/billNavClient.xhtml...
So it seems it’s not enforced.
If you have a janitor in CO, every job is an improvement and potential promotion. Having to first go to that janitor for every position is going to wear out FAST. Posting for the CTO position? Go to janitor first. Posting for CEO position - same thing - that's what is going to drive folks away.
Unfortunately it looks like companies can just state wide ranges (which don't help applicants understand the market any better) and they can also omit other compensation elements beyond the base salary component (stocks, cash bonuses). That makes the disclosed information not very useful.
The biggest strength for me is that I can ask them and they have to tell me. When I was job searching before I'd ask the recruiter and then they'd always ask me what my target is. Not showing your cards is like negotiation 101. Plus it's made fielding messages and emails easier. If they provide a range, I can just easily say "This position sounds interesting but unfortunately the salary range isn't what I'm looking for currently."
You're right that a lot of positions aren't great about this, I've literally seen "Pay starting at $0.01" or ranges literally $30k to +$200k. But I've seen it as a benefit in the negotiation process and it's just been easier to field recruiters.
> total comp
This law only covers salary as far as I know, so equity and such aren't included. Total comp is different.
It's not perfect, but I feel like it's wildly helpful for a ton of people. It helps transparency for a lot of positions and I feel like it rewards honest companies. If equity and bonuses are a big part I think it reduces the overhead for talking about salary to clear the way to talk more about equity and bonuses.
It's also in the early stages and CO is the only state with a law that requires disclosure up-front to my knowledge (someone mentioned CA has a law, but they only have to answer if asked). It feels like a fantastic first-step to me.
This is a space that unions fill well compared to for-profit recruiting agencies. The organizations give engineers control and leverage in hiring, and can help them up-negotiate. They also allow employers to hire from pools of skilled talent.
Whereas with recruiting agencies, the customers are employers, not the candidates. Incentives are not aligned to give candidates control.
People at the recruiting agency usually get paid a commission as a percentage of the first year of the hiree’s salary (not taken from your paycheck, but just in general). Which means, it is in their interest to get you as high of a comp as possible. Which also means that I dont have to bs around with the recruiter about my salary expectations or anything like that. I give them the upper range of what i want, they tell me either “sounds good” or “they are easily willing to pay more, so why not try $X instead” or “they cannot pay that much, but there is this other company that we work with that can offer you as much.”
And if you work with one recruiter consistently, it gets even easier, because now that the recruiter knows my total comp expectations, she pre-filters opportunities for me and only contacts me if there is a position she is aware of that pays at least as much as I want or more.
Of course there are downsides too, for example, the limited choice of companies that the recruiting agency is working with at any given time, so i still have to find some opportunities and interview with those companies on my own. But as a supplementary option, working with a good recruiting agency has been quite nice for me.
P.S. The type of a recruiting agency i am talking about is the one that is focused on a specific sector (in my case, fintech/quant), so it might be a completely different story for a “generic software dev” recruiting agency.
Beware: this is not universally true. What you typically see is that it leads to closing as many candidates as possible. A recruiter can make more by closing 2 candidates at 80% of max comp than 1 candidate at 100% in the same time.
This is a well known behavior in real estate sales, which have a similar commission structure.
Even worse, real estate pays both sides so realtors are heavily incentivized to get their buyers to purchase from their own sellers so they can get both sides of the commission. Instead of paying them 3% to act on your behalf and do a bunch of leg work they get double and now they can do less work and aren't strictly working for your benefit.
Working on commission just leads to wanting to make a deal happen. Even ignoring the time aspect of it, if a recruiter has much better odds of making a placement at 80% max comp compared to 100% max comp, the 80% is still objectively better. Why risk not closing the deal when the expected return is negative?
I am saying that because I’ve been working with her for at least 3 years now, and I am yet to take a single offer from the companies she set me up with (either due to them getting cold feet last second or simply due to me not passing some of those interviews or me declining due to some red flags with the companies). If it was all just about high volume for her, it would have been in her interest to drop involvement with me after the 3rd unsuccessful attempt to set me up with a fintech company.
Collective bargaining is inherently about eliminating competition between workers. That is, it creates a labour monopoly to balance out a labour monopsony. That might (might!) be a good thing for autoworkers. It would be an unmitigated disaster for software engineers, and the tech sector in general.
People who write code are unbelievably productive compared to just about any other job function, and software engineers (who we'll define as people who write code for tech companies) are even more so. Part of eliminating competition between workers is making sure that nobody works harder than anyone else. That would be devastating to the productivity of software engineers, which would make them less valuable to tech companies. That would lead to a decline in salaries, and general stagnation of tech.
Another aspect of preventing competition is seniority. One of the archetypes of the tech world is the wunderkind—the brilliant kid fresh out of school that rockets to the top. A union would tell that kid, "sorry, you'll have to pay your dues as a junior engineer for 5 more years, then you'll be eligible for promotion. In the meantime, we'll give that job to cwp, he's got the required seniority."
Just think how this would affect tech companies. Sorry, you can't pay more to attract the best. If you're really careful about interviewing, you might be able to find some people with both skill and seniority, but it'll be tough because smart, ambitious people have all gone to Wall St or whatever industry has the best opportunities.
A yet every movie star in the US is in the same union as B-movie and infomercial actors.
> Another aspect of preventing competition is seniority.
Weird, considering pro-athlete unions regularly help kids out of college quickly become multi-millionaires.
> One of the archetypes of the tech world is the wunderkind—the brilliant kid fresh out of school that rockets to the top.
Weird, considering unions have been at the forefront of education, apprenticeships and training that helps kids out of school rocket to the top, the same apprenticeships and training that companies refuse to fund themselves.
> Sorry, you can't pay more to attract the best
Weird, considering SAG-AFTRA regularly helps movie stars up-negotiate multi-million dollar contracts.
However, I dropped out completely because TripleByte wouldn't let me see companies without entering in a desired comp number. My recruiter at triplebyte said I could just put in "$1 or $1,000,000" to get past it, but that just feels like the same kind of corporate HR bullshit that people have complained about for years. Just Google "recruiter won't proceed without desired salary" and you'll understand how many folks dislike that question.
The experience made me feel that Triplebyte wasn't interested in putting me in control of the process at all if they have such a user hostile requirement and require me to work around it by putting in fake numbers. I understand that many people will have no problem providing this info, but I prefer to see what companies offer me because it tells me how they value me and the position they're hiring for.
Almost all recruiters have been fine with that, but a couple haven't. To such people I wish a good day and go my own way. There are plenty of other recruiters in the sea.
1 - I get real-world interview practice, and that always helps me in future interviews. Many companies ask the same or similar questions, and the more I'll be ready for them being asked again. In addition, when I'm surprised by a question I'll research it thoroughly when I get home and won't be surprised by that question again.
2 - I get a bit of an inside peek in to how the companies I interview at work. That's a privilege not afforded to many outsiders, and how some of these companies (some of which are world class) can and has helped me at other companies.
3 - A company might bring me in to interview for one position, but after the interview might determine I'm better suited for another. If I didn't interview with them they'd never know.
4 - As I interview with people I get a chance to make a good impression on them, so even if they don't wind up hiring me for that position they might want me when a different position opens up (whether at that company or another company the people I interviewed with moved to).
5 - By letting companies make the first offer I myself get a good salary survey of the minimum what all the companies I interview with are willing to pay for someone with my qualifications in that position. Of course I try to do my own research ahead of time, but getting salary numbers without getting a job offer is not always possible or reliable.
6 - If I blurt out a number first I'll it may put me at a disadvantage in the negotiation, as they'll just start the negotiation from what I say even if they would have been happy to have paid way more.
I hear this a lot, but how much interview practice do you really need? After the third or fourth set of interviews, they start to really blur together for me.
None of your points are false, but if you're casting a wide net and taking every interview without any idea if the company can even afford you, that's a lot of hours spent. The last time I looked for a job, I interviewed at only four companies and spent an average of probably 5-6 hours on each, and it was exhausting. I can't imagine making a habit out of it while also working full-time.
But, in my case, doing many interviews helped me understand what negative signals I am emitting during interviews [according to the interviewers]. There's nothing like an outsider's perspective, even if you disagree with it.
---
Example: in one of my last interviews I was told that I am too "cocky" and "over-confident" because when I was given real-time modifications to a homework assignment (that I completed before) I was smiling and saying "oh, that's easy!" and proceeded to make the changes live in my editor -- while sharing screen.
This feedback left a very WTF feeling in me. Dude, I am (a) enthusiastic about the task and (b) have enough experience to show you, in real time, that I can modify the code right there and then to suit the new requirements, and (c) chatting with the interviewing team while doing it... and all you can gleam from that is that I am overconfident and cocky? Seriously, get a grip.
For the record, I strongly disagree with their take. But it gave me a very interesting perspective -- namely to scan for the more introverted / likely-insecure people in the interviewing dev team and try and act a little more modestly to not rub them the wrong way.
It's a fact of life that most people suck at interviewing -- as I hope that I demonstrated here. One of the guys in that team simply dislikes certain kinds of people and he lets this go against his better professional judgement about my abilities. Not cool, right? Especially when you also take into account that the CTO who also attended all interviews agreed that my tech acumen is top-notch and they seriously couldn't catch me with my pants down no matter what they did.
So again, this helps me gather experience and culture and makes me more adaptive in the future.
Does it hurt your ego? Absolutely. Does it shake your preconceived notions? Much more than one would think beforehand. Is it exhausting? Hell yes, sometimes after an interview I didn't want to do another one for a week.
But, is it also very valuable to help you improve your interviewing and negotiation skills? Yes!
That's why I did lot of interviews the last time around. And I'd do it again.
The nice thing about the explosion of remote work due to COVID is that engineering salaries are beginning to normalize across the US, and there is now a lot more salary data available online, so it's easier than ever to see what you're "worth" on the market on average. This at least gives you an anchor when having these conversations, so you're not going in blind, and gives you a lot more leverage if they low-ball you.
[0]: Yes, I'm aware of the drawbacks of self-reported data, but it's still broadly useful in aggregate.
[1]: I'm making up numbers, but something like $120k - $200k, with the lower number being the minimum you'd realistically take. Usually recruiters are honest up front if they can't even afford your absolute minimum, but it still leaves you with a lot of negotiation room once the offer comes in.
Why even give an upper limit? It's not like you'd refuse $400k if it was offered to you, would you?
Most companies will naturally start negotiation near the lower end of your range, even if they would have started way higher had you let them make the offer first. I really don't see an upside to this strategy.
Because if you just give one number, even if that's your absolute minimum, the entire salary conversation is going to be anchored around that one number, and you're going to have a harder time asking for 50+% more than that. If you give a broad range, it's easier to justify countering with something closer to the top of that range.
> It's not like you'd refuse $400k if it was offered to you, would you?
Of course not, but nobody has ever offered me that. :) In all seriousness, yes, you run the risk of your range coming in below what the company would be willing to pay, but in my experience (and those of friends and acquaintances), outside of FAANG, I have found that to be extremely rare if your range is wide enough. The companies that pay super well are generally big, and thus well-known to pay that much, and I wouldn't give them any salary numbers at all because I'll have a pretty good idea of what they'll pay me with a bit of research prior to interviewing.
Of course, anecdote != data, so YMMV and all that. This is just my experience interviewing at mostly smaller companies.
Agreed though in practice that just moves the pre-interview conversation from a comp discussion to a leveling discussion.
Luckily levels.fyi has fairly accurate data for those same companies.
That time would not be "wasted" if the company had done due diligence on what the position and candidate were "worth" and revealed their target (or target range) to the candidate.
The real reason, I'm sure, is that companies think they hold the upper hand in the negotiation. And a lot of time with a lot of candidates that's true. It's clearly _not_ true for the class of candidate who's realistically got a $300k+ target. You are totally not going to successfully headhunt anyone in the top 1% or 5% of FAANG engineers if you try to dick them around with recruiting power imbalance power plays. _They_ don't need your bullshit. They will totally walk away from whatever time you've "wasted" so far trying to recruit them. You need them _way_ more than they need you.
(And I'm pretty sure that generalises a lot further down the experience/renumeration food chain than most devs realise. Fresh grads or devs with only a few years of experience might need to play the stupid recruiter games, but most companies trying to fill a mid level or senior role need the right candidates way more than those candidates need any one specific job. You can and should be able and willing to walk away from any potential job offer over stupid games like this...)
I used to be much more relaxed about this, but if you get burned doing a few rounds of interviews and it was a waste you wise up pretty quickly - teams do not like time wasted.
I've also found that folks who can't / won't talk money - usually not great hires for a bunch of reasons.
We're happy to talk through the different positions, levels, progressions and comp ranges for them all. But you have to have a clue about what type of comp would work for you (early on in initial call or two) or it's just not worth continuing forward.
You say "I've also found that folks who can't / won't talk money - usually not great hires for a bunch of reasons."
The same goes in my experience for companies which insist on talking salary before the job offer. It's a red flag.
"you have to have a clue about what type of comp would work for you (early on in initial call or two) or it's just not worth continuing forward"
Of course I have a clue, but I'm not going to put myself at a disadvantage in the negotiation by revealing it before I've been given a firm job offer and heard a number from the company first.
Also, in my experience recruiters are the ones who push hardest to hear salary expectations up front. They know it'll give them a leg up in negotiations, and some of them are even paid specifically to get that number (as one recruiter was brazen enough to reveal to me).
In this case, obviously a job seeker does have many possible employers to apply to.
So no, AFAICS you are wrong and the original thesis of "the company holds the advantage" is correct.
___
[1]: Consider the connotations of those words.
If you're happy to provide a range, great, if I'm still there doesn't that generally indicate that the range is in line with my expectations?
I guess your comment kind of makes sense, I can see how corporate might interpret 'this person looks out for themselves' as a potentially bad hire.
That’s because it benefits you as an employer.
I’ve been on the hiring side and I know there’s a budget. If someone is too high you want to push them out right away.
My flatmate in Singapore about 6 years ago was looking for a new job. Her salary was about 2200/m and she said the recruiter kept asking about salary.
I told her not to discuss it. She doesn’t /need/ a new job. If the company really values her then they will give her an offer. They have a budget. They offered her around 4600/m.
If she told them she was on 2200 it’s guaranteed they would have given her only a small bump.
Companies should never ask for salary info. Because it’s only used to get cheap labour.
Your clients know their budgets. Why not just be upfront about that to the candidates. Tell us your budget or reasonable expected range your client can pay. Then if (when) we find that it's well below what some other hungry companies are paying, we can save you and us time by passing.
If you know your client is paying low, then just be open about it. Something like, "Ok, this client is a bit below market (and tell the upper limit they will pay), but they have these good things to offer." If the client has no good things to offer, and the pay is low, then you're still likely to not seal the deal with a candidate no matter what. So just cut the game and get to the point.
This makes me think that all jobs position should have rather narrow target ranges which are stated up-front. Then you can discuss if you are too high level/low level for that job and range.
It seems to be terrible waste of time going through interview process to learn that they won't even come close to what you want. I would not do that to myself.
at senior levels especially, you should know what you want. comp bands are informed by huge data troves now and any edge you may have gotten before is gone due to the fixed and tight bands. just state your requirements and insist on “turnabout is fair play” ie they need to tell you the band. do this during the prescreen and save everyone’s time. if you kill it in the interview you still easily can negotiate to top of band.
If your desired comp number actually is $1M/year, will that just flag you as a joke to Triplebyte companies? With post-pandemic stock price increases a lot of engineers at FAANGs are making more than that now.
For any reasonable definition of "a lot", no they absolutely are not. A lot of people with 2+ years of experience who've either been promoted once and received a strong annual review, or switched jobs as a mid-level industry hire are making in the 200s and 300s, a few pushing into the 400s. Much more than that and you're looking at the top 1% (or less) of engineers at the top companies. Not a lot.
Source: was a mid level engineer at a FAANG until I left late last year, knew seniors and principals and their comp. Nobody was making $1M/year.
Nobody that makes 2x what their coworkers do at the same job/level casually discloses their comp at the water cooler.
A) your company is engaged in disturbing monopolism; B) your company is engaged in disturbing labor exploitation and arbitrage; or C) your company is engaged in disturbing environmental destruction.
Would love to be proven wrong. Am I?
Edit: option D) is that American cardiologists are not as exclusive and skilled a labor force as I would prefer to believe ;)
They’re all doing well, but even Amazon hasn’t doubled their stock since prepandemic.
There are definitely more people making 1m/yr at FAANGs than their were last year, but I agree with others that it’s not “a lot.”
AMZN in May 2019 = $1775, today = $3496, 96% appreciation.
AAPL in May 2019 = $43.77, today = $130.87, 198% appreciation.
GOOG in May 2019 = $1103, today = $2508, 127% appreciation.
That's all over the last 2 years. Market low was apparently Dec 2018, before the Fed started cutting interest rates, so the numbers are a bit bigger since then. They were pretty steady through 2017-2019, so anyone who got grants or refreshers during those time periods has benefitted from that appreciation.
Also, staff level is a tiny slice of the engineers at FAANG. So still not “a lot” even if your stock appreciation numbers were correct.
I don't think this part is off. Why I never went to Triplebyte and will not go, I think doing tests is dumb and don't wanna do it. Like original commenter said, you can't change model that quickly.
It's incredible how mileages vary. I recently had to strike companies off my list which demanded to know my last salary.
I offered them my salary expectations, but they would absolutely not accept that, and preferred to take the fact that I would not tell them the exact salary at my last position as a reason to decline to immediately stop the interview (with half an hour left in the call).
This wasn't a one-off either, I would say about 5% of companies (biased sample) try and get away with shit like this.
(This was German-speaking Europe)
In which case the correct answer is "Das geht Sie überhaupt nichts an."
As CEO, you:
* Grossly expanded profiles without user permission.
* Laid off tons of staff right before the COVID lockdowns. (What has been your personal returns since April 1, 2020 by the way?)
* You have been privy to discrimination happening in client company on-sites yet did nothing to the client companies.
You’re a CFO, an investor. You’re not a CEO, and you certainly do not have my trust as a developer.
In 2015, candidates with the same schooling scored about 20 percentage points lower than 2020, the last candidates I used the test for. The number of questions doubled, and then the test got much easier, by eliminating more challenging programming questions and replacing them with questions with giveaway context. By comparison, according to my data, about N=43, until about 2018 being a senior versus a junior in college CS programs is worth about 10 percentage points on the test; going to Harvard instead of Berkeley is also worth about 10 percentage points. In 2020, the last tests had no predictive features.
I stopped using the test, because it became too easy and too noisy to be informative.
I recognize some of the coded language in the blog post. There are definitely more lucrative opportunities in recruiting for DEI. I don't know if it will last. If you're still jittery about the public-profile-by-default thing, which by the way, was totally irrelevant and overblown IMO, this may not be a pivot for you.
One thing I see in the data is that at MIT, women and men performed the same, controlling for seniority. This wasn't true at the 3 other universities that produced enough data to measure.
That said, what really is the best way to hire candidates? I'm not convinced having binders full of engineers is special, there are almost always more candidates than jobs, at least 5:1, in every non-credentialed industry vertical. Anyone who has worked at a jobs (or indeed any matching platform, like the Common App or Tinder) knows that.
Then there's this long thing about asymmetries or whatever, warble garble about missing information... It has never, not once been my experience that someone seeking a subordinate role at a typical private company with preferences like "pair programming" or whatever have ever been better than someone with no preferences at all.
Maybe it helps to engage in the vanity of whatever trendy workplace trend is hot for whatever vertical. But like, if you're being intellectually honest, if you thought pair programming was important, you'd pair program at TripleByte, but you don't, you know in your heart of hearts none of that shit matters, so why are you putting stuff like that into your search system?
Indeed and ZipRecruiter are ad arb companies. They don't care. Private universities lead, not lag, DEI at giant companies, so it's hard to see how to compete against them in that core business. It will still come down to a real defensible opinion.
Do you have more valuable inventory than ZipRecruiter for DEI candidates? Who knows. What an uninteresting question. Apple also hires people who just make shit up on their resumes, I know two - though they weren't engineers.
If this were true for "qualified" candidates, there wouldn't be a labor shortage, right? ;-)
They told me the actual number of qualified candidates was usually just a handful or less.
I doubt this is true in SV, but I don't live in SV.
That's interesting to me. Most reputable rankings put Berkeley at #2 or #3, and Harvard is usually around #7 or #8 (and US News puts them at #16!). I wonder what the cause of that discrepancy is.
From what I can tell, anyone who is a Harvard student can declare the CS major if they pass the pre-reqs. At Berkeley, passing the pre-reqs doesn't get you into the major -- you still have to apply for a limited number of spots, which generally means having a 3.8 GPA or better in your pre-reqs. So effectively both programs have the same acceptance rate.
Berkeley EECS is sub-7%.
Most clinical doctors have social skills and crystalized intelligence from domain expertise, but aren't typically mistakable for particle physicists.
People don't necessarily require the proper sheepskins to possess fluid, crystallized, and/or other domains of intelligence AND the skills, personality, and experience relevant to excelling at a particular STEM role.
Pedigree is mostly used for social filtering and business leadership board packing, but if someone wanted to create an elite monoculture of staff, lacking in cognitive and personality diversity, by all means, go right ahead.
PS. I. Don't even get me started about a car full of CS IITians talking about the JEE and high-placers who seem normal on the way to snow country for snowboarding and gambling.
PS. II. I bombed an Apple interview for a mid-career role by being too intelligent and too maverick compared to the group of compliant, I hate to say, yuppies. It was an interview panel of about 9 people and they were just speechless. The other time I bombed an interview for being too smart was about 10 years before that when I was 20 at the old Borders bookstore in Palo Alto. Moral of the story: it's important to play dumb where appropriate because most people are relative-intelligence insecure.
PS. III. Sorry, reader, for the rambling and discontinuous thoughts. Absurd endocrine values of unknown etiology currently... doctor appointments pending. ):
That sounds awfully specific.
Are you sure they were speechless due to your high intelligence and maverick qualities?
There is a large pool of people who think that they are programmers, but are not. This is exactly what the infamous fizzbuzz test was intended to weed out. I personally know a top notch recruiter who credits her success due to learning early in her career how to filter most of the junk out. For example a "Perl programmer" who listed Microsoft Word as a skill was probably not worth looking at. (Note, this is an example from 20 years ago. But the principle remains.)
If you create an engineer focused site and DON'T find a way to segment engineers by ability, you'll find that your negotiating power is based on the average of your candidate pool. Which means that you'll discover why traditional recruiters kowtow to companies and not the other way around.
& maybe I'm wrong but I thought the entire value proposition of TB was that their pre screening and testing process made the whole process much less noisy for your hiring managers. If you still end up needing technical interviews and aren't guaranteed that all the candidates you get have gone through some technical screening, what's the point?
> Going forward, our assessments are a purely optional means by which engineers can show their skills to employers (who overwhelmingly tell us that they trust our scoring), rather than a requirement for entry.
I think the idea is that an employer can still filter out the unfizzbuzzables, but has the option to ignore that for candidates whose stellar track record is a stronger signal.
I'm a little bit skeptical, because i think there are lots of people with apparently strong CVs who are still weak when you actually put them to the test. But maybe not at the very top end; i am really thinking about people with experience in various investment banks, which is perhaps less of a signal than Apple then Netflix or something.
The problem is we're all spending weeks and weeks on weeding out fakers. We've moved to a quick pop quiz right after we get an apply. This takes out 85% of the candidates (it makes me sad when I look at how simple the stuff is that weeds them out...). Two interviews are more than enough to land on a hire after that.
Our company is a conversational recruiting software company (you apply, we say hi, and screen candidates in real time and then hook the candidates up with an interview). We started doing the screening in our chat flow and it works really well for us.
Sure, having metrics on how often a company ships code, etc. would be helpful, but frankly I have about 10 companies on my list that I'd work for, and I doubt any of them would give you this data.
What I'd really like is not having to practice leet code problems for 3 months before I interview. The older I get, the less motivated I am to do this. I've worked in senior engineering roles at big tech (FANG) with 3+ years tenure. I don't think I managed to not be able to do my job and fooled people for that long.
I've interviewed people I KNOW could do the job, but they didn't get the optimal solution, or were nervous, or N things in the loop, and we ended up not hiring them. There must be some incentive to create a system that lowers these false negatives.
I simply do not care about things like, “what does malloc return”. I do not care what people know about bloom filters in Postgres. The 99.99% of web developers don’t have to know these things. I do not care if you can implement Tic-Tac-Toe.
It’s poor interviewing, which is the entire product they’ve been offering.
I’m generally quite pro immigration but when BigTech goes to DC and bellyaches about how they need higher quotas because they can’t find enough workers it really sticks in my craw.
Reminds me a bit of the classic example of chutzpah—-the boy who killed his parents, then threw himself on the mercy of the courts, asking for leniency because he was an orphan.
I am looking forward to progress toward a hiring process that puts engineers in control, such as by allowing them to accurately record their current role in their profile.
But the transparency you need to extract is mainly financial, that’s the big asymmetry. I think people care more about money than they say they do, and less about release cadence or test coverage or whatever than they say they do.
So triplebyte should be called “get me a raise.com”. I tell you what I’m making, smash all your tests, and you find me somebody who wants to offer me e.g $50k more or $100k more, and that WILL be the offer I get. And if the company reneges on that offer then it’s flagged to warn others, and if they do well by their employees they are rewarded as well.
That said, the idea of "we will offer X to anyone with Y quiz scores" as a showing a company can make is pretty cool, and is definitely in a "company makes public conditional guarantees" space that we're interested in exploring.
Does this guy have a patreon?
how do I reply to recruiter contacts? applied to a job, received a request for interview from company. Nowhere to reply on the platform? How do I get in contact with them?
Should there not be a reply button right there with recruiter's message?
p.s. I already emailed support but, my past experience tells me they won't reply.
Funny but whenever I think of Triplebyte, all I remember is seeing their ads everywhere. I am personally a fan of Indeed and they do a good job sending leads and applicants other than good old Linkedin.
> We got jobs for over 1000 engineers
Given how many years Triplebyte has been running, "over 1000" seems surprisingly low. I wish them luck with their pivot.
dev)
> I forgot how to implement the zig operation in a splay tree.
employee at Triplebyte)
> To put some hard numbers to this: further down this thread, there's a post about how "any engineer" could answer a question the poster thinks is too easy. I looked up the question in our back end and, in fact, barely a third of people who take our quiz get it right (the correct answer isn't even the most common one!).
So there 1000 job matches and at most (("hundreds of thousands of engineers on our platform" / 3) - 1000) who were incentivized to answer these contrived problems, did so correctly and still couldn't be matched.
So even among the large pool of engineers who have gone thru the process, met some arbitrary threshold of engineer-ness, there's still a huge mismatch between corporate/prospective employee expectations, that I'm not sure will be able to be overcome quickly even with these new initiatives, but it's interesting that they are being pursed now (not surprisingly after the shift in working environment after massive government restrictions on freedom uber alles).
Yeah, different companies want different things and are ok with different things. My best experiences have been with companies that have taken less than 2 weeks to hire me with no testing whatsoever: just a couple of calls. Other companies want and only select for the contrived coding tests and programming jeopardy, and that's fine, but I want nothing to do with them at all.
I would be interested in using triplebyte if they explored something like this: https://news.ycombinator.com/item?id=27545446
Maybe even allowing 3rd parties to bet on specific hiring decisions (in addition to what is described in the link) could add another layer of incentivized feedback loops to the process (esp if the people making the hire/fire before 3 months are made known) and add another layer of market driven signals for them to pay attentions to. Maybe make it more expensive to chase certain types that are in higher demand, and less expensive to chase others types that are in low demand (not the salary to the employee, just the payment for filtering). Maybe less expensive for shorter refund windows, and more expensive for longer refund windows.
For sure. That's very likely why they're making this pivot. Exclusivity was their goal, but it just turned out to not be a very profitable goal for them, it seems. I was basically just saying it seemed to be a "theory" issue rather than an "implementation/execution" issue.
$25k*10k = $25M / 10 years = $2.5M / year. That is peanuts compared to how many people they employ. Plus the ads they run...
I guess they didn’t hit product-market fit with candidates. Two sided markets are tough.
It’s possible that they just couldn’t code with someone watching but the effect was the same.
Note that I didn’t have control over the interview process, so I couldn’t switch to a take home or something else.
The resumes seemed like they had relevant experience. That’s why I pushed them forward to the tech screen.
That's worthwhile.
When I was fresh out of university, I had trouble finding a job. I had worked part-time at a small company through the last few semesters, but that company blew up just before I graduated. Every interviewer I ever met with was impressed because I really knew my stuff, but had trouble getting in the door due to an unimpressive CV and awkward mannerisms.
I did very well at Triplebyte, scoring in their top bucket and getting top marks in several of the sub-categories. Those metrics are relative to all engineers, which would put me even higher amongst recent graduates. I also did very well on every software project I worked on and scored top marks on several other objective tests, like the Major Field Test.
I was exactly the sort of candidate their system should have been able to help. I never even got an email out of it, let alone an interview.
Waste of time.
To be clear, I'm only addressing the comment about how people thinking highly of them are gullible. I had a great experience, twice, within a few months, and that included going through a time of huge uncertainty.
That being said, I was not happy to see each of the changes they've made over the last year or so, and this does not strike me a good direction for engineers. I suppose I have to agree that they are not doing this from a position of market leadership. Perhaps they were providing asymmetric value to engineers, and at the end of the day the engineers are not the ones paying for the service.
This is a very good insight. Unless candidates pay more than companies, the incentives are off.
One. The problem domain they are in is on its way out, and they've decided to defect instead of being the ones left holding the bag.
Two. They hate what they do for a living.
The first company to stop making buggy whips probably did not do it because they weren't selling any. The fact that they ceded market share to their top competitors probably sunk the barb in deeper for them: Yeah people are buying cars but our sales numbers are still going up so why should we change?
In tech we see companies all the time end up being complicit in the destruction of their own industry by reducing its relevance in some manner or other, and by exiting early they have more options of destination. If I jump to the same industry as a competitor a year later I just look like a copycat. While some copycats copy other people, only better, they are the exception to the rule. Most churn out uninspiring derivative work, and if they can't at least do it cheaper then they end up on the scrapheap of history.
At that point, you can either keep going with what you're doing (and run into an inevitable wall later on) or you can foresee the future problems and pivot before they become unmanageable (which is what we did). Relatively few startups find their final model from minute one.
Fortunately, most of the dirt thrown at recruiting teams is wrong in the first place. I don't know any hiring managers who make lowball offers that have a low chance of acceptance - the costs of the process, the risks of hiring someone who starts mad at you, and the value of winning someone who is actually worth hiring, are all way more important than some incremental cash saved.
Well, presumably the idea is that engineers, especially high quality ones, will be drawn to the recruiting platform that has more pro-engineer policies. And so if you avoid using it, you lose access to that subset of engineers.
It's still a chicken and egg problem, though, because engineers have little incentive to join until there's enough companies.
However, I think that even if they build an attractive, differentiated experience, the advantage is unstable. The problem is that "traditionally strong" applicants (those who already have positive ROI from existing processes) are disadvantaged by being exclusive to one platform. So, from a hiring perspective, TripleByte's differentiation in terms of unique resumes is going to be from people who are harder to assess.
I'm actually fine with that as a value case - we've had great success hiring from bootcamps. "We help you reach strong nontraditional applicants" is a great value proposition. But it's not the same as a position of power to dictate the hiring process. Maybe a better pitch would be: we run a process that helps great non-traditional talent and great non-traditional companies find each other.
And as a HM, I was never interesting in using them, and we continue to use our own internal recruiters or local recruiters who know the market.
Having worked at a startup that was trying to disrupt the hiring/recruiting market back in 2010. I'm curious to how effective they were....if at all.
Edit: update, maybe I'm super skeptical because of all their cheesy ads that were all over Reddit for so long?
On related matters. I know several companies that have pivoted out of the "hiring/recruiting" space because apparently that space is not easy to scale/automate.
My experience is different. I've been through the TB process three times, back in 2017 where they flew me to SF from Asia for Flexport, Mixpanel and two smaller startups, and then again this year where I went to the final stage with FB production engineering and TB itself. And in between as well i created yet another profile on the system after shelving prior ones for a fresh start. I have a non traditional background, (ie, an "outsider" developer, chem degree and self taught hobbyist programmer since young), i take my craft seriously, have a bunch of popular open source products and a successful low touch indie business. But i don't consider future success guaranteed so i like to keep talking to companies to find that great opportunity for me. I know my tech skills in some areas are not the best so i like to stay in the loop, stay up to date and relevant and stay in touch with what companies are looking for. By far the best hiring process experiences I've had have all been through the TB platform. And my "open rate" for applications and inbound interest on TB is way higher than any other methods I've tried (SO jobs, WWR, HN). The quality of companies, hiring process and level of interest I've received via TB has been epic and, at least in the past, even the TB support staff are super helpful going so far as to provide personalized feedback on profile improvements and resumes. I'm not an expert at getting hired but i couldn't have asked for anything more from them. They were fantastic. I only have great and wonderful memories of the time i spent in hiring processes of companies i "met" via TB. So this is my experience of it, and this is despite the fact that i have never ever received an offer from any company I have met on TB, including from TB itself. I don't blame them. I just consider my own skills are not at the right level, and the fit is not right for the companies I've met, while i work to improve myself and my skills. That's all. It's a simple and beautiful experience I've had with them. Soooo different to hiring through other platforms (where funnily enough i have recieved offers). I've met and talked to ammon and many other TB team members and they have all been cool, friendly, passionate people. From what I'm reading of this pivot, they're being vulnerable and open to learning, and using what they've learned so far to create something even better. I'm not gullible, I'm not saying they will succeed. But they're trying. And they're earnest and sincere. What more can you ask for? This has been my deep experience with them. I think the most vocal critics are talking merely theoretically without any experience of TB nor deep nor accurate understanding of who they are, what they do and what they stand for. To them i say, give it a try. Investigate before contempt! ;P ;) xx
I've been wishing for something like this for years. Especially for early-career engineers or people from non-traditional backgrounds, this is insanely valuable because it helps you to know avoid wasting time on applications that will never go anywhere; avoid typing in your resume, line by line, into another form after already submitting your pdf. Avoid writing a cover letter to a job that's already been filled. Avoid applying to a posting that's really just out there for a company to "gauge" interest, not for filling a real role.
If an application is rejected, fine. But getting a follow-up request 7 months after submitting a resume into a black hole is ridiculous, and I think any system that decreases the information asymmetry between the applicant and the employer that allows people to intelligently approach their career search is going to be tapping into something truly valuable.
I'm in this industry and doing well because of recruiter reaching out to me (multiple times) and not vice versa. The whole idea of applying for jobs just doesn't seem to work at all.
On the other end a recruiter reached out to me and I'll be doing the final interview with their VP next week.
I don't think we can draw conclusion at that stage (we are a dataset of 2..). But I'd love to have access to the LinkedIn dataset to figure out if applying for jobs is broken at scale.
It's ugly and mimicks real life... more effort gives better results but life isn't fair and never has been. Luck factors but always fight on multiple fronts.
My major are slightly mismatched though.
Then I stopped bothering when it's super obvious that isn't working. Particularly, on Linkedin you could see the number of people that has applied (maybe a paid feature, i don't remember now). And for any entry level software engineering and data science position it's always in the 100s.
The one recruiter that I had contact with did more work than all of these. I've since switched jobs (to one of the FAANG) and that was through recruiter too.
I also don't think field matters until one gets to PhD levels, which I did not.
Like you, we thought (correctly, as it turns out) that it would appeal to people early in their career. What surprised us is that very senior engineers also told us they loved it - we didn't think it was as much of a concern for them, but sometimes your users surprise you.
It's also a great example of how obvious low-hanging fruit gets missed until you start thinking strictly about "what can we do to make job seekers' lives better?". It's not a particularly innovative or difficult feature, and yet major job sites with a dozen times our engineering resources still haven't done it.
The last time I was a hiring manager, I committed to giving feedback to anyone I rejected after the very first screening step. This was about 25-30% of applicants, which (even for a role in considerably less demand than engineering jobs) ended up being many dozens of emails. It was a giant pain in the ass - and it's pretty emotionally draining because you're turning down people who would really, really benefit from getting hired. I did it anyway, because I'd promised to, but it was not fun.
Next time I look for a job, I will definitely be targeting both these companies.
I wouldn't charge for the service, however, unless building it as a freemium model.
/economist hat
Often enterprise HR is paranoid of honestly evaluating anyone for anything, to the point that many HR teams tell one another and engineering hiring managers that it’s “illegal” to assess candidate abilities on the way in the door, at all.
If you think about joining a dev team where no one checked if any candidate could FizzBuzz, you can imagine the workplace environment that can end up with.
Triplebyte is able to provide engineering managers with a stack of pre-vetted resumes so an enterprise can interview by its lowest common denominator HR policies, while still having a prayer a team will at least be made of candidates who can code.
The value of this to an enterprise stuck in this position is hard to overstate.
Hopefully along with expanding the talent pool per this post, Triplebyte can figure out a way to get well paid for this ability to help land actual coders on actual dev teams despite enterprise HR.
The need for this is huge.
// I see a comment below from a Triplebyte PM about score matching: https://news.ycombinator.com/item?id=27542621 … That would be slick: land candidates that genuinely raise the bar, but aren’t so far ahead as to be speaking a foreign language to the team. Of course, this would require also assessing the target team, and oops, we’re right back up against that HR policy…
There's an old quote: "Nobody ever got fired for buying IBM."
Large companies tend to be more risk-averse, so if there are two companies you can use for a service — $PromisingStartup and IBM — large companies will generally make a decision that's optimal to the specific decision-maker, not optimal to the company.
There are two options if you, as the buyer, choose either $PromisingStartup or IBM: success or failure.
If IBM or the $PromisingStartup succeed, then you've done your job.
If IBM fails, you can tell your manager "Who could've guessed! It's IBM!"
If $PromisingStartup fails, you'll have a harder time explaining your decision, and the fault will be with you.
The "Nobody ever got fired for choosing IBM" idea is useful to look at every decision large companies make, whether it's pivoting, choosing a SaaS product, or hiring.
TripleByte, in its current form, has been beneficial to candidates as well by giving enterprise employees a justifiable signal towards hiring them regardless of pedigree.
That sounds hilarious How could a dev team not be able to write 5 lines?
Its the same reason why a top notch, genious writer can turn out to be terrible at giving speeches, even if they're reading from a script.
Performing an interview is a different skill from writing code.
I think this is true for the day-long leetcode interviews, but to succeed as an engineer, even at the entry level, I think you need to be able to talk through how you would solve fizzbuzz. A big part of the job is communicating trade offs with other engineers.
I think if I could describe my personal experiences, I'd say that on a subconscious level, I know that how I speak and articulate details will be observed by the interviewer through a cultural lens more often than not. This may be unique to my personal background, so maybe this is different for other groups. But interviewers often get hung up on the use of different language/terminology, the types of jokes/jests that enter the conversation, how I respond to their language. And there's sometimes been an unwillingness to allow different terminology, perhaps because that's all the interviewer themselves knows and understands.
All that to say, a code interview doesn't always come down to the quality of the code.
Such an eye-opening experience! Now that I’m on the other side of the table these days, I understand the importance of making interviewees feel comfortable and asking questions that allow them to start off strong.
I got shitlisted at one place for being the only dev with the mathematical insight to know that multiplying pesos by a dollars:yen ratio doesn't yield a correct conversion of pesos to yen. They let that one slide without fixing it, but the final straw that got me frogmarched was refactoring an if-else to a switch.
They saw me as a god for fixing their shit and offered to hire me. No thanks!
From the position of a candidate this sounds like a poorly run company I'd have no interest in working for. I wouldn't want to interview with them.
Which turned out fine. I found a job where my pre-computer programming work experience was relevant, and here a couple jobs later I'm working heavily with and on parsers.
So. Count me as one engineer they rejected who doesn't hold that against them.
I'm pessimistic about this pivot though, for a couple reasons. One, it smells desperate. Recruiting is sales, and in sales, desperation is the kiss of death. I can't read that blog post without coming away with the strong sense that if this doesn't work, it's over for them. If I can see it, anyone can see it.
Two, companies aren't going to want to work with a firm that's allowing engineers to collect detailed information on douchey behaviour that the company might engage in. Does Triplebyte have the leverage and moxie to make them engage anyway? Probably not, see point one.
I wish them well, because recruiting is awful, interviewing is broken, and engineers deserve to have a better time of it given how demand-driven the market is, and likely will remain for quite some time.
Can't help thinking they'll be writing about their incredible journey within a year or two.
I have a total of 6 profile views. So you’re truly not missing much.
I would think big tech companies wouldn't touch companies who might fold in the next 6 months with a 6 foot pole.
On the other hand, if a company does not engage in douchey behaviour, then using Triplebyte could be a way to signal that.
This fails a bit if companies can be credibly accused of douchey behaviour when they haven't been engaged in it. I don't know what Triplebyte's plan is here.
> Does Triplebyte have the leverage and moxie to make them engage anyway? Probably not, see point one.
Indeed. They labour the point that programmers have the upper hand in hiring, but that doesn't translate to Triplebyte having the upper hand!
The most friction when switching jobs is the interview process -- and the question is why? I've been writing code for a decade+ now, working on startups, for known tech companies, public open-source, and have written a freakin' book! But, oh, my bad, I couldn't figure out a solution to your optimization question. I forgot how to implement the zig operation in a splay tree. I might simply not function well under pressure. Maybe I'm having a bad day.
Companies are missing out on literal geniuses by using outdated hiring practices. And, I get it, Google doesn't care. Amazon doesn't care. (There's an argument that they should.) They're huge and get X,XXX applicants daily. But why are small startups using the same hiring methodologies? It quite literally makes no sense. They're shooting themselves in the foot. I'm very passionate about this, and I'm working on a book on how to hire engineers. Maybe I'll actually finish it one of these days :)
And yet they will still email you, year after year… as a dev that's worked in finance/academic research labs and for low funded/bootstraped startups all over the world for the past 10 years… I'll never get into these big companies because I simply try to find companies that need work done yesterday, see my experience (maybe reach out to past corps i've done work for), and give me an offer for at least a short term contract (and these big companies will never do this).
The worst is when I've think I found an interesting company (that on more than one occasion, publicly likes to complain about not having enough devs… lol), start talking a bit and they send me a triplebyte link… maybe it will be different this time, but I doubt it because these behaviors are too ingrained in a lot of corporate processes (even if its mostly "big co does this, me small co must copy"-NPC type thinking)…
Past companies and open source contributions are not perfect indicators. I've seen people with stellar resumes fail to even know the syntax for an 'if' statement in any language of their choosing. I've had people absolutely ace takehome exams, yet in person don't know how to write a function.
False positives are also much more expensive than false negatives, so companies would rather accidentally weed out good candidates than risk hiring bad ones.
That's maybe a little unfair (plenty of people who aren't engineers take our quiz out of curiosity), but the basic point here - that companies are frustrated because they keep getting fizzbuzz-incapable applicants - is more-or-less accurate. That has been (and continues to be) a big part of our offering on the other side of our platform: in the same way that we can provide data about company behavior to engineers, we can provide independent verification of an applicant's skills to companies.
Wow, sounds like you either need to change your sourcing or improve your early screening. If that many poor candidates are making it to the interview stage something is wrong with the process.
Most either don't show up on time or fail fizzbuzz.
If this is actually true, which I highly doubt (I've interviewed people, too), your process is terrible at preselection. I've heard so much about this "programmer charlatan" that lies on his resume and ends up blowing up million dollar systems, but have yet to see any tangible proof.
> Past companies and open source contributions are not perfect indicators.
This is self-contradictory: how/why would any open source project let someone that doesn't know fizzbuzz contribute to their codebase? I run a tiny throwaway open source project (~600 GH stars) and even my reviews are pretty stringent. This point of view is not consistent. It's like saying "I know people that ran in marathons, but couldn't even run for half a mile in my interview."
I've seen similar. I wouldn't say 80%, but somewhere in the 20% range of people on phone screens. If I ask a similar modulo question that isn't obviously a rephrased FizzBuzz (i.e. "write a function that returns every 4th item in a list"), that percentage goes up dramatically. A lot of candidates memorize FizzBuzz without actually understanding what's happening and how to use it.
> This is self-contradictory: how/why would any open source project let someone that doesn't know fizzbuzz contribute to their codebase? I run a tiny throwaway open source project (~600 GH stars) and even my reviews are pretty stringent. This point of view is not consistent. It's like saying "I know people that ran in marathons, but couldn't even run for half a mile in my interview."
There are no time constraints, and no way to tell how much of their contribution is from their own knowledge and how much is copied from StackOverflow/other open source projects/etc. The point of view is consistent; I can't run a marathon, but I can walk 26 miles. That's fine for the local charity marathon, but it's not going to cut it if I want to do competitive marathons. The constraints and expectations are wildly different.
Even ignoring the fact that you keep moving the goalposts and are taking my "marathon" analogy way too literally, the argument that any kind of real-world coding involves "time constraints" akin to a 30-minute stress-ridden whiteboard interview is just bonkers.
I can run a marathon but I can't run 100. Your interview is one of many that I am making time over lunch. I'm tired and getting a perfect mark on a take home isn't as important as getting it done quickly so I can find time to apply to other positions before I go back to work.
If you are not paying someone to do the interview you are not going to get anyone's best. Even if you pay people are trying to juggle things to get the interview to work.
And to the parent post. 50% of people being late means they took the time out to talk to you and come to your offices and didn't gage the time or didn't think it mattered. That doesn't make them bad programmers and shouldn't even matter to the interviewer. Was the car service the employer used late picking the employee or did they have to come on their own?
I dunno about the back half (blowing up million dollar system) but the front half, certainly.
We used to ask "write min()" at a previous employ, and the pass rate was probably <50%. I still to this day structure my coding question around "it has to get them to write a for() loop". It's not trick questions (I understand why people don't like those, and those can die out, yes.) it's things like "here's a sample grammar, how would we go about parsing it?" which is something that I've used any number of times in my career.¹ Other candidates fail to display understanding, let alone deep understanding, about things like HTTP or Linux. Not exactly niche topics, and during this example period my employer at the time was clearly posting for "backend software engineer". And not just one or the other, but all of them, simultaneously. Since I can't establish any proof of anything, so… what other choice is there, but to pass?
¹while there are parsing libraries, yes, and I'll use those first / when I can, I would also say that 50% of the time, for whatever reason — better error reporting, better control — I end up going with a partially or mostly hand-rolled state machine or recursive descent parser. Also, while I wish people would just encode their data in something like JSON, or YAML, or whatever, it doesn't really matter, they. keep. making. new. grammars. Often inside those other grammars. Just today I had to deal with a config — a YAML config! — that wanted "key values". Turns out, it wanted strings formatted like "key:value", not YAML mappings.
With a bad pipeline, it's extremely likely, especially if you cheap out and try the bootcamp route (there's a reason Lambda is desperate enough they will "loan" a student you can try before you hire!).
It costs nothing to send a resume after all...
They could maybe just be making small or simple changes. Or perhaps mostly working on markup, templates, CSS, etc.
It's possible they know a bit of programming but just don't happen to know how to check if a number is evenly divisible by another number. This isn't something an open source project would necessarily know you don't know. (I agree it's something any programmer should know, but I can see how someone could contribute certain things to certain projects without that knowledge gap being discovered.)
I learned about the modulo operator very early on, but if I didn't know it existed, I'd admittedly struggle with the question for a bit and would seem like I don't know how to program; especially in a pressured interview environment. If it were Python or something I think I'd come up with something awkward like `if i / divisor == float(int(i / divisor))` within a minute or a few minutes, but who knows, maybe the pressure would mess with me.
Of the people that turn up late to the interview, what percentage pass fizzbuzz?
(N.B. I was being a bit hyperbolic, but, yes, for example, I highly doubt Steve Jobs could.) The spirit of my point is that most smart people don't care about the kind of idiotic minutia typical hiring panels ask. They care about interesting/creative problems.
Tongue in cheek, good luck on the pivot guys, it is a tough problem.
https://keithschwarz.com/interesting/code/?dir=find-duplicat...
I have Github repo. I have my own portfolio. I have wrote flutter app deployed to play store to demonstrate I can function individually and still can write production ready code. And All the above just on my own spare time.
But every company I have applied told me my skill set doesn't aligned with them. I get response from them that there is no traction on my resume. While some friends get interview after interview and others don't.
I am not saying that I am the best but we need different process where entry is given and now show me what you can do.
* The blog post complains that even experienced engineers they interviewed couldn't write some simple code on paper. Did anyone stop and think that maybe it's writing code on paper in an interview setting that's the problem? Stop and think about this. Imagine writing code like you usually do in a text editor. And then writing code with pen and paper. Backspace? Can't do that. Copy/paste some line to move code? Can't do that. Add some newlines? Can't do that. Run the code using the compiler? Can't do that. Of course most developers, even experienced ones, will do poorly under these conditions, this is not how code is normally written. On top of the already stressful situation of an interview, there's this suddenly completely foreign and limited way to write code.
* Even the blog post only mentioned fizzbuzz as a quick, easy question to filter out zero coding ability. Another way to say this, it was meant to filter out coding skill <= 0. Not skill <= 50 or skill <= 90. Zero. After that, the interview was supposed to move on to things that actually matter for software engineering. The present day interview has strayed extremely far from this. These days, you get asked two leetcode questions during an interview and even if you perfectly answer the first one, if you don't get the second one then you're an automatic fail. That was certainly not the intent of the original fizzbuzz, since answering any leetcode clearly shows coding ability. And the interview never has a chance to move on to the part that actually discusses software engineering skills. Or even if it does and they do very well on those interviews, the failure of only getting 1/2 leetcode questions correct overrules everything else since they must avoid "false positives".
I moved from engineering to management over 5 years ago now and still in interviews for positions like "head of" in 100+ people companies or other management roles in bigger companies I got asked the usual algorithm questions.
I just have zero interest to prepare for these interviews, I don't have the time or the motivation, and I honestly just don't care about the answers. Most of my job, and my team's job, has nothing to do with them.
The world might be different in FAANG, and admittedly I have not worked there (though I have worked for other top companies) – but most companies are not at that scale, and they should not use the same metrics, particularly for management roles.
Our principles at the time were very similar to what Triplebyte is trying to do now: give power back to developers.
We build a version of "hired" (did they die?) before "hired" even exist. A matching platform that was matching people with companies that was sharing both their desires and values. Small/big company. Consulting/Product company. How much $$ do you want? What technologies you're interested in? Where do you want to work? All of this anonymously of course.
It was all fine and dandy, the company actually still exists and generates revenue, but I'm overall very bearish on the recruiting space.
It's a sales game. It's a numbers game. And on top of this the people top companies REALLY want to hire aren't hanging out in these recruiting platforms. So as a platform you're only feeding average/average low candidates to companies, and in the meantime, companies that have in house or externalized sourcing resources get better results by going over linkedin and their employees network.
It then becomes a financial game for companies.
And platforms that are not linkeding rarely win.
Maybe I'm wrong here, but it gives me the impression that Triplebyte doesn't believe in it's ability to get candidates into jobs. If companies pay (a not small sum) when candidates actually get hired, then I feel that incentives are aligned. If it's an annual fee (with no option for contingency) then I wonder what incentive the company has to make sure what I care about (making a hire) is the thing they care about.
90% of the other major players (AngelList, Hired, etc) in the market offer this... so why not Triplebyte? Perhaps I'm missing something.
I wanted to use triplebyte - but I only want to make 3 hires per year.
-You vet engineers with however you/or individuals engineers see fit (test/ past work/ opensource / interviews/ etc), they highlight what they want + maybe what your data says would help them stand out for the "type" of dev they are.
- force companies to pay you $xk - $1xk to hire them right away if they "match" with someone they select, but with the ability to get a refund if they fire them within 3 months (but like 60 business days to get a refund, and allow them to keep picking but reset the window if they match with someone else again).
- keep in touch with those employees during that window
That way, companies will eventually get their money if they make a bad hire and still be willing to use you again, you can collect more data on the larger part of the employment life cycle and basically get 60 day interest free loans at worst.
"But it means that once a company signs up, they want to make as many hires as possible through us.", right, but as a hiring manager that doesn't make sense to me... I don't care that we're making hires to justify where we spend our sourcing money. I care that we're making great hires, so I want to cast as wide a net as possible. I'm not going to shut my eyes to other sources because I've paid for triplebyte. But maybe in bigger companies I wouldn't be the one making purchasing decisions? And the folks making those decisions would be pushing HMs to use Triplebyte?
It is interesting, how the incentives align and don't align sometimes. Hard to say if I'm even in the target market (raised $15m, hiring a few folks this year in the "hardest" category, #1 problem is qualified folks at top-o-funnel).
Maybe I also have a personal penchant for seeing companies who bet on themselves and clearly align incentives :shrug:
Then the company is incentivized to use you for junior hires. Maybe you'll get a few freeloaders who only want to hire juniors, but even those will eventually be seniors who love your platform, and at the same time, will boost your stats on the number of juniors you got jobs for.
We have a big problem in the culture of employing developers that is closely related to how the startup market operates. The market is willing to shovel money at a problem rather than spend time, therefore the market will pay someone with specific experience in their problem far more than pay someone less and have more time to learn the domain. This results in siphoning of experienced developers away from new developers, which artificially reduces the number of experienced devs. The market is creating the problem for short term profit, and then turning around, lying and saying "there's not enough devs".
Sorry, but I call major bullshit on this. Sure, if you have a D average at a low tier school, perhaps. But if you are a competent coder who got good grades in school, I see many companies ready to hire new grads left and right (obviously folks who graduated during the height of the pandemic had different issues to deal with). I don't know if we're quite at the "anyone who can fog a mirror can get a job" stage like we were in the original dot com boom, but we're not that far away.
Companies advertise they're hiring. Like they advertise that they're profitable, make the best products, and are better than their competition. That doesn't mean it's true. Many positions are "nice to have", so they will stay there until they get someone with far more experience on paper than they actually need - someone willing to get paid less of course.
People lie, especially companies. That's the nature of the market. Even if we had 5x the devs than employers, employers would still say otherwise. This is because more competition between applicants means you can pay people less.
Infact even during the dotcom boom my sister actually got a degree in "web development", and nobody wanted anything to do with her. I don't know who comes out with these lies about "high demand" employment but so far all I've seen is confirmation bias by those already employed - ignoring all the rest of the people who can't get anywhere, leave it behind, and go work at something else.
https://lh6.googleusercontent.com/5n4R_Osks_fVsVmedxJKsp9zPu...
That's empirical evidence of a tight labor market that's difficult to fake. Why else would salaries rise so quickly from such a high base? Even if employers lie (they do), the salary numbers from BLS are separate from that.
Of course, there are always people who slip through the cracks unfortunately. But avoiding anecdotes and looking purely at the numbers, it's difficult to find a labor market in the USA that's tighter than the one for software engineers.
This can be explain by FAANG paying more, or people being more risk averse of having bad devs (ie hiring less, but paying more for more skilled).
* Reluctance to move / restrictions on where they want to move (only SF, Seattle, NYC)
* Extreme focus on product development (and not applying for the operations jobs that keep many large orgs running).
* Focus on working for a high salary at a big tech company (and not applying to jobs at companies that aren't seen as big tech... say... this job - https://csx.taleo.net/careersection/basic_faceted_search/job... )
I believe that there is extreme saturation in certain desirable sectors of the field and "where is everyone?" in other areas.
I've moved across country twice and from one end of a state to another twice... its a pain, but I like new places.
One of the companies that I worked for (smallish city in rural midwest). There were a few tech jobs there (and a reasonable sized tech company)... several of my co-workers had never been further than about 100-200 miles away from where they were born (and then, only as vacation).
Yea... the reluctance to move is a problem.
I attribute part of this to my (gen X) and the "this is how things are" that I got from my parents (boomers).
My father's family had a few moves through the years... not too many, but a few. My mother's family moved several times (and once even to another territory) as part of my grandfather's career. Once my parents were married, they moved twice before settling down.
For me, when I graduated college, it was almost expected that I move somewhere else for a time. And I did. My brother too... and my sister. My brother (gen X) has moved five times since graduating college and my sister (gen X / millennial cusp) has also had five major moves since graduating college.
I believe there's a generational shift in that expectation of moving out - possibly from the financial insecurities that the dot com crash, '09 recession, and the '20 recession have instilled in the millennials and gen Z as they look for jobs... though that's just a guess.
> rather than looking at the three cities that have the most jobs.
There are a lot of cities/metro areas that have a lot of jobs that people tend to pass by. https://www.bls.gov/oes/current/oes151251.htm - note that there are other classifications that are applicable, but I can't get a page that has a set of them together... so just using this one as an example.
If you look at the raw employment figures, Dallas, Chicago, DC, and LA are way up there too.
If you expand this to the location quotient (based on how many jobs there are compared too other jobs) you'll find many other cities have quite a few jobs too. Huntsville, Durham, and Madison are also up there.
The reason we didn't talk about it as much in the main post is that our approach to the problem of credentials hasn't changed all that much. We still have quizzes, and someone who is having a hard time getting their foot in the door (a) is likely to be willing to put in the effort to take them and (b) will benefit from doing so. That's not a complete solution to the problem (it helps people with skills get hired, but doesn't in itself help people get skills that you only get on the job), but it's better than the status quo.
We haven't forgotten about people struggling as they look for their first job. (And I certainly never will: to get personal for a sec, my first job search was a miserable multi-year affair that nearly killed me.) We've got some experimental irons in the fire here (e.g. a program that basically highlights people who haven't gotten as much attention as they 'should' based on their skills) that we're not quite ready to talk about in great detail. If they work, you'll hear more about them soon. If they don't, we'll try something else.
This was me when I tried Triplebyte. I wish it had worked out. I don't really blame TB for the failure, but I think there was a lot of optimism on the part of TB that wasn't shared by the companies I interviewed with. TB was just another placement option for every one of them, and I found it challenging to overcome what felt like walking into several onsite interviews completely blind to what was expected of me.
Yeah I don’t think this is true. I teach undergraduate CS and our data shows (which we publish at admissions so is public) we regularly have close to 100% job placement within 6 months, with 90+% of that being CS related. I know many of the students personally and they are almost all landing good software development jobs. I think we have a good program but I hear the same from colleagues at other schools.
“Good software development jobs” of course is a relative term. CS related does not mean Amazon at >$100k, although a few achieve that.
There's good programs out there, but they are in the minority. If you run a good program, that's great. In my experience most places offering compsci are there to take the federal loan money and hand you a piece of paper.
If you allow everyone in, regardless of if they are qualified to be a senior level engineer, will this not just be another job hunt platform where (when I look for another job) I'd still have to do another technical interview at the company itself?
How will companies know now who has been vetted as qualified and who is just lying on their resume? Maybe it's shitty to think this way but there is already a ton of sites out there that allow me to send my resume (or allow a company to view mine) before starting a traditional hiring process that Triplebyte prevented, allowing companies to connect with qualified high level engineers and know the person would fit their skill needs. The way it was companies mainly were just testing for cultural fits, etc.
TripleByte wants that growth rate more than anything else: therefore, this.
I tried TripleByte and wanted to like it, but mostly it didn’t seem like anything special. The whole concept of getting pre-screened, getting multiple offers, and jumping straight to final interviews just didn’t pan out. The last company I interviewed with basically treated my application like any other pipeline. I had a standard recruiter call, hiring manager call, and several technical interviews before getting an offer. And the experience with other companies was similar. From the outside looking in, it appeared that there was no special track for TripleByte applicants. In the end it felt like the TripleByte process simply made the interview process longer, not shorter. Eventually I found it faster & more effective to pursue companies on my own.
TL;DR — companies I met through TripleByte ran me through the standard process I’d experience through any third party recruiter without the personal touch one might get from a good third party recruiter.
To be clear, we still have quizzes and the ability to show your performance on them to companies. The fact that they’re not completely mandatory to use Triplebyte doesn’t mean that they’re not important or that we won’t build further features around them.
Right now, companies can offer an expedited process as part of their reach-out. We’re working on ways to productize this a little bit better. We didn’t announce the feature in this post because it isn’t done yet, but one thing we’re looking at is allowing companies to set individual score thresholds for various types of expedited process (e.g. “if you scored a 3 on algos and a 4 on back-end, we’ll definitely get on the phone with you” or “if you have a 5 on python we’ll skip tech screens for you”). We certainly recognize that FastTrack was valuable to a subset of engineers, and we want to recreate the value it offered in a way that is a little bit more sensitive to the specific company and specific engineer in question. We imagine, for example, that less-prestigious companies (who are more concerned about attracting applicants) will probably set lower thresholds than the Apples of the world. Under the old system, we had to set a single threshold, which would necessarily either be too low for prestige companies or too high for everybody else.
And just to lean a little bit back into the pitch we’re trying to make here: if you’re concerned about interviewing process, wouldn’t it be nice to be able to search companies by how their interview process works? That’s not an axis on which companies meaningfully compete right now, but with the right incentives, companies will cut a lot of the annoying hassle that they currently have no reason to get rid of.
If a company asks me if I have questions, and I ask them, and they ignore me, I'm done.
I wasn't looking for a job, but thought the test results would be interesting. Who takes a test and doesn't get results?
I used them two times. The first one was very early on, where I was given a home assignment and interviewed on it by Aaron himself, if I am not mistaken. The dude was awesome at interviewing, and knew exactly how to probe to get a better understanding of your skills.
That was when they were promising that you can interview with them so you don't have to do technical interviews with the companies. I thought it was an awesome concept and could really reshape hiring in the tech industry.
Second time was a couple of years ago, where the model has already changed a bit. Passed the first round and one of the companies that I could interview for was triplebyte themselves.
What a disappointment! The only difference in the interview process than the rest of the companies was that they gave you a laptop and asked you to do practical coding instead of whiteboard generic algorithm solving.
Some of the interviewers themselves were junior members of the stuff with 0 experience in interviewing.
is that they say they have companies like Apple and TrueCar posting job listings and hiring but they actually never respond and all you get is offers for companies between 10-200 people in size (aka, startups)
It's a great platform, the indicator letting me know companies aren't going to respond is great.
It just doesn't fix the problem that... the quality of companies on Triplebyte aren't companies most senior/lead/architect/advisory level engineers are willing to settle for. I'm being unnecessarily harsh/biased. I'm sure plenty of people use the platform with great success. I've just come to accept "early/mid-stage startups" job offers from Triplebyte and not much else. Not necessarily "garbage" but... for sure lots of risk.
The "super week" of onsite interviews was interesting, probably increased my overall interviewing skill, but was probably one of the more stressful individual weeks in my life.
Scheduled a practice interview and the only slot available was ~6AM my time. Nobody showed up and I wasn't informed that they wouldn't be showing up.
After reaching out about it -- they tried to tell me that my interview scored poorly, and that I would need to retry again in a few months. After some back-and-forth they realized both that it was practice interview and that the person didn't show up. So they rescheduled the practice interview.
When the interview did happen -- this was for ML stuff -- the interviewer was just no great. We spent so much time discussing the differences in terms we used -- mine largely coming from University, there's from I'm assuming their formal education -- that much of the interview was wasted. It literally came down to me deriving what we were not agreeing on for them to understand we were talking about the exact same thing.
I then had to reach out several times for my results, I'm assuming due to the fact that you're allotted one practice interview, and technically I had two(?). When I finally got my results I was again informed that my results were not good enough, and I'd need to wait to reapply. I gently informed them that it was practice interview, and the representative apologized their mistake.
When I reviewed my results... the interviewer didn't rate me too well, largely due to our differences in terminology. They also didn't like my coding style -- even though no one has ever complained to me before -- and rated the coding exercise poorly even though I was able to perform what was asked of me.
I just gave up.
Then several months later, I got an email about being TripleByte certified or whatever.
The whole thing was a really bizarre experience.
People hate overly abstract/CS-focused whiteboard problems (mostly for good reason), isn't practical coding the obvious better alternative?
Also, dissing those who have passed the bar as "engineers who like tests" is disrespectful. When they first launched, passing the bar makes you (in their words) a highly qualified engineer, and now passing the bar means nothing except that you only "like tests"...lol
Fundamentally, I think this invalidates Triplebyte's business model. Companies don't really care if someone passes your tests. They still put you through LC type interviews onsite. They simply save a phone screen.
We still have the quiz, and we still show performance on it prominently on your profile (provided you've chosen to share those scores). In fact, we've put a lot of energy into improving the trust companies place in our quizzes over the past year precisely because we think that skills data is important.
> Also, dissing those who have passed the bar as "engineers who like tests" is disrespectful. When they first launched, passing the bar makes you (in their words) a highly qualified engineer, and now passing the bar means nothing except that you only "like tests"...lol
Heh, yeah, that's fair, at least to a point.
Our quizzes are predictive. We know this from a lot of objective data, it's what you'd expect subjectively, and companies do tell us that they trust and value that data. That does not mean that our tests have no bias towards certain personality types. Different people respond differently to testing, and that does have a differential effect, not because our tests are bad but because testing is inherently somewhat artificial.
I actually taught test prep before joining Triplebyte years ago, so I've been this first-hand: it was not uncommon for a student who'd been doing well in practice sessions to crumple under the pressure of the real thing. That doesn't mean the test is bad, it just means that some people fare better on tests than others for reasons other than just their raw ability. When we say "likes tests", that's more what we mean: not that that's the only reason someone does well on a test, but that people who perform well in isolated, pressured environments do better relative to their skills than others.
I think that goes to another ability. If we need a candidate to gracefully handle a 3 AM production outage, we should expect them to handle an interview.
So while in theory I like the idea of this meritocratic minimum bar granting special privileges (I liked it enough to join Triplebyte!), in practice we seemed to be fighting an uphill battle with all but a few candidates each month.
On the other hand, I have wasted a lot of time in conversations with recruiters only to be let down at the end by mismatched expectations. I'm interested to see if this new approach can make a meaningful difference in that problem.
Tell HN: Interviewed with Triplebyte? Your profile is about to become public (1543 points) https://news.ycombinator.com/item?id=23279837
Tell HN: Triplebyte reverses, emails apology (1030 points) https://news.ycombinator.com/item?id=23303037
Because she fucking nailed it with Triplebyte.
I used Triplebyte. I really liked it. The leads were not as great or as numerous as I was hoping, and I felt like I still had to do pseudo-technical interviews anyway. Fucking COVID, man.
I hope this pivot works for them.
> Triplebyte has hundreds of thousands of engineers on our platform, and that means we can flip the script on companies. The collective power of thousands of engineers is enough to change their incentives in a way that individual engineers cannot.
> We can change their incentives directly by rewarding or punishing certain behavior. For example, companies aren’t normally incentivized to provide salary and culture data. But we can force their hand by promoting transparent companies in our search rankings. When a company’s access to thousands of engineers is on the line, their incentives are very different. The same goes for honesty: a company often has no reason to be honest with any one engineer, but we can disincentivize lying by making their behavior with one engineer affect their access to the next.
So... a union? In digital, online format?
I'm glad that someone there is now realising the advantages of joining forces to negotiate workers' conditions. It was about time high-tech engineers noticed this.
So hiring through TripleByte would be conducted in some fixed-length time intervals (e.g. 1 month). Companies specify skills they're hiring for, interested engineers interview for the skills that TripleByte can screen for and supply a minimum salary (invisible to companies), and companies hiring that month bid starting at the min salary for the engineer with all other hiring companies.
If the market determines the hiring timeline, recruiters can't put you on hold if they're interested-but-not-sure, and can't use exploding offer tactics. It also pushes companies to be more honest about salary ranges and required skills. TripleByte would be responsible for providing feedback rather than the companies.
I found an awesome startup to work at. And now, years later, I work at a FAANG company, working on tons of interesting problems. Thank you triplebyte, and good luck with the pivot!
I love this, but I'm curious about the incentives here. Triplebyte only makes money from companies, not engineers. In the long run, can engineer-centric intentions override a business model based around companies and recruiters staying happy? I hope so!
Triplebyte has always seemed scummy and this obvious--and unacknowledged--contradiction makes me trust them even less.
We do also have differentiating features on the company side. It's not like we're abandoning the idea of building features for companies or anything.
The job search, like other matching problems, is often not zero sum. So building (say) a better search interface for companies benefits engineers as well, because both sides of the market have an interest in a good match. What we're saying here is that in the cases where things -are- zero sum, the power - and therefore the incentives - lie with engineers and not with employers.
https://news.ycombinator.com/item?id=27501675
Our bet is that LinkedIn is going to fragment. They are just not creating a hiring process that most engineers like. People tend to either get ghosted, or overwhelmed with low-relevance inbound (almost no one gets the "right" amount of attention). Companies need to go where the best engineers are, not the other way around. So I think our long-term incentive is to fix these problems.
In any case, I'm committed to giving this a try. There is danger that we get pulled toward building for companies. I want top guard against this by being public about what we're doing, and "showing our work" as we go.
> There is danger that we get pulled toward building for companies.
A common phrase thrown around with free services is "if you're not paying for it, you are the product".
> For example, companies aren’t normally incentivized to provide salary and culture data. But we can force their hand by promoting transparent companies in our search rankings.
Maybe I didn't read the article carefully enough, but are you planning to continue charging companies $15k - $30k for the ability to access candidates on your platform?
If so, companies are still your customers. And if you're building and optimizing your product for people who aren't your customers, your real customers (companies) may not be happy with you which will hurt retention, etc.
Maybe I skimmed the article too quickly, but it seems strange to charge companies $20k+ to post a job on your platform, and then actively do things that "force their hand". It might be a net benefit to the engineers on the platform, but I wonder how it will work from a business model perspective since you're potentially creating adversarial relationships with your "real" customers (companies paying you to access your candidates).
Edit: But maybe it's by design, if you actively remove employers who aren't abiding by your philosophy. Although again, that means turning away customers, which means turning away revenue, which in my mind raises questions about the overall business model. A lot of conflicting interests.
That does not fully express my motivations. I am an engineer and find the idea of making the process better for engineers more exciting than making it better for companies. But it explains how I think the incentives work.
maybe (and i know that this is an outré thing to say these days) that was the point.
To use a rough biological analogy, I think it's generally a lot easier to have a big, rapidly growing company if you take an r-selection rather than a K-selection approach.
At least good companies do this anyway.
Also, executive search recruiters kind-of do what you're describing. Exec leadership is a small-ish world, and the recruiting cycle for, say, a CTO can take months. However, there's a lot of commission on the other end of that. So recruiters basically act as brokers.
I much prefer the imperfect systems that at least pretend to be fair and objective-ish.
Edit: just saw the comment that TB is keeping the quiz. Would consider using the platform in the future as long as that pre-qualification track is available and that this new direction is just a superset on the existing TB features.
On the other hand, when I applied to TripleByte, their feedback to me was "we think your skills are fine, but we want somebody who can perform well in an interview". Which is exactly the opposite of the value you're attributing to them here.
Quoting them, for reference:
> We really appreciate you taking the time to work on the take home project. We're aware this requires a substantial time commitment and we are really grateful that you invested the time in completing it. We thought you wrote a great, very full featured regular expression matcher. It was especially impressive how much you dug into the academics behind regular languages.
> However we made the decision because we felt that while going through the project together during the interview, we didn't see the fluency of programming when adding to it that we had hoped for. While we specifically designed the take home project track to help overcome the difficulties of coding under time pressure with someone watching, we do still need to see a certain level of programming during the interview.
- In borderline cases, we didn't want to risk our credibility and companies didn't want to skip tech screens. So competent-but-not-amazing engineers got shut out.
- Our quiz is not perfect, so engineers who didn't fit what we were quizzing got shut out.
- Not everyone needs us to vouch for them. Companies (reasonably) trust that someone with a degree from a top school and years of prestigious experiences can probably do fizzbuzz, and demanding that they prove it was a barrier to them using Triplebyte.
We think it's better to have more granular ability for companies and engineers to decide what mattes to them. A prestigious company can say "we'll only talk to people who took the quiz and got a top score" (and that score is us saying "this person deserves a job"). One that desperately needs the headcount can talk to the borderline cases or decide they don't want to put the extra barrier of a quiz in the way.
Like most markets, we think the hiring market on our platform works best when it's able to respond to local conditions.
If you're not willing to refer people to companies based on their score, how can you interpret the score as you saying "this person deserves a job"?
How does the new ability to make that recommendation address the original problem?
Do not believe anyone who tells you they can force employers to change their methodology when it comes to salary and compensation revelation. Recruiting agencies are so far outside that wheelhouse that it's damned near scandalous to even pretend the statement has any validity.
Also, I think it's important to differentiate staff recruiters from outside recruiters.
Staff recruiters are HR people. They're on the job to shove square pegs into square holes.
Outside recruiters are salespeople. They're on the job to explain why a round peg can reasonably fit in a square hole.
This is your career. So act like it.
If 10 different people were each going to give you a hundred dollars if you described yourself to them in a way that would obviously ingratiate yourself to them (person with cat photo, person with dog photo, person with photo of their kids, etc) most would immediately come up with reasons to make themselves1 appeal to their tastes.
But people keep on submitting the same dumb resume over and over again and expect a different result. Accentuate the things in your resume that matter to the company you're trying to get a gig at. Yes, for each of them. If you would do it for a grand, you can do it for a career.
1) Look at relative data, not raw ratings. Written reviews or raw numerical scores are what show the nightclub effect the most. What we care about is a ranking of companies (showing the better companies first in search results). This may not be as impacted by the problem. If sour grapes from people who failed interviews overwhelm other signals, we can only look at the scores from people who pass (or normalize the two groups separately).
2) Ask very specific questions (and maybe provide dummy options to attract sour grapes). I suspect that questions like "did this company say they could meet at $250 salary and then offer you less?" will get more accurate answers than "does this company have a toxic culture?". (We do actually want to get culture data, but I want to be really specific about the culture questions we are asking).
3) Use objective facts, not subject opinions. For example, we can tell when a company ghosts a candidate (because they reply through a proxy email that we control). So we're listing the 'ghost rate' for each company.
Anyway, a ghost rate would be greatly appreciated.
I'm pretty sure many salty people will just lie anyway, to punish the company. Source: I've been on the internet a long time.
I think this model of 'punishing' companies would've worked when you had a high bar for being on the platform in the first place, but if anyone can get on there, you'll have a lot of low quality candidates who are very mad that nobody will take them, and they'll overwhelm the reviews.
Remember that there's a survivorship bias at work: the lower quality people will take longer to get hired, which means they do a lot more interviews, which means a lot more reviews on your platform. Essentially, it's a reverse meritocracy when it comes to evaluating companies.
Maybe this would be useful for folks just starting their career, but on the hiring side of this, in the last year I haven’t made an offer to a candidate who didn’t already have at least one other offer on the table.
Overall I think there are a lot of misconceptions about how people are hired from the recruiter side — which is why most of these platforms see the recruiter as the target customer, not engineers. I get that y’all are trying to change that, but without insight into the hiring practices at any given company (e.g. my company gives a lot of latitude for salary negotiations but every hiring manager negotiated differently) it’s going to be hard to pull a signal from the noise.
We look at the job posting as a brand marketing — all we’re really trying to do is get you to apply and fill the funnel. The whole hiring process is designed to be a calibration on role, cultural fit, compensation, etc.
Seems almost like the bait and switch that the OP company is railing against in the blog post.
I'm curious to see how you guys have managed it without disgruntled employees quitting within 18 months.
I work in consulting so high turnover is just expected. We’re actually a lot lower than the industry average. The job isn’t for everyone, but there are people who thrive in this environment. For everyone else who isn’t a good fit, we get a year or so worth of billable hours, they get exposed to a lot of different scenarios, and we give folks a lot of runway to find something they do like.
It’s just the nature of the beast though. If I get a candidate who is legitimately at level, I bring them in at level. There’s no need to be shady about it, but job search sites are what they are and that’s the game you play.
That being said, the experienced engineers who can get that kind of attention have other desiderata in their job search beyond just "get an offer". "Get -an- offer" is a goal more typical of a new graduate or someone trying to break into the industry than it is a goal of experienced engineers.
To people who are sure they can get a job, there's a big difference between a 150k offer at a company that has none of the workplace traits they want and a 200k offer at a company that they'd genuinely love to work for. And right now, they can't necessarily tell the difference between those jobs before going through a bunch of application steps. The example we mention in the post - of a highly credentialed and very senior engineer who kept having his time wasted by recruiters who refused to admit that they couldn't offer what he was worth - wasn't cherry-picked. It came up organically while we were doing user research, because it's a really common problem, and was echoed further by a number of other senior engineers we spoke to.
(Of course, we care about the folks just starting their career, too, so I don't want to totally ignore the importance of solving problems for them. But I think we are solving an important problem for senior devs here, too!)
Anecdote: Recruiter R from Company A reaches out to me, we have a conversation and she says HM (hiring manager) loves your profile and wants to talk couple of days later and so we schedule a time slot to talk to HM. Next day, R calls and says HM wants to know why I left a previous employer X and I just blurted out the truth which was that X made it very difficult for me to take FMLA when my kid was born and I got pissed and left. Shortly after this I received a rejection message. This is classic discrimination as defined by EEOC.
I replied to R and said this is discrimination and she denied it. I posted my experience on Glassdoor and my post was flagged and removed. Nowadays I am honest but not truthful.
No response after getting the "wow, nice quiz result" email. No further emails. No replies for months to my gentle inquiries. Eventually they did reply, saying they were sorry but they didn't need help.
I have significant doubts that they were hiring tech interviewers in the first place. I think instead they were gathering market data at our expense.
Furthermore, there were detailed stories from users on Glassdoor about their internal policies and attitudes, mostly painting their leader as lacking scruples and leadership ability. Perhaps this was all fake, but if so it was really convincing and thorough.
I'm curious if anyone has had legitimately good experiences with them.
I have started to re-write my resume for each application and that has yielded much better results in terms of quality of the company and number of interviews.
If you are submitting the same resume to multiple companies, you are doing it wrong.
But glad you figured out an approach that worked for you!
It would be a nice touch though.
The hiring manager side of me on the other hand continues to remember that mishires are expensive. I've also very skeptical in that once a month some staffing agency comes along, tells us they have some fancy heuristic for matching good candidates to our jobs, and then unleashes a tsunami of mediocre leads into our recruiting database. They'll then proceed to act upset and surprised when we don't want to intervene sixty percent of them and most of those we do interview wash out. I guess TB can somehow promise that a certain baseline technical competence is present, but even that's of moderate value. If I really need a candidate who is skilled at X, I might be willing to sacrifice some skill in Y because I already have two engineers strong in Y. I never found TB sufficiently granular in how you configure the automatic screens to allow for that. Secondly, it's totally useless in soft skill assessment.
The cynic in me just sees a lot of this TB now trying to force you to expend internal resources to rank the unqualified people or people whose skill sets don't align with what you want (forcing you to respond with data on how you handled the application) because they can't crack the nut of doing it themselves. So they're going to leverage the customer's recruiting teams to code the data for them.
I had some bad luck. My nodejs build broke after a recent update on my machine, which I didn't realize until right before the interview. During a test with a different language, I tried to define a constant with the same name as a built-in function and ran into a vague compiler error (something about missing parentheses, ugh). This language has case-insensitive function names to compound the confusion. I unfortunately looked up how to define a constant in the docs. Their conclusion was that "I was uncomfortable in the language," despite having used it for 10 years. There was some other feedback which I felt was inaccurate, I think I just had a bad day, and obviously didn't convey my knowledge and experience well. I could see why a recruiter would hard pass on me for some of the stumbles, since their main goal is to forward candidates who interview well. It hurt to get rejected.
I mean, you went into a combat scenario, and chose a weapon absolutely notorious for jamming.
If a company has a bad interview experience, a bad culture, or the management is impopular, if the work life balance is bad, or anything... you'll be able to read about that in those websites.
For example, a recruiter contacted me last year regarding an opportunity for a fancy senior role at a large company (Sr Staff engineer)... I checked the Glassdoor reviews for that company and found many testimonies from people who were contacted with the exact same narrative, but at the end of the interview process they were offered vastly lower levels, so clearly that was their intention all along. Because of that, I assessed that interviewing there was going to be a waste of my time and I ghosted the recruiter. Developers win.
Of course, you need to take some reviews with a grain of salt, but if everyone is saying the same thing, you will at least have a sample of what you are getting into.
Having said this, I am not sure Triplebyte will ever be able to effectively solve or mitigate the problems Ammon is describing. It is possible that he needs to continue talking to developers, and I am not trying to say this in a rude way. There are also many more problems that need help fixing that Ammon did not describe.
Having a conversation with a developer about different topics can give you some insights on the breadth and depth of the knowledge of a developer, and many interviews omit this. I think interviews should be "adaptive" tests, that get as progressively more difficult until you reach your limit.
I recommended TripleByte to several friends whom I considered reasonably skilled and currently undervalued. None of them passed the test, however, so they got little value out of service (except for the interview feedback, which at the time was unusually excellent). And I ultimately needed TripleByte less than they did, since someone who could pass the old TripleByte test also has a high chance of eventually passing on-sites at Google, Facebook, etc.
So I understand the pivot, although it will certainly be harder to differentiate yourselves in this new space. Good luck.
Triplebyte has existed for, what, 4 years? That actually doesn't sound like very many.
> Triplebyte has hundreds of thousands of engineers on our platform
Hmm.
For unemployed candidates, we should do away with interviews altogether and do paid 2-week trials instead. After the first 2-3 PRs it's pretty clear if someone has the skills you need and to what degree. It would also help the candidate decide if they like the company.
Employed candidates are harder to evaluate and I don't think there's a single formula that will work for every company.
My takeaway from my experience finding a SW engr job was that actual technical capability was really only a second- or third-tier consideration in companies' hiring decisions. (This is often misaligned with the companies' actual business needs, but that's not what materially drives hiring decisions either.) So, although TB nailed TFOO the competence assessment & pre-screening problem, Amdahl's Law still applied to the whole enchilada.
What TB should do is monetize their experience placing candidates over the last 5+ years and just sell training & maybe some standardized SaaS products to HR departments -- especially at startups! -- that regularly need to hire SWEs/SDEs (yes I'm a FAANG-er now). Because the real dysfunction in tech hiring originates & thrives in those departments, and...something about robbing banks because that's where the money is...
That capable people failed to make one solution work, doesn't mean no solution can work. Now they're going to try another.
That's a lot less than I expected.
What am I looking for?
1: I want my skills evaluated in a standard way by some neutral licensing board, just like engineers who build dams or surgeons.
2: I want job interviews based on mutual interest, not proving competency.
3: I want my colleagues to be certified, just like me.
4: I want legal standards in place that give me leverage against employers who are cutting corners, or managers who "know enough to be dangerous."
Example: My wife, a doctor, had to go through an intense process to be certified. The vast majority of medical school graduates pass certification. She only applied for a handful of jobs. The interview was based on mutual interest because the certification process proved she was competent. All her colleagues are competent, and her hospital is well run. There are legal guidelines that make it hard for her hospital to cut corners or make her do something unethical.
For most developers, the worst they can do is bankrupt the company they work for, and even that usually requires multiple compounding screwups and a bad luck roll.
I personally think the gap between our standards for doctoring and our standards for software development makes a lot of sense.
I do think people building life-sustaining software should be held to a higher standard, but that's pretty rare.
Also, are we sure there is a shortage of engineers, or simply a shortage of engineers that can pass the hiring interviews and qualifications?
I think a lot of companies don't always mind waiting to find someone who passes there test, be it a newer grad, or an existing engineers that has now prepped up better.
I still don't understand exactly what benefits it brings to me as a programmer though. I'm about to start preparing for interviews with 25+ years under my belt, after not working for 2 years and I've accepted the double-edges nature of these interviews. I know I need to LeetCode for 3 months to be competitive and memorize as many of the answers as I can. I know that some companies will ghost but others will be forthcoming. I know that in big companies, different teams have different recruiting processes, so it's hard to put an entire company in a single basket.
I have friends at all FAANG, and they are very open with their salaries and TC, so I don't know what Triplebyte would bring to the table besides friction, so be perfectly honest.
In addition, as my username suggests, I really like the idea of a representative for Engineers during the job search process, but I also want to make sure I'm the customer of this representative. If I'm paying TB then I think the incentives are aligned. If they're just a job board then their new stance is opposed to the interests of their true customers (hiring companies) and that won't work out too well for them.
My firend cracked interview and joined one of this and for others like me have no traction (6 months in) on the resume at all after using hired/linkedin/direct careers/cold emails. Doesn't matter you have github with real projects/portfolio with good responsive design .......
If this process is broken and we also don't know what are we doing wrong then how all them are going to find all these talent they need??
The interview itself was nice and ok, the feedback was really appreciated (after all this more or less reflects what high paying programming recruiters are looking for). But I felt like there was no place for people with my background.
I was rejected when I did the quiz at university, trying to get a job.
What made me a bit angry is how happily they rejected me with an automated email, after I put effort and time into doing their quiz... I didn't feel like recommending them, or even visiting the site ever again.
Glad to see they're finally aware of how this approach doesn't benefit them as much as it could of if people were still invited into the system.
What Triplebyte is trying to do is give more information back to job seekers. But it will be limited to the companies they actually work with (and punishing your own clients sounds like a peculiar business model...)
Why not build a Glassdoor for hiring, that would cover the entire industry? Anyone can go and describe the hiring process they went through, if they heard back or not, etc.
Does such a thing already exist? If not, why not?
I felt like Triplebyte worked hard to create a good experience for me, but that overall the companies I interviewed with just viewed TB as another placement company. Other than getting to skip the fairly easy phone screen phase, I didn't feel like I got much value from the experience at all, and I had to put a lot of work into it.
Ever thought of building a recruiting recommendation software product, instead of just selling people? You have the data.
edit: I want to be more constructive with my feedback. You have four years of data on hundreds of thousands of candidates. I don't know what the legal implications are regarding using that stuff as training data, but.. embeddings. doc2vec. Thank me later.
The quizzes weren't quite a fit for ML, nor was the interviewer or job density there for the domain, but the immediate half-hour call with the promise I'd passed a technical screen was extremely efficient, even if Triplebyte lacked any basic level of market liquidity.
I can also report I haven’t received a single TripleByte email since the issue was last brought up :) Thank you for standing by that.
Copying this from the article. Doesn't this sound like a union? Unions generally lead to mediocrity though.
Does this not mean that even if you apply from TripleByte that you now still need to go through FizzBuzz and all that?
So it sounds like the hiring manager can choose to skip a fizzbuzz call when they know you already passed.
Long story short I bear no ill-will towards the people at Triplebyte but will not attempt to use them again.
It's a tricky situation because you have to balance being accepting of people with non-traditional backgrounds while also making sure they can do the basics. I liked that TripleByte could condense DAYS of interviews into a few hours.
In general, long tech screens are hard on people that can't afford to take off a full day, so it's tricky for people already working in crushing jobs that need a change. The TripleByte quiz worked nicely for this.
On the flip side, if their tech screen is filtering people for reasons other than ability, it's removing possibly good candidates who deserve a shot at something.
The new approach is sounding like it's trying to satisfy the latter, which is good for improving inclusivity, but won't be to everyone's liking.
"Triplebyte, to my mind, did an admirable job of trying to solve the credentialing problem. But their approach was not without shortcomings: 1) not everyone wanted to take their lengthy quiz, even though the quiz was well done, and 2) scaling up an army of interviewers, all of whom had to be trained in exactly the same way, was non-trivial and not cheap. These challenges were surmountable, but the challenge that wasn’t arose from the second issue that all companies had to face: lack of candidate autonomy, which was driven in part by a lack of faith in the credential.
Once you passed Triplebyte’s assessment process, just like at Hired, you had to interact with a talent advocate (Triplebyte labeled them “talent managers,” but again they’re just recruiters). The talent manager would examine your background and short-list you for some companies of their choosing, where you’d then go onsite. You could have some input into which companies you spoke to, but it was limited, and if you didn’t meet the company’s (often somewhat arbitrary) criteria, no matter how well you did on the assessment, its doors were closed to you.
As with Hired, having an army of recruiters AND interviewers working for you makes achieving SaaS margins impossible, and then, you’ve essentially become a tech-enabled recruiting firm (albeit this time one with much better performance data!).
Just like Hired, Triplebyte eventually moved away from the auction marketplace model, fired most of their talent managers and interviewers, and fulfilled their destiny, becoming a glorified LinkedIn Recruiter clone. Recruiters could search for candidates, just like on LinkedIn Recruiter, based on their pedigree, languages known, and so forth. One thing Triplebyte still does differently, however, is to leverage their aforementioned coding quiz to annotate candidate profiles. (Presumably, they are using their historical interview data, from when candidates had to do BOTH, to predict how people will perform in interviews.) The limitation, of course, is that great people will be unlikely to take the quiz in the first place, especially now that it no longer fast-tracks them to an onsite."
https://blog.alinelerner.com/ive-been-an-engineer-and-a-recr...
“ In short, there is more demand for engineers than there is supply, and that makes engineers powerful.”
This statement is very telling that TB leaders got tunnel vision, a very SV, startup, software centric tunnel vision. Let me rephrase your statement to what you really meant.
In short, there is more demand for “experienced Software” engineers “in startups “ than there is supply “in Silicon Valley”, and that makes engineers “,with pedigree of FAANG or top schools or well connected,” “very” powerful.
Not that there's not room for another player in the space, but it's just a good example of how to do it well.
So unless there is a simple business model shift to charging the candidates, all of their words are empty. THIS is why they made the mistake of making profiles public, because they were thinking about their customer (THE COMPANIES) and not their product (THE CANDIDATES)
"If you're not paying for it, you're not the customer; you're the product being sold." Tim Oreilly Tweet
Sidebar: Take the Quiz!
Maybe they should rollout the update before announcing it?
The biggest problem with engineering job searches is that they're a huge time sink. I'm a senior engineer with Google on my resume, so I don't have much trouble getting in the door or passing interviews, but the process is such a slog. If I were to take PTO for a week and cram my schedule full of job search, I could probably get 3-4 interview processes done per week. That's not a lot, considering that it's good to line up offers and that many people have limited PTO. Even if your PTO is "unlimited", it looks suspicious to be taking random days off for on-sites.
What this means is that I try to keep doing my job and schedule phone screens and recruiter calls in the gaps when I know I won't get meetings. I try not to have to block time on my work calendar for these in case someone wonders "why does Troy suddenly have so many random meetings all of the sudden". The scheduling process with companies is a pain, they ask for 2-3 time windows for the phone screen, then often take a couple days to pick one. So I have huge time blocks reserved for them that I can't give to other companies I am interviewing with. It's stressful to try to slot all this in. Add to that the fact that recruiters want to do everything over the phone (which is very unnatural for me), so I have to also schedule little interruptions for 15 minute "chats" to "check in" and prepare me for the next round. I appreciate these things but at the same time, I'd appreciate fewer of them.
When it comes to the actual interviews, some companies seem to be fine with one phone screen, some need 2-3. Sometimes they are an hour, sometimes 45 minutes. Some companies see that I work at Google and deduce that I might be competent enough to skip the phone screens entirely. I appreciate that.
It's the same with on-sites. Some are a half day, and some are more of a full day. It's not a big difference in practice because I couldn't really take a half day off from 11-3. Many companies have a "no interview day" which is great for their employees but hard to work around, particularly if it falls on "non-suspicious" PTO days like Monday or Friday. Sometimes I wish we could just do several phone screens instead to make scheduling easier.
I'm not sure how to solve this, but it's the biggest pain point by far and if someone could solve it I would absolutely use their platform. I want fewer phone calls, fewer hours of interviewing, and a clear understanding of the time commitment ahead of time.
I also wish companies would be up front about WLB so I don't waste time interviewing, only to learn they work "maybe 50-55 hours a week". That's not a "little more" than 40 guys, that's more than 6 full work days every week. You can't just ask at the outset because some people perceive that as a red flag.
I can't believe this. I must be a garbage engineer then, because I found the US job market for software engineering roles to be absolutely soul crushingly difficult. I am not the only one, my friends did too.
I have given up several times and then restarted job search. I have been grinding leetcode but companies want nothing less than complete perfection.
I actually thought something like triplebyte would be useful, give a test once and have the score be valid for a couple of years, like a GRE for algos and data structures.
I really don't get it.
The "solution" to the big problem might be a 10 page PDF (for reasonable fee or free) to engineers on how to play the game to their own advantage. Of course no one really profits big from this enough to keep a startups lights on. But if all developers played the game well it helps everyone, even the few that don't play it (like a vaccine).
I wonder what 'fad-diet' solution this company I've never heard of is pitching for either (A) getting me an engineer or (B) getting me a job as an engineer. Oh, let me guess, you're going to find those magical engineers who come from non-traditional backgrounds or magically making it possible to both "identify the top 5% of Ruby on Rails developers with no fewer than 2 years of React Devops Cloud Experience"[0].
Going from that, if you RTA -- I was pretty horrified to hear what they thought would be effective in this space. I realize you'll have to take my word for it that there were literal sentences in there spoken as "good ideas" that would have literally had me thinking "oh, so this company's some form of scam" had I happened upon them before they chose to reverse these ideas. I gotta say, public profiles, entrance exams in an attempt to "hack hiring"[2] don't know what this company is. I have to take everyone's word for it that they're somehow big in this space as I haven't looked for a job, nor been responsible for finding viable job candidates[1], so that's probably me.
I was kind-of torn about his apology. It was really un?-professional (I mean that as a compliment but I can't find a way to say 'not corporate-fake' either in 'sorry-not-sorry', 'deflection' or with what appeared to be much concern for what the lawyers have to say on the subject. I don't know that the lawyers would have to say much on the subject but I'm told that this is often the reason why companies manage always manage to communicate "Sorry,... (assholes...)" every time they apologize. There's always that miserable thing, they're going along "We apologize to our customers ..." it's looking good "... for the inconvenience this outage has caused." Seems simple enough, except you've taken no responsibility -- the outage was to your software, or your hardware, running on wires that you pay for which you are supposed to have redundancy for, and all of this stuff is outside of your core competency but there are many companies who can help if you're willing to spend. And I'm sure when the issue happened, there was a key choice that had you spending a large amount of money and me getting a flight home; which you also chose to inconvenience me over.
But Triplebyte wasn't Southwest. I think the author went a little over the top on the apology, almost crossing into the "trying too hard", except that the things he was apologizing for -- basically, being blind to some pretty obvious problems -- were kind of spot on. Please don't think I'm trying to throw some sort of slant compliment; I realize I sort-of sounded like a dick.
There's no malice in that statement; I've been more blind to things I should have know that were right in front of me than a silly business model decision made too quickly. I am certain I have not apologized this clearly/bluntly.
And the thing of it is, if it hadn't been done so well, I probably would have read the first two paragraphs, closed the tab and made a mental note to never touch them -- basically, having them go from "I don't know who they are" to "probably a scam". It's the risk a company takes when their apology gets attention beyond "people who (currently) care". I am not currently hunting, but I'm all around it on the "need to hire someone" side and I have no guarantee I'll keep my current job, forever.
Thinking about it -- I've now read a very long post and written a very long comment about a company that I had never heard of and upon hearing about them, couldn't care less about their services (the space, yes, deeply). They'll probably be the first address I put into the bar if my needs change in the next year or so because I can't name a single other place I'd actually look. This guy might be an evil genius; if I suddenly have a strong desire to change employment in the next three months, I call witchcraft and there'll be pitch forks and hay, or...something. :)
[0] I must have been in a really bad mood because HN has never been a place that promotes that crap to the level of this post, so that should have been my signal that there was something of value here, and I'm glad I stuck it out.
[1] Mostly only interviewing; we have a pretty spectacular individual who has extensive knowledge about hiring for the rather broadly scoped kinds of work that just my tiny team in the 100-or-so company that employs me. I have a process that, though I am always tweaking, the major points are about as perfect a process. I've commented like this on that subject many times in the past if you want to be super bored. :)
[2] I'm confident in my abilities as a software engineer; push comes to shove, there's many, many ways to make enough money to live reasonably in most parts of the US. I've never had problems making money writing software, either directly for an employer or on contract, independently. Neither my wife nor myself have completed a 4-year degree (though I was one semester away; long story), she did a job that does not require a degree -- became a Realtor -- before we met, six-figures in 2003? I think. The most Senior guys on infrastructure pulled that where I worked (they paid a little under average for the role/area but like 1-2%).
i'm not sure about the viability of their "search criteria" though - things like test coverage, release cadence etc are team specific, so it's hard to define it at a company-level, other than maybe small startups
my advice for triplebyte is to partner with leetcode and establish an industry-wide coding certificate, kind of like those SAT or GRE tests that you pass once and apply everywhere.
Take a look at the question they just asked me when I was doing their General Coding Assessment:
What is the output of the following function? (1m 12s)
function foo(a, b) {
a += 1
b.push(1)
}
const a = 0
const b = []
foo(a, b)
console.log(a, b)The only thing that bugs me about that question is that "the output of the following function" is confusingly worded. It's the sort of question that a competent candidate might get nervous about and start to overthink: wait, that isn't a function, it's a code snippet. The function here is foo – so maybe they mean "what's the output of foo"? But then what does "the output of a function" mean? I suppose they mean return value? Is this a trick question to see if I remember what push returns? Damn it, does it return the entire array or just the pushed element?
That's what my mind usually does with questions like that and I'm far from the only one. Since the goal of the test is to screen for basic competence, it ought not to filter out people who could answer the question perfectly fine if it were being asked clearly, but who also perceive corner cases and ambiguities. Such a skill should make you more likely to pass such a test, not less. Therefore the question ought to say something like "What does the following code snippet write to the console?"
The timer is what is bad about this.
I do not care about the question.
As you say, any competent programmer could solve this in a few seconds, if they are proficient, or minutes if they are not.
In my opinion, they should not put a count down because they are interrupting the programmer’s flow. If they really want to measure time, they do it without interrupting the programmer, hide the timer, let them take as much time as they want/need to solve these problems. Then, at the end of the quiz, show them how long it took them to give a proper solution, and take in consideration that time to score them.
If Alice solved 10 problems in 10 minutes, and Bob solved the same problems in 15 minutes, maybe there is something there that is worth highlighting in their profiles. Maybe Alice is able to analyze this type of questions much faster than Bob, and if that matters to the recruiters or potential employers, then allow the candidate to use that in their favor.
However, if Bob actually knew the answer, but they ran out of time to select an option, that seems unfair.
You've found exactly what they are looking for.
Also, it's pretty bad manners to leak test questions imo. You probably agreed not to in the TOS that you accepted.
That's a tricky question, and if you're not good enough to understand why they're asking that then you fail!
a and b are shadowed in the foo function so the function only acts on the shadowed (and copied) variables. Once you return from foo, the const a and b are unchanged.
Not a great question as a pass or fail test imho because if you use shadowed names often enough to be able to parse this code in your head, you’re writing bad code. There is a reason why shadowing names is a bad practice: it’s hard to figure out what the end result is and account for side effects! But as a discussion on all the points above then it would teach the recruiter something about what the candidate knows rather than using this as a trivia question.
At least I assume that's how const works; I've not used const in JS.
There may be some edge cases I'm not covering here, but JavaScript passes primitives by value (strings, numbers, etc) and non-primitives (objects, arrays, etc) by passing a copy of the reference.
What this means is if you reassign the reference inside of the method, it will only affect that scope, because the reference itself is a copy. If you modify the properties of the non-primitive, it will be modified, because the copy of the reference points to the same non-primitive.
Since primitives are passed by value then any modifications are not reflected outside of the method.
To the other poster below, in JavaScript, const with an array (object, etc) simply prevents reassignment of the variable, the array can still be modified.