by Harj on 5/31/18, 6:00 PM with 60 comments
by KirinDave on 5/31/18, 9:38 PM
On Reddit right now they're running this weird faux-linear-algebra thing where they imply that they can build a vector of your skills vs. a vector of job requirements and get a meaningful answer via the dot product.
Which is a bit like saying you can predict the weather by taking the dot product of a vector of ocean and air. What even are the units? What does any of this even mean?
Hiring is a challenging, multi-dimensional thing. It involves a high-risk and ideally informed decision by multiple parties. Doing it effectively is hard. Doing it effectively and respectfully is harder still. And yet TripleByte comes in and says, "We sound vaguely like machine learning. We got this."
Honestly, they make HackerRank, which was another extremely sketchy organization making a lot of very questionable decisions, look reasonable by comparison.
by yosito on 5/31/18, 9:09 PM
by compumike on 5/31/18, 6:03 PM
But most don't seem to understand that mathematically, there's a tradeoff for a higher false negative rate.
Screening false negatives are people who would have done well in an interview, but don't make it to that stage. These are more hidden to the company, but quite expensive for the hiring process, and painful for people who are wrongly rejected. If we put this in probabilistic terms, I hope we can have a deeper conversation about what's happening and how this issue impacts engineers.
by crispyambulance on 5/31/18, 8:53 PM
Hiring is _intrinsically_ a subjective act.
People are very much a moving target. They change over time. Work experiences, even bad ones, shape one's skills and ability to cope in organizations. Almost everybody has a bad-fit job at one time or another. The experience of a bad-fit is actually important to the growth of the individual and, I think, their coworkers and employers.
by acconrad on 5/31/18, 9:09 PM
1. I did "exceptionally well" but I don't know how many I got right or in what percentile I fell in. Why? Firstly, I'm immediately suspect if I actually did that well. For all I know, the "top percent" could just be anyone in the top 50%. And the top 50% only gets 10 questions right. But more importantly, I already stated in the beginning I was taking it for fun, so why can't I learn what I got wrong so I can improve? I imagine there were areas I got wrong that were repeated, which brings me to my next point...
2. Why so React-centric? I happen to use it but plenty of people use Angular or Vue. You can't expect them to know about the React events lifecycle.
3. Okay, so I don't live in SF or NYC. But I know some of those bigger companies have offices in Boston, where I do live. Why can't you make that work if you already know what companies are in your pipeline and which offices they have? Seems super expensive and wasteful to lose out on a great engineer when you know your list of 200 companies totally has an office not in SF or NYC (e.g. Facebook).
4. Okay, so I want to work remote. Why can't I just take your final Google hangouts exam so that way you have on file "okay cool this person is great, we can fast track this person." If you green light companies who are remote-friendly, you don't have to worry about this issue. Plus isn't TripleByte a YC company, working with other YC companies? I know for a fact that GitLab is also YC and is a remote-friendly company. Not to mention you've advertised for remote engineers! https://news.ycombinator.com/item?id=15066073
I dunno, given that the article is all about touting how effective the 30 question exam is at screening out candidates, you'd think you'd want to do something useful with that quiz instead of locking people out, even if they don't fit your current criteria.
by kofejnik on 5/31/18, 9:38 PM
So, in the end, I've had 4 hangouts sessions over 3 weeks, plus time spent preparing, and was rejected with no feedback at all. I'm still curious, was it the bloom filter?
by asadlionpk on 5/31/18, 10:41 PM
Sadly, this might impress some dumb CEO to use them though.
by burnte on 6/1/18, 1:26 AM
Two years ago I was hiring for a sysadmin. My HR department put my requirements up on Indeed. I got 70 resumes that passed their screening. Of those 70 I found 5 that I wanted to interview, 3 that showed up, and none were hirable. I left a company several months ago, couldn't deal with the management anymore, and the past few months of job searching have been excruciating.
We need more technical people screening resumes and comparing to actual job requirements.
by closed on 5/31/18, 8:17 PM
Really enjoyed the build up from simple cases, to more complex models!
If you're interested in the statistics behind estimating skill, and how well questions tell apart novices from experts, check out item response theory :)
by mlthoughts2018 on 5/31/18, 10:47 PM
The discussions about making these pipelines more quantitative, with assessments and quizzes, always couches it with a tacit assumption that the worst outcome would be to actually hire someone who fails at one of these interviews. Rejecting a good person sucks, as they say, but not as much as hiring one of the multitude of sneaky, low-skilled fakers out there.
And of course, everybody's got their hot new take on how to spot the supposedly huge population of fakers.
What I have learned is two-fold:
(1) That person who aced all your interviews and finally looked like the perfect person to hire probably just spent 3-6 months utterly failing at a bunch of other interviews, just to get into "interview shape," refresh on all the nonsense hazing-style whiteboard trivia about data structures that they had never needed in years at their job, etc. So it's totally asinine to believe that someone passing through all your filters must be the sort of person who would rarely fail some filters. That person almost surely did fail filters, and the companies where they failed believe they dodged a costly false-positive bullet, while you believe you just made an offer to the greatest engineer. Hopefully you can see the myopia here.
(2) The cost of passing up a good-but-failed-at-interview-trivia engineer is often far greater than the cost of hiring them. For one thing, "suboptimal-at-interviews" engineers are pretty damn good engineers, and they can do things that differ from esoteric algorithm trivia, such as helping your business make money. Another thing is that many engineers can generalize what they learn, generalize from example code or templates, etc., very efficiently. So while they might reveal a weakness by failing part of an interview (and everybody has such weaknesses), why do you really care? They can probably become an expert on that weakness topic in a matter of months if they work on it every day, or if you have existing employees who can mentor them.
But the biggest thing is part of what Paul Graham wrote in "Great Hackers": good engineers tend to cluster and want to work with other good engineers.
So if you're sitting there without already having a few good engineers on your team, then most likely, the cost to mistakenly rejecting a great candidate who happened to have a bad day, or a great candidate who happens to hate writing tree algorithms on whiteboards, leaves you running a huge risk of losing out on a good engineer who could help kickstart the phenomenon of getting the next good engineer.
When your team is in this stage, you absolutely can manage with a few "dud" hires who need a lot of help or who have skill gaps in key areas. The cost of adding them to the team and managing their "suboptimality" is far less than the continued search costs brought on by rejecting good candidates with and overly risk averse hiring threshold, and leaving your team in the state of affairs where it still doesn't have a good engineer to help attract more.
In other words, the loss function penalizes false negatives more severely than the combined penalty from effort spent on true negatives and suboptimality / management costs of false positives.
But all these skeezy interview-as-aservice businesses what you to believe that the opposite is true, that if you accidentally hire a "faker" because your hiring process was too easy, then Cthulhu is going to rise out of the sea and lay waste to your company.
Of course they want you to believe that. That's how they make money. Preying on your fears over what would happen if you just unclench and treat candidates like human beings with strengths and flaws and don't hold them up to ludicrous standards that lead to self-selecting macho 22-year-olds getting hired because they just spent 10 months on leetcode.
When you start to realize this, it becomes obvious that onerous code tests, brainless data structure esoterica, hazing-style coding interviews, and especially businesses that offer to outsource that nonsense, like TripleByte, is all just snake oil junk.
by greatamerican on 5/31/18, 9:01 PM