from Hacker News

We can't judge another programmer's abilities in a 60 minute interview

by passenger on 5/2/19, 6:35 PM with 148 comments

  • by andrewstuart on 5/2/19, 9:57 PM

    I am both a recruiter and a highly experienced developer and software designer.

    I once sent one of the very best programmers I know for a job interview.

    I have met a small handful of outstanding programmers in my > 30 years in IT and this guy was in the top five, near the top.

    I had personally worked directly with him for 5 years and he was unquestionably one of the most talented programmers there is, and a nice guy, easy to get along with and great to work with. Incredibly productive, and capable of the most incredible feats of software development.

    The client interviewed him and said ....... no thanks.

    There you go - I have just explained everything you need to know about technical recruiting. Give it some thought - the more you think about it, the more it reveals the deepest truth about YOU and your software hiring processes.

    NO-ONE - no-one ever thinks they make the wrong call when they reject a candidate - including you.

    If you want to know what happened after that - I called the CEO who I knew quite well and essentially said to him "What the fuck are you doing? I sent you one of the best programmers there is - and I know this from first hand personal experience over five years - and your interviewers rejected him?". They employed him because of that and he became their chief technical architect pretty quickly.

  • by rsweeney21 on 5/2/19, 8:06 PM

    Post author here:

    My new company helps companies fill contract software positions. We're a lot like a recruiting or staffing agency, but trying to do it in a new, less spammy way. It's been REALLY frustrating working with hiring managers. I used to be one of these frustrating hiring managers so I guess I had it coming.

    I had a hiring manager tell me that he can tell in the first 15 min of an interview if someone is a great engineer or not. I remember thinking the same thing. I felt like I had this special intuition that I had developed over the years. That's just total BS. There is NO WAY you can know that and you don't have a "special gift".

    I don't have a great solution yet, but for senior engineers I've come to trust previous work experience the candidate has much more than my gut, my culture fit questions and my stupid coding questions. We had a hiring manager that was looking for a strong back end engineer turn down the engineer that built the search backend for yelp because he didn't like his answer to "how would you program the code in an elevator."

    So my new company Facet (www.facetdev.com) is basically built around helping senior engineers, with strong experience building products at scale, find contract work.

  • by MikeTaylor on 5/2/19, 7:11 PM

    Obviously interviewing isn't 100% accurate. Almost nothing is. You definitely can't tell in 60 minutes whether someone will be a success at your company.

    But you often can tell, with very very nearly 100% confidence if someone will not be a success. And that is the purpose of the interview. To filter out the "definitely not"s — the programmers who can't write fizzbuzz. And yes, I have seen plenty of these.

    And interview is not the be-all and end-all of recruiting: of course it isn't. But I'm seeing a pendulum swinging a long way in the other direction now, and a lot of people talking as though interviews are worthless. They're really not.

  • by nocitrek on 5/2/19, 9:25 PM

    I conduct the technical part of interview in my company. There are many candidates which charm our director and HR. They recite the buzzwords flawlessly and their resumes are top notch. When they come to technical part of interview, they charm us as well. Until they start coding. We ask two easy coding tasks (one programming, one data science) and so many candidates have issue with transfering algorithm they conjured on the spot into code; or are unable to solve problem on abstract level (and not hard-code the values for problem given to them). It is not uncommon that we identify sweet talkers which would have negative productivity for the company (meaning we would spend more time/money into fixing their code than they would bring value). I am firm believer that software developers must pass technical part of the interview.

  • by sagitariusrex on 5/2/19, 7:17 PM

  • by 0x445442 on 5/2/19, 9:12 PM

    One of the biggest issues is the industry as a whole operates on a completely different hubris level. I don't think any of my professional references has ever been contacted with the purposes of gaining significant insight into my prior accomplishments or to get information about what it was like to work with me. Nope, all we need is some fancy data structure and algorithm knowledge and a white board to find that great candidate.

    And those that do care about past accomplishments seem to be more interested in public Github repos.

    However, it's my guess there's some sort of inflection point because I doubt Pike or van Rossum did any white board problems when they were hired at Google. My guess is their interviews were much closer to the way the rest of the world, the world outside software development, conducts interviews.

  • by larrik on 5/2/19, 7:11 PM

    It's better to let good/great people slip away then to get a bad apple onto the team. Or worse, a mediocre one who never does anything wrong but also isn't contributing positively either.

  • by movax on 5/2/19, 9:53 PM

    Daniel Kahneman, a Nobel laureate noted for his work in judgment and decision making, stated in an interview earlier this year that interviews are terrible at choosing the best candidates. Unfortunately, he didn’t articulate alternatives.

  • by mjevans on 5/2/19, 9:24 PM

    Maybe making the 'extended interview' process more formal would be a good thing.

    Formally have an internal 'contract' worker process where anyone they might want can be hired in to a trial pool, and projects can bring in the trial worker for small things quickly (maybe writing unit tests, or a fresh set of eyes, etc).

    Every month review performance, make a decision to offer job as X, keep on for another month, or let go.

    As a different part of this, alternating weekends might also be a more focused part of this, let someone that ALREADY has a job have a similar trial process on just some weekends, maybe with a more remote-work focus.

  • by thtthings on 5/2/19, 9:08 PM

    It's more accurate to say that the current format of asking algorithm questions is not the best way to judge programmers ability. All it tests is how much has the programmer memorized for the interview and not really how he codes in the real world.

    But i dont see this changing as most interviewers have been through the system and are likely average programmers themselves.

  • by stratigos on 5/3/19, 10:56 PM

    I am approaching 1.5 decades as an employed software engineer. Interviewing culture seems to change a little over the years, but usually in other wacky directions that leave me feeling entirely disengaged from the industry. I also very much love my vocation, though it alone is not enough to keep me enduring through the insanity of technical interviews , "cultural fit" assessments, and the violence of linear reductionism.

    Articles and discussion threads like this are pretty much the only reason Im willing to interview at all (and thus remain in the industry). So thanks for sharing, Im glad to know other people are aware of how bonkers this is.

    The long version takes years of explaining.

    The short version is: I am highly allergic to cognitive dissonance.

  • by llamataboot on 5/2/19, 8:10 PM

    I'm interested in where the break-even point is. With some interview processes now stretching over 15-20 hours or even more, it seems at some point the process is not helping make a better decision and just demanding extra resources from both the hiring company and the interviewees.

  • by starpilot on 5/2/19, 8:15 PM

    This is why onsite rounds last 4-7 hours instead, while speaking to 5-10 engineers.

  • by bdcravens on 5/2/19, 8:58 PM

    The underlying assumption is that Daniel would have been a great fit. Can this be assumed? Even as a good engineer, his claim to fame is founding the Prime Air team. Would there have been a "What started as an idea over coffee with Gur..." moment at Netflix? Would he have just become a random developer instead? Worth nothing that Daniel actually went from Microsoft to Amazon after not being hired at Netflix, so perhaps he would have similarly churned at Netflix.

    This really isn't your typical "but whiteboard interviews suck!" situation. (not to say that they don't ....)

  • by gumby on 5/2/19, 11:17 PM

    I feel that the best we can do is "does this person look good on reference and/or paper? Then if so, in the phone screen and then interview: is there any reason to believe the stuff on paper might not be true?" (that latter happens a surprising amount).

    Also: "is this person a jerk?" though you have to account for the awkward things someone might accidentally say when they feel they're in a high-pressure situation.

  • by vram22 on 5/3/19, 12:08 AM

    It's interesting that some people find that many so-called "programmers" cannot write FizzBuzz (see links in post, apart from a fun FizzBuzz version in Python):

    FizzBuzz in Python with nested conditional expressions and a generator expression:

    https://jugad2.blogspot.com/2018/12/fizzbuzz-in-python-with-...

  • by julienreszka on 5/3/19, 3:45 AM

    Sometimes I think we would be better off hiring people on the spot at random.

    In France we have a legal maximal 4 months trial period for employees so... This gives enough time I think.

    https://www.service-public.fr/particuliers/vosdroits/F1643

  • by stunt on 5/3/19, 6:27 AM

    It really depends. When you are hiring senior programmers, don’t set the bar very high, then you can filter out majority of bad applications in a 60mins interview. For both hard skills and soft skills.

    Then you need to spend more time for the remaining ones to find the right people.

  • by baconmania on 5/3/19, 3:58 AM

    What's interesting is that most thoughtful people in the industry agree that our conventions around interviewing make for an almost hilariously arbitrary hiring process, but very few large software orgs have actually changed the way they hire.

  • by combatentropy on 5/3/19, 5:03 AM

    Portfolio, portfolio, portfolio.

    I could talk to someone for hours and still misjudge whether I would enjoy coding on the same project with them.

    Show me reams of code that they wrote, and I will know immediately.

    It's too bad that usually isn't possible.

  • by alkonaut on 5/2/19, 9:37 PM

    We can’t. But we also can’t interview people for weeks. So we do one hour and we go with our gut feeling. It’s a terrible way to hire, possibly the worst, except for all the other ones.

  • by EliRivers on 5/2/19, 7:30 PM

    No worries. There's always that probationary period in which either party can recognise it's not a match and either do something about it, or just give it up. How long will it take you to decide? A week? Two weeks? A month?

  • by LeicaLatte on 5/2/19, 7:59 PM

    Of course we can. Its just that we can't do it right.

  • by segmondy on 5/2/19, 8:48 PM

    You can judge programmer's ability in 60 minutes. Can someone program? 60 minutes is way more than enough. How great of a programmer are they? You're going to need more than 60 minutes, unless already had a chance to look at their prior code base.

    What's difficult to judge is, are they organized? are they hard working? how do they handle stress? how's their communication and attitude? do they write great documentation? do they test their code? are they a great team player? how creative are they?

    Due to time constraint of an interview, someone might be more sloppy than usual. Their might be the interview stress which is different from regular work stress, interview stress can impact their communication, etc. But if we focus plainly on technical skills, you can flush that out in 60 minutes.

    I'm a hiring manager, and I have never failed in hiring someone who didn't have the required tech skills. Technical ability I can always hit spot on, the tough one is determination and soft skills. Some of the folks I took chance on, who displayed terrible soft skills during interview turned out to be champs, communicate great, work hard, etc. A few that were very eloquent during interviews were a disappointment, sloppy in development, poor communication in practice, etc.

    People also grow, so someone that was a bad fit/interview for you, could in a few years grow and become great. Potential is also difficult to spot.