by Harj on 2/18/19, 6:04 PM with 511 comments
by drblast on 2/18/19, 9:11 PM
In my email I have an "interview prep packet" from them that essentially tells me to brush up on algorithms and read Cracking the Coding Interview to prepare for their interview process.
I'm fairly happy in my job. If they offered more money or a really interesting project I'd consider working for them. But I'm pretty lazy about redo-ing college algorithms class during my free time at home to go work there, so I probably won't.
There's an opportunity cost with interviews like this where an M.S. and long career of getting shit done counts for very little and memorization of undergrad level topics that you can look up in two minutes in Knuth if you have a problem that requires it can make or break an interview.
I've made a career fixing a ton of horribly shitty, inefficient code that's been produced exclusively by people who pass these interviews.
by bunderbunder on 2/18/19, 7:39 PM
Once upon a time, skill at doing these sorts of problems might have correlated (imperfectly) with general aptitude as a programmer or software engineer. But the very act of trying to leverage that correlation for hiring purposes probably also made it go away. Now you've got a whole lot of people practicing hard on these sorts of problems, spending huge chunks of their free time grinding away on Project Euler and Advent of Code and HackerRank. That muddies the quality of this stuff as a proxy for what it was originally trying to detect: natural aptitude. I'm guessing having time to level grind like that also correlates inversely with other traits that are desirable in a programmer.
by zinclozenge on 2/18/19, 6:30 PM
edit: now if you'll excuse me, I need to do some dynamic programming problems.
by esoterica on 2/18/19, 9:00 PM
The tech industry's preferred form of gatekeeping is asking people to do algorithmic puzzles, which is far tamer and less exclusionary than what other industries do. If you believe that the only thing standing between you and 400k/year is a few dozen hours of practicing leetcode, why are you whining about it instead of taking advantage of the situation to wildly enrich yourself with a fairly modest amount of effort?
by PopeDotNinja on 2/18/19, 7:33 PM
I seem to do OK when an interviewer:
- asks me a question that sounds like a real problem, not a contrived one (although on occasion I'll have fun with a contrived puzzle if the interviewer has a sense of humor & makes the process light hearted)
- doesn't push me down a path that requires me to implement a "simpler" solution I'd never consider (e.g. asking me a question that clearly wants an O(N) solution & then pushes me to try the O(2^n solution first)
- talks like a person with a problem, and not as someone who clearly knows what they want & simply won't say it
- doesn't try to "see how I think", because I code as much in my head as I do on a screen, meaning most of the code I throw into a text editor is the latest thought in a stream of random ideas until I get to one that works
- doesn't constantly interrupt me
- states their actual expectations, such as "I don't expect you to finish, what I'm really looking for is X"
by dcplogic on 2/18/19, 6:26 PM
So, as a courtesy we figured, why not spend a few hours extra with this applicant in the programming test. We set up a laptop with a clean Ubuntu install, devised a programming test that was quite involved. Not algorithmic hard, just more complex than what can normally be done within a 20-minute whiteboard interview. We expected it to take at least 2-3 hours. Google/Stack overflow/etc access was allowed and encouraged. "Just act as like you would normally do when solving a problem."
We spent like 2x4 hours devising this problem, based on our codebase (cutting out something somewhat easily digestible and making it able to run standalone).
It took like one hour to get productive. Explaining the problem, setting up editors, compilers, etc.
We took turns, but most of the time someone in the interview team (of two) sat next to the guy. We did give him some alone time.
This is probably nothing new in terms of interviewing techniques, but to us it was such a revelation. We learned so much more about the applicant. Perhaps it worked well with this guy because he happened to be a bit more outgoing than our typical successful applicant. We'd never felt so confident about giving someone an offer before.
I'm really looking forward towards testing out this approach with local candidates to see if we can replicate this "data gathering success".
by joker3 on 2/18/19, 7:29 PM
There's also a question about how to mix question difficulty. Should you ask nothing but easy questions, or is it good to throw in a harder question or two to see how the candidate reacts to something they can't answer? I can see a good interviewer getting a lot of signal out of that, but in the hands of a bad interviewer it would not work well.
by davidhyde on 2/18/19, 6:41 PM
by Zelphyr on 2/18/19, 7:19 PM
I'd be interested to know how many of these interviewers actually think they're able to identify a solid candidate this way? Not to mention, are they even factoring in how many people don't test well but are otherwise superb software engineers?
Ultimately it seems like there is a soft element to interviewing that is being tossed out now, which is: do I think we can work with this guy/gal? Are they someone that can become part of our team on a personal level? Can they get good work done? Fizz Buzz can't tell you that. What can tell you that is experience. It's a hard-to-put-your-finger-on-it X-factor that I think companies think they can ignore.
by je42 on 2/18/19, 7:35 PM
> how would you explain the with statement to a junior developer ? then increasingly difficult questions that go into the language runtime/concepts.
one other favourite question of mine is:
> Imagine, you got a standard website the serves data from a database. When a customer types in the url into the browser bar what needs to happen until the customer see the website.
> Go as deep as you can in the answering the question.
When you got an answer, you'll see frontend engineers explain more about the browser, while backend engineers talk more about the backend.
There was one very senior engineer, that actually talked about the ethernet layer, he talked for more than 15min. Most medior engineers are done in 5mins. ;)
by acroback on 2/18/19, 8:16 PM
1. A candidate who can solve puzzles but is not willing to do the dirty work with team, solving production issues, doing debugging, bug fixing with usual stuff. 2. Another Candidate, who is willing to learn, is ready to work with team and do the dirty work.
I am on a hiring committee as a Tech Lead, and I always try to weed out 1.
Works great, we hire as interns and then assess them. Someone from Google who we hired full time, was detrimental to team's morale, grunting, complaining about code, complaining about food and what not.
Another experienced smartass was self centered on his skills and didn't want to teach junior engineers anything or willing to admit he needs to update his skills. The moment he realized his skills have no values, started attending pointless conferences. Now his LinkedIn profile has "aware of block chain technology", "attended machine learning seminars".
I said not to interviewing at Google and FB because I don't have cycles to spend months on leetcode. Did I erred? Perhaps. But I am sure neither can provide me same work quality I execute in my current mid size company. I regret nothing :).
by SkyPuncher on 2/18/19, 7:07 PM
1. The problem is pretty well understood (but does offer room for interpretation).
2. Provides time to cover all key aspects (Frontend, Backend, Database, Networking, Debugging, Testing, Caching, etc) in at least some capacity. In particular, it shows you what areas the developers focus on.
3. Provides a more relaxed/realist environment. It's also more accommodating to developers switching stacks - familiar with good programming patterns but not the specifics of stack (e.g. "Here's how I'd do [some specific task] in [other stack]. How do I do it here").
4. It's clearly a throw away task so there's no concern about "interview labor". It can also be pre-prepped so you don't have to worry about jumping too far in.
5. You can cut short with bad candidates and expand the problem for more complex candidates.
by komali2 on 2/18/19, 6:30 PM
I got bit in the ass by this one as triplebyte itself. They asked me to make a tic tac toe game, and gave me iirc 30 minutes (less?) to do it. Except, it wasn't "build a tic tac to" game, first it was "draw a board to the console," "take user input from the console," etc a bunch of instructions in a convoluted path that perhaps another engineer would do when knowing from the outset that the goal was to build a tic tac toe game in 30 minutes, but not me.
So we'd get to a portion where I'd be writing a quick test on user input, or extrapolating something to a function, and the interviewer would say "don't worry about that, just worry about {getting the grid to print to console or whatever}."
Later on I got my feedback and they said they were disappointed with my user input tests and repeated, extractable code in the tic tac toe portion.
Triplebyte is trying to do good things in the interview space but I think they're still learning. All in all my interview with them was about as positive an experience as a harried and bad interview could be, from my perspective.
by wiremine on 2/18/19, 7:50 PM
1. Phone screen which takes 15 or 20 minutes. 2. The candidate fills out an essay, including showing us some code they're proud of. 3. If the essay ticks the boxes we conduct a 1 hour on site interview. We use the same a set of questions for every candidate, so the investment is easy to manage, and our team has a shared set of expectations on what is good or bad. 4. If the interview goes well, we give them a take home assignment. Takes between 2 and 6 hours, depending on how experienced the candidate is. Problem is in C and/or Python (or both) 5. We wrap up with a 2 to 3 hour onsite interview. We walk through the assignment and have a deeper conversation about culture and fit.
The results have been positive for us: we've made some great hires and weeded out some candidates who weren't a good fit.
We've also been able to scale it down to the process we use for interns.
The 1 hour interview has some typical programming interview questions, but we wrap them into a real-world example. The goal isn't to prove they know how to program, but more about allowing them to show us how they think/work out a problem.
by dahart on 2/18/19, 8:51 PM
More important than question difficulty to me is attitude, and I’d love to see whether attitude is measurable and how it compares to later performance, but curiosity and optimism and communication really do go further than right or wrong on math and engineer questions for me. That point might even be tired already, I know people say it all the time, but I’m going to keep saying it because we still have blog posts on question difficulty, when easy vs hard engineering questions are pretty low on my list of what matters when I’m hiring.
by antibland on 2/18/19, 6:42 PM
The other candidates, after answering some of the harder questions incorrectly, seemed very upset with themselves. They knew they were cracking a bit under pressure, but actually showed that they knew the answers when we chatted further. I hired 3/4 of those people because of how well I felt they'd do given the opportunity. All three became leads within a year and a half.
I think personality has a lot to do with outcomes. If you are someone who shows they are hungry to learn and knows how to improve their skills, I will never dismiss you for screwing up a few coding questions.
by paul7986 on 2/18/19, 6:44 PM
by angarg12 on 2/18/19, 8:38 PM
I find it hard to reconcile these two experiences. How can I thrive at a top tech company while failing to solve an 'easy' coding challenge. It makes me concerned about what would be of me if I had to look for a new job now.
by minimaxir on 2/18/19, 6:47 PM
Now, the ads ask simpler things like floating point precision and function variable scoping (https://www.facebook.com/triplebyte/ads/?ref=page_internal ); legit problems, but not sure if they are an indicator of how good a developer they'd be in the real world working on a CRUD app.
by curiousDog on 2/18/19, 6:51 PM
My worst interview ever was with Facebook when a non-native, new college grad gave me a Leetcode hard problem in half-broken english and went back to his work without even looking up or walking with me through the problem.
by aantix on 2/18/19, 7:18 PM
Give the candidate a project with 300,000 loc, tell them to make the most local change possible that fixes the reported bug. Update the tests to reflect the new logic.
Bonus: discuss architectural changes that would have resolved the bug and/or improved performance.
by xiphias2 on 2/18/19, 6:17 PM
The philosophy at Google is that it's better to filter out 3 good engineers than to let in a bad one. The consequence of this is that it's really hard to get kicked out of Google.
The other part (whether it's more important to work on long easier questions to see how the candidate works on a large code base) is orthogonal reasoning, and that part may be true, depending on what type of engineers somebody is looking for.
by taylodl on 2/18/19, 6:45 PM
by Waterluvian on 2/18/19, 8:02 PM
"Right now our roboticists use a hacked together QT based GUI to manage customer robot fleet data. It takes 1-10 minutes to load on a slow network and is hard to add more features to. I know you have far less information than you'd want but walk through your thought process for how you'd replace this system over the next 12 months. We can make assumptions."
And then the next 15 mins can be an organic conversation about the problem space. You can direct the conversation into corners most relevant to their potential role: "you mentioned using web tech because we discussed how all usage is across the internet. Can you talk about the merits of Http vs. websocket?" "How would you ensure that we don't accidentally take every single customer offline if we centralised our data store?" "What kinds of UI technology would lend itself to robot mapping? Can we just use Google Maps?"
If you really need to dig deeper into technical prowess, find something relevant in your conversation and dig deep into it. "We talked about saving changes to floor layout. Can you whiteboard/laptop how you might implement undo/redo for floor elements?"
by wglb on 2/18/19, 6:25 PM
by dmolony on 2/18/19, 10:18 PM
by high_derivative on 2/18/19, 7:50 PM
After receiving an offer from a big tech company, the interviewing process has already completely turned me off from the idea of working there.
Now despite this being a dream position for many and me having no alternative but to take it currently, the smug interviewers have already gotten me in the corporate mind-set: No matter, the reputation and salary, treat it like any other job, do not bother being loyal - they will not be.
So the terrible interview process has at least the advantage of reminding future employees what they are signing up for.
I wonder what kind of psychological filtering is at play. Do employees feel loyalty after an interview process that is best described as hazing? Are they projecting the humiliation they experienced when interviewing future employees? That's always been my impression.
by cshg on 2/18/19, 6:23 PM
by rajeshmr on 2/18/19, 7:47 PM
Also, platforms like hackerrank are adding fuel to fire. I read the CEO write somewhere that he wished the below "were taught in schools :
1) Communicating complex ideas with clarity 2) Systems thinking 3) Grunt work tasks 4) Boundaryless thinking 5) Self-awareness / EQ"
Please note almost all of these are not evaluated on their platform (they profit from coding tests) or during interviews and almost all are soft / intangible skills (skills which are not immediately obvious about the candidate during a typical programming interview). [ Side note : some could say that coding tests are the problem they have chosen to solve - in which case, why are they worried about these skills ? Are the companies seeing coders crack the tests on their platform, while not performing well on the above skills post hiring ? We could only speculate. ]
All good work is done by teams, and to be effective in a team requires a lot of intangibles which aren't even assessed in a typical interview.
A better approach could be from this article: https://leerob.io/blog/technical-recruiting-is-broken/
Or : A couple of weeks of work with a task being assigned and the mentor or interviewer looking at how the candidate is approaching the problem and whether he is able to solve the problem within the time constraints (an easy task shouldn't take long, and a hard problem shouldn't be short circuited to give a sub-optimal solution.) and other such observable traits can be evaluated.
My two cents!
EDIT : Poor wording above (i.e., couple of weeks). A task should be assigned and evaluated post a time (which is ideal for task completion as per the interviewer). No constant interaction with the candidate and spending loads of time with that candidate - that isn't scalable when the demand-supply equation is imbalanced already.
EDIT 2 : The idea above is not about spending weeks for recruitment. The idea was about being practical about the kind of questions / tasks that are given during interview (example : code a feature or fix an issue we have, as another user has suggested well in the comments). Took me a while to realize we have missed the point I tried to convey for the logistics of how it should be done.
by gumby on 2/18/19, 7:48 PM
Questions about the standard library of the programming language in question are good. Questions about the dusty corners of said library: bad.
And don't ask about floating point: most likely the candidate won't really know more than the usual things; anybody who really does understand them will probably give answers over the interviewer's head :-).
by zinckiwi on 2/18/19, 7:39 PM
- A take home (2-3 hours) task for retrieving tabular data from an API and displaying it. Here I'm looking for general framework chops, readability, some design sense.
- An in-person (~45m) not-quite-pair programming task, with a real computer, tools, editor etc., for doing a typical UI operation, e.g. truncating text. Starts simple and gets more complex as time allows: make a function to truncate text to x chars; now add an ellipsis only if truncation occurred; now make sure not to truncate in the middle of a word, etc.
by systematical on 2/18/19, 6:27 PM
It was worded far worse than that. What exactly is that telling you about the engineer?
On the flip side I'm asked to code full fledged applications but not to spend too much time on them... okay...
Another time I was asked to code a luhn algorithm. Oh and do it while a room of people watches you on giant screen cause that's what your day to day job will look like... I failed miserably and still got the job. What?!?!?
by innocentoldguy on 2/18/19, 7:01 PM
I recently interviewed managers at two different technology companies, both nationally known, for a report I am working on. Most managers admitted their technical interviews were flawed, but didn't know any other way to assess skills. They also admitted that a significant number of people they recruit refuse to even take the technical challenge and end up working elsewhere.
In interviewing a couple dozen engineers, I found most just don't want to waste their evenings and weekends on a technical puzzle for a job, especially when there are a lot of companies out there who don't bother with them, so they end up searching for companies that don't waste their time with technical challenges.
Another funny thing I discovered during my research is that just under half of the employees at both companies I've interviewed so far were not able to successfully complete their own technical challenges.
Another problem with technical challenges is that often times the interviewer knows less about the topic than the interviewee. I recently went through the interview process with a local technology company who uses Elixir and Go (both of which I know). During the onsite interview, the interviewer kept saying things like, "Don't forget to..." or "You forgot..." I kept explaining that I didn't need to do as he was suggesting. In the end, my code worked, my tests worked, and I passed the interview. In spite of this, I was rejected because the interviewer, "Wasn't feeling it."
I still have a lot of research to do, but I haven't found anything, so far, that suggests that technical interviews predictably result in top-talent getting hired. It seems to be the same crapshoot interviewing people without using technical challenges is, because in the end, most people decide within the first couple of minutes if they like someone and hire based on that, regardless of the rest of the interview process.
by g9yuayon on 2/18/19, 10:45 PM
It's just unfortunate that there's so much prepping materials online nowadays that the programming puzzles have become ineffective. It gets worse as many interviewers were not good enough to ask follow-up questions. For instance, addition with big integers is a pretty easy interview question, right? But if a candidate can go as deep as this article: https://bearssl.org/bigint.html, I can be pretty confident that the candidate is really really good.
That said, I personally don't find it necessary to join the rat race. Instead, I'd suggest engineers just take time to thoroughly study just one book on algorithm designs. In fact, an introductory book, such as Kleinberg's Algorithm Design or Udi Manber's Introduction to Algorithms, will be good enough. It may not get you into Google, but it will likely get you into another damn good company. The best part of this approach is that passing interview is really just the byproduct of you trying to become a better engineer.
by charliewrites on 2/18/19, 6:05 PM
by weird on 2/18/19, 6:47 PM
by sytelus on 2/19/19, 2:07 AM
Example:
1. Write function that multiplies two integers.
2. What if these numbers were real numbers but computer can only operate on integers? How do we use same number of bytes as ints to hold a real number?
3. What if I wanted infinite precision? What would be run time of your algorithm and storage complexity? (don't insist that candidate must hit the known optimal).
4. Can I have complex numbers as well?
5. Imagine complex numbers not only has "i" but also "j" and "k". How do we handle this?
It is astonishing how many candidates won't be able to move past #2.
The key is to look at how candidate approaches handling complexity, create representations and use it to craft clean solutions. Whether they eventually arrive at known optimal/great answers is unimportant.
by pps43 on 2/18/19, 6:29 PM
by ggggtez on 2/19/19, 4:11 AM
This was the key sentence if anyone missed it. "Optimal" for whom, exactly? Not for FAANG certainly. They don't need to worry about filtering too many candidates out, because they have a nearly infinite pool of applicants, and infinite money to conduct a search.
They can ask as difficult questions as they want, because they can pass hundreds and thousands of qualified candidates, and still have plenty more where that came from.
Edit: If you consider "optimal" to be the expected cost of a hire compared to the expected profit, it is fully plausible that if your margins are big enough, asking hard questions is the most effective way to ensure low false-positives. But as everyone knows, comments on articles about interviews are never about the economics, it's only about human ego of feeling rejected.
by StreamBright on 2/18/19, 7:52 PM
by kylestlb on 2/18/19, 6:57 PM
Switching to more practical, simpler problems allowed me to really observe how they work and solve a coding problem. As the article said, I was also able to add requirements or features to the problem which let me see how the candidate adapts to changing requirements, or refactors their own solution to handle a new edge case. Simpler is generally better if you are timeboxed to 45 minutes.
by duxup on 2/18/19, 8:35 PM
Until one finally gave me a fairly straightforward homework style project. It was probably more apt for a noob and I threw myself into it and submitted my work and explained what it over the phone. I was in the office the next day and we talked about it and I had an offer.
The homework style interviews are understandably controversial, but at least I got to show my work, me doing my work, my thought process, outside of a few moments at a keyboard.
by alboy on 2/18/19, 10:51 PM
by echevil on 2/18/19, 6:59 PM
by austincheney on 2/19/19, 2:45 AM
At one of my jobs I nailed it as the top candidate, by far, of the 79 people they interviewed in person. The interviewers were looking for competence, freedom from frameworks, experience and so forth. I have in this line of work more than 20 years and do it as a hobby so I nailed the interview.
At the job though I worked with a bunch of fresh juniors who only know how to write code the one way they learned in school. According to them I am shitty developer because I didn’t write code in the one way they understand.
Who qualifies the outcome?
by dudul on 2/18/19, 6:54 PM
Now, 13 years later, I mostly rely on "homework" type exercises. I think they address most of the issues. They are more "real world", no time pressure, etc. However, even those now are being heavily criticized. What's left to be used?
by rdiddly on 2/18/19, 8:40 PM
by jorblumesea on 2/18/19, 8:36 PM
Perhaps Facebook needs and wants engineers that can bust out A* on the spot, but I doubt Nordstrom or Starbucks needs that level of talent.
This has changed the field where now to even get an average job at an average company you need to study at the new normal, whose interview ideas were designed to look for the top 1% of the field.
by spullara on 2/18/19, 7:43 PM
by dilyevsky on 2/18/19, 8:25 PM
As I gained more experience (300 interviews and counting, baby) I realized that I pick up more on candidates skills that are not directly related to their performance on particular question. At this point the question itself is just a conversation starter and so it’s better if it’s simpler and more broad because it leaves lots more avenues for the conversation to go.
by graphememes on 2/18/19, 8:18 PM
This shows you way more than what a technical question can answer.
There are edge cases to this. Expertise positions specifically.
by insie5 on 2/18/19, 10:27 PM
by kappi on 2/19/19, 1:34 AM
by wolfgke on 2/18/19, 9:05 PM
I cannot believe that any interview situation is comfortable.
by throwawayinter2 on 2/18/19, 7:55 PM
by harry8 on 2/19/19, 12:53 AM
Questions you can answer immediately simply don't test if you can do this. I have no idea how you would test to see if you can do this.
by ummonk on 2/18/19, 7:02 PM
by crispytx on 2/18/19, 9:13 PM
by dfilppi on 2/18/19, 7:41 PM
by cbau on 2/18/19, 7:45 PM
Would Triplebyte mind sharing the data? I'd love if the numbers could speak for themselves, rather than having to rely on an interpretation of the numbers.
by skybrian on 2/18/19, 6:56 PM
by forrestthewoods on 2/18/19, 8:04 PM
by jamestimmins on 2/18/19, 7:07 PM
Interesting piece!
by RomanPushkin on 2/18/19, 7:20 PM
"Hello, my name is David. I would fail to write bubble sort on a whiteboard"
by otabdeveloper2 on 2/18/19, 8:33 PM
by avatarbl on 2/18/19, 6:51 PM
by drugme on 2/18/19, 6:22 PM
* Rejecting far more candidates than you need to -- so you can feel like you're hiring "the top 1 percent"
* Giving yourself the feeling that you have an objective hiring process (when really you don't)
* Making your own team members feel like they're super brilliant and special when really they're not
That's what the modern hiring process is designed to do. And in fact it works quite well, to serve this purpose.
by gammateam on 2/18/19, 6:32 PM
- Workers are going to be around for 15 months or less and they have domain expertise on 1 stack already and I don't need to screen for how they would hypothetically function across all stacks
- Worker's process and resource finding skills are more indicative of the time they will spend on a task
- Worker's process includes collaborate use of version control and code reviews, if they pass the screening but can't really integrate on these things then thats what will get them booted from the team
It isn't always more expensive to have a not great developer. Look in your organization and see if what I experience is true for you, and you'll save everyone a lot of time.
by crimsonalucard on 2/18/19, 7:15 PM
by james_s_tayler on 2/19/19, 6:05 AM
I'm pretty sure it's either once a week or once every two weeks.