HN (Hacker News) Discussion: https://news.ycombinator.com/item?id=16673369
If you do like it, what does it do well? If not, what are you doing to help change it or why is that difficult?
I’m asking this coming fresh out of a recent series of interviews I did. After spending about 5 months studying for today’s software dev interviews (detailed below) and generally despising the process, it makes me think that most people (?) think that the current state of software interviews is sane and that most everyone else is way better at them than I am. I originally wrote this post for my alma mater’s FB computer science group but would like to get more perspectives on this.
This is my attempt at rebooting this thread in perhaps a more constructive manner: (https://news.ycombinator.com/item?id=12667174)
Welcome to the Tech Interview Torture Chamber
This is what ~140 hours of post-work day interview preparation looks like for a mix of 100+ medium and hard LeetCode problems (left stack): https://i.imgur.com/uaa8T5z.jpg
I interviewed at over 10 companies in the period of 2 months. I did over 16 technical screens and 2 take homes. I moved onto on-sites for 70% of these technical screens. I did 4 on-sites and received 2 offers, one of which I accepted.
I have 4 years of work experience in backend and frontend web applications, working mainly in Java and Javascript with a B.S. in CS. I spent 1.5 years at a large company, 2.5 years at a startup / small company. All prep was done after working 45-50 hours a week. I was never great at solving the algorithms and data structures problems in timed situations. The thought of getting stuck while the interviewer interrupts me every 15 seconds with “it can be done in constant time!” still wakes me up in cold sweat at night.
I was asked all sorts of questions that had barely anything to do with my actual day to day experience, often in an arrogant and condescending tone.
Examples of interviewer interactions:
- After I caught my own logical error and corrected it: “OK, now we’re back to *REAL* engineering.” – 3 years of experience at a social networking company
- After asking if the interviewer would like me to finish coding the current problem or move onto the next since we only had 10 minutes left: “Well, as a software engineer, I would expect you to produce code that ACTUALLY runs?” – 1.5 years experience S.E. at a ride sharing company.
I actually solved it and the next problem in the time but I still didn’t make it to the on-site. ¯\_(ツ)_/¯ - “This isn’t the way I would have done it, but I know for sure that there’s something wrong with your solution, even though I can’t find what it is.” – 4 years experience at some startup, a proud MIT alumni. I just smiled and nodded as I retreated into my inner mind and screamed at the top of my lungs.
- After whiffing a problem and asking about my experience: “Did you simply configure this application or did you actually code it?” – 20+ years experience at a social networking company. Yes, I did design and implement the whole thing myself. Don’t be an asshole.
Personal learnings
– Much less anxiety when doing interview problems in front of an interviewer. For me this was the biggest thing. A lot of this comes from realizing that people can ask you anything under the sun and if you get a problem you’ve never seen before, you are probably fucked in the sense that it’s sort of unreasonable to solve it in 25 minutes under extreme pressure.
There are so many other factors that are out of your control including the interviewer’s mood, if they like you, laugh at your jokes, are biased toward sex/race/orientation, experience of the interviewer in interviewing, how much they want to help you or make you struggle, etc.
Uh OK. At this point, there’s more to a job than reversing a binary tree in 5 minutes or applying dynamic programming to word break. Does not figuring this stuff in the allotted time limit out make me a bad developer?
– Learned to “play along” to get through the interview. I always try to reason through a disagreement or gap with the interviewer, but sometimes they are hell bent on getting you to do things their way – even if it’s wrong. If it’s not an important detail, see if you can come back to it later. If you are at an impasse, just nod and use the interviewer’s own language on them.
– Did I actually get better at algorithms and data structures?
I suppose…I think it made me more familiar with some algorithms or data structures I don’t normally use. I think it is enough to be exposed to the concepts, know that they exist, and search them up online. If you can code, you are at least capable of understanding arbitrary implementations given enough time. If anything, I think it protects against the “If you always use a hammer, then everything looks like a nail” type ideology. Now, is it really required for you to do 140 hours of leetcode? I’m not sure, isn’t that what design and code reviews are for?
What the current state of our industry’s hiring process tells us about candidates, from my personal experience:
– You are an algorithms and data structures savant and use all of them on a regular basis.
– You studied a lot of the problems in Cracking the Coding Interview, Elements of Programming Interviews, LeetCode, HackerRank, Project Euler, InterviewBit, etc.
– You lucked out because you have already seen or done max-stack before. Congratulations on your 300k / year salary.
What it doesn’t tell me, having been both an interviewer and an interviewee:
– How does this person think when I don’t interrupt them every 30 seconds to “see how they think” while they write code on on the top left part of the whiteboard without IDE support as I stare at them for 20 minutes straight.
– How does this person get unstuck when they have access to multiple resources, instead of relying solely on the interviewer?
– What kind of quality work does this person produce in a normal work environment where someone isn’t breathing down their neck?
– Can I work with this person on projects that span weeks, months, or years?
Some of these issues can be mitigated by an empathetic and experienced interviewer but I still think these interviews are incredibly stressful.
Look, obviously no interview process is perfect because no test is perfect. Take-homes are extremely time consuming and applicants are dis-incentivized to do them in the interest of time*. They are also extremely risky in the sense of burn out. You can easily spend 12+ hours on a project, get rejected, and have zero feedback provided as to why. Thanks for not paying me for my work, shall I publish your take-home on GitHub? Not saying I would do this, it’s just an incredibly frustrating experience.
[* One way to get around this is to give a HackerRank time-boxed for 3 to 6 hours.]
Making the process better
I still think work samples are the gold standard (echoing the sentiments of tptacek). I know people who are trained and/or naturally good at these algorithmic problems that have expressed that this process is not particularly good – just ask the former CTO of Stripe or anyone else who works there.
3-4 hour work sample problem
My suggestion, which I’ve implemented with good success, is to create 3-4 hour problems that allow a candidate to bring in their own laptop (or rent one), get full access to the internet, access to their favorite IDEs, a set of requirements, and let them sit in a room with 2-3 other interviewers who will work on their own tasks alongside the candidate to simulate a work environment. The candidate can ask questions and treat the engineers as PMs, fellow engineers, managers, etc.
Some examples:
- Implement a service that does X using Y technology, where Y is disclosed to the candidate ahead of time. i.e. Y could be one of {Spring, Kafka, RabbitMQ, Redis, PostgreSQL, MongoDB, Terraform, React, Redux, Webpack, Babel, …}
- Tell the candidate to look at one of several areas of these technologies so you can constrain the problem and also their time for preparation.
- Create a rubric for scoring the problem based on what your team cares most about.
- Dog food your own problem set on your employees. If they can’t pass it under the same conditions as your candidates, re-assess the scope and difficulty of the on-site assignment.
- Gather empirical data on the performance of your employees hired through this process. If you don’t have a good system for figuring out if your employees are needs improvement, meeting expectations (good), exceeding expectations (great), you need to fix that in parallel if not first.
- This process is not set it and forget it! You continue to make new problems and rotate them in and out as necessary when someone eventually posts the “optimal” solution online. Every interview question eventually falls victim to Goodhart’s Law: “When a measure becomes a target, it ceases to be a good measure.”
The Code Review
Show the candidate some of your technical debt. Yes, the nasty garbage code you guys wrote back in the day, not the pristine new microservice you built that connects to the legacy backend anyway. Get the lawyers involved if you need to, or sufficiently distort the original code so proprietary details are obscured. Ask them to code review it by:
- Reading through it and ask clarifying questions.
- Suggest improvements and explain the trade-offs. This should give you a good sense of how good a candidate is at analyzing a problem, making technical decisions, and giving constructive criticism.
- How to evaluate trade-offs: even asking candidates to produce pro con lists for multiple approaches to a problem and forcing them to choose one has been very elucidating.
You must create a scoring rubric and you need to keep rotating the problems in and out. This is not easy and will be an enormous time investment. You have to come up with meaningful on-site projects that will reflect the skills and qualities that YOUR team is actually interested in. Nobody else is going to be able to tell you what to ask your candidates, since YOU have to figure that out because it’s YOUR team.
And this is where I feel like most companies just become completely lazy. “Oh, let’s copy Google/Microsoft/Facebook/Amazon to mimic their success because they hire only the best, we too must hire only the best!” (cue “We only hire the trendiest” https://news.ycombinator.com/item?id=15591441) No, a lot of companies succeed due to a *multitude* of factors. And something that people don’t often tell you is that the biggest factor in success isn’t always the quality of the underlying code, it’s the idea and the sales people. Obviously there is no product without engineering, but you can go a long way running on garbage implementations that are incredibly difficult to refactor or make changes to simply because they *work*. Been there, done that.
If your process actually ends up selecting for ACM championship winners, well that’s fine. That was a conscious decision that your team made because those skills demonstrate a strong, *measurable* correlation with good performance on your team.
Thoughts? Refutations? I’m genuinely interested in how we got to this state of affairs aside from the need to “scale hiring up”.
metalogue 
