The Software Engineer’s Bar Exam

I follow a bunch of legal blogs, and one of the things they consistently complain about is the content of the bar exam. As Ilya Somin describes it, “the bar exam is primarily a test of memorization that covers huge amounts of material most lawyers won’t need.” As a consequence, time and effort spent preparing for the bar exam do very little to make one a better lawyer.

I’m getting my own taste of what that’s like.

In the near future, I will be looking for a new software development job (why is a story for another day), and everyone is telling me I should prepare for what’s called a “coding interview.” That’s where potential employers give you small programming problems to solve and expect you to write the code for the solutions during the interview. A typical question might be something like,

• Generate a list of all the prime numbers from 1 to 1000.
• Given two strings of letters, figure out the smallest number of characters you’d need to delete from either string to make them anagrams.
• Given an array of integers, how many sets of three integers can you find — in order, but not necessarily consecutive — where each is a specific ratio of the previous one? For example, if the integers are 3, 4, 9, 12, 27, 36 and the specified ratio is 3, there are two sets: (3,9,27) and (4,12,36).

Those are certainly problems you can solve with a short computer program but — as with many of the items not on the bar exam — they’re not really representative of the kinds of tasks most software engineers do every day.

First of all, in most industrial strength applications, the core problem-solving code is only a small part of whole software system, which will also include code for:

• getting input from the user
• validating the input
• storing the data someplace safe
• retrieving the data
• displaying the data to the user
• logging activity for later analysis
• handling errors (and logging them too)
• bulk-importing data
• bulk-exporting results
• querying stored results
• graphing results
• testing individual pieces of the code for correctness
• testing the integrated system for correctness
• building the code into the final product
• deploying the product

…and so on. Even in products that revolve around algorithmic solutions to certain problems, the algorithms themselves are only a small part of the code.

Second, one of the biggest goals of software engineering for the last few decades has been code reuse. This is the idea that we should be able to build software as a collection of reusable components, so we don’t have to reinvent everything from scratch. The industry’s success at this goal is murky, because we really like inventing new things from scratch. Nevertheless, there are also a whole lot of code libraries out there for doing things someone else has figured out how to do.

The C++ language, for example, comes with a Standard Template Library that defines a collection of useful data structures and includes a bunch of algorithms for working on those structures at a very low-level. If you’re looking for somebody else’s solution to a more complex problem, the Boost collection includes over 80 libraries of useful C++ code. Beyond that, there are resources like Faraz Fallahi’s Awesome C++ curated collection of code libraries. Or Erik Rigtorp’s Awesome C++ collection, which takes a different approach.

If you’re not using C++, there are similar collections for every other widely-used modern language. As a professional C#/.NET developer, I can solve a lot of problems with one or more of the 149,000 packages at nuget, and as I teach myself Python, I’m finding solutions amid the 175,000 packages at pypi. Of course, just as most programming is not about writing core problem-solving algorithms, most of those packages are intended to address other programming concerns. Even so, there’s still a lot of canned algoritm code available.

Third, if you can’t find working code in your language of choice, you can still find descriptions of tons of useful algorithms. In college, I learned algorithms from the classic Data Structures and Algorithms by Aho, Ullman, and Hopcroft, which I still have on my bookshelf. (The classic algorithms really haven’t changed much.) Some more modern books are Algorithms by Sedgewick and Wayne, The Algorithm Design Manual by Steven S Skiena, and the comprehensive CLRS book (Introduction to Algorithms by Thomas H. Cormen, Clifford Stein, Ronald L. Rivest, and Charles E. Leiserson.) There are also textbooks focused on algorithms for specific problem domains such as cryptography, bioinformatics, and image processing.

It takes work to translate the books’ algorithm descriptions into working code, but it’s still easier than inventing the algorithms from scratch. You can also find lots of books on algorithms that give examples in a real-world programming language. These are often less-rigorous than the more theoretical books, but they are less likely to have errors because the code can be tested just by running it against test data.

(If you need to find seriously cutting-edge algorithms, you will likely have to search through academic journals, although if you have the kind of job where that’s necessary, then you probably really do need a good understanding of algorithms.)

Fourth, even if you do have one of the minority of software development jobs that requires constructing algorithms on a regular basis, you won’t write the code the way it’s done in a coding interview. In the real world, developers write code in bits and pieces, and they test it as they build it. They start with simple cases and then they handle the edge cases. They move snippets around to make the code faster or cleaner, using performance profiles to study the effects of the changes. They have meetings to clear up requirements, they subject their work to code reviews, and they write documentation.

Finally, professional software development is done with tools to make the job easier, and a modern software developer interacts with them constantly. (I recently realized I had forgotten the format for writing a “for loop” in C# because I have been using the editor’s template to write them for me.) Switching to a text editor, a testing website, or worst of all, a whiteboard, tests the applicant in highly artificial conditions.

Despite all that, employers still do these kinds of interviews, and to be honest, they’re not as useless as I have been making them sound.

Evaluating software development candidates isn’t the easiest thing in the world, so even an imperfect evaluation is better than nothing. Watching how a job applicant works through a software problem will give you an idea of how they think when solving software problems, and stripping away all the real-world issues keeps the interview focused on core problem solving skills. Ultimately, employers want to avoid hiring dumb people, and there aren’t many dumb people who can succeed at this kind of challenge.

Unfortunately, there are a fair number of smart, hard-working developers who aren’t very good at these kinds of problems. Algorithms courses are a standard component of a Computer Science Bachelor’s degree, but I know a number of good software developers who didn’t learn their craft in college, and this sort of interview is worrisome to them, because they fear their hard-won practical knowledge will be of little use.

I actually have two degrees in Computer Science, but I earned them a long time ago, so it’s been many years since I had to solve these kinds of problems. I’m a little worried too.

For better or worse, this situation has lead to a small ecosystem of interview preparation materials. If you think you might be facing a coding interview, the canonical preparation guide is Cracking the Coding Interview by Gayle Laakmann McDowell. This gives an overview of the coding interview process — with specific descriptions of interviewing practices at major software companies like Microsoft, Amazon, Google, Apple, Facebook, and Palantir — and some general guidance to solving these kinds of problems. It also includes 189 questions similar to those that might be on interviews.

I’ve read through sections of that book, but I haven’t tried to answer more than a couple of the example questions. Instead, I’ve been practicing my algorithm skills using a site called HackerRank, where developers can try writing solutions to small programming problems.

I’ve been working on the problems in Visual Studio (Microsoft’s development environment) and then copying the relevant code into HackerRank. Some would argue that I should be working the problems on paper, to simulate the whiteboard experience, but I think my time is best spent reacquainting my brain with solving these kinds of problems, and working in a less efficient way would just slow down the learning process.

The HackerRank site includes an Interview Preparation Kit that collects 69 coding problems, and I’ve worked my way through 50 of them so far. It’s been a very nostalgic experience, since the last time I did so much of this kind of programming was in algorithms classes in college. I’ve kind of missed it, and it’s got me thinking more broadly about the kinds of jobs I might be interested in.

So, that’s what I’ve been up to in my spare time. It’s been keeping me busy enough that I haven’t had time to work on my Model Justice project, although I plan to get back to it one of these days.

And if you happen to need an experienced .NET Senior Software Engineer/Project Lead for a full-time remote position, you know where to find me.