It is nearly impossible to evaluate a full-stack developer, data science candidate, or devops candidate without seeing some code. In-person interviews and other subjective forms of evaluation are inaccurate sources of bias. So, looking at actual work is critical for employers to get the right people in the door.
What do if you are a candidate
Having an actual portfolio can set developers, data science, and devops job-seekers apart from other applicants. However, candidates may not have publicly referenceable work—everything they’ve done is covered by NDA or intellectual property contract clauses. They may also be too entry-level to have any referenceable work at all.
One way for candidates to create a portfolio is to build something useful that they put on GitHub (or whatever replaces it once Microsoft screws it up the way it did Yammer) that employers can look at to set them apart from the other applicants. As a hiring manager, I fast-track people with GitHub-hosted work I can look at.
What to do if you are hiring
As an employer, after you’ve used such public work to screen candidates, you should consider creating a project for the applicants who passed the initial public-code test to complete. Such projects let you evaluate the candidates in your specific context, and it gives the candidates a better sense of the work they’ll do.
Here are some project ideas for evaluating different types of candidates.
Test projects for full-stack and app developers
All the projects should have a clear specification and description and possibly screen diagrams for the UI.
- Granny’s address book: Write a simple application to keep addresses for your grandma. She doesn’t need security (she’s running it on her computer), scalability, or any of the bells and whistles. This application shows that you can write a simple CRUD application to test database, HTML, code organization, and unit test skills. It also determines your native programming language.
- REST stock quotes: Consider having web developer candidates write an application to get stock and financial information from public companies. This project shows if they can consume an API that they don’t directly control and design a simple interface. It also tests code organization and other skills necessary to do a full-stack job. I use this tst myself when evaluate web developercandidates.
- Image search application: For more senior data engineer candidates, consider asking them to create an image search application. This tests if they can use a well-defined API (like Google Vision), have ETL skills, and can deal with complex data issues in a non-RDBMS. You can even use it to test their UI development skills if you ask them to create a front end.
Test projects for data scientists
- Preferences application: Ask data scientist and data science developer candidates to predict user preferences or make recommendations. The idea is to test their familiarity with modern machine learning techniques. Possible data sets include Best Buy, Movie Lens, and really anything with some kind of indirect or direct preference data. They should be able to take any of these datasets and create “recommendations” for the user.
- Game AI: This is another project to test familiarity with data science concepts, including deep learning and rules systems. You can use basic games like chess, but there are open source games as well. (Just make sure your selection has a reasonable scope; Freeciv, for example, is a bit of complicated game whose AI took years to develop.) A challenge is finding an appropriate language and interface. This also tests one of the first problems a developer faces: dealing with someone else’s build environment.
Test projects for devops
For devops candidates, assume all the projects require a script in Ansible, Puppet, Chef, or a similar technology. Accomplishing the project without actual devops tools is not sufficient.
- Host and secure on Amazon Web Services or Google Cloud Engine: Pick an open source application or platform component like Solr, Couchbase, or MariaDB and have the person install it securely in the cloud. This can be done using a “free” or trial node.
- Detect and scale: Give more advanced devops candidates a test script that hits a REST API and generates data. Have them create a script to detect overutilization that will automatically shard and create a second node. Pick something more stateless like an application server for a simpler test or something more like Solr for an advanced test. Morally, you should provide the nodes because this can be a more costly evaluation that the candidates should not bear the charges for.
- Replicate and provide redundancy: Using a similar test script as for the detect-and-scale test, have the candidate also set up WAN replication (aka CDCR, aka “replication”) for disaster recovery. The result should be only the “acceptable” level of data loss inherent in the technology and its latency tolerance.