Once we recycle our school notebooks and leave our classroom behind, there are no more rubrics. There is rarely a syllabus or a well-written assignment. Instead we are met with vague requests and requirements which grow over time. Scope-creep it is often called. But, sometimes it isn’t scope creep. Sometimes there are questions that no one thought to ask until they saw the next set of data. Or, perhaps, there are opportunities to grow an analysis into a different direction because new information or new strategy is available. As a younger data scientist or analyst, there are many skills you can develop to help you navigate this amorphous quagmire of ongoing project requirements via questions.
Let’s put ourselves into a Demo meeting. And let’s say you carefully built out an algorithm for section A of your company’s portfolio. It’s tuned and measured against a hold-out set in section A. You gathered the data carefully and managed several exceptions and ‘gotchas’. You put together a nice notebook and successfully walked your leader and peers through the content. Now people are starting to ask questions. These mysterious questions are offering you insight into what you should add to your metaphorical homework assignment page. Somehow you need to translate all these questions into tasks that you can tackle and report back on. Let’s walk through a few of the most common questions and what you should take from those questions.
First, if the questions are about the specific details of your project, then you need to address them and know the answers. These often sound like:
- “Did you filter out for exception Y?”
- “Which table did you pull this data from?”
- “Does this include data A or data B as the training set?”
- “This numbers looks high, I’m worried there is a bug somewhere.”
For this kind of question, know that you are probably going to learn something OR your team is going to learn something about you. Maybe you made a mistake because no one told you that data A is missing element Y during that time period. This is nothing personal, there’s a lot to remember and not everyone will remember all the gotchas in the data. Just say something like, “great, I will take a note of that is fix it. Thanks for sharing that information, I hadn’t learned that yet.” On the other hand, maybe your work is great and you can answer every question with the desired answer. In that case, your team learned something about you and your coding. Chances are good that next time you’ll have fewer questions, because it means that your team is beginning to trust your work.
One way or another, you need to know the answers to all of these technical questions and be able to speak intelligently about the work that you did.
Second, there are questions which can (eventually) be expected. In industrial data science, the most common question is “Does this new solution solve a valuable problem?” This question can come in many formats:
- What’s the value proposition of this model?
- How much of the problem is this algorithm going to identify?
- Is the model output precisely aligned with our goals?
Anticipating these questions is an important step in the first few years of industrial work. Which version of this question does your leader like to ask? It’s very likely that your technical leader will ask a different version of the question than your non-technical leader. I encourage you to take your metaphorical pencil and add “value estimation” as a task within the project. Most leaders will not explicitly ask you to do this analysis (Because, of course, they already believe that the thing you are doing is worthwhile or they would not have asked you to do it!). But, the analysis on size of prize is critical to understanding how good your solution is. If you don’t know how big the problem is, then you’ll have no idea when you are done. And you won’t know what to write down on your annual self-review!
Eventually, you’ll want to prepare in advance to be able to include the answer to the value question within your presentation.
Lastly, there are questions which cannot be expected. These are often from your leader who, presumably, has more context than you. Now, let’s go back to that Demo meeting where you talked through your work on training a model for section A. Let’s pretend that the first question your leader asks is, “How well would this algorithm perform on section B?” This question feels off topic and totally random! How DARE my leader not comment on anything that I actually did and only ask me about something that specifically is not in scope! It feels so rude!
A leader will often ask a sideways question when your work is already solid. The “tangential” question actually says several wonderful things about your work. Chiefly, it says that: your leader isn’t concerned about the details of your solution. It means you proved yourself competent and your methods sound. I guarantee that if your work was suspicious then your leader would be quizzing you about your methodology (see above). But, unlike school, where your leader (I mean teacher!) needs to prove that you know what you are doing to grade you, your leader will eventually trust that your work is sound. They don’t need to see all the details to believe that the project was done appropriately. This frees them up to think about additional applications or that conversation with the section B leader who was desperately concerned about the same thing the section A leader is, but whom didn’t bother talking to your leader until yesterday.
So, take heart, no applicable questions is a good thing in this situation. If your leader skips the first two types of questions, I promise your leader isn’t trying to snub you. In fact, it’s the opposite! They trust you so much that they just forgot to compliment you on your solution before diving into the next idea.
Social Mathematics 