My wife Sam and I love our morning coffee ritual. Before the 2020 pandemic, we'd drive down to the local cafe and have a chat over a cup of espresso coffee before work. When the first lock-downs started our local cafe became takeaway only. So we adapted to the restrictions but our ritual became more of a hassle. Rushing off to get the coffees and returning back home took up far too much extra time. Looking around for alternatives I decided to try making stovetop coffee using a Bialetti kettle. The first time I tried making our coffees with the Bialetti it took 3 times longer than I expected.
The process involves putting water into a chamber below your ground coffee and screwing your kettle on top. It works by building up enough pressure for water to be forced up through your coffee grounds into the pot above. The first time I tried to make coffee this way I didn't tighten the kettle firmly enough and it took ages for the brewed coffee to start appearing out the top. When I finally had the coffee ready it was almost time for Sam to leave for work. I learnt quickly that the fastest way to make coffee with a Bialetti is to boil your water first, then fill the chamber and make sure the seal is as tight as possible. The next day, after correcting my mistakes I had piping hot and delicious coffees served in plenty of time. Changing our coffee routine at first felt like a bit of a backwards move but it was the feeling of regression that was a clear sign of new opportunity.
In the same way, making mistakes in your move to data science from subject matter expert (SME) can feel like regression. Yet when you start leveraging what you have learned throughout your career with your new data science skills, you soon realise that you have a huge advantage over graduates without this domain knowledge.
Let's find out why it's important to become a "SME-powered" data scientist" in the following 3-parts.
1.Subject matter expertise
2.Creating results that matter
3.Seeing opportunities right in front of you
Your subject matter expertise is your secret weapon
Over time we develop particular skills in the work we do. This might be specialised and involve formal training and certification, credentials and experience. Yet when making a transition into a new career it seems a shame to through all of this experience and skill out the door. Especially if you have invested significant time and capital in acquiring those skills. So don't let these skills go to waste. Instead of starting out from scratch as just another machine learning beginner leverage the power of your subject matter expertise.
But what if you need a change and have grown tired or disillusioned with your current career?
It's true that for some a clean break from the past occupation will be the best choice but you need to make this decision carefully. Maybe it was the location where you were working, a work environment you didn't like or you may just be wanting a change. It's this last desire for change that has likely brought you to consider the field of machine learning and data science in the first place.
It can be a mistake to just drop everything you already know because this is what makes you unique. Instead of walking away from stuff you've been doing over a long time why not use that knowledge and experience in a different way? Leveraging what you already know and understand is the fastest track to making a successful transition into machine learning and data science. Once you have the new machine learning skills in place then you can investigate new topics all you like.
It's a common trap to over-estimate what you can achieve in the short-term but to under-estimate what you can achieve in the long run. For this reason it helps in the short term to leverage what you already know. This will help keep you grounded in what you know and can already do. Use your subject matter expertise on a machine learning project to get the results you know SME's like yourself will care about. Which brings us to part 2 can creating results that matter.
Creating results that only SMEs know and care about can help you stand out
There are many classical machine learning projects available to the beginner. For example, one common machine learning project is to design a system that learns to recognise cats from a series of photographs in a database available for anyone who is interested. This type of learning project might be fun and you will learn useful skills but this can be a mistake if you want to build up your portfolio. Unless you have created some very novel and powerful algorithms (highly unlikely when you are new to data science) the end result may just not be that interesting to anyone else besides your most ardent supporters. Most people can already tell the difference very nicely thanks very much.
So instead, let's consider what you might do if you are a radiologist for example. You might have skills in the interpretation of MRI images that have taken many years to fine tune. Imagine how much more valuable it would be to develop a machine learning algorithm that can automatically provide notes and interpretation that are normally performed by a subject matter expert such as yourself. You may even have access to your own private data source to train your models. This kind of result is only possible if you have access to the knowhow of a SME. As a subject matter expert you are in a unique position of applying your new data science skills to your domain and creating results that can really make a difference. Which brings us to part three, how to identify the opportunities for interesting results by applying what you already know.
Seeing opportunities can be easier in your own domain
It makes practical sense to apply your newly acquired skills in machine learning and data science to your existing field of expertise. For example, if you are currently working in marketing there are certain types of information you have access to and questions you'd like to answer. For instance, how long do new members spend in your forum? What courses do they take? Where did they come from? How long do they remain a member? All of this data typically gets averaged and you might develop some metrics about turnover, churn and lifetime value.
The most useful areas for investigation with a machine learning approach will likely be the very things you have wondered about or struggled with every day in your familiar domains. The power of data science and machine learning is that you can uncover fresh insights and abilities that may rekindle your creativity and interest for your own field of expertise. The key is that you use your subject matter expertise to come up with better questions and the important features to measure.
Here is a summary of the key reasons to become a "SME-powered" data scientist
- We tend to over-estimate how much we can achieve in the short term and this can cause frustration and a sense of angst and fear that we are out of our depth. But this is actually a very good sign that you are stretching your boundaries and learning. Get used to feeling uncomfortable.
- We generally under-estimate how much we can achieve in the long term. After one year of daily practice and learning I feel confident in my ability to efficiently build a machine learning system for regression style problems. Not only in my engineering field of expertise but for a much broader set of problems where data science can help us predict the future and make better decisions today.
- Leveraging your existing subject matter expertise is an essential part of a successful transition into the field of machine learning. Your unique subject matter expertise and experience cannot be easily replicated by other data science graduates.
- As a SME you will already have an uncanny sense of what an important result will look like in your field of expertise. Use this uniqueness as a guidepost as you develop your machine learning skills.
- Identify opportunities amongst the boring and recurring problems you needed to address in your day to day work as a subject matter expert.
Next Steps
We introduced the concept of identifying important feature toward the end of this article. We will be considering a "SME's perspective" on feature engineering in an upcoming article.