An edited version of this essay was published in The Batch, a newsletter by DeepLearning.AI. Here’s the original version—which do you prefer?

Machine learning progresses fast. Word2vec (2013) replaced the use of one-hot encoding with word embeddings. Transformers and self-attention (2017) parallelized a previously sequential training process and was adopted across most NLP tasks and even computer vision and recommendation systems. Spark (2014) eclipsed Hadoop as the big data tool of choice, while TensorFlow (2015) and PyTorch (2016) succeeded Theano and Torch.

How do we keep up with the deluge of new papers and tools? Here’s what worked for me as I continue to learn about machine learning (or really, anything in general).

Try something different in each new project. If you typically use pandas for data processing, try Spark or Dask. If you usually rely on decision trees for machine learning, experiment with deep learning. While your exploration may not pay off immediately in the current project, it paves the way towards solving previously unapproachable problems (e.g., big data that can’t fit in memory, machine learning on unstructured data).

Regardless of whether it works, share what you learned with your team so they also benefit from your exploration. A 10-minute demo or short blog post will do. Just be sure to balance the explore-exploit trade-off and time-box yourself, so the project doesn’t get derailed.

Do a personal project that stretches you. It’s a practical way to gain hands-on experience outside of work. It could be learning a new programming language, trying a new framework, or building an app. Pick a project that aligns with your interests. This makes it more fun and thus more likely to finish. Also, good projects are scary—if you’re not worried about failing, it’s probably not challenging enough to provide much learning. Personally, I’m motivated to finish projects where I can help others, learn something new, and have fun along the way.

In addition, pick a pace that’s sustainable for you instead of trying to do as many projects as possible. I tend to do one project a year, completing it over three to six months. This leaves room for vacations and when work or life gets hectic. Track progress via milestones/features and quantifiable output (e.g., lines of code, mistakes made lessons learned). Wrap up your project by sharing your experience via a blog post or at a meetup. This reinforces your learning and as a bonus, adds to your portfolio.

Attend meetups and conferences. You get to hear from, and interact with, people who push the boundaries on research and applying machine learning in industry. I usually find out about such events from my peers and tech Twitter. You can also search on meetup.com. There are also specialized conferences on tools (e.g., Spark Summit) and domains (e.g., RecSys) that tend to be more in-depth. I try to attend one or two conferences a year, and a meetup each month.

After a conference or meetup, consolidate your knowledge by writing about what you learned. (Do you see the pattern around writing here?) For example, here are some great recaps of RecSys 2021. And if you hear of a technique or tool that excites you, make time to apply it at work or in a personal project to take your learning further.

Read papers, consistently. It’s one of the simplest ways to stay up to date and widen your perspective. If you’re not already doing this, conduct a literature review for each new project at work: How have others solved similar problems? What were the challenges and what worked? A few days of lit review prevents unnecessary reinventing of the wheel and saves weeks of effort. Another way to build the habit is to organize a paper reading group with your team. Every two weeks, someone picks a relevant paper to present and discuss. For a start, here’s a list of papers/tech blogs around applying machine learning in industry.

If you’re starting to build the habit, it can be daunting to read scientific papers. I find the classic three-pass approach helpful:

• First pass (5 - 10 minutes): Is the paper relevant to my current work?
• Second pass (1 hour): What is the main thrust of the paper?
• Third pass (4 - 5 hours): How can I virtually re-implement the paper?

Find mentors who are a few steps ahead. Where do you see yourself in 3 years? Focused on analytics, research, or MLOps? Your goal will influence what you learn. I found it helpful to consult people who were 2 - 3 steps ahead: What did you learn and demonstrate to make the transition (e.g., from data analyst to data scientist)? Of the most effective people in this role, what skills/traits do they have? The key is to find people who are just slightly ahead. For example, if you want to switch from data engineering to ML engineering, someone who made the change recently can provide more relevant advice than a CTO or head of ML.

For a start, reach out to one person each month. And if you hit it off with them, ask to have a follow-up chat in a quarter or two where you can share your progress. Consider them your informal mentors. Start with finding mentors within your company—they share the same context and can provide more relevant advice. LinkedIn, being an online resume, is another helpful resource. Also, here are some example questions and responses from mentors in the field of ML.

Finally, adopt a Beginner’s mind. Regardless of our experience and expertise, we can adopt a beginner’s mind by staying curious and constantly trying new things—someone with a beginner’s mind is likely already practicing some combination of the above.

In the beginner’s mind, there are many possibilities, but in the expert’s there are few. —Shunryu Suzuki

Thus, I challenge you to stay a beginner and learn one new thing each week, either by reading a paper, trying a new tool/technique, or hacking a prototype. After a year or two, you’ll be surprised when problems that were previously out of reach are now child’s play.

P.S. What other ways have you found helpful for learning?

Thanks to Yang Xinyi for reading drafts of this.

If you found this useful, please cite this write-up as:

Yan, Ziyou. (Jan 2022). How to Keep Learning about Machine Learning. eugeneyan.com. https://eugeneyan.com/writing/how-to-keep-learning/.

or

@article{yan2022learning,
  title   = {How to Keep Learning about Machine Learning},
  author  = {Yan, Ziyou},
  journal = {eugeneyan.com},
  year    = {2022},
  month   = {Jan},
  url     = {https://eugeneyan.com/writing/how-to-keep-learning/}
}