Key themes in my work

New here? These are topics I write & speak about. Alternatively, browse tags or search.

Machine Learning Systems in Industry

Exploring ML systems in industry and how they're implemented.

• Patterns for LLM Systems: Evals, RAG, fine-tuning, caching, guardrails, defensive UX.
• System Design for RecSys & Search: Offline vs. online, retrieval vs. ranking.
• Real-time Retrieval: Examples from various companies and how to build an MVP.
• Patterns for Personalization: Via bandits, sequences, graphs, and user embeddings.
• Reinforcement Learning for Recsys: Long-term rewards and explore-exploit.
• Bandits for RecSys: Industry examples, warm-start, off-policy evaluation.
• Search Query Matching: Via lexical, graph, and representation learning methods.
• Content Moderation: Collecting labels, data augmentation, cascade pattern, etc.
• Bootstrapping Data Labels: With semi, active, and weakly supervised learning.
• Feature Stores: As a hierarchy of needs (e.g., access, serving, integrity, etc.)
• Data Discovery Platforms: How they help with find data and open source options.

Machine Learning Techniques

Surveys on machine learning methods.

• Evals for Abstractive Summaries: Reference, context, and preference-based metrics.
• Intuition on Attention: Why Q, K, V vectors, multiple heads and layers, etc.
• Counterfactual Evaluation & IPS: Aka the observational vs. interventional problem
• Measuring and Mitigating Position Bias: Accounting for bias in ordered results.
• Measuring Serendipity in Recommendations: Diversity, novelty, unexpectedness, etc.
• A Brief Survey of NLP: From RNN to Word2Vec to Transformer to BERT to T5.
• A Survey of Text-to-Image: Diffusion, conditioning, guidance, and latent space.

Machine Learning & Engineering

Practices at the intersection of ML and engineering.

• LLM Problems & Patterns: External vs. internal LLMs; data vs. non-data patterns.
• ML Design Patterns: Patterns in code & systems such as factory, decorator, proxy, etc.
• ML Design Patterns for Systems: HITL, hard mining, reframe, cascade, flywheel, etc.
• Challenges with ML in Production and A Practical Guide to Overcome them.
• Python Patterns: Lesser seen patterns such as mixins, relative imports, etc.
• ML Testing: Implementation, expected learned behavior, and evaluation metrics.
• Pipeline Testing: Managing brittle tests in data & machine learning pipelines.
• Python Project Setup: With a workflow of checks that run with each  git push.

Weekend Prototypes

I don't get to hack as much as I want, but when I do, they're a ton of fun.

• Obsidian-Copilot: A Prototype Assistant for Writing and Reflecting.
• LLM UXs: Interacting with LLMs with Minimal Chat.
• Raspberry-LLM: Dr. Seuss headlines, HackerNews trolls, etc.
• LLMs to Research, Reflect, and Plan: LLMs, document retrieval, and Discord.
• ApplyingML.com: Ghost knowledge of applying machine learning.
• RecSys with Graphs & NLP: Node2Vec, Gensim word2vec, PyTorch word2vec.
• Baseline RecSys in PyTorch: The humble matrix factorization.
• SortMySkills: Sorting 50 skills to identify your passion and strengths.
• Image Search: Using image embeddings and cosine similarity.
• Image Classification: Transfer learning via Keras, Theano, and AWS.
• Title Classification: Data acquisition, data prep, building a frontend.

Mechanisms for Business, Product, and Tech Teams

Processes and tools for effective projects and teams.

• Mechanisms for ML Projects: Pilot & copilot, literature reviews, and more.
• Mechanisms for technical teams: Weekly debriefs, learning sessions, WBRs, etc.
• The First Rule of Machine Learning: Start without machine learning.
• Weekly 15-5s: Increase visibility and earn trust with this 15-minute habit.
• Scrum for Data Science: Parts of Scrum that fit the data science process well.
• Agile & Data Science: What works and what doesn’t and tweaks to make it work
• What I Do Before, During, and After a data science project for team productivity.

Learning & Career

Practices that worked well for me and general advice.

• Onboarding Effectively: Mindset, 100-day plan, and balancing learning and doing.
• Data Science Red Flags: What to look out for when interviewing with a new team.
• Read Papers: How it helps you be a better data scientist and my approach.
• Influencing without Authority: Socratic method, earning trust, finding advocates, etc.
• Comparing Data/ML Roles: Data/Applied/Research scientist, and ML engineer.
• Georgia Tech OMSCS FAQ: A guide to applying for and completing OMSCS.

Writing Tips

Especially in the context of a career in tech and data.

• Writing Lessons You Don’t Learn at School: And my writing syllabus.
• Writing Online for Non-Writers: First, write for yourself. And other advice.
• Writing Advice from the Pros: Lessons from David Perell and Sahil Lavingia
• Writing for Tech Careers: Why write, my writing process, and writing at work.
• Why, What, How: A simple framework to make your writing more effective.
• Design Docs for ML Systems: Applying Why-What-How to ML design docs.

Featured Talks

Talks that've received the most positive feedback and engagement.

• RecSys Keynote: Is the juice worth the squeeze, for real-time recommendations?
• System Design for RecSys & Search: Candidate retrieval and ranking in industry.
• The Importance of Writing in Tech: Why and how I write, and writing at Amazon.
• OLX Product & Tech Keynote: How Asia’s tech giants scale & the SuperApp strategy.

Ideas & Opinions

Random ideas and unnecessarily strong opinions.

• The 85% Rule: When pushing harder != better outcomes.
• Beginner’s Mind: A mindset for continued learning and growth.
• In Defence of Simplicity: Pushing back on the cult of complexity.
• End-to-End Data Science: Why it works and examples from Stitchfix and Netflix.
• Machine Learning & Life Lessons: Seven things ML taught me about life.
• My Seven Habits: The seven habits that improved my last decade.

Summaries & Notes

Summaries and permanent notes, tidied up for public consumption.

• Uncommon Uses of Python: A few things from the most popular python libraries.
• RecSys 2022: 3 fav papers, 17 summaries, and many lessons.
• RecSys 2021: Simple models, tech stacks, and ops practices.
• RecSys 2020: Bias, sequential models, bandits, robust offline evaluation.
• Spark+AI Summit 2020: Application-agnostic and application specific talks.

Other resources

That are mostly scattered across the internet.

• applied-ml: Papers and tech blogs on real-world machine learning in industry.
• applyingml: Papers, guides, and interviews on how to apply ML effectively.
• open-llms: Open LLMs available for commercial use.
• ml-design-docs: Template of design docs for machine learning systems.
• testing-ml: Examples of implementation & behavioral tests for ML code.
• python-collab-template: Template with tests, type checks, linting, etc.
• papermill-mlflow: Experimentation workflow for machine learning.
• 1-on-1s: Questions to ask during 1-on-1s, from my time as a manager.

172 posts, 25 talks, 13 prototypes, 320,185 words, and countless hours.

Join 5,700+ readers getting updates on machine learning, RecSys, LLMs, and engineering.

Get email updates