Writing
- Patterns for Building Cybersecurity Evals
A sandboxed target, inputs that influence task difficulty, tools, and a grader.
- Using LLMs to Secure Source Code
Build a threat model, discover vulnerabilities, verify, triage, and patch.
- How to Work and Compound with AI
Context as infra, taste as config, verify for autonomy, scale via delegation, close the loop.
- 2025 Year in Review
An eventful year of progress in health and career, while making time for travel and reflection.
- Product Evals in Three Simple Steps
Label some data, align LLM-evaluators, and run the eval harness with each change.
- Advice for New Principal Tech ICs (i.e., Notes to Myself)
Based on what I've learned from role models and mentors in Amazon
- Training an LLM-RecSys Hybrid for Steerable Recs with Semantic IDs
An LLM that can converse in English & item IDs, and make recommendations w/o retrieval or tools.
- Evaluating Long-Context Question & Answer Systems
Evaluation metrics, how to build eval datasets, eval methodology, and a review of several benchmarks.
- Exceptional Leadership: Some Qualities, Behaviors, and Styles
What makes a good leader? What do good leaders do? And commando, soldier, and police leadership.
- Building News Agents for Daily News Recaps with MCP, Q, and tmux
Learning to automate simple agentic workflows with Amazon Q CLI, Anthropic MCP, and tmux.
- An LLM-as-Judge Won't Save The Product—Fixing Your Process Will
Applying the scientific method, building via eval-driven development, and monitoring AI output.
- Frequently Asked Questions about My Writing Process
How I started, why I write, who I write for, how I write, and more.
- Improving Recommendation Systems & Search in the Age of LLMs
Model architectures, data generation, training paradigms, and unified frameworks inspired by LLMs.
- Building AI Reading Club: Features & Behind the Scenes
Exploring how an AI-powered reading experience could look like.
- 2024 Year in Review
A peaceful year of steady progress on my craft and health.
- A Spark of the Anti-AI Backlash (on Bluesky)
How the sharing of 1M Bluesky posts uncovered the strong anti-AI sentiment on Bluesky.
- Seemingly Paradoxical Rules of Writing
With regard to writing, there are many rules and also no rules at all.
- How to Run a Weekly Paper Club (and Build a Learning Community)
Benefits of running a weekly paper club, how to start one, and how to read and facilitate papers.
- My Minimal MacBook Pro Setup Guide
Setting up my new MacBook Pro from scratch
- 39 Lessons on Building ML Systems, Scaling, Execution, and More
ML systems, production & scaling, execution & collaboration, building for users, conference etiquette.
- AlignEval: Building an App to Make Evals Easy, Fun, and Automated
Look at and label your data, build and evaluate your LLM-evaluator, and optimize it against your labels.
- Building the Same App Using Various Web Frameworks
FastAPI, FastHTML, Next.js, SvelteKit, and thoughts on how coding assistants influence builders' choices.
- Evaluating the Effectiveness of LLM-Evaluators (aka LLM-as-Judge)
Use cases, techniques, alignment, finetuning, and critiques against LLM-evaluators.
- How to Interview and Hire ML/AI Engineers
What to interview for, how to structure the phone screen, interview loop, and debrief, and a few tips.
- Prompting Fundamentals and How to Apply them Effectively
Structured input/output, prefilling, n-shots prompting, chain-of-thought, reducing hallucinations, etc.
- What We've Learned From A Year of Building with LLMs
From the tactical nuts & bolts to the operational day-to-day to the long-term business strategy.
- Building an AI Coach to Help Tame My Monkey Mind
Building an AI coach with speech-to-text, text-to-speech, an LLM, and a virtual number.
- Task-Specific LLM Evals that Do & Don't Work
Evals for classification, summarization, translation, copyright regurgitation, and toxicity.
- Don't Mock Machine Learning Models In Unit Tests
How unit testing machine learning code differs from typical software practices
- How to Generate and Use Synthetic Data for Finetuning
Overcoming the bottleneck of human annotations in instruction-tuning, preference-tuning, and pretraining.
- Language Modeling Reading List (to Start Your Paper Club)
Some fundamental papers and a one-sentence summary for each; start your own paper club!
- 2023 Year in Review
An expanded charter, lots of writing and speaking, and finally learning to snowboard.
- Push Notifications: What to Push, What Not to Push, and How Often
Sending helpful & engaging pushes, filtering annoying pushes, and finding the frequency sweet spot.
- Out-of-Domain Finetuning to Bootstrap Hallucination Detection
How to use open-source, permissive-use data and collect less labeled samples for our tasks.
- Reflections on AI Engineer Summit 2023
The biggest deployment challenges, backward compatibility, multi-modality, and SF work ethic.
- Evaluation & Hallucination Detection for Abstractive Summaries
Reference, context, and preference-based metrics, self-consistency, and catching hallucinations.
- How to Match LLM Patterns to Problems
Distinguishing problems with external vs. internal LLMs, and data vs non-data patterns
- Patterns for Building LLM-based Systems & Products
Evals, RAG, fine-tuning, caching, guardrails, defensive UX, and collecting user feedback.
- Obsidian-Copilot: An Assistant for Writing & Reflecting
Writing drafts via retrieval-augmented generation. Also reflecting on the week's journal entries.
- Some Intuition on Attention and the Transformer
What's the big deal, intuition on query-key-value vectors, multiple heads, multiple layers, and more.
- Open-LLMs - A list of LLMs for Commercial Use
It started with a question that had no clear answer, and led to eight PRs from the community.
- Interacting with LLMs with Minimal Chat
Should chat be the main UX for LLMs? I don't think so and believe we can do better.
- More Design Patterns For Machine Learning Systems
9 patterns including HITL, hard mining, reframing, cascade, data flywheel, business rules layer, and more.
- Raspberry-LLM - Making My Raspberry Pico a Little Smarter
Generating Dr. Seuss headlines, fake WSJ quotes, HackerNews troll comments, and more.
- Experimenting with LLMs to Research, Reflect, and Plan
Also, shortcomings in document retrieval and how to overcome them with search & recsys techniques.
- LLM-powered Biographies
Asking LLMs to generate biographies to get a sense of how they memorize and regurgitate.
- How to Write Data Labeling/Annotation Guidelines
Writing good instructions to achieve high precision and throughput.
- Content Moderation & Fraud Detection - Patterns in Industry
Collecting ground truth, data augmentation, cascading heuristics and models, and more.
- Mechanisms for Effective Technical Teams
End of week debrief, weekly business review, monthly learning sessions, and quarter review.
- Mechanisms for Effective Machine Learning Projects
Pilot & copilot, literature review, methodology review, and timeboxing.
- Goodbye Roam Research, Hello Obsidian
How to migrate and sync notes & images across devices
- What To Do If Dependency Teams Can’t Help
Seeking first to understand, earning trust, and preparing for away team work.
- 2022 in Review & 2023 Goals
Travelled, wrote, and learned a lot, L5 -> L6, gave a keynote at RecSyS, and started a meetup.
- Autoencoders and Diffusers: A Brief Comparison
A quick overview of variational and denoising autoencoders and comparing them to diffusers.
- Text-to-Image: Diffusion, Text Conditioning, Guidance, Latent Space
The fundamentals of text-to-image generation, relevant papers, and experimenting with DDPM.
- RecSys 2022: Recap, Favorite Papers, and Lessons
My three favorite papers, 17 paper summaries, and ML and non-ML lessons.
- Writing Robust Tests for Data & Machine Learning Pipelines
Or why I should write fewer integration tests.
- Simplicity is An Advantage but Sadly Complexity Sells Better
Pushing back on the cult of complexity.
- Uncommon Uses of Python in Commonly Used Libraries
Some off-the-beaten uses of Python learned from reading libraries.
- Why You Should Write Weekly 15-5s
15 minutes a week to document your work, increase visibility, and earn trust.
- Design Patterns in Machine Learning Code and Systems
Understanding and spotting patterns to use code and components as intended.
- What I Wish I Knew About Onboarding Effectively
Mindset, 100-day plan, and balancing learning and taking action to earn trust.
- Bandits for Recommender Systems
Industry examples, exploration strategies, warm-starting, off-policy evaluation, and more.
- How to Measure and Mitigate Position Bias
Introducing randomness and/or learning from inherent randomness to mitigate position bias.
- Counterfactual Evaluation for Recommendation Systems
Thinking about recsys as interventional vs. observational, and inverse propensity scoring.
- Traversing High-Level Intent and Low-Level Requirements
How they differ and why they work better in different situations.
- Data Science Project Quick-Start
Hard-won lessons on how to start data science projects effectively.
- Mailbag: How to Define a Data Team's Vision and Roadmap
I'm heading into a team lead role and would like to define the vision and roadmap.
- Red Flags to Look Out for When Joining a Data Team
What to consider for in terms of data, roadmap, role, manager, tooling, etc.
- How to Keep Learning about Machine Learning
Beyond getting that starting role, how does one continue growing in the field?
- 2021 Year in Review
Met most of my goals, adopted a puppy, and built ApplyingML.com.
- Informal Mentors Grew into ApplyingML.com!
More than two dozen interviews with ML Practitioners sharing their stories and advice
- 5 Lessons I Learned from Writing Online (Guest post by Susan Shu)
Susan shares 5 lessons she gained from writing online in public over the past year.
- What I Learned from Writing Online - For Fellow Non-Writers
Write before you're ready, write for yourself, quantity over quality, and a few other lessons.
- RecSys 2021 - Papers and Talks to Chew on
Simple baselines, ideas, tech stacks, and packages to try.
- The First Rule of Machine Learning: Start without Machine Learning
Why this is the first rule, some baseline heuristics, and when to move on to machine learning.
- Reinforcement Learning for Recommendations and Search
Focusing on long-term rewards, exploration, and frequently updated item.
- Bootstrapping Labels via ___ Supervision & Human-In-The-Loop
How to generate labels from scratch with semi, active, and weakly supervised learning.
- Mailbag: How to Bootstrap Labels for Relevant Docs in Search
Building semantic search; how to calculate recall when relevant documents are unknown.
- Influencing without Authority for Data Scientists
Show them the data, the Socratic method, earning trust, and more.
- System Design for Recommendations and Search
Breaking it into offline vs. online environments, and candidate retrieval vs. ranking steps.
- Patterns for Personalization in Recommendations and Search
A whirlwind tour of bandits, embedding+MLP, sequences, graph, and user embeddings.
- The Metagame of Applying Machine Learning
How to go from knowing machine learning to applying it at work to drive impact.
- Search: Query Matching via Lexical, Graph, and Embedding Methods
An overview and comparison of the various approaches, with examples from industry search systems.
- My Impostor Syndrome Stories (Guest Post by Susan Shu)
Even high achieving individuals experience impostor syndrome; here's how Susan learned to manage it.
- How to Live with Chronic Imposter Syndrome
More education, achievements, and awards don't shoo away imposter syndrome. Here's what might help.
- Planning Your Career: Values and Superpowers
What do you deeply care about? What do you excel at? Build a career out of that.
- Choosing Problems in Data Science and Machine Learning
Short vs. long-term gain, incremental vs. disruptive innovation, and resume-driven development.
- Seven Habits that Shaped My Last Decade
I wish I started sooner. All have improved my life and several have compounding effects.
- How to Write Design Docs for Machine Learning Systems
Pointers to think through your methodology and implementation, and the review process.
- How to Write Better with The Why, What, How Framework
Three documents I write (one-pager, design doc, after-action review) and how I structure them.
- Feature Stores: A Hierarchy of Needs
Access, serving, integrity, convenience, autopilot; use what you need.
- How to Win a Data Hackathon (Hacklytics 2021)
What the top teams did to win the 36-hour data hackathon. No, not machine learning.
- Growing and Running Your Data Science Team
What I learned about hiring and training, and fostering innovation, discipline, and camaraderie.
- You Don't Really Need Another MOOC
Stop procrastinating, go off the happy path, learn just-in-time, and get your hands dirty.
- Mailbag: How to Get Experienced DS Resume Noticed by Recruiters?
How to increase the chances of getting called up by recruiters?
- Real-time Machine Learning For Recommendations
Why real-time? How have China & US companies built them? How to design & build an MVP?
- 2021 Roadmap: Sharing, Helping, and Living More
A public roadmap to track and share my progress; nothing mission or work-related.
- 2020 Retrospective: New Country, New Role, New Habit
Wrapping up 2020 with writing and site statistics, graphs, and a word cloud.
- Catch the Flying Daggers
A short story on flying daggers and life's challenges.
- How I’m Reflecting on 2020 and Planning for 2021
Time to clear the cache, evaluate existing processes, and start new threads.
- Alexey Grigorev on His Career, Data Science, and Writing
How he switched from engineering to data science, what "senior" means, and how writing helps.
- Mailbag: What's the Architecture for your Blog?
How did you set up your site and what's an easy way to replicate it?
- What Machine Learning Can Teach Us About Life - 7 Lessons
Data cleaning, transfer learning, overfitting, ensembling, and more.
- How to Prevent or Deal with a Data Science Role or Title Mismatch
Interview questions you should ask and how to evolve your job scope.
- Applied / Research Scientist, ML Engineer: What’s the Difference?
A personal take on their deliverables and skills, and what it means for the industry and your team.
- Chip Huyen on Her Career, Writing, and Machine Learning
Setbacks she faced, overcoming them, and how writing changed her life.
- Data Discovery Platforms and Their Open Source Solutions
What questions do they answer? How do they compare? What open-source solutions are available?
- Why I switched from Netlify back to GitHub Pages
DNS server snafus led to email & security issues. Also, limited free build minutes monthly.
- Why Have a Data Science Portfolio and What It Shows
Not 'How to build a data science portfolio', but 'Whys' and 'Whats'.
- How to Install Google Scalable Nearest Neighbors (ScaNN) on Mac
Step-by-step walkthrough on the environment, compilers, and installation for ScaNN.
- How Prototyping Can Help You to Get Buy-In
Building prototypes helped get buy-in when roadmaps & design docs failed.
- Is Writing as Important as Coding?
As careers grow, how does the balance between writing & coding change? Hear from 4 tech leaders.
- RecSys 2020: Takeaways and Notable Papers
Emphasis on bias, more sequential models & bandits, robust offline evaluation, and recsys in the wild.
- Appreciating the Present
What if the alternative was nothingness?
- Routines and Tools to Optimize My Day (Guest Post by Susan Shu)
For years I've refined my routines and found tools to manage my time. Here I share it with readers.
- How to Accomplish More with Less - Useful Tools & Routines
My tools for organization and creation, autopilot routines, and Maker's schedule
- Migrating Site Comments to Utterances
A step-by-step of how to migrate from json comments to Utterances.
- How to Test Machine Learning Code and Systems
Checking for correct implementation, expected learned behaviour, and satisfactory performance.
- Mailbag: Parsing Fields from PDFs—When to Use Machine Learning?
Should I switch from a regex-based to ML-based solution on my application?
- How Reading Papers Helps You Be a More Effective Data Scientist
Why read papers, what papers to read, and how to read them.
- Mailbag: I'm Now a Senior DS—How should I Approach this?
Becoming a senior after three years and dealing with imposter syndrome.
- Embrace Beginner's Mind; Avoid The Wrong Way To Be An Expert
How not to become an expert beginner and to progress through beginner, intermediate, and so on.
- NLP for Supervised Learning - A Brief Survey
Examining the broad strokes of NLP progress and comparing between models
- Unpopular Opinion: Data Scientists Should be More End-to-End
Why (and why not) be more end-to-end, how to, and Stitch Fix and Netflix's experience
- Adding a Checkbox & Download Button to a FastAPI-HTML app
Updating our FastAPI app to let users select options and download results.
- What I Did Not Learn About Writing In School
Surprising lessons I picked up from the best books, essays, and videos on writing non-fiction.
- Georgia Tech's OMSCS FAQ (based on my experience)
Why OMSCS? How can I get accepted? How much time needed? Did it help your career? And more...
- How to Set Up a HTML App with FastAPI, Jinja, Forms & Templates
I couldn't find any guides on serving HTML with FastAPI, thus I wrote this to plug the hole on the internet.
- Why You Need to Follow Up After Your Data Science Project
Ever revisit a project & replicate the results the first time round? Me neither. Thus I adopted these habits.
- What I Do During A Data Science Project To Deliver Success
It's not enough to have a good strategy and plan. Execution is just as important.
- How to Update a GitHub Profile README Automatically
I wanted to add my recent writing to my GitHub Profile README but was too lazy to do manual updates.
- The 85% Rule: When Giving It Your 100% Gets You Less than 85%
I thought giving it my all led to maximum outcomes; then I learnt about the 85% rule.
- My Notes From Spark+AI Summit 2020 (Application-Specific Talks)
Part II of the previous write-up, this time on applications and frameworks of Spark in production
- My Notes From Spark+AI Summit 2020 (Application-Agnostic Talks)
Sharing my notes & practical knowledge from the conference for people who don't have the time.
- How to Set Up a Python Project For Automation and Collaboration
After this article, we'll have a workflow of tests and checks that run automatically with each git push.
- Mailbag: Qns on the Intersection of Data Science and Business
Does DS have business requirements? When does it make sense to split DS and DE??
- Why Are My Airflow Jobs Running “One Day Late”?
A curious discussion made me realize my expert blind spot. And no, Airflow is not late.
- What I Do Before a Data Science Project to Ensure Success
Haste makes waste. Diving into a data science problem may not be the fastest route to getting it done.
- What I Love about Scrum for Data Science
Initially, I didn't like it. But over time, it grew on me. Here's why.
- How to Apply Crocker's Law for Feedback and Growth
Crocker's Law, cognitive dissonance, and how to receive (uncomfortable) feedback better.
- A Practical Guide to Maintaining Machine Learning in Production
Can maintaining machine learning in production be easier? I go through some practical tips.
- 6 Little-Known Challenges After Deploying Machine Learning
I thought deploying machine learning was hard. Then I had to maintain multiple systems in prod.
- How to Write: Advice from David Perell and Sahil Lavingia
An expansion of my Twitter thread that went viral.
- A Hackathon Where the Dinkiest Idea Won. Why?
What I Learnt about evaluating ideas from first-hand participation in a hackathon.
- Serendipity: Accuracy’s Unpopular Best Friend in Recommenders
What I learned about measuring diversity, novelty, surprise, and serendipity from 10+ papers.
- How to Give a Kick-Ass Data Science Talk
Why you should give a talk and some tips from five years of speaking and hosting meet-ups.
- Commando, Soldier, Police and Your Career Choices
Should I join a start-up? Which offer should I accept? A simple metaphor to guide your decisions.
- Stop Taking Regular Notes; Use a Zettelkasten Instead
Using a Zettelkasten helps you make connections between notes, improving learning and memory.
- Writing is Learning: How I Learned an Easier Way to Write
Writing begins before actually writing; it's a cycle of reading -> note-taking -> writing.
- Simpler Experimentation with Jupyter, Papermill, and MLflow
Automate your experimentation workflow to minimize effort and iterate faster.
- My Journey from Psych Grad to Leading Data Science at Lazada
How hard work, many failures, and a bit of luck got me into the field and up the ladder.
- Beating the Baseline Recommender with Graph & NLP in Pytorch
Beating the baseline using Graph & NLP techniques on PyTorch, AUC improvement of ~21% (Part 2 of 2).
- Building a Strong Baseline Recommender in PyTorch, on a Laptop
Building a baseline recsys based on data scraped off Amazon. Warning - Lots of charts! (Part 1 of 2).
- OMSCS CS6200 (Introduction to OS) Review and Tips
OMSCS CS6200 (Introduction to OS) - Moving data from one process to another, multi-threaded.
- OMSCS CS6750 (Human Computer Interaction) Review and Tips
OMSCS CS6750 (Human Computer Interaction) - You are not your user! Or how to build great products.
- Goodbye Wordpress, Hello Jekyll!
Moving off wordpress and hosting for free on GitHub. And gaining full customization!
- OMSCS CS6440 (Intro to Health Informatics) Review and Tips
OMSCS CS6440 (Intro to Health Informatics) - A primer on key tech and standards in healthtech.
- OMSCS CS7646 (Machine Learning for Trading) Review and Tips
OMSCS CS7646 (Machine Learning for Trading) - Don't sell your house to trade algorithmically.
- What does a Data Scientist really do?
No, you don't need a PhD or 10+ years of experience.
- Data Science and Agile (Frameworks for Effectiveness)
Taking the best from agile and modifying it to fit the data science process (Part 2 of 2).
- Data Science and Agile (What Works, and What Doesn't)
A deeper look into the strengths and weaknesses of Agile in Data Science projects (Part 1 of 2).
- OMSCS CS6601 (Artificial Intelligence) Review and Tips
OMSCS CS6601 (Artificial Intelligence) - First, start with the simplest solution, and then add intelligence.
- OMSCS CS6460 (Education Technology) Review and Tips
OMSCS CS6460 (Education Technology) - How to scale education widely through technology.
- OMSCS CS7642 (Reinforcement Learning) Review and Tips
OMSCS CS7642 (Reinforcement Learning) - Landing rockets (fun!) via deep Q-Learning (and its variants).
- Building a Strong Data Science Team Culture
Culture >> Hierarchy, Process, Bureaucracy.
- OMSCS CS7641 (Machine Learning) Review and Tips
OMSCS CS7641 (Machine Learning) - Revisiting the fundamentals and learning new techniques.
- My first 100 days as Data Science Lead
How being a Lead / Manager is different from being an individual contributor.
- OMSCS CS6300 (Software Development Process) Review and Tips
OMSCS CS6300 (Software Development Process) - Java and collaboratively developing an Android app.
- How to get started in Data Science
Tools and skills to pick up, and how to practice them.
- OMSCS CS6476 (Computer Vision) Review and Tips
OMSCS CS6476 Computer Vision - Performing computer vision tasks with ONLY numpy.
- One way to help a data science team innovate successfully
If things are not failing, you're not innovating enough. - Elon Musk
- Product Categorization API Part 3: Creating an API
Or how to put machine learning models into production.
- Image search is now live!
A web app to find similar products based on image.
- Product Classification API Part 2: Data Preparation
Cleaning up text and messing with ascii (urgh!)
- Image classification API is now live!
A simple web app to classify fashion images into Amazon categories.
- I'm going back to school
Got accepted into Georgia Tech's Computer Science Masters!
- SortMySkills is now live!
A card sorting game to discover youl passion by identifying skills you like and dislike.
- Product Classification API Part 1: Data Acquisition
Parsing json and formatting product titles and categories.
- Thoughts on Functional Programming in Scala Course (Coursera)
Learning Scala from Martin Odersky, father of Scala.
- First post!
Time to start writing.
- DataKind Singapore’s Latest Project Accelerator
Guest post of how DataKind SG worked with NGOs to frame their problems and suggests solutions