Tag:
all
(208)Advice for New Principal Tech ICs (i.e., Notes to Myself)
Based on what I've learned from role models and mentors in Amazon
19 Oct 2025 Ā· 14 min Ā· leadership engineering career
Training an LLM-RecSys Hybrid for Steerable Recs with Semantic IDs
An LLM that can converse in English & item IDs, and make recommendations w/o retrieval or tools.
Evaluating Long-Context Question & Answer Systems
Evaluation metrics, how to build eval datasets, eval methodology, and a review of several benchmarks.
AI Engineer 2025 - Improving RecSys & Search with LLM techniques
Recsys & search are converging with LLMs via semantic IDs, data augmentation, and unified foundation models.
04 Jun 2025 Ā· 1 min Ā· recsys llm engineering production
Exceptional Leadership: Some Qualities, Behaviors, and Styles
What makes a good leader? What do good leaders do? And commando, soldier, and police leadership.
18 May 2025 Ā· 4 min Ā· leadership
Building News Agents for Daily News Recaps with MCP, Q, and tmux
Learning to automate simple agentic workflows with Amazon Q CLI, Anthropic MCP, and tmux.
An LLM-as-Judge Won't Save The ProductāFixing Your Process Will
Applying the scientific method, building via eval-driven development, and monitoring AI output.
20 Apr 2025 Ā· 5 min Ā· eval llm engineering
Frequently Asked Questions about My Writing Process
How I started, why I write, who I write for, how I write, and more.
30 Mar 2025 Ā· 13 min Ā· writing
NVIDIA GTC 2025 - Building LLM-Powered Applications
Chip Huyen and I share what we've learned, best practices, and insights at NVIDIA GTC 2025.
18 Mar 2025 Ā· 1 min Ā· llm engineering production
Improving Recommendation Systems & Search in the Age of LLMs
Model architectures, data generation, training paradigms, and unified frameworks inspired by LLMs.
Building AI Reading Club: Features & Behind the Scenes
Exploring how an AI-powered reading experience could look like.
2024 Year in Review
A peaceful year of steady progress on my craft and health.
22 Dec 2024 Ā· 6 min Ā· life
A Spark of the Anti-AI Butlerian Jihad (on Bluesky)
How the sharing of 1M Bluesky posts uncovered the strong anti-AI sentiment on Bluesky.
Seemingly Paradoxical Rules of Writing
With regard to writing, there are many rules and also no rules at all.
How to Run a Weekly Paper Club (and Build a Learning Community)
Benefits of running a weekly paper club, how to start one, and how to read and facilitate papers.
24 Nov 2024 Ā· 6 min Ā· learning productivity mechanism
My Minimal MacBook Pro Setup Guide
Setting up my new MacBook Pro from scratch
17 Nov 2024 Ā· 5 min Ā· engineering misc
39 Lessons on Building ML Systems, Scaling, Execution, and More
ML systems, production & scaling, execution & collaboration, building for users, conference etiquette.
03 Nov 2024 Ā· 10 min Ā· machinelearning engineering production leadership
AlignEval: Building an App to Make Evals Easy, Fun, and Automated
Look at and label your data, build and evaluate your LLM-evaluator, and optimize it against your labels.
Weights & Biases LLM-Evaluator Hackathon - Hackathon Judge
Being a human judge at the Weights & Biases LLM-as-a-Judge Hackathon
Building the Same App Using Various Web Frameworks
FastAPI, FastHTML, Next.js, SvelteKit, and thoughts on how coding assistants influence builders' choices.
08 Sep 2024 Ā· 8 min Ā· learning engineering python š
Evaluating the Effectiveness of LLM-Evaluators (aka LLM-as-Judge)
Use cases, techniques, alignment, finetuning, and critiques against LLM-evaluators.
18 Aug 2024 Ā· 49 min Ā· llm eval production survey š„
How to Interview and Hire ML/AI Engineers
What to interview for, how to structure the phone screen, interview loop, and debrief, and a few tips.
07 Jul 2024 Ā· 21 min Ā· machinelearning career leadership š„
AI Engineer 2024 Keynote - What We Learned from a Year of LLMs
Special double-feature closing keynote from the 6 authors of the hit O'Reilly article on Applied LLMs.
27 Jun 2024 Ā· 2 min Ā· llm ai engineering production
Netflix PRS 2024 - Applying LLMs to Recommendation Experiences
Challenges and lessons from deploying LLM experiences: evals, scalability, guardrails.
31 May 2024 Ā· 2 min Ā· llm engineering production leadership
Prompting Fundamentals and How to Apply them Effectively
Structured input/output, prefilling, n-shots prompting, chain-of-thought, reducing hallucinations, etc.
26 May 2024 Ā· 17 min Ā· llm production š„
What We've Learned From A Year of Building with LLMs
From the tactical nuts & bolts to the operational day-to-day to the long-term business strategy.
12 May 2024 Ā· 1 min Ā· llm engineering production leadership š„
Building an AI Coach to Help Tame My Monkey Mind
Building an AI coach with speech-to-text, text-to-speech, an LLM, and a virtual number.
Task-Specific LLM Evals that Do & Don't Work
Evals for classification, summarization, translation, copyright regurgitation, and toxicity.
Don't Mock Machine Learning Models In Unit Tests
How unit testing machine learning code differs from typical software practices
25 Feb 2024 Ā· 6 min Ā· machinelearning engineering python
How to Generate and Use Synthetic Data for Finetuning
Overcoming the bottleneck of human annotations in instruction-tuning, preference-tuning, and pretraining.
Language Modeling Reading List (to Start Your Paper Club)
Some fundamental papers and a one-sentence summary for each; start your own paper club!
2023 Year in Review
An expanded charter, lots of writing and speaking, and finally learning to snowboard.
31 Dec 2023 Ā· 8 min Ā· life
Push Notifications: What to Push, What Not to Push, and How Often
Sending helpful & engaging pushes, filtering annoying pushes, and finding the frequency sweet spot.
24 Dec 2023 Ā· 18 min Ā· teardown recsys machinelearning production
Out-of-Domain Finetuning to Bootstrap Hallucination Detection
How to use open-source, permissive-use data and collect less labeled samples for our tasks.
05 Nov 2023 Ā· 12 min Ā· llm eval machinelearning python
Reflections on AI Engineer Summit 2023
The biggest deployment challenges, backward compatibility, multi-modality, and SF work ethic.
AI Engineer 2023 Keynote - Building Blocks for LLM Systems
Evals, retrieval-augmented generation, guardrails, and collecting feedback; all that good stuff.
09 Oct 2023 Ā· 17 min Ā· llm ai engineering production
Evaluation & Hallucination Detection for Abstractive Summaries
Reference, context, and preference-based metrics, self-consistency, and catching hallucinations.
How to Match LLM Patterns to Problems
Distinguishing problems with external vs. internal LLMs, and data vs non-data patterns
13 Aug 2023 Ā· 6 min Ā· llm production
Patterns for Building LLM-based Systems & Products
Evals, RAG, fine-tuning, caching, guardrails, defensive UX, and collecting user feedback.
30 Jul 2023 Ā· 66 min Ā· llm engineering production š„
Obsidian-Copilot: An Assistant for Writing & Reflecting
Writing drafts via retrieval-augmented generation. Also reflecting on the week's journal entries.
11 Jun 2023 Ā· 6 min Ā· llm engineering š
Some Intuition on Attention and the Transformer
What's the big deal, intuition on query-key-value vectors, multiple heads, multiple layers, and more.
21 May 2023 Ā· 8 min Ā· deeplearning llm
Open-LLMs - A list of LLMs for Commercial Use
It started with a question that had no clear answer, and led to eight PRs from the community.
Interacting with LLMs with Minimal Chat
Should chat be the main UX for LLMs? I don't think so and believe we can do better.
More Design Patterns For Machine Learning Systems
9 patterns including HITL, hard mining, reframing, cascade, data flywheel, business rules layer, and more.
23 Apr 2023 Ā· 20 min Ā· machinelearning engineering production recsys
Raspberry-LLM - Making My Raspberry Pico a Little Smarter
Generating Dr. Seuss headlines, fake WSJ quotes, HackerNews troll comments, and more.
Experimenting with LLMs to Research, Reflect, and Plan
Also, shortcomings in document retrieval and how to overcome them with search & recsys techniques.
09 Apr 2023 Ā· 14 min Ā· llm deeplearning learning š š„
LLM-powered Biographies
Asking LLMs to generate biographies to get a sense of how they memorize and regurgitate.
How to Write Data Labeling/Annotation Guidelines
Writing good instructions to achieve high precision and throughput.
12 Mar 2023 Ā· 6 min Ā· machinelearning mechanism survey
Content Moderation & Fraud Detection - Patterns in Industry
Collecting ground truth, data augmentation, cascading heuristics and models, and more.
26 Feb 2023 Ā· 16 min Ā· teardown machinelearning production
Mechanisms for Effective Technical Teams
End of week debrief, weekly business review, monthly learning sessions, and quarter review.
05 Feb 2023 Ā· 7 min Ā· mechanism leadership
Mechanisms for Effective Machine Learning Projects
Pilot & copilot, literature review, methodology review, and timeboxing.
22 Jan 2023 Ā· 7 min Ā· mechanism machinelearning productivity
Goodbye Roam Research, Hello Obsidian
How to migrate and sync notes & images across devices
15 Jan 2023 Ā· 2 min Ā· til productivity
What To Do If Dependency Teams Canāt Help
Seeking first to understand, earning trust, and preparing for away team work.
08 Jan 2023 Ā· 3 min Ā· leadership misc
2022 in Review & 2023 Goals
Travelled, wrote, and learned a lot, L5 -> L6, gave a keynote at RecSyS, and started a meetup.
24 Dec 2022 Ā· 8 min Ā· life
Autoencoders and Diffusers: A Brief Comparison
A quick overview of variational and denoising autoencoders and comparing them to diffusers.
11 Dec 2022 Ā· 3 min Ā· deeplearning
Text-to-Image: Diffusion, Text Conditioning, Guidance, Latent Space
The fundamentals of text-to-image generation, relevant papers, and experimenting with DDPM.
27 Nov 2022 Ā· 19 min Ā· deeplearning llm survey
RecSys 2022: Recap, Favorite Papers, and Lessons
My three favorite papers, 17 paper summaries, and ML and non-ML lessons.
02 Oct 2022 Ā· 14 min Ā· recsys engineering production
RecSys 2022 Keynote - Is the Juice Worth the Squeeze?
Invited keynote at the Workshop for Online Recommender Systems and User Modeling (ORSUM)
23 Sep 2022 Ā· 2 min Ā· recsys engineering production
Writing Robust Tests for Data & Machine Learning Pipelines
Or why I should write fewer integration tests.
04 Sep 2022 Ā· 19 min Ā· engineering machinelearning production š©·
Simplicity is An Advantage but Sadly Complexity Sells Better
Pushing back on the cult of complexity.
14 Aug 2022 Ā· 10 min Ā· machinelearning engineering production š„
Uncommon Uses of Python in Commonly Used Libraries
Some off-the-beaten uses of Python learned from reading libraries.
31 Jul 2022 Ā· 10 min Ā· python engineering š„
Why You Should Write Weekly 15-5s
15 minutes a week to document your work, increase visibility, and earn trust.
26 Jun 2022 Ā· 4 min Ā· mechanism productivity career
Design Patterns in Machine Learning Code and Systems
Understanding and spotting patterns to use code and components as intended.
12 Jun 2022 Ā· 13 min Ā· machinelearning engineering python š„
What I Wish I Knew About Onboarding Effectively
Mindset, 100-day plan, and balancing learning and taking action to earn trust.
Bandits for Recommender Systems
Industry examples, exploration strategies, warm-starting, off-policy evaluation, and more.
08 May 2022 Ā· 14 min Ā· teardown recsys machinelearning
How to Measure and Mitigate Position Bias
Introducing randomness and/or learning from inherent randomness to mitigate position bias.
17 Apr 2022 Ā· 7 min Ā· recsys
Counterfactual Evaluation for Recommendation Systems
Thinking about recsys as interventional vs. observational, and inverse propensity scoring.
10 Apr 2022 Ā· 8 min Ā· recsys eval machinelearning
Traversing High-Level Intent and Low-Level Requirements
How they differ and why they work better in different situations.
20 Mar 2022 Ā· 7 min Ā· engineering productivity misc
Data Science Project Quick-Start
Hard-won lessons on how to start data science projects effectively.
06 Mar 2022 Ā· 7 min Ā· datascience engineering productivity
Mailbag: How to Define a Data Team's Vision and Roadmap
I'm heading into a team lead role and would like to define the vision and roadmap.
18 Feb 2022 Ā· 3 min Ā· leadership datascience š¬
Red Flags to Look Out for When Joining a Data Team
What to consider for in terms of data, roadmap, role, manager, tooling, etc.
13 Feb 2022 Ā· 8 min Ā· datascience machinelearning career
How to Keep Learning about Machine Learning
Beyond getting that starting role, how does one continue growing in the field?
19 Jan 2022 Ā· 6 min Ā· learning career machinelearning
The Data Scientist Show - Building end-to-end ML systems
Daliana and I had a 2hr chat on all things data science and machine learning.
02 Dec 2021 Ā· 1 min Ā· datascience machinelearning career
2021 Year in Review
Met most of my goals, adopted a puppy, and built ApplyingML.com.
28 Nov 2021 Ā· 7 min Ā· productivity life
Informal Mentors Grew into ApplyingML.com!
More than two dozen interviews with ML Practitioners sharing their stories and advice
25 Nov 2021 Ā· 1 min Ā· machinelearning career š
5 Lessons I Learned from Writing Online (Guest post by Susan Shu)
Susan shares 5 lessons she gained from writing online in public over the past year.
07 Nov 2021 Ā· 6 min Ā· writing
What I Learned from Writing Online - For Fellow Non-Writers
Write before you're ready, write for yourself, quantity over quality, and a few other lessons.
17 Oct 2021 Ā· 7 min Ā· writing
RecSys 2021 - Papers and Talks to Chew on
Simple baselines, ideas, tech stacks, and packages to try.
03 Oct 2021 Ā· 5 min Ā· recsys deeplearning production survey
The First Rule of Machine Learning: Start without Machine Learning
Why this is the first rule, some baseline heuristics, and when to move on to machine learning.
19 Sep 2021 Ā· 8 min Ā· machinelearning š„
MLOps Community - System Design for RecSys & Search
An overview of system design, candidate retrieval, and ranking, with industry examples.
15 Sep 2021 Ā· 1 min Ā· recsys machinelearning production
Reinforcement Learning for Recommendations and Search
Focusing on long-term rewards, exploration, and frequently updated item.
05 Sep 2021 Ā· 13 min Ā· teardown recsys deeplearning
Amazon Science - Eugene Yan and the Art of Writing about Science
Why the Amazon applied scientist takes the time to break down his work for readers.
02 Aug 2021 Ā· 6 min Ā· writing
Bootstrapping Labels via ___ Supervision & Human-In-The-Loop
How to generate labels from scratch with semi, active, and weakly supervised learning.
01 Aug 2021 Ā· 12 min Ā· teardown machinelearning
Mailbag: How to Bootstrap Labels for Relevant Docs in Search
Building semantic search; how to calculate recall when relevant documents are unknown.
20 Jul 2021 Ā· 1 min Ā· machinelearning š¬
SF Big Analytics - System Design for RecSys & Search
Why real-time RecSys? What does the system design look like in industry? How to build an MVP?
13 Jul 2021 Ā· 1 min Ā· recsys machinelearning production
Influencing without Authority for Data Scientists
Show them the data, the Socratic method, earning trust, and more.
04 Jul 2021 Ā· 9 min Ā· leadership
System Design for Recommendations and Search
Breaking it into offline vs. online environments, and candidate retrieval vs. ranking steps.
27 Jun 2021 Ā· 13 min Ā· teardown production engineering recsys š„
Patterns for Personalization in Recommendations and Search
A whirlwind tour of bandits, embedding+MLP, sequences, graph, and user embeddings.
13 Jun 2021 Ā· 25 min Ā· teardown recsys machinelearning deeplearning
Towards Data Science - Author Spotlight with Eugene Yan
My favourite project, how I write weekly and how you can start, and content I would like to see more of.
02 Jun 2021 Ā· 11 min Ā· career writing datascience
The Metagame of Applying Machine Learning
How to go from knowing machine learning to applying it at work to drive impact.
02 May 2021 Ā· 12 min Ā· machinelearning career š©·
Search: Query Matching via Lexical, Graph, and Embedding Methods
An overview and comparison of the various approaches, with examples from industry search systems.
25 Apr 2021 Ā· 21 min Ā· teardown machinelearning production š„
My Impostor Syndrome Stories (Guest Post by Susan Shu)
Even high achieving individuals experience impostor syndrome; here's how Susan learned to manage it.
How to Live with Chronic Imposter Syndrome
More education, achievements, and awards don't shoo away imposter syndrome. Here's what might help.
Planning Your Career: Values and Superpowers
What do you deeply care about? What do you excel at? Build a career out of that.
Bukalapak - Fireside Chat with the Data Science team
We discussed about how to build and run data teams and engage better with business.
28 Mar 2021 Ā· 10 min Ā· datascience leadership
TalkPython - What ML can Teach Us About Life
Mike and I take a philosophical detour on Talk Python and discuss life lessons from machine learning.
26 Mar 2021 Ā· 1 min Ā· machinelearning life
Choosing Problems in Data Science and Machine Learning
Short vs. long-term gain, incremental vs. disruptive innovation, and resume-driven development.
21 Mar 2021 Ā· 12 min Ā· datascience machinelearning leadership
Seven Habits that Shaped My Last Decade
I wish I started sooner. All have improved my life and several have compounding effects.
How to Write Design Docs for Machine Learning Systems
Pointers to think through your methodology and implementation, and the review process.
07 Mar 2021 Ā· 15 min Ā· writing machinelearning engineering
How to Write Better with The Why, What, How Framework
Three documents I write (one-pager, design doc, after-action review) and how I structure them.
28 Feb 2021 Ā· 10 min Ā· writing engineering productivity š©· š„
Feature Stores: A Hierarchy of Needs
Access, serving, integrity, convenience, autopilot; use what you need.
21 Feb 2021 Ā· 19 min Ā· teardown machinelearning engineering š„
How to Win a Data Hackathon (Hacklytics 2021)
What the top teams did to win the 36-hour data hackathon. No, not machine learning.
14 Feb 2021 Ā· 6 min Ā· datascience engineering misc
DataTalksClub - Building an ML System; Behind the Scenes
Design and architecture, tech stack, methodology, results, and lessons learned.
07 Feb 2021 Ā· 5 min Ā· machinelearning production
Growing and Running Your Data Science Team
What I learned about hiring and training, and fostering innovation, discipline, and camaraderie.
31 Jan 2021 Ā· 16 min Ā· leadership datascience
You Don't Really Need Another MOOC
Stop procrastinating, go off the happy path, learn just-in-time, and get your hands dirty.
24 Jan 2021 Ā· 9 min Ā· learning
DataTalksClub - The Importance Of Writing In A Tech Career
Why did I start writing? What's my writing process? What's the writing culture at Amazon like?
Mailbag: How to Get Experienced DS Resume Noticed by Recruiters?
How to increase the chances of getting called up by recruiters?
16 Jan 2021 Ā· 5 min Ā· career datascience š¬
Real-time Machine Learning For Recommendations
Why real-time? How have China & US companies built them? How to design & build an MVP?
10 Jan 2021 Ā· 21 min Ā· teardown machinelearning recsys production š„
2021 Roadmap: Sharing, Helping, and Living More
A public roadmap to track and share my progress; nothing mission or work-related.
03 Jan 2021 Ā· 4 min Ā· productivity life
2020 Retrospective: New Country, New Role, New Habit
Wrapping up 2020 with writing and site statistics, graphs, and a word cloud.
20 Dec 2020 Ā· 8 min Ā· productivity life
Catch the Flying Daggers
A short story on flying daggers and life's challenges.
How Iām Reflecting on 2020 and Planning for 2021
Time to clear the cache, evaluate existing processes, and start new threads.
06 Dec 2020 Ā· 4 min Ā· productivity life
Alexey Grigorev on His Career, Data Science, and Writing
How he switched from engineering to data science, what "senior" means, and how writing helps.
Mailbag: What's the Architecture for your Blog?
How did you set up your site and what's an easy way to replicate it?
What Machine Learning Can Teach Us About Life - 7 Lessons
Data cleaning, transfer learning, overfitting, ensembling, and more.
22 Nov 2020 Ā· 11 min Ā· machinelearning life
How to Prevent or Deal with a Data Science Role or Title Mismatch
Interview questions you should ask and how to evolve your job scope.
15 Nov 2020 Ā· 8 min Ā· datascience career
Applied / Research Scientist, ML Engineer: Whatās the Difference?
A personal take on their deliverables and skills, and what it means for the industry and your team.
08 Nov 2020 Ā· 11 min Ā· datascience machinelearning engineering career
Chip Huyen on Her Career, Writing, and Machine Learning
Setbacks she faced, overcoming them, and how writing changed her life.
01 Nov 2020 Ā· 11 min Ā· career machinelearning writing
Data Discovery Platforms and Their Open Source Solutions
What questions do they answer? How do they compare? What open-source solutions are available?
25 Oct 2020 Ā· 16 min Ā· teardown datascience engineering
Why I switched from Netlify back to GitHub Pages
DNS server snafus led to email & security issues. Also, limited free build minutes monthly.
21 Oct 2020 Ā· 3 min Ā· misc
Why Have a Data Science Portfolio and What It Shows
Not 'How to build a data science portfolio', but 'Whys' and 'Whats'.
18 Oct 2020 Ā· 15 min Ā· datascience learning career
How to Install Google Scalable Nearest Neighbors (ScaNN) on Mac
Step-by-step walkthrough on the environment, compilers, and installation for ScaNN.
14 Oct 2020 Ā· 3 min Ā· python machinelearning til
How Prototyping Can Help You to Get Buy-In
Building prototypes helped get buy-in when roadmaps & design docs failed.
11 Oct 2020 Ā· 7 min Ā· productivity
Is Writing as Important as Coding?
As careers grow, how does the balance between writing & coding change? Hear from 4 tech leaders.
04 Oct 2020 Ā· 13 min Ā· writing leadership
RecSys 2020: Takeaways and Notable Papers
Emphasis on bias, more sequential models & bandits, robust offline evaluation, and recsys in the wild.
27 Sep 2020 Ā· 16 min Ā· recsys deeplearning production survey
CareerFair - Day-to-day as an Applied Scientist at Amazon
What's an average day like? What's great about the role? How's working in Amazon?
21 Sep 2020 Ā· 8 min Ā· datascience career
Routines and Tools to Optimize My Day (Guest Post by Susan Shu)
For years I've refined my routines and found tools to manage my time. Here I share it with readers.
20 Sep 2020 Ā· 16 min Ā· productivity
How to Accomplish More with Less - Useful Tools & Routines
My tools for organization and creation, autopilot routines, and Maker's schedule
13 Sep 2020 Ā· 11 min Ā· productivity
Migrating Site Comments to Utterances
A step-by-step of how to migrate from json comments to Utterances.
How to Test Machine Learning Code and Systems
Checking for correct implementation, expected learned behaviour, and satisfactory performance.
06 Sep 2020 Ā· 14 min Ā· machinelearning engineering python
Mailbag: Parsing Fields from PDFsāWhen to Use Machine Learning?
Should I switch from a regex-based to ML-based solution on my application?
04 Sep 2020 Ā· 4 min Ā· machinelearning š¬
Datacast Podcast - Effective Data Science with Eugene Yan
My chat with James Le about my experience, leadership, agile, ML in production, writing, and more.
03 Sep 2020 Ā· 17 min Ā· datascience leadership career lazada
How Reading Papers Helps You Be a More Effective Data Scientist
Why read papers, what papers to read, and how to read them.
30 Aug 2020 Ā· 6 min Ā· learning
Mailbag: I'm Now a Senior DSāHow should I Approach this?
Becoming a senior after three years and dealing with imposter syndrome.
27 Aug 2020 Ā· 2 min Ā· career datascience š¬
Embrace Beginner's Mind; Avoid The Wrong Way To Be An Expert
How not to become an expert beginner and to progress through beginner, intermediate, and so on.
23 Aug 2020 Ā· 11 min Ā· learning
NLP for Supervised Learning - A Brief Survey
Examining the broad strokes of NLP progress and comparing between models
16 Aug 2020 Ā· 23 min Ā· llm deeplearning survey
Unpopular Opinion: Data Scientists Should be More End-to-End
Why (and why not) be more end-to-end, how to, and Stitch Fix and Netflix's experience
09 Aug 2020 Ā· 17 min Ā· datascience machinelearning leadership š„
Adding a Checkbox & Download Button to a FastAPI-HTML app
Updating our FastAPI app to let users select options and download results.
05 Aug 2020 Ā· 3 min Ā· engineering python til
What I Did Not Learn About Writing In School
Surprising lessons I picked up from the best books, essays, and videos on writing non-fiction.
Georgia Tech's OMSCS FAQ (based on my experience)
Why OMSCS? How can I get accepted? How much time needed? Did it help your career? And more...
How to Set Up a HTML App with FastAPI, Jinja, Forms & Templates
I couldn't find any guides on serving HTML with FastAPI, thus I wrote this to plug the hole on the internet.
23 Jul 2020 Ā· 3 min Ā· engineering python til
Why You Need to Follow Up After Your Data Science Project
Ever revisit a project & replicate the results the first time round? Me neither. Thus I adopted these habits.
19 Jul 2020 Ā· 12 min Ā· mechanism datascience productivity
What I Do During A Data Science Project To Deliver Success
It's not enough to have a good strategy and plan. Execution is just as important.
12 Jul 2020 Ā· 7 min Ā· mechanism datascience productivity
How to Update a GitHub Profile README Automatically
I wanted to add my recent writing to my GitHub Profile README but was too lazy to do manual updates.
11 Jul 2020 Ā· 3 min Ā· engineering python til
The 85% Rule: When Giving It Your 100% Gets You Less than 85%
I thought giving it my all led to maximum outcomes; then I learnt about the 85% rule.
09 Jul 2020 Ā· 2 min Ā· productivity life š©·
My Notes From Spark+AI Summit 2020 (Application-Specific Talks)
Part II of the previous write-up, this time on applications and frameworks of Spark in production
05 Jul 2020 Ā· 15 min Ā· machinelearning deeplearning production survey
My Notes From Spark+AI Summit 2020 (Application-Agnostic Talks)
Sharing my notes & practical knowledge from the conference for people who don't have the time.
28 Jun 2020 Ā· 11 min Ā· machinelearning deeplearning survey
How to Set Up a Python Project For Automation and Collaboration
After this article, we'll have a workflow of tests and checks that run automatically with each git push.
21 Jun 2020 Ā· 20 min Ā· engineering production python productivity
Mailbag: Qns on the Intersection of Data Science and Business
Does DS have business requirements? When does it make sense to split DS and DE??
21 Jun 2020 Ā· 3 min Ā· datascience š¬
Why Are My Airflow Jobs Running āOne Day Lateā?
A curious discussion made me realize my expert blind spot. And no, Airflow is not late.
17 Jun 2020 Ā· 3 min Ā· engineering production til
What I Do Before a Data Science Project to Ensure Success
Haste makes waste. Diving into a data science problem may not be the fastest route to getting it done.
15 Jun 2020 Ā· 12 min Ā· mechanism datascience productivity
What I Love about Scrum for Data Science
Initially, I didn't like it. But over time, it grew on me. Here's why.
07 Jun 2020 Ā· 10 min Ā· mechanism agile leadership datascience
How to Apply Crocker's Law for Feedback and Growth
Crocker's Law, cognitive dissonance, and how to receive (uncomfortable) feedback better.
A Practical Guide to Maintaining Machine Learning in Production
Can maintaining machine learning in production be easier? I go through some practical tips.
25 May 2020 Ā· 16 min Ā· machinelearning engineering production
6 Little-Known Challenges After Deploying Machine Learning
I thought deploying machine learning was hard. Then I had to maintain multiple systems in prod.
18 May 2020 Ā· 14 min Ā· machinelearning engineering production
How to Write: Advice from David Perell and Sahil Lavingia
An expansion of my Twitter thread that went viral.
09 May 2020 Ā· 4 min Ā· writing
A Hackathon Where the Dinkiest Idea Won. Why?
What I Learnt about evaluating ideas from first-hand participation in a hackathon.
03 May 2020 Ā· 7 min Ā· datascience lazada
Serendipity: Accuracyās Unpopular Best Friend in Recommenders
What I learned about measuring diversity, novelty, surprise, and serendipity from 10+ papers.
How to Give a Kick-Ass Data Science Talk
Why you should give a talk and some tips from five years of speaking and hosting meet-ups.
18 Apr 2020 Ā· 7 min Ā· datascience writing
Commando, Soldier, Police and Your Career Choices
Should I join a start-up? Which offer should I accept? A simple metaphor to guide your decisions.
12 Apr 2020 Ā· 6 min Ā· career
Stop Taking Regular Notes; Use a Zettelkasten Instead
Using a Zettelkasten helps you make connections between notes, improving learning and memory.
05 Apr 2020 Ā· 6 min Ā· writing learning productivity š„
Writing is Learning: How I Learned an Easier Way to Write
Writing begins before actually writing; it's a cycle of reading -> note-taking -> writing.
Simpler Experimentation with Jupyter, Papermill, and MLflow
Automate your experimentation workflow to minimize effort and iterate faster.
15 Mar 2020 Ā· 6 min Ā· productivity python
My Journey from Psych Grad to Leading Data Science at Lazada
How hard work, many failures, and a bit of luck got me into the field and up the ladder.
DataScience SG Meetup - RecSys, Beyond the Baseline
Comparing baselines (matrix factorization) against novel approaches using graphs & NLP.
14 Jan 2020 Ā· 2 min Ā· recsys machinelearning
Beating the Baseline Recommender with Graph & NLP in Pytorch
Beating the baseline using Graph & NLP techniques on PyTorch, AUC improvement of ~21% (Part 2 of 2).
13 Jan 2020 Ā· 17 min Ā· recsys deeplearning python š
Building a Strong Baseline Recommender in PyTorch, on a Laptop
Building a baseline recsys based on data scraped off Amazon. Warning - Lots of charts! (Part 1 of 2).
06 Jan 2020 Ā· 14 min Ā· recsys deeplearning python š
OMSCS CS6200 (Introduction to OS) Review and Tips
OMSCS CS6200 (Introduction to OS) - Moving data from one process to another, multi-threaded.
15 Dec 2019 Ā· 7 min Ā· omscs learning engineering
DataScience SG x ODSC Meetup - Applying ML to Healthcare
In-depth sharing on how to put machine learning systems into production.
09 Oct 2019 Ā· 4 min Ā· machinelearning production
OLX Prod Tech 2019 Keynote - Asia's Tech Giants & SuperApps
Keynote on how Asia's tech giants scale and their SuperApp strategy.
03 Oct 2019 Ā· 3 min Ā· machinelearning engineering leadership
OMSCS CS6750 (Human Computer Interaction) Review and Tips
OMSCS CS6750 (Human Computer Interaction) - You are not your user! Or how to build great products.
Goodbye Wordpress, Hello Jekyll!
Moving off wordpress and hosting for free on GitHub. And gaining full customization!
25 Aug 2019 Ā· 1 min Ā· misc
OMSCS CS6440 (Intro to Health Informatics) Review and Tips
OMSCS CS6440 (Intro to Health Informatics) - A primer on key tech and standards in healthtech.
OMSCS CS7646 (Machine Learning for Trading) Review and Tips
OMSCS CS7646 (Machine Learning for Trading) - Don't sell your house to trade algorithmically.
11 May 2019 Ā· 9 min Ā· omscs learning machinelearning python
What does a Data Scientist really do?
No, you don't need a PhD or 10+ years of experience.
30 Apr 2019 Ā· 8 min Ā· career datascience
DATAx - A Production ML system for SEA's Biggest Hospital Group
How we built an ML system to predict hospitalization costs at admission; sharing at DATAx Conference.
06 Mar 2019 Ā· 4 min Ā· production machinelearning
Data Science and Agile (Frameworks for Effectiveness)
Taking the best from agile and modifying it to fit the data science process (Part 2 of 2).
02 Feb 2019 Ā· 14 min Ā· mechanism agile datascience productivity
Data Science and Agile (What Works, and What Doesn't)
A deeper look into the strengths and weaknesses of Agile in Data Science projects (Part 1 of 2).
26 Jan 2019 Ā· 13 min Ā· agile datascience productivity š„
DataScience SG Meetup - Panel On the Different Roles in Data
What's the difference between a data scientist, data engineer, and ML engineer? A panel at Google.
17 Jan 2019 Ā· 4 min Ā· datascience career
OMSCS CS6601 (Artificial Intelligence) Review and Tips
OMSCS CS6601 (Artificial Intelligence) - First, start with the simplest solution, and then add intelligence.
20 Dec 2018 Ā· 8 min Ā· omscs learning machinelearning python
GovTech Conference - Data Science and AgileāCan or Not?
Yes, Agile can be adopted by data science teams. Moderating a panel at GovTech STACK.
28 Oct 2018 Ā· 6 min Ā· agile datascience
OMSCS CS6460 (Education Technology) Review and Tips
OMSCS CS6460 (Education Technology) - How to scale education widely through technology.
OMSCS CS7642 (Reinforcement Learning) Review and Tips
OMSCS CS7642 (Reinforcement Learning) - Landing rockets (fun!) via deep Q-Learning (and its variants).
30 Jul 2018 Ā· 6 min Ā· omscs learning deeplearning python
Big Data & Analytics Summit - Data Science Challenges @ Lazada
Technical challenges easy compared to business and people issues. Sharing at the BDA Summit.
21 Jun 2018 Ā· 8 min Ā· datascience leadership lazada
Building a Strong Data Science Team Culture
Culture >> Hierarchy, Process, Bureaucracy.
12 May 2018 Ā· 5 min Ā· leadership datascience lazada
INSEAD Lunchtime Talks - How Lazada uses Data
And my idiosyncratic journey to VP of Data Science at Lazada (Alibaba). A Lunchtime chat at INSEAD.
25 Apr 2018 Ā· 2 min Ā· datascience lazada
OMSCS CS7641 (Machine Learning) Review and Tips
OMSCS CS7641 (Machine Learning) - Revisiting the fundamentals and learning new techniques.
27 Dec 2017 Ā· 4 min Ā· omscs learning machinelearning python
My first 100 days as Data Science Lead
How being a Lead / Manager is different from being an individual contributor.
25 Sep 2017 Ā· 5 min Ā· leadership datascience lazada
SMU - What is Data Analytics and How do I get into it?
What is data science, how to pick it up, and how to enter the field? A discussion with SMU undergrads.
26 Aug 2017 Ā· 1 min Ā· datascience career
OMSCS CS6300 (Software Development Process) Review and Tips
OMSCS CS6300 (Software Development Process) - Java and collaboratively developing an Android app.
13 Aug 2017 Ā· 5 min Ā· omscs learning engineering
Tech in Asia - My Journey in Data Science and Advice for others
Sharing about why data science, data science myths, a typical day, and more with TIA.
26 Jul 2017 Ā· 4 min Ā· datascience career
SMU Masters in IT - How to get started in Data Science
Tools and skills to pick up and how to practice them. An Invited Talk with Masters in IT candidates.
26 Jun 2017 Ā· 1 min Ā· datascience learning career
How to get started in Data Science
Tools and skills to pick up, and how to practice them.
25 Jun 2017 Ā· 9 min Ā· datascience learning career
OMSCS CS6476 (Computer Vision) Review and Tips
OMSCS CS6476 Computer Vision - Performing computer vision tasks with ONLY numpy.
15 May 2017 Ā· 5 min Ā· omscs learning deeplearning python
One way to help a data science team innovate successfully
If things are not failing, you're not innovating enough. - Elon Musk
19 Feb 2017 Ā· 3 min Ā· leadership datascience lazada
Product Categorization API Part 3: Creating an API
Or how to put machine learning models into production.
13 Feb 2017 Ā· 8 min Ā· machinelearning production python š
Image search is now live!
A web app to find similar products based on image.
14 Jan 2017 Ā· 4 min Ā· deeplearning python production š
Product Classification API Part 2: Data Preparation
Cleaning up text and messing with ascii (urgh!)
11 Dec 2016 Ā· 8 min Ā· machinelearning python š
Strata x Hadoop 2016 - How Lazada Ranks Products
How Lazada ranks products to improve customer experience and conversion at Strata 2016.
09 Dec 2016 Ā· 1 min Ā· machinelearning lazada production
Image classification API is now live!
A simple web app to classify fashion images into Amazon categories.
27 Nov 2016 Ā· 2 min Ā· deeplearning python production š
I'm going back to school
Got accepted into Georgia Tech's Computer Science Masters!
SortMySkills is now live!
A card sorting game to discover youl passion by identifying skills you like and dislike.
Product Classification API Part 1: Data Acquisition
Parsing json and formatting product titles and categories.
11 Oct 2016 Ā· 9 min Ā· machinelearning python š
Thoughts on Functional Programming in Scala Course (Coursera)
Learning Scala from Martin Odersky, father of Scala.
31 Jul 2016 Ā· 4 min Ā· learning
DataKind Singaporeās Latest Project Accelerator
Guest post of how DataKind SG worked with NGOs to frame their problems and suggests solutions
17 Sep 2015 Ā· 8 min Ā· datascience
DataScience SG Meetup - How we got top 3% in Kaggle
Sharing about my first data science competition at DataScience SG.
20 Jun 2015 Ā· 1 min Ā· machinelearning
Join 11,800+ readers getting updates on machine learning, RecSys, LLMs, and engineering.