Tag:
survey
(13)Evaluating Long-Context Question & Answer Systems
Evaluation metrics, how to build eval datasets, eval methodology, and a review of several benchmarks.
Evaluating the Effectiveness of LLM-Evaluators (aka LLM-as-Judge)
Use cases, techniques, alignment, finetuning, and critiques against LLM-evaluators.
18 Aug 2024 Ā· 49 min Ā· llm eval production survey š„
Task-Specific LLM Evals that Do & Don't Work
Evals for classification, summarization, translation, copyright regurgitation, and toxicity.
How to Generate and Use Synthetic Data for Finetuning
Overcoming the bottleneck of human annotations in instruction-tuning, preference-tuning, and pretraining.
Evaluation & Hallucination Detection for Abstractive Summaries
Reference, context, and preference-based metrics, self-consistency, and catching hallucinations.
How to Write Data Labeling/Annotation Guidelines
Writing good instructions to achieve high precision and throughput.
12 Mar 2023 Ā· 6 min Ā· machinelearning mechanism survey
Text-to-Image: Diffusion, Text Conditioning, Guidance, Latent Space
The fundamentals of text-to-image generation, relevant papers, and experimenting with DDPM.
27 Nov 2022 Ā· 19 min Ā· deeplearning llm survey
RecSys 2021 - Papers and Talks to Chew on
Simple baselines, ideas, tech stacks, and packages to try.
03 Oct 2021 Ā· 5 min Ā· recsys deeplearning production survey
RecSys 2020: Takeaways and Notable Papers
Emphasis on bias, more sequential models & bandits, robust offline evaluation, and recsys in the wild.
27 Sep 2020 Ā· 16 min Ā· recsys deeplearning production survey
NLP for Supervised Learning - A Brief Survey
Examining the broad strokes of NLP progress and comparing between models
16 Aug 2020 Ā· 23 min Ā· llm deeplearning survey
My Notes From Spark+AI Summit 2020 (Application-Specific Talks)
Part II of the previous write-up, this time on applications and frameworks of Spark in production
05 Jul 2020 Ā· 15 min Ā· machinelearning deeplearning production survey
My Notes From Spark+AI Summit 2020 (Application-Agnostic Talks)
Sharing my notes & practical knowledge from the conference for people who don't have the time.
28 Jun 2020 Ā· 11 min Ā· machinelearning deeplearning survey
Serendipity: Accuracyās Unpopular Best Friend in Recommenders
What I learned about measuring diversity, novelty, surprise, and serendipity from 10+ papers.
Join 11,300+ readers getting updates on machine learning, RecSys, LLMs, and engineering.