#survey
13 posts
- Evaluating Long-Context Question & Answer Systems
Evaluation metrics, how to build eval datasets, eval methodology, and a review of several benchmarks.
- Evaluating the Effectiveness of LLM-Evaluators (aka LLM-as-Judge)
Use cases, techniques, alignment, finetuning, and critiques against LLM-evaluators.
- Task-Specific LLM Evals that Do & Don't Work
Evals for classification, summarization, translation, copyright regurgitation, and toxicity.
- How to Generate and Use Synthetic Data for Finetuning
Overcoming the bottleneck of human annotations in instruction-tuning, preference-tuning, and pretraining.
- Evaluation & Hallucination Detection for Abstractive Summaries
Reference, context, and preference-based metrics, self-consistency, and catching hallucinations.
- How to Write Data Labeling/Annotation Guidelines
Writing good instructions to achieve high precision and throughput.
- Text-to-Image: Diffusion, Text Conditioning, Guidance, Latent Space
The fundamentals of text-to-image generation, relevant papers, and experimenting with DDPM.
- RecSys 2021 - Papers and Talks to Chew on
Simple baselines, ideas, tech stacks, and packages to try.
- RecSys 2020: Takeaways and Notable Papers
Emphasis on bias, more sequential models & bandits, robust offline evaluation, and recsys in the wild.
- NLP for Supervised Learning - A Brief Survey
Examining the broad strokes of NLP progress and comparing between models
- My Notes From Spark+AI Summit 2020 (Application-Specific Talks)
Part II of the previous write-up, this time on applications and frameworks of Spark in production
- My Notes From Spark+AI Summit 2020 (Application-Agnostic Talks)
Sharing my notes & practical knowledge from the conference for people who don't have the time.
- Serendipity: Accuracy’s Unpopular Best Friend in Recommenders
What I learned about measuring diversity, novelty, surprise, and serendipity from 10+ papers.