Tag:

eval

(5)

Evaluating the Effectiveness of LLM-Evaluators (aka LLM-as-Judge)

Use cases, techniques, alignment, finetuning, and critiques against LLM-evaluators.

18 Aug 2024  ·  49 min  ·  llm eval

Task-Specific LLM Evals that Do & Don't Work

Evals for classification, summarization, translation, copyright regurgitation, and toxicity.

31 Mar 2024  ·  33 min  ·  llm eval machinelearning

Out-of-Domain Finetuning to Bootstrap Hallucination Detection

How to use open-source, permissive-use data and collect less labeled samples for our tasks.

05 Nov 2023  ·  12 min  ·  llm eval machinelearning

Evaluation & Hallucination Detection for Abstractive Summaries

Reference, context, and preference-based metrics, self-consistency, and catching hallucinations.

03 Sep 2023  ·  23 min  ·  llm eval survey

Counterfactual Evaluation for Recommendation Systems

Thinking about recsys as interventional vs. observational, and inverse propensity scoring.

10 Apr 2022  ·  8 min  ·  recsys eval machinelearning

Join 8,500+ readers getting updates on machine learning, RecSys, LLMs, and engineering.