Can Large Language Models Be an Alternative to Human Evaluations?
Paper
•
2305.01937
•
Published
•
2
LLM System Evaluations Papers
A collection of recent papers on NLG evaluations, very applicable to components of LLM systems.
-
-
Decontextualization: Making Sentences Stand-Alone
Paper • 2102.05169 • Published -
RARR: Researching and Revising What Language Models Say, Using Language Models
Paper • 2210.08726 • Published • 1 -
SummEval: Re-evaluating Summarization Evaluation
Paper • 2007.12626 • Published -
QAFactEval: Improved QA-Based Factual Consistency Evaluation for Summarization
Paper • 2112.08542 • Published -
SummaC: Re-Visiting NLI-based Models for Inconsistency Detection in Summarization
Paper • 2111.09525 • Published -
Neural Text Summarization: A Critical Evaluation
Paper • 1908.08960 • Published -
GPTScore: Evaluate as You Desire
Paper • 2302.04166 • Published • 1 -
ChatGPT Outperforms Crowd-Workers for Text-Annotation Tasks
Paper • 2303.15056 • Published -
G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment
Paper • 2303.16634 • Published • 3 -
Large Language Models Are State-of-the-Art Evaluators of Translation Quality
Paper • 2302.14520 • Published -
RAGAS: Automated Evaluation of Retrieval Augmented Generation
Paper • 2309.15217 • Published • 3 -
Is ChatGPT a Good NLG Evaluator? A Preliminary Study
Paper • 2303.04048 • Published -
Measuring Attribution in Natural Language Generation Models
Paper • 2112.12870 • Published -
ARES: An Automated Evaluation Framework for Retrieval-Augmented Generation Systems
Paper • 2311.09476 • Published • 5 -
L-Eval: Instituting Standardized Evaluation for Long Context Language Models
Paper • 2307.11088 • Published • 4 -
FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation
Paper • 2305.14251 • Published • 2 -
Leveraging Large Language Models for NLG Evaluation: A Survey
Paper • 2401.07103 • Published • 4