Evaluation (commonly referred to as an “eval”) measures an AI model's performance on a specific set of benchmark tasks. Researchers use evals to assess their models' strengths and weaknesses by comparing the model's answers to the correct answers for each task, computing the accuracy, and identifying areas for improvement. However, researchers should heed Goodhart’s Law when optimizing performance on evals: using a measure as a target can render it ineffective. A model may excel at tasks in the dataset but fail to generalize across the domain, or hidden capabilities may go unnoticed if certain examples are excluded. To assess strengths and weaknesses, individuals and organizations should curate diverse evaluation datasets. As the old adage goes, “what gets measured gets managed.”
Disclaimer: Our global network of contributors to the AI & Human Rights Index is currently writing these articles and glossary entries. This particular page is currently in the recruitment and research stage. Please return later to see where this page is in the editorial workflow. Thank you! We look forward to learning with and from you.