The Ada Lovelace Algorithmic Lectures are a series of interactive lectures in which we branch, explain and prove... in an algorithmic way. For each edition, we invite one or more researchers specialising in large language models from outside the host institution, Vrije Universiteit Brussel (VUB), whose work has caught our attention. This could be a critical point of view, interesting position or just excellent research that aligns with the interests of the Data Lab. A core aspect of our scientific ambitions is to push the boundaries of foundational AI research. We are renowned for our interdisciplinary approach, integrating physics, mathematics, information theory, and AI to tackle fundamental scientific challenges. Through this series, we aim to increase collaboration and discussion between scientists who satisfy the aforementioned constraints, in honour of the world's first software engineer, Ada Lovelace.
20/05/2025
Dr. Desi R. Ivanova on the reliability of Large Language Models and their evaluations.
TLDR: Desi reminded us to check the conditions of the central limit theorem (CLT) when using error bars in LLM evaluations and restarted the broader discussion on how to measure and improve reliability of LLMs.
Paper: Position: Don’t Use the CLT in LLM Evals With Fewer Than a Few Hundred Datapoints. Slides: Reliability of LLMs and LLM Evaluations.