23 April 2026, 4pm - 6pm

Czech Academy of Sciences, Národní 3, Prague 1, Hall 205
Public lecture, free entry

Abstract:
In the space of a few years, Large Language Models (LLMs) have become a staple of Artificial Intelligence systems. Used by hundreds of millions of people on the planet, they are often heralded as a technological revolution. But the willingness of end users to believe in their promises is only matched by the lack of transparency surrounding their architecture. In this talk, Aurelie Herbelot will give an overview of LLMs from both an engineering and a scientific perspective, with the aim of clarifying the ontological status of the underlying algorithm.

The first part of the talk will be dedicated to the engineering question and will provide a description of the algorithm behind LLMs: the Transformer. Issued from the Google research labs in 2017, the Transformer mixes well-known insights from the Natural Language Processing and Computer Vision in a neural network architecture, capitalising on the scale of the model. Aurelie Herbelot’s exposition of the algorithm will involve a tiny version of the Transformer, specifically developed for educational purposes and designed to be “opened up” by non-experts. The system will be trained from scratch during the session, thus illustrating how (and what) a language model actually learns from its data. 

The second part of the talk will focus on scientific matters and discuss the epistemological underpinning of the Transformer. In particular, Aurelie Herbelot will ask whether it can be regarded as a “model” in the strong sense of the term. She will show that, beyond the vaguest behaviourism, the architecture fails to encode any known theory of language and thus cannot be taken as a scientific model. On this basis, Herbelot will cast doubts on the suitability of LLMs to be used for any kind of knowledge inquiry. She will finish by briefly considering the use of Transformers for a range of everyday tasks such as literature research and drafting reports, and ask how the ontological status of the algorithm impacts on its real-world application. Against the background of Large Language Models, Herbelot will plead for the development of Small Models of Language.
 
Aurelie Herbelot is a computational semanticist. Having spent 18 years in academic research, she now runs Denotation UG, a small company invested in bringing sustainable AI systems to the real world. Previously, she was assistant professor at the Center for Mind/Brain Sciences, University of Trento (Italy). Aurelie obtained a PhD in Natural Language Processing from the University of Cambridge, after which she was an Alexander von Humboldt Fellow in Potsdam, and a postdoc in Cambridge, Stuttgart, and the Center for Mind/Brain Sciences in Trento. She briefly moved to the Universitat Pompeu Fabra in Barcelona as a Marie Skłodowska-Curie fellow and returned to Trento in 2018 as a faculty member. In 2023, she left academia to found Denotation UG because she wanted to provide an alternative to the dominant discourse about AI. For more information, please consult Aurelie’s website: www.aurelieherbelot.net.

This event is co-organized with AVU Emergent Technologies Research Group and AI in Context (AI v kontextu) Group.