Speaker: Dimitris Papailiopoulos
Title: Looped Transformers as Programmable Computers
Abstract: In this talk, I will present an exploration of transformer networks as programmable computers, highlighting their capacity for algorithmic reasoning. I will discuss looped transformers as universal computers, demonstrating how they can be programmed with specific weights to execute general-purpose programs. Next, I will talk about some preliminary thoughts on setting up a “Turing Machine Test” that assesses the computational abilities of frozen LLMs like GPT4, with the goal to get insights into their computational abilities. Finally, we’ll explore the emergence of algorithmic properties in LMs through the lens of simple arithmetic case studies.
April 19, 2023
12:30 pm (1h)
Orchard View Room, Virtual
Diimitris Tsipras, Victor Blanco