Systems | Information | Learning | Optimization
 

SILO: Looped Transformers as Programmable Computers

Speaker: Dimitris Papailiopoulos
Title: Looped Transformers as Programmable Computers
Abstract: In this talk, I will present an exploration of transformer networks as programmable computers, highlighting their capacity for algorithmic reasoning. I will discuss looped transformers as universal computers, demonstrating how they can be programmed with specific weights to execute general-purpose programs. Next, I will talk about some preliminary thoughts on setting up a “Turing Machine Test” that assesses the computational abilities of frozen LLMs like GPT4, with the goal to get insights into their computational abilities. Finally, we’ll explore the emergence of algorithmic properties in LMs through the lens of simple arithmetic case studies.

April 19 @ 12:30
12:30 pm (1h)

Orchard View Room, Virtual

Diimitris Tsipras, Victor Blanco