Reading Group

The reading group in the WiSe24/25 will be every week on monday evenings, starting on the 21st of October. Exact time and room is not fixed yet.

As last semester, we will use the time to explore two topics. The first will be LLM's and the second is TBD.

Past

July 1, 2024

On the importance of initialization and momentum in deep learning

Ilya Sutskever, James Martens, George Dahl, Geoffrey Hinton

July 1, 2024

Adam: A Method for Stochastic Optimization

Diederik P. Kingma, Jimmy Ba

June 18, 2024

Stochastic Gradient Descend

appliedprobability

June 18, 2024

Adding Gradient Noise Improves Learning for Very Deep Networks

Arvind Neelakantan, Luke Vilnis, Quoc V. Le, Ilya Sutskever, Lukasz Kaiser, Karol Kurach, James Martens

June 4, 2024

Revealing the Dark Secrets of Masked Image Modeling

Zhenda Xie, Zigang Geng, Jingcheng Hu, Zheng Zhang, Han Hu, Yue Cao

June 4, 2024

Can We Break Free from Strong Data Augmentations in Self-Supervised Learning?

Shruthi Gowda1, Elahe Arani, Bahram Zonooz

May 21, 2024

Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecture

Mahmoud Assran, Quentin Duval, Ishan Misra, Piotr Bojanowski, Pascal Vincent, Michael Rabbat, Yann LeCun, Nicolas Ballas

May 21, 2024

What do Self Supervised Vision Transformers Learn?

Namuk Park, Wonjae Kim, Byeongho Heo, Taekyung Kim, Sangdoo Yun

May 7, 2024

Emerging Properties in Self-Supervised Vision Transformers

Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, Armand Joulin

May 7, 2024

Vision Transformers Need Registers

Timothée Darcet, Maxime Oquab, Julien Mairal, Piotr Bojanowski

April 23, 2024

Intriguing Properties of Contrastive Losses

Ting Chen, Calvin Luo, Lala Li

April 23, 2024

Bootstrap your own latent: A new approach to self-supervised Learning

Jean-Bastien Grill, Florian Strub, Florent Altché, Corentin Tallec, Pierre H. Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhaohan Daniel Guo, Mohammad Gheshlaghi Azar, Bilal Piot, Koray Kavukcuoglu, Rémi Munos, Michal Valko

February 6, 2024

MT3 Multi-Task Multitrack Music Transcription

Josh Gardner, Ian Simon, Ethan Manilow, Curtis Hawthorne, Jesse Engel

January 30, 2024

MusicLM: Generating Music From Text

Andrea Agostinelli, Timo I. Denk, Zalán Borsos, Jesse Engel, Mauro Verzetti, Antoine Caillon, Qingqing Huang, Aren Jansen, Adam Roberts, Marco Tagliasacchi, Matt Sharifi, Neil Zeghidour, Christian Frank

January 16, 2024

Jukebox A Generative Model for Music

Prafulla Dhariwal, Heewoo Jun, Christine Payne, Jong Wook Kim, Alec Radford, Ilya Sutskever

January 9, 2024

Neural Discrete Representation Learning

Aaron van den Oord, Oriol Vinyals, Koray Kavukcuoglu

December 19, 2023

RL-Duet - Online Music Accompaniment Generation Using Deep Reinforcement Learning

Nan Jiang, Sheng Jin, Zhiyao Duan, Changshui Zhang

December 12, 2023

Agent57: Outperforming the Atari Human Benchmark

Adrià Puigdomènech Badia, Bilal Piot, Steven Kapturowski, Pablo Sprechmann, Alex Vitvitskyi, Daniel Guo, Charles Blundell

December 5, 2023

Rainbow: Combining Improvements in Deep Reinforcement Learning

Matteo Hessel, Joseph Modayil, Hado van Hasselt, Tom Schaul, Georg Ostrovski, Will Dabney, Dan Horgan, Bilal Piot, Mohammad Azar, David Silver

November 24, 2023

Noisy Networks for Exploration

Meire Fortunato, Mohammad Gheshlaghi Azar, Bilal Piot, Jacob Menick, Ian Osband, Alex Graves, Vlad Mnih, Remi Munos, Demis Hassabis, Olivier Pietquin, Charles Blundell, Shane Legg

November 15, 2023

A Distributional Perspective on Reinforcement Learning

Marc G. Bellemare, Will Dabney, Rémi Munos

November 7, 2023

Prioritized Experience Replay

Tom Schaul, John Quan, Ioannis Antonoglou, David Silver

November 1, 2023

Deep Reinforcement Learning with Double Q-learning

Hado van Hasselt, Arthur Guez, David Silver

October 30, 2023

Dueling Network Architectures for Deep Reinforcement Learning

Ziyu Wang, Tom Schaul, Matteo Hessel, Hado van Hasselt, Marc Lanctot, Nando de Freitas

October 22, 2023

Human-level Control through Deep Reinforcement Learning

Volodymyr Mnih, K. Kavukcuoglu, David Silver, Andrei A. Rusu, J. Veness, Marc G. Bellemare, Alex Graves, Martin A. Riedmiller, A. Fidjeland, Georg Ostrovski, Stig Petersen, Charlie Beattie, Amir Sadik, Ioannis Antonoglou, Helen King, D. Kumaran, Daan Wierstra, S. Legg, D. Hassabis