Attention in Deep Learning and Physics Applications

• Pierre Baldi (University of California, Irvine)

Recently, so-called "attention mechanisms" have been added to deep learning architectures leading-to-state- of-the-art results in natural language processing using transformer architectures. In this seminar, we will present the fundamental principles behind attention mechanisms and show how attention-based architectures can be applied effectively to physics problems.