New Study Suggests Self-Attention Layers Could Replace Convolutional Layers on Vision Tasks

Inspired by the performance of attention mechanisms in NLP, researchers have explored the possibility of applying them to vision tasks.