Selim Kuzucu

Transformers are the primary architecture of choice on a plethora of downstream applications, from computer vision to natural language processing. An important driver behind their success is the self-attention mechanism, which models the relationships between tokens. Traditionally, input images are tokenized through considering their parts as constant-size non-overlapping patches for vision transformers. On the other hand, numerous contemporary works propose dynamic or greedy tokenization strategies. This project will focus on tokenization for vision transformers with the aim of improving their performance and efficiency in downstream applications.