Nice refreshing list :)
I didn't think "Attention > Convolution" was a prevalent myth, given how integral convolutions are to SOTA image classifiers and GANs (if anything, I believe attention is unde-utilised here and due to grow in usage a lot)
I didn't think "Attention > Convolution" was a prevalent myth, given how integral convolutions are to SOTA image classifiers and GANs (if anything, I believe attention is unde-utilised here and due to grow in usage a lot)