Brief:

Before you read this page, you can skim over Transformer , Self-Attention & Multi-Head Attention.

• The Basic Structure

• The Advantages of Convolutional Self-Attention

Reference:

1907.00235.pdf