But not all transformer applications require both the encoder and decoder module. For example, the GPT family of large language models uses stacks of decoder modules to generate text.

Cross-attention connects encoder and decoder components in a model and during translation. For example, it allows the English word “strawberry” to relate to the French word “fraise.” ...

The first of these technologies is a translation model architecture — a hybrid architecture consisting of a Transformer encoder and a recurrent neural network (RNN) decoder implemented in Lingvo ...