Self-attention is naturally permutation equivariant, therefore, we maythink of them as set-encoders rather than sequence encoders. However, for modalities where thedata does follow a specific ordering, for example agent state across different time steps, it is ben-eficial to break permutation equivariance and utilize the sequence information. This is commonlydone through positional embeddings. For simplicity, we add learned positional embeddings for allmodalities. As not all modalities are ordered, the learned positional embeddings are initially set tozero, letting the model learn if it is necessary to utilize the ordering within a modality.