Receptance Weighted Key Value (RWKV), thatcombines the efficient parallelizable trainingof transformers with the efficient inference ofRNNs.
RWKV - Receptance Weighted Key Value = hybrid approach. Parallelization in training of transformers but efficient inference of RNNs.