We gratefully acknowledge support from
the Simons Foundation and member institutions.

Alexander M. Rush is qualified to endorse.

Pretraining Without Attention

Alexander M. Rush: Is registered as an author of this paper.
Can endorse for cs.AI, cs.CL, cs.LG, cs.NE. (why?)

Junxiong Wang, **g Nathan Yan and Albert Gu are not registered as owners of this paper. (why?)