We gratefully acknowledge support from
the Simons Foundation and member institutions.

Xinpeng Wang is qualified to endorse.

How to Distill your BERT: An Empirical Study on the Impact of Weight Initialisation and Distillation Objectives

Xinpeng Wang: Is registered as an author of this paper.
Can endorse for cs.AI, cs.CL, cs.CV, cs.LG. (why?)

Leonie Weissweiler, Hinrich Schütze and Barbara Plank are not registered as owners of this paper. (why?)