MCNC: Manifold Constrained Network Compression
Authors:
Chayne Thrash,
Ali Abbasi,
Parsa Nooralinejad,
Soroush Abbasi Koohpayegani,
Reed Andreas,
Hamed Pirsiavash,
Soheil Kolouri
Abstract:
The outstanding performance of large foundational models across diverse tasks-from computer vision to speech and natural language processing-has significantly increased their demand. However, storing and transmitting these models pose significant challenges due to their massive size (e.g., 350GB for GPT-3). Recent literature has focused on compressing the original weights or reducing the number of…
▽ More
The outstanding performance of large foundational models across diverse tasks-from computer vision to speech and natural language processing-has significantly increased their demand. However, storing and transmitting these models pose significant challenges due to their massive size (e.g., 350GB for GPT-3). Recent literature has focused on compressing the original weights or reducing the number of parameters required for fine-tuning these models. These compression methods typically involve constraining the parameter space, for example, through low-rank reparametrization (e.g., LoRA) or quantization (e.g., QLoRA) during model training. In this paper, we present MCNC as a novel model compression method that constrains the parameter space to low-dimensional pre-defined and frozen nonlinear manifolds, which effectively cover this space. Given the prevalence of good solutions in over-parameterized deep neural networks, we show that by constraining the parameter space to our proposed manifold, we can identify high-quality solutions while achieving unprecedented compression rates across a wide variety of tasks. Through extensive experiments in computer vision and natural language processing tasks, we demonstrate that our method, MCNC, significantly outperforms state-of-the-art baselines in terms of compression, accuracy, and/or model reconstruction time.
△ Less
Submitted 27 June, 2024;
originally announced June 2024.
Revival of the magnetar PSR J1622-4950: observations with MeerKAT, Parkes, XMM-Newton, Swift, Chandra, and NuSTAR
Authors:
F. Camilo,
P. Scholz,
M. Serylak,
S. Buchner,
M. Merryfield,
V. M. Kaspi,
R. F. Archibald,
M. Bailes,
A. Jameson,
W. van Straten,
J. Sarkissian,
J. E. Reynolds,
S. Johnston,
G. Hobbs,
T. D. Abbott,
R. M. Adam,
G. B. Adams,
T. Alberts,
R. Andreas,
K. M. B. Asad,
D. E. Baker,
T. Baloyi,
E. F. Bauermeister,
T. Baxana,
T. G. H. Bennett
, et al. (183 additional authors not shown)
Abstract:
New radio (MeerKAT and Parkes) and X-ray (XMM-Newton, Swift, Chandra, and NuSTAR) observations of PSR J1622-4950 indicate that the magnetar, in a quiescent state since at least early 2015, reactivated between 2017 March 19 and April 5. The radio flux density, while variable, is approximately 100x larger than during its dormant state. The X-ray flux one month after reactivation was at least 800x la…
▽ More
New radio (MeerKAT and Parkes) and X-ray (XMM-Newton, Swift, Chandra, and NuSTAR) observations of PSR J1622-4950 indicate that the magnetar, in a quiescent state since at least early 2015, reactivated between 2017 March 19 and April 5. The radio flux density, while variable, is approximately 100x larger than during its dormant state. The X-ray flux one month after reactivation was at least 800x larger than during quiescence, and has been decaying exponentially on a 111+/-19 day timescale. This high-flux state, together with a radio-derived rotational ephemeris, enabled for the first time the detection of X-ray pulsations for this magnetar. At 5%, the 0.3-6 keV pulsed fraction is comparable to the smallest observed for magnetars. The overall pulsar geometry inferred from polarized radio emission appears to be broadly consistent with that determined 6-8 years earlier. However, rotating vector model fits suggest that we are now seeing radio emission from a different location in the magnetosphere than previously. This indicates a novel way in which radio emission from magnetars can differ from that of ordinary pulsars. The torque on the neutron star is varying rapidly and unsteadily, as is common for magnetars following outburst, having changed by a factor of 7 within six months of reactivation.
△ Less
Submitted 5 April, 2018;
originally announced April 2018.
