-
Quantization-Guided Training for Compact TinyML Models
Authors:
Sedigh Ghamari,
Koray Ozcan,
Thu Dinh,
Andrey Melnikov,
Juan Carvajal,
Jan Ernst,
Sek Chai
Abstract:
We propose a Quantization Guided Training (QGT) method to guide DNN training towards optimized low-bit-precision targets and reach extreme compression levels below 8-bit precision. Unlike standard quantization-aware training (QAT) approaches, QGT uses customized regularization to encourage weight values towards a distribution that maximizes accuracy while reducing quantization errors. One of the m…
▽ More
We propose a Quantization Guided Training (QGT) method to guide DNN training towards optimized low-bit-precision targets and reach extreme compression levels below 8-bit precision. Unlike standard quantization-aware training (QAT) approaches, QGT uses customized regularization to encourage weight values towards a distribution that maximizes accuracy while reducing quantization errors. One of the main benefits of this approach is the ability to identify compression bottlenecks. We validate QGT using state-of-the-art model architectures on vision datasets. We also demonstrate the effectiveness of QGT with an 81KB tiny model for person detection down to 2-bit precision (representing 17.7x size reduction), while maintaining an accuracy drop of only 3% compared to a floating-point baseline.
△ Less
Submitted 10 March, 2021;
originally announced March 2021.
-
Renormalization group analysis on a neck-narrowing Lifshitz transition in the presence of weak short-range interactions in two dimensions
Authors:
Sedigh Ghamari,
Sung-Sik Lee,
Catherine Kallin
Abstract:
We study a system of weakly interacting electrons described by the energy dispersion $ξ(\mathbf{k}) = k_x^2 - k_y^2 - μ$ in two dimensions within a renormalization group approach. This energy dispersion exhibits a neck-narrowing Lifshitz transition at the critical chemical potential $μ_c=0$ where a van Hove singularity develops. Implementing a systematic renormalization group analysis of this syst…
▽ More
We study a system of weakly interacting electrons described by the energy dispersion $ξ(\mathbf{k}) = k_x^2 - k_y^2 - μ$ in two dimensions within a renormalization group approach. This energy dispersion exhibits a neck-narrowing Lifshitz transition at the critical chemical potential $μ_c=0$ where a van Hove singularity develops. Implementing a systematic renormalization group analysis of this system has long been hampered by the appearance of nonlocal terms in the Wilsonian effective action. We demonstrate that non-locality at the critical point is intrinsic, and the locality of the effective action can be maintained only away from the critical point. We also point out that it is crucial to introduce a large momentum cutoff to keep locality even away from the critical point. Based on a local renormalization group scheme employed near the critical point, we show that, as the energy scale $E$ is lowered, an attractive four-fermion interaction grows as $\log^2 E$ for $E > μ$, whereas it retains the usual BCS growth, $-\log E$, for $E < μ$. Starting away from the critical point, this fast growth of the pairing interaction suggests that the system becomes unstable toward a superconducting state well before the critical point is reached.
△ Less
Submitted 29 July, 2015; v1 submitted 24 July, 2014;
originally announced July 2014.
-
Order in a Spatially Anisotropic Triangular Antiferromagnet
Authors:
Sedigh Ghamari,
Catherine Kallin,
Sung-Sik Lee,
Erik S. Sørensen
Abstract:
The phase diagram of the spin-1/2 Heisenberg antiferromagnet on an anisotropic triangular lattice of weakly coupled chains, a model relevant to Cs2CuCl4, is investigated using a renormalization group analysis, which includes marginal couplings important for connecting to numerical studies of this model. In particular, the relative stability of incommensurate spiral spin-density order and collinear…
▽ More
The phase diagram of the spin-1/2 Heisenberg antiferromagnet on an anisotropic triangular lattice of weakly coupled chains, a model relevant to Cs2CuCl4, is investigated using a renormalization group analysis, which includes marginal couplings important for connecting to numerical studies of this model. In particular, the relative stability of incommensurate spiral spin-density order and collinear antiferromagnetic order is studied. While incommensurate spiral order is found to exist over most of the phase diagram in the presence of a Dzyaloshinskii-Moriya (DM) interaction, at small interchain and extremely weak DM couplings, collinear antiferromagnetic order can survive. Our results imply that Cs2CuCl4 is well within the part of the phase diagram where spiral order is stable. The implications of the renormalization group analysis for numerical studies, many of which have found spin-liquidlike behavior, are discussed.
△ Less
Submitted 21 November, 2011; v1 submitted 15 August, 2011;
originally announced August 2011.
