-
AdaWCT: Adaptive Whitening and Coloring Style Injection
Authors:
Antoine Dufour,
Yohan Poirier-Ginter,
Alexandre Lessard,
Ryan Smith,
Michael Lockyer,
Jean-Francois Lalonde
Abstract:
Adaptive instance normalization (AdaIN) has become the standard method for style injection: by re-normalizing features through scale-and-shift operations, it has found widespread use in style transfer, image generation, and image-to-image translation. In this work, we present a generalization of AdaIN which relies on the whitening and coloring transformation (WCT) which we dub AdaWCT, that we appl…
▽ More
Adaptive instance normalization (AdaIN) has become the standard method for style injection: by re-normalizing features through scale-and-shift operations, it has found widespread use in style transfer, image generation, and image-to-image translation. In this work, we present a generalization of AdaIN which relies on the whitening and coloring transformation (WCT) which we dub AdaWCT, that we apply for style injection in large GANs. We show, through experiments on the StarGANv2 architecture, that this generalization, albeit conceptually simple, results in significant improvements in the quality of the generated images.
△ Less
Submitted 1 August, 2022;
originally announced August 2022.
-
Overparameterization Improves StyleGAN Inversion
Authors:
Yohan Poirier-Ginter,
Alexandre Lessard,
Ryan Smith,
Jean-François Lalonde
Abstract:
Deep generative models like StyleGAN hold the promise of semantic image editing: modifying images by their content, rather than their pixel values. Unfortunately, working with arbitrary images requires inverting the StyleGAN generator, which has remained challenging so far. Existing inversion approaches obtain promising yet imperfect results, having to trade-off between reconstruction quality and…
▽ More
Deep generative models like StyleGAN hold the promise of semantic image editing: modifying images by their content, rather than their pixel values. Unfortunately, working with arbitrary images requires inverting the StyleGAN generator, which has remained challenging so far. Existing inversion approaches obtain promising yet imperfect results, having to trade-off between reconstruction quality and downstream editability. To improve quality, these approaches must resort to various techniques that extend the model latent space after training. Taking a step back, we observe that these methods essentially all propose, in one way or another, to increase the number of free parameters. This suggests that inversion might be difficult because it is underconstrained. In this work, we address this directly and dramatically overparameterize the latent space, before training, with simple changes to the original StyleGAN architecture. Our overparameterization increases the available degrees of freedom, which in turn facilitates inversion. We show that this allows us to obtain near-perfect image reconstruction without the need for encoders nor for altering the latent space after training. Our approach also retains editability, which we demonstrate by realistically interpolating between images.
△ Less
Submitted 12 May, 2022;
originally announced May 2022.
-
Magnetocrystalline Anisotropy Energy of Transition Metal Thin Films: A Non-perturbative Theory
Authors:
A. Lessard,
T. H. Moos,
W. Huebner
Abstract:
The magnetocrystalline anisotropy energy E(anis) of free-standing monolayers and thin films of Fe and Ni is determined using two different semi-empirical schemes. Within a tight-binding calculation for the 3d bands alone, we analyze in detail the relation between bandstructure and E(anis), treating spin-orbit coupling non-pertubatively. We find important contributions to E(anis) due to the lifti…
▽ More
The magnetocrystalline anisotropy energy E(anis) of free-standing monolayers and thin films of Fe and Ni is determined using two different semi-empirical schemes. Within a tight-binding calculation for the 3d bands alone, we analyze in detail the relation between bandstructure and E(anis), treating spin-orbit coupling non-pertubatively. We find important contributions to E(anis) due to the lifting of band degeneracies near the Fermi level by SOC. The important role of degeneracies is supported by the calculation of the electron temperature dependence of the magnetocrystalline anisotropy energy, which decreases with the temperature increasing on a scale of several hundred K. In general, E(anis) scales with the square of the SOC constant. Including 4s bands and s-d hybridization, the combined interpolation scheme yields anisotropy energies that quantitatively agree well with experiments for Fe and Ni monolayers on Cu(001). Finally, the anisotropy energy is calculated for systems of up to 14 layers. Even after including s-bands and for multilayers, the importance of degeneracies persists. Considering a fixed fct-Fe structure, we find a reorientation of the magnetization from perpendicular to in-plane at about 4 layers. For Ni, we find the correct in-plane easy-axis for the monolayer. However, since the anisotropy energy remains nearly constant, we do not find the experimentally observed reorientation.
△ Less
Submitted 3 September, 1996;
originally announced September 1996.
