Showing 1–50 of 92 results for author: Chan, E

AuNR-SMA: Automated Gold Nanorod Spectral Morphology Analysis Pipeline

Authors: Samuel P. Gleason, Jakob C. Dahl, Mahmoud Elzouka, Xingzhi Wang, Dana O. Byrne, Mumtaz Gababa, Hannah Cho, Ravi Prasher, Sean Lubner, Emory Chan, A. Paul Alivisatos

Abstract: The development of a colloidal synthesis procedure to produce nanomaterials of a specific size with high shape and size purity is often a time consuming, iterative process. This is often due to the time, resource and expertise intensive characterization methods required for quantitative determination of nanomaterial size and shape. Absorption spectroscopy is often the easiest method of colloidal n… ▽ More The development of a colloidal synthesis procedure to produce nanomaterials of a specific size with high shape and size purity is often a time consuming, iterative process. This is often due to the time, resource and expertise intensive characterization methods required for quantitative determination of nanomaterial size and shape. Absorption spectroscopy is often the easiest method of colloidal nanomaterial characterization, however, due to the lack of a reliable method to extract nanoparticle shapes from absorption spectroscopy, it is generally treated as a more qualitative measure for metal nanoparticles. This work demonstrates a gold nanorod (AuNR) spectral morphology analysis (SMA) tool, AuNR-SMA, which is a fast and accurate method to extract quantitative information about an AuNR sample's structural parameters from its absorption spectra. We apply AuNR-SMA in three distinct applications. First, we demonstrate its utility as an automated analysis tool in a high throughput AuNR synthesis procedure by generating quantitative size information from optical spectra. Second, we use the predictions generated by this model to train a machine learning model capable of predicting the resulting AuNR size distributions from the reaction conditions used to synthesize them. Third, we turn this model to spectra extracted from the literature where no size distributions are reported to impute unreported quantitative information of AuNR synthesis. This approach can potentially be extended to any other nanocrystal system where the absorption spectra are size dependent and accurate numerical simulation of the absorption spectra is possible. In addition, this pipeline could be integrated into automated synthesis apparatuses to provide interpretable data from simple measurements and help explore the synthesis science of nanoparticles in a rational manner or facilitate closed-loop workflows. △ Less

Submitted 11 July, 2024; originally announced July 2024.

Expressions for weight 2 cusp forms in holomorphic eta quotients

Authors: Elisabeth Chan, Lewis Combes

Abstract: We attempt to compute expressions in terms of the Dedekind eta function for all weight 2 new cusp forms with level up to 100, using methods of Allen et. al. In cases where no expression exists, we raise the level instead of the weight, meaning our eta quotients are always holomorphic. Of the forms we examine, we find expressions for all but 4. We also present methods to find expressions with relat… ▽ More We attempt to compute expressions in terms of the Dedekind eta function for all weight 2 new cusp forms with level up to 100, using methods of Allen et. al. In cases where no expression exists, we raise the level instead of the weight, meaning our eta quotients are always holomorphic. Of the forms we examine, we find expressions for all but 4. We also present methods to find expressions with relatively few terms, and how these expressions can be used to demonstrate zeroes of modular forms. △ Less

Submitted 8 July, 2024; originally announced July 2024.

MSC Class: 11F03; 11F20; 11Y40

Unsupervised Analysis of Alzheimer's Disease Signatures using 3D Deformable Autoencoders

Authors: Mehmet Yigit Avci, Emily Chan, Veronika Zimmer, Daniel Rueckert, Benedikt Wiestler, Julia A. Schnabel, Cosmin I. Bercea

Abstract: With the increasing incidence of neurodegenerative diseases such as Alzheimer's Disease (AD), there is a need for further research that enhances detection and monitoring of the diseases. We present MORPHADE (Morphological Autoencoders for Alzheimer's Disease Detection), a novel unsupervised learning approach which uses deformations to allow the analysis of 3D T1-weighted brain images. To the best… ▽ More With the increasing incidence of neurodegenerative diseases such as Alzheimer's Disease (AD), there is a need for further research that enhances detection and monitoring of the diseases. We present MORPHADE (Morphological Autoencoders for Alzheimer's Disease Detection), a novel unsupervised learning approach which uses deformations to allow the analysis of 3D T1-weighted brain images. To the best of our knowledge, this is the first use of deformations with deep unsupervised learning to not only detect, but also localize and assess the severity of structural changes in the brain due to AD. We obtain markedly higher anomaly scores in clinically important areas of the brain in subjects with AD compared to healthy controls, showcasing that our method is able to effectively locate AD-related atrophy. We additionally observe a visual correlation between the severity of atrophy highlighted in our anomaly maps and medial temporal lobe atrophy scores evaluated by a clinical expert. Finally, our method achieves an AUROC of 0.80 in detecting AD, out-performing several supervised and unsupervised baselines. We believe our framework shows promise as a tool towards improved understanding, monitoring and detection of AD. To support further research and application, we have made our code publicly available at github.com/ci-ber/MORPHADE. △ Less

Submitted 4 July, 2024; originally announced July 2024.

Comments: 11 pages, 5 figures

Direct Observation of Morphological and Chemical Changes During the Oxidation of Model Inorganic Ligand-Capped Particles

Authors: Maximilian Jaugstetter, Xiao Qi, Emory Chan, Miquel Salmeron, Kevin R. Wilson, Slavomír Nemšák, Hendrik Bluhm

Abstract: Functionalization and volatilization are competing reactions during the oxidation of carbonaceous materials and are important processes in many different areas of science and technology. Here we present a combined ambient pressure X-ray photoelectron spectroscopy (APXPS) and grazing incidence X-ray scattering (GIXS) investigation of the oxidation of oleic acid ligands surrounding NaYF4 nanoparticl… ▽ More Functionalization and volatilization are competing reactions during the oxidation of carbonaceous materials and are important processes in many different areas of science and technology. Here we present a combined ambient pressure X-ray photoelectron spectroscopy (APXPS) and grazing incidence X-ray scattering (GIXS) investigation of the oxidation of oleic acid ligands surrounding NaYF4 nanoparticles (NPs) deposited onto SiOx/Si substrates. While APXPS monitors the evolution of the oxidation products, GIXS provides insight into the morphology of the ligands and particles before and after the oxidation. Our investigation shows that the oxidation of the oleic acid ligands proceeds at O2 partial pressures of below 1 mbar in the presence of X-rays, with the oxidation eventually reaching a steady state in which mainly CHx and -COOH functional groups are observed. The scattering data reveal that the oxidation and volatilization reaction proceeds preferentially on the side of the particle facing the gas phase, leading to the formation of a chemically and morphologically asymmetric ligand layer. This comprehensive picture of the oxidation process could only be obtained by combining the X-ray scattering and APXPS data. The investigation presented here lays the foundation for further studies of the stability of NP layers in the presence of reactive trace gasses and ionizing radiation, and for other nanoscale systems where chemical and morphological changes happen simultaneously and cannot be understood in isolation. △ Less

Submitted 30 June, 2024; originally announced July 2024.

Comments: 18 pages, 8 figures

Interpretable Representation Learning of Cardiac MRI via Attribute Regularization

Authors: Maxime Di Folco, Cosmin I. Bercea, Emily Chan, Julia A. Schnabel

Abstract: Interpretability is essential in medical imaging to ensure that clinicians can comprehend and trust artificial intelligence models. Several approaches have been recently considered to encode attributes in the latent space to enhance its interpretability. Notably, attribute regularization aims to encode a set of attributes along the dimensions of a latent representation. However, this approach is b… ▽ More Interpretability is essential in medical imaging to ensure that clinicians can comprehend and trust artificial intelligence models. Several approaches have been recently considered to encode attributes in the latent space to enhance its interpretability. Notably, attribute regularization aims to encode a set of attributes along the dimensions of a latent representation. However, this approach is based on Variational AutoEncoder and suffers from blurry reconstruction. In this paper, we propose an Attributed-regularized Soft Introspective Variational Autoencoder that combines attribute regularization of the latent space within the framework of an adversarially trained variational autoencoder. We demonstrate on short-axis cardiac Magnetic Resonance images of the UK Biobank the ability of the proposed method to address blurry reconstruction issues of variational autoencoder methods while preserving the latent space interpretability. △ Less

Submitted 5 July, 2024; v1 submitted 12 June, 2024; originally announced June 2024.

Comments: arXiv admin note: substantial text overlap with arXiv:2312.08915

Solving Inverse Problems in Protein Space Using Diffusion-Based Priors

Authors: Axel Levy, Eric R. Chan, Sara Fridovich-Keil, Frédéric Poitevin, Ellen D. Zhong, Gordon Wetzstein

Abstract: The interaction of a protein with its environment can be understood and controlled via its 3D structure. Experimental methods for protein structure determination, such as X-ray crystallography or cryogenic electron microscopy, shed light on biological processes but introduce challenging inverse problems. Learning-based approaches have emerged as accurate and efficient methods to solve these invers… ▽ More The interaction of a protein with its environment can be understood and controlled via its 3D structure. Experimental methods for protein structure determination, such as X-ray crystallography or cryogenic electron microscopy, shed light on biological processes but introduce challenging inverse problems. Learning-based approaches have emerged as accurate and efficient methods to solve these inverse problems for 3D structure determination, but are specialized for a predefined type of measurement. Here, we introduce a versatile framework to turn raw biophysical measurements of varying types into 3D atomic models. Our method combines a physics-based forward model of the measurement process with a pretrained generative model providing a task-agnostic, data-driven prior. Our method outperforms posterior sampling baselines on both linear and non-linear inverse problems. In particular, it is the first diffusion-based method for refining atomic models from cryo-EM density maps. △ Less

Submitted 6 June, 2024; originally announced June 2024.

Logic Query of Thoughts: Guiding Large Language Models to Answer Complex Logic Queries with Knowledge Graphs

Authors: Lihui Liu, Zihao Wang, Ruizhong Qiu, Yikun Ban, Eunice Chan, Yangqiu Song, **grui He, Hanghang Tong

Abstract: Despite the superb performance in many tasks, large language models (LLMs) bear the risk of generating hallucination or even wrong answers when confronted with tasks that demand the accuracy of knowledge. The issue becomes even more noticeable when addressing logic queries that require multiple logic reasoning steps. On the other hand, knowledge graph (KG) based question answering methods are capa… ▽ More Despite the superb performance in many tasks, large language models (LLMs) bear the risk of generating hallucination or even wrong answers when confronted with tasks that demand the accuracy of knowledge. The issue becomes even more noticeable when addressing logic queries that require multiple logic reasoning steps. On the other hand, knowledge graph (KG) based question answering methods are capable of accurately identifying the correct answers with the help of knowledge graph, yet its accuracy could quickly deteriorate when the knowledge graph itself is sparse and incomplete. It remains a critical challenge on how to integrate knowledge graph reasoning with LLMs in a mutually beneficial way so as to mitigate both the hallucination problem of LLMs as well as the incompleteness issue of knowledge graphs. In this paper, we propose 'Logic-Query-of-Thoughts' (LGOT) which is the first of its kind to combine LLMs with knowledge graph based logic query reasoning. LGOT seamlessly combines knowledge graph reasoning and LLMs, effectively breaking down complex logic queries into easy to answer subquestions. Through the utilization of both knowledge graph reasoning and LLMs, it successfully derives answers for each subquestion. By aggregating these results and selecting the highest quality candidate answers for each step, LGOT achieves accurate results to complex questions. Our experimental findings demonstrate substantial performance enhancements, with up to 20% improvement over ChatGPT. △ Less

Submitted 13 April, 2024; v1 submitted 17 March, 2024; originally announced April 2024.

Classification of Nasopharyngeal Cases using DenseNet Deep Learning Architecture

Authors: W. S. H. M. W. Ahmad, M. F. A. Fauzi, M. K. Abdullahi, Jenny T. H. Lee, N. S. A. Basry, A Yahaya, A. M. Ismail, A. Adam, Elaine W. L. Chan, F. S. Abas

Abstract: Nasopharyngeal carcinoma (NPC) is one of the understudied yet deadliest cancers in South East Asia. In Malaysia, the prevalence is identified mainly in Sarawak, among the ethnic of Bidayuh. NPC is often late-diagnosed because it is asymptomatic at the early stage. There are several tissue representations from the nasopharynx biopsy, such as nasopharyngeal inflammation (NPI), lymphoid hyperplasia (… ▽ More Nasopharyngeal carcinoma (NPC) is one of the understudied yet deadliest cancers in South East Asia. In Malaysia, the prevalence is identified mainly in Sarawak, among the ethnic of Bidayuh. NPC is often late-diagnosed because it is asymptomatic at the early stage. There are several tissue representations from the nasopharynx biopsy, such as nasopharyngeal inflammation (NPI), lymphoid hyperplasia (LHP), nasopharyngeal carcinoma (NPC) and normal tissue. This paper is our first initiative to identify the difference between NPC, NPI and normal cases. Seven whole slide images (WSIs) with gigapixel resolutions from seven different patients and two hospitals were experimented with using two test setups, consisting of a different set of images. The tissue regions are patched into smaller blocks and classified using DenseNet architecture with 21 dense layers. Two tests are carried out, each for proof of concept (Test 1) and real-test scenario (Test 2). The accuracy achieved for NPC class is 94.8% for Test 1 and 67.0% for Test 2. △ Less

Submitted 4 April, 2024; originally announced April 2024.

Comments: This article has been accepted in the Journal of Engineering Science and Technology (JESTEC) and awaiting publication

Infrared nanosensors of pico- to micro-newton forces

Authors: Natalie Fardian-Melamed, Artiom Skripka, Changhwan Lee, Benedikt Ursprung, Thomas P. Darlington, Ayelet Teitelboim, Xiao Qi, Maoji Wang, Jordan M. Gerton, Bruce E. Cohen, Emory M. Chan, P. James Schuck

Abstract: Mechanical force is an essential feature for many physical and biological processes.1-12 Remote measurement of mechanical signals with high sensitivity and spatial resolution is needed for diverse applications, including robotics,13 biophysics,14-20 energy storage,21-24 and medicine.25-27 Nanoscale luminescent force sensors excel at measuring piconewton forces,28-32 while larger sensors have prove… ▽ More Mechanical force is an essential feature for many physical and biological processes.1-12 Remote measurement of mechanical signals with high sensitivity and spatial resolution is needed for diverse applications, including robotics,13 biophysics,14-20 energy storage,21-24 and medicine.25-27 Nanoscale luminescent force sensors excel at measuring piconewton forces,28-32 while larger sensors have proven powerful in probing micronewton forces.33,34 However, large gaps remain in the force magnitudes that can be probed remotely from subsurface or interfacial sites, and no individual, non-invasive sensor is capable of measuring over the large dynamic range needed to understand many systems.35,36 Here, we demonstrate Tm3+-doped avalanching nanoparticle37 force sensors that can be addressed remotely by deeply penetrating near-infrared (NIR) light and can detect piconewton to micronewton forces with a dynamic range spanning more than four orders of magnitude. Using atomic force microscopy coupled with single-nanoparticle optical spectroscopy, we characterize the mechanical sensitivity of the photon avalanching process and reveal its exceptional force responsiveness. By manipulating the Tm3+ concentrations and energy transfer within the nanosensors, we demonstrate different optical force-sensing modalities, including mechanobrightening and mechanochromism. The adaptability of these nanoscale optical force sensors, along with their multiscale sensing capability, enable operation in the dynamic and versatile environments present in real-world, complex structures spanning biological organisms to nanoelectromechanical systems (NEMS). △ Less

Submitted 2 April, 2024; originally announced April 2024.

Intrinsic Optical Bistability of Photon Avalanching Nanocrystals

Authors: Artiom Skripka, Zhuolei Zhang, Xiao Qi, Benedikt Ursprung, Peter Ercius, Bruce E. Cohen, P. James Schuck, Daniel Jaque, Emory M. Chan

Abstract: Optically bistable materials respond to a single input with two possible optical outputs, contingent upon excitation history. Such materials would be ideal for optical switching and memory, yet limited understanding of intrinsic optical bistability (IOB) prevents development of nanoscale IOB materials suitable for devices. Here, we demonstrate IOB in Nd3+-doped KPb2Cl5 avalanching nanoparticles (A… ▽ More Optically bistable materials respond to a single input with two possible optical outputs, contingent upon excitation history. Such materials would be ideal for optical switching and memory, yet limited understanding of intrinsic optical bistability (IOB) prevents development of nanoscale IOB materials suitable for devices. Here, we demonstrate IOB in Nd3+-doped KPb2Cl5 avalanching nanoparticles (ANPs), which switch with high contrast between luminescent and non-luminescent states, with hysteresis characteristic of bistability. We elucidate a nonthermal mechanism in which IOB originates from suppressed nonradiative relaxation in Nd3+ ions and from the positive feedback of photon avalanching, resulting in extreme, >200th-order optical nonlinearities. Modulation of laser pulsing tunes hysteresis widths, and dual-laser excitation enables transistor-like optical switching. This control over nanoscale IOB establishes ANPs for photonic devices in which light is used to manipulate light. △ Less

Submitted 6 March, 2024; originally announced March 2024.

Dirac mass induced by optical gain and loss

Authors: Letian Yu, Haoran Xue, Ruixiang Guo, Eng Aik Chan, Yun Yong Terh, Cesare Soci, Baile Zhang, Y. D. Chong

Abstract: Mass is commonly regarded as an intrinsic property of matter, but modern physics reveals particle masses to have complex origins, such as the Higgs mechanism in high-energy physics. In crystal lattices such as graphene, relativistic Dirac particles can exist as low-energy quasiparticles with masses imparted by lattice symmetry-breaking perturbations. These mass-generating mechanisms all assume Her… ▽ More Mass is commonly regarded as an intrinsic property of matter, but modern physics reveals particle masses to have complex origins, such as the Higgs mechanism in high-energy physics. In crystal lattices such as graphene, relativistic Dirac particles can exist as low-energy quasiparticles with masses imparted by lattice symmetry-breaking perturbations. These mass-generating mechanisms all assume Hermiticity, or the conservation of energy in detail. Using a photonic synthetic lattice, we show experimentally that Dirac masses can be generated via non-Hermitian perturbations based on optical gain and loss. We then explore how the space-time engineering of the gain/loss-induced Dirac mass affects the quasiparticles. As we show, the quasiparticles undergo Klein tunnelling at spatial boundaries, but a local breaking of a non-Hermitian symmetry can produce a novel flux nonconservation effect at the domain walls. At a temporal boundary that abruptly flips the sign of the Dirac mass, we observe a variant of the time reflection phenomenon: in the nonrelativistic limit, the Dirac quasiparticle reverses its velocity, while in the relativistic limit the original velocity is retained. △ Less

Submitted 14 April, 2024; v1 submitted 27 January, 2024; originally announced January 2024.

Acoustic lattice instabilities at the magneto-structural transition in Fe$_{1.057(7)}$Te

Authors: K. Guratinder, E. Chan, E. E. Rodriguez, J. A. Rodriguez-Rivera, U. Stuhr, A. Stunault, R. Travers, M. A. Green, N. Qureshi, C. Stock

Abstract: Fe$_{1.057(7)}$Te undergoes a first-order tetragonal to monoclinc structural transition at T$_{S} \sim 70$ K, breaking the C$_{4}$ lattice symmetry and simultaneously breaking time reversal symmetry with bicollinear magnetic order. We investigate the soft acoustic lattice dynamics near this combined magneto-structural transition. We apply spherically neutron polarimetry to study the static magneti… ▽ More Fe$_{1.057(7)}$Te undergoes a first-order tetragonal to monoclinc structural transition at T$_{S} \sim 70$ K, breaking the C$_{4}$ lattice symmetry and simultaneously breaking time reversal symmetry with bicollinear magnetic order. We investigate the soft acoustic lattice dynamics near this combined magneto-structural transition. We apply spherically neutron polarimetry to study the static magnetism near this transition, characterized with x-ray powder diffraction, and find no evidence of static incommensurate magnetic correlations near the onset of monoclinic and bicollinear antiferromagnetic order. This fixes the position of our single crystal sample in the Fe$_{1+x}$Te phase diagram in the magnetic bicollinear region and illustrates that our sample statically undergoes a transition from a paramagnetic phase to a low-temperature bicollinear phase. We then apply neutron spectroscopy to study the acoustic phonons, related to elastic deformations of the lattice. We find a temperature dependent soft acoustic branch for phonons propagating along [010] and polarized along [100]. The slope of this acoustic phonon branch is sensitive to the elastic constant $C_{66}$ and the shear modulus. The temperature dependence of this branch displays a softening with a minimum near the magneto-structural transition of T$_{S}$ $\sim$ 70 K and a recovery within the magnetically ordered low temperature phase. Soft acoustic instabilities are present in the collinear phases of the chalcogenides Fe$_{1+x}$Te where nematic order found in Fe$_{1+δ}$Se is absent. We speculate, based on localized single-ion magnetism, that the relative energy scale of magnetic spin-orbital coupling on the Fe$^{2+}$ transition metal ion is important for the presence of a nematicity in the chalcogenides. △ Less

Submitted 10 December, 2023; v1 submitted 28 November, 2023; originally announced November 2023.

Comments: (13 pages, 10 figures, to be published in Physical Review B)

Arbitrary Engineering of Spatial Caustics with 3D-printed Metasurfaces

Authors: Xiaoyan Zhou, Hongtao Wang, Shuxi Liu, Hao Wang, John You En Chan, Cheng-Feng Pan, Daomu Zhao, Joel K. W. Yang, Cheng-Wei Qiu

Abstract: Caustics occur in diverse physical systems, spanning the nano-scale in electron microscopy to astronomical-scale in gravitational lensing. As envelopes of rays, optical caustics result in sharp edges or extended networks. Caustics in structured light, characterized by complex-amplitude distributions, have innovated numerous applications including particle manipulation, high-resolution imaging tech… ▽ More Caustics occur in diverse physical systems, spanning the nano-scale in electron microscopy to astronomical-scale in gravitational lensing. As envelopes of rays, optical caustics result in sharp edges or extended networks. Caustics in structured light, characterized by complex-amplitude distributions, have innovated numerous applications including particle manipulation, high-resolution imaging techniques, and optical communication. However, these applications have encountered limitations due to a major challenge in engineering caustic fields with customizable propagation trajectories and in-plane intensity profiles. Here, we introduce the compensation phase via 3D-printed metasurfaces to shape caustic fields with curved trajectories in free space. The in-plane caustic patterns can be preserved or morphed from one structure to another during propagation. Large-scale fabrication of these metasurfaces is enabled by the fast-prototy** and cost-effective two-photon polymerization lithography. Our optical elements with the ultra-thin profile and sub-millimeter extension offer a compact solution to generating caustic structured light for beam sha**, high-resolution microscopy, and light-matter-interaction studies. △ Less

Submitted 27 November, 2023; originally announced November 2023.

Retrieving positions of closely packed sub-wavelength nanoparticles from their diffraction patterns

Authors: Benquan Wang, Ruyi An, Eng Aik Chan, Giorgio Adamo, **-Kyu So, Yewen Li, Zexiang Shen, Bo An, Nikolay I. Zheludev

Abstract: Distinguishing two objects or point sources located closer than the Rayleigh distance is impossible in conventional microscopy. Understandably, the task becomes increasingly harder with a growing number of particles placed in close proximity. It has been recently demonstrated that subwavelength nanoparticles in closely packed clusters can be counted by AI-enabled analysis of the diffraction patter… ▽ More Distinguishing two objects or point sources located closer than the Rayleigh distance is impossible in conventional microscopy. Understandably, the task becomes increasingly harder with a growing number of particles placed in close proximity. It has been recently demonstrated that subwavelength nanoparticles in closely packed clusters can be counted by AI-enabled analysis of the diffraction patterns of coherent light scattered by the cluster. Here we show that deep learning analysis can determine the actual position of the nanoparticle in the cluster of subwavelength particles from a sing-shot diffraction pattern even if they are separated by distances below the Rayleigh resolution limit of a conventional microscope. △ Less

Submitted 17 November, 2023; originally announced November 2023.

Comments: 6 pages, 3 figures

ZeroNVS: Zero-Shot 360-Degree View Synthesis from a Single Image

Authors: Kyle Sargent, Zizhang Li, Tanmay Shah, Charles Herrmann, Hong-Xing Yu, Yunzhi Zhang, Eric Ryan Chan, Dmitry Lagun, Li Fei-Fei, Deqing Sun, Jiajun Wu

Abstract: We introduce a 3D-aware diffusion model, ZeroNVS, for single-image novel view synthesis for in-the-wild scenes. While existing methods are designed for single objects with masked backgrounds, we propose new techniques to address challenges introduced by in-the-wild multi-object scenes with complex backgrounds. Specifically, we train a generative prior on a mixture of data sources that capture obje… ▽ More We introduce a 3D-aware diffusion model, ZeroNVS, for single-image novel view synthesis for in-the-wild scenes. While existing methods are designed for single objects with masked backgrounds, we propose new techniques to address challenges introduced by in-the-wild multi-object scenes with complex backgrounds. Specifically, we train a generative prior on a mixture of data sources that capture object-centric, indoor, and outdoor scenes. To address issues from data mixture such as depth-scale ambiguity, we propose a novel camera conditioning parameterization and normalization scheme. Further, we observe that Score Distillation Sampling (SDS) tends to truncate the distribution of complex backgrounds during distillation of 360-degree scenes, and propose "SDS anchoring" to improve the diversity of synthesized novel views. Our model sets a new state-of-the-art result in LPIPS on the DTU dataset in the zero-shot setting, even outperforming methods specifically trained on DTU. We further adapt the challenging Mip-NeRF 360 dataset as a new benchmark for single-image novel view synthesis, and demonstrate strong performance in this setting. Our code and data are at http://kylesargent.github.io/zeronvs/ △ Less

Submitted 23 April, 2024; v1 submitted 27 October, 2023; originally announced October 2023.

Comments: Accepted to CVPR 2024. 12 pages

State of the Art on Diffusion Models for Visual Computing

Authors: Ryan Po, Wang Yifan, Vladislav Golyanik, Kfir Aberman, Jonathan T. Barron, Amit H. Bermano, Eric Ryan Chan, Tali Dekel, Aleksander Holynski, Angjoo Kanazawa, C. Karen Liu, Lingjie Liu, Ben Mildenhall, Matthias Nießner, Björn Ommer, Christian Theobalt, Peter Wonka, Gordon Wetzstein

Abstract: The field of visual computing is rapidly advancing due to the emergence of generative artificial intelligence (AI), which unlocks unprecedented capabilities for the generation, editing, and reconstruction of images, videos, and 3D scenes. In these domains, diffusion models are the generative AI architecture of choice. Within the last year alone, the literature on diffusion-based tools and applicat… ▽ More The field of visual computing is rapidly advancing due to the emergence of generative artificial intelligence (AI), which unlocks unprecedented capabilities for the generation, editing, and reconstruction of images, videos, and 3D scenes. In these domains, diffusion models are the generative AI architecture of choice. Within the last year alone, the literature on diffusion-based tools and applications has seen exponential growth and relevant papers are published across the computer graphics, computer vision, and AI communities with new works appearing daily on arXiv. This rapid growth of the field makes it difficult to keep up with all recent developments. The goal of this state-of-the-art report (STAR) is to introduce the basic mathematical concepts of diffusion models, implementation details and design choices of the popular Stable Diffusion model, as well as overview important aspects of these generative AI tools, including personalization, conditioning, inversion, among others. Moreover, we give a comprehensive overview of the rapidly growing literature on diffusion-based generation and editing, categorized by the type of generated medium, including 2D images, videos, 3D objects, locomotion, and 4D scenes. Finally, we discuss available datasets, metrics, open challenges, and social implications. This STAR provides an intuitive starting point to explore this exciting topic for researchers, artists, and practitioners alike. △ Less

Submitted 11 October, 2023; originally announced October 2023.

Evaluating the Sensitivity of Mortality Attributable to Pollution to Modeling Choices: A Case Study for Colorado

Authors: Priyanka N. deSouza, Susan Anenberg, Neal Fann, Lisa M. McKenzie, Elizabeth Chan, Ananya Roy, Jose L. Jimenez, William Raich, Henry Roman, Patrick L. Kinney

Abstract: We evaluated the sensitivity of estimated PM2.5 and NO2 health impacts to varying key input parameters and assumptions including: 1) the spatial scale at which impacts are estimated, 2) using either a single concentration-response function (CRF) or using racial/ethnic group specific CRFs from the same epidemiologic study, 3) assigning exposure to residents based on home, instead of home and work l… ▽ More We evaluated the sensitivity of estimated PM2.5 and NO2 health impacts to varying key input parameters and assumptions including: 1) the spatial scale at which impacts are estimated, 2) using either a single concentration-response function (CRF) or using racial/ethnic group specific CRFs from the same epidemiologic study, 3) assigning exposure to residents based on home, instead of home and work locations. This analysis was carried out for the state of Colorado. We found that the spatial scale of the analysis influences the magnitude of NO2, but not PM2.5, attributable deaths. Using county-level predictions instead of 1 km2 predictions of NO2 resulted in a lower estimate of mortality attributable to NO2 by ~ 50% for all of Colorado for each year between 2000-2020. Using an all-population CRF instead of racial/ethnic group specific CRFs results in a higher estimate of annual mortality attributable to PM2.5 by a factor 1.3 for the white population and a lower estimate of mortality attributable to PM2.5 by factors of 0.4 and 0.8 for Black and Hispanic residents, respectively. Using racial/ethnic group specific CRFs did not result in a different estimation of NO2 attributable mortality for white residents, but led to lower estimates of mortality by a factor of ~ 0.5 for Black residents, and by a factor of 2.9 for to Hispanic residents. Using NO2 based on home instead of home and workplace locations results in a smaller estimate of annual mortality attributable to NO2 for all of Colorado by ~0.980 each year and 0.997 for PM2.5. △ Less

Submitted 24 September, 2023; originally announced September 2023.

Comments: 24 pages, 6 figures, 2 tables

Ensuring User-side Fairness in Dynamic Recommender Systems

Authors: Hyunsik Yoo, Zhichen Zeng, Jian Kang, Ruizhong Qiu, David Zhou, Zhining Liu, Fei Wang, Charlie Xu, Eunice Chan, Hanghang Tong

Abstract: User-side group fairness is crucial for modern recommender systems, aiming to alleviate performance disparities among user groups defined by sensitive attributes like gender, race, or age. In the ever-evolving landscape of user-item interactions, continual adaptation to newly collected data is crucial for recommender systems to stay aligned with the latest user preferences. However, we observe tha… ▽ More User-side group fairness is crucial for modern recommender systems, aiming to alleviate performance disparities among user groups defined by sensitive attributes like gender, race, or age. In the ever-evolving landscape of user-item interactions, continual adaptation to newly collected data is crucial for recommender systems to stay aligned with the latest user preferences. However, we observe that such continual adaptation often exacerbates performance disparities. This necessitates a thorough investigation into user-side fairness in dynamic recommender systems, an area that has been unexplored in the literature. This problem is challenging due to distribution shifts, frequent model updates, and non-differentiability of ranking metrics. To our knowledge, this paper presents the first principled study on ensuring user-side fairness in dynamic recommender systems. We start with theoretical analyses on fine-tuning v.s. retraining, showing that the best practice is incremental fine-tuning with restart. Guided by our theoretical analyses, we propose FAir Dynamic rEcommender (FADE), an end-to-end fine-tuning framework to dynamically ensure user-side fairness over time. To overcome the non-differentiability of recommendation metrics in the fairness loss, we further introduce Differentiable Hit (DH) as an improvement over the recent NeuralNDCG method, not only alleviating its gradient vanishing issue but also achieving higher efficiency. Besides that, we also address the instability issue of the fairness loss by leveraging the competing nature between the recommendation loss and the fairness loss. Through extensive experiments on real-world datasets, we demonstrate that FADE effectively and efficiently reduces performance disparities with little sacrifice in the overall recommendation performance. △ Less

Submitted 31 March, 2024; v1 submitted 29 August, 2023; originally announced August 2023.

Comments: 19 pages, 20 figures, 2 tables, ACM Web Conference 2024

3D Printed Multilayer Structures for High Numerical Aperture Achromatic Metalenses

Authors: Cheng-Feng Pan, Hao Wang, Hongtao Wang, Parvathi Nair S, Qifeng Ruan, Simon Wredh, Yujie Ke, John You En Chan, Wang Zhang, Cheng-Wei Qiu, Joel K. W. Yang

Abstract: Flat optics consisting of nanostructures of high-refractive-index materials produce lenses with thin form factors that tend to operate only at specific wavelengths. Recent attempts to achieve achromatic lenses uncover a trade-off between the numerical aperture (NA) and bandwidth, which limits performance. Here we propose a new approach to design high NA, broadband and polarization-insensitive mult… ▽ More Flat optics consisting of nanostructures of high-refractive-index materials produce lenses with thin form factors that tend to operate only at specific wavelengths. Recent attempts to achieve achromatic lenses uncover a trade-off between the numerical aperture (NA) and bandwidth, which limits performance. Here we propose a new approach to design high NA, broadband and polarization-insensitive multilayer achromatic metalenses (MAM). We combine topology optimization and full wave simulations to inversely design MAMs and fabricate the structures in low-refractive-index materials by two-photon polymerization lithography. MAMs measuring 20 micrometer in diameter operating in the visible range of 400-800 nm with 0.5 NA and 0.7 NA were achieved with efficiencies of up to 42%. We demonstrate broadband imaging performance of the fabricated MAM under white light, and RGB narrowband illuminations. These results highlight the potential of the 3D printed multilayer structures for realizing broadband and multi-functional meta-devices with inverse design. △ Less

Submitted 27 August, 2023; originally announced August 2023.

Comments: 37 pages, 15 figures

Effect of Mindfulness and Mindful Art on Beginners and Experienced Meditators

Authors: Koonlin Eunice Chan, Joy Bose

Abstract: Mindfulness meditation has been proven to be effective in treating a range of mental and physical conditions. Mindful Art is a type of mindfulness meditation that comprises sessions of drawing, painting and sculpturing with mindfulness for a given length of time. To date, the efficacy of mindful art has not been systematically studied. In this paper, we describe an experimental pilot study on two… ▽ More Mindfulness meditation has been proven to be effective in treating a range of mental and physical conditions. Mindful Art is a type of mindfulness meditation that comprises sessions of drawing, painting and sculpturing with mindfulness for a given length of time. To date, the efficacy of mindful art has not been systematically studied. In this paper, we describe an experimental pilot study on two groups of participants, a beginner group of 21 participants and an experienced meditation group of 9 participants, who had previously practiced mindfulness meditation for more than one year. The beginner group was instructed in mindfulness sitting and moving meditation, while the experienced group was instructed in mindful art making in addition to mindfulness meditation. The instructions were delivered remotely over Tencent Conference and WeChat. The sessions were of 90 minutes duration each, twice per week, with 45 minutes of home practice daily and the length of the study was 21 days. The blood pressure, pulse rate and breathing rates, as well as the subjective degree of relaxation were recorded at every session. At the end of the study, the experienced group reported higher average difference in breath rate and relaxation within each session, while the beginner group reported a greater degree of improvement in breath rate and relaxation over the period of the study, although their scores were lower on average than the experienced group. △ Less

Submitted 24 August, 2023; originally announced August 2023.

Comments: 5 pages, 15 figures

ACM Class: H.5

The minimal computational substrate of fluid intelligence

Authors: Amy PK Nelson, Joe Mole, Guilherme Pombo, Robert J Gray, James K Ruffle, Edgar Chan, Geraint E Rees, Lisa Cipolotti, Parashkev Nachev

Abstract: The quantification of cognitive powers rests on identifying a behavioural task that depends on them. Such dependence cannot be assured, for the powers a task invokes cannot be experimentally controlled or constrained a priori, resulting in unknown vulnerability to failure of specificity and generalisability. Evaluating a compact version of Raven's Advanced Progressive Matrices (RAPM), a widely use… ▽ More The quantification of cognitive powers rests on identifying a behavioural task that depends on them. Such dependence cannot be assured, for the powers a task invokes cannot be experimentally controlled or constrained a priori, resulting in unknown vulnerability to failure of specificity and generalisability. Evaluating a compact version of Raven's Advanced Progressive Matrices (RAPM), a widely used clinical test of fluid intelligence, we show that LaMa, a self-supervised artificial neural network trained solely on the completion of partially masked images of natural environmental scenes, achieves human-level test scores a prima vista, without any task-specific inductive bias or training. Compared with cohorts of healthy and focally lesioned participants, LaMa exhibits human-like variation with item difficulty, and produces errors characteristic of right frontal lobe damage under degradation of its ability to integrate global spatial patterns. LaMa's narrow training and limited capacity -- comparable to the nervous system of the fruit fly -- suggest RAPM may be open to computationally simple solutions that need not necessarily invoke abstract reasoning. △ Less

Submitted 14 August, 2023; originally announced August 2023.

Comments: 26 pages, 5 figures

A generalized approach to photon avalanche upconversion in luminescent nanocrystals

Authors: Artiom Skripka, Minji Lee, Xiao Qi, Jia-Ahn Pan, Haoran Yang, Changhwan Lee, P. James Schuck, Bruce E. Cohen, Daniel Jaque, Emory M. Chan

Abstract: Photon avalanching nanoparticles (ANPs) exhibit extremely nonlinear upconverted emission valuable for sub-diffraction imaging, nanoscale sensing, and optical computing. Avalanching has been demonstrated with Tm3+, Nd3+ or Pr3+-doped nanocrystals, but their emission is limited to 600 and 800 nm, restricting applications. Here, we utilize Gd3+-assisted energy migration to tune the emission wavelengt… ▽ More Photon avalanching nanoparticles (ANPs) exhibit extremely nonlinear upconverted emission valuable for sub-diffraction imaging, nanoscale sensing, and optical computing. Avalanching has been demonstrated with Tm3+, Nd3+ or Pr3+-doped nanocrystals, but their emission is limited to 600 and 800 nm, restricting applications. Here, we utilize Gd3+-assisted energy migration to tune the emission wavelengths of Tm3+-sensitized ANPs and generate highly nonlinear emission of Eu3+, Tb3+, Ho3+, and Er3+ ions. The upconversion intensities of these spectrally discrete ANPs scale with the nonlinearity factor s = 10-17 under 1064 nm excitation at power densities as low as 6 kW/cm2. This strategy for imprinting avalanche behavior on remote emitters can be extended to fluorophores adjacent to ANPs, as we demonstrate with CdS/CdSe/CdS core/shell/shell quantum dots. ANPs with rationally designed energy transfer networks provide the means to transform conventional linear emitters into a highly nonlinear ones, expanding the use of photon avalanching in biological, chemical, and photonic applications. △ Less

Submitted 9 June, 2023; originally announced June 2023.

Comments: 13 pages, 5 figures

Single-Shot Implicit Morphable Faces with Consistent Texture Parameterization

Authors: Connor Z. Lin, Koki Nagano, Jan Kautz, Eric R. Chan, Umar Iqbal, Leonidas Guibas, Gordon Wetzstein, Sameh Khamis

Abstract: There is a growing demand for the accessible creation of high-quality 3D avatars that are animatable and customizable. Although 3D morphable models provide intuitive control for editing and animation, and robustness for single-view face reconstruction, they cannot easily capture geometric and appearance details. Methods based on neural implicit representations, such as signed distance functions (S… ▽ More There is a growing demand for the accessible creation of high-quality 3D avatars that are animatable and customizable. Although 3D morphable models provide intuitive control for editing and animation, and robustness for single-view face reconstruction, they cannot easily capture geometric and appearance details. Methods based on neural implicit representations, such as signed distance functions (SDF) or neural radiance fields, approach photo-realism, but are difficult to animate and do not generalize well to unseen data. To tackle this problem, we propose a novel method for constructing implicit 3D morphable face models that are both generalizable and intuitive for editing. Trained from a collection of high-quality 3D scans, our face model is parameterized by geometry, expression, and texture latent codes with a learned SDF and explicit UV texture parameterization. Once trained, we can reconstruct an avatar from a single in-the-wild image by leveraging the learned prior to project the image into the latent space of our model. Our implicit morphable face models can be used to render an avatar from novel views, animate facial expressions by modifying expression codes, and edit textures by directly painting on the learned UV-texture maps. We demonstrate quantitatively and qualitatively that our method improves upon photo-realism, geometry, and expression accuracy compared to state-of-the-art methods. △ Less

Submitted 4 May, 2023; originally announced May 2023.

Comments: SIGGRAPH 2023, Project Page: https://research.nvidia.com/labs/toronto-ai/ssif

Real-Time Radiance Fields for Single-Image Portrait View Synthesis

Authors: Alex Trevithick, Matthew Chan, Michael Stengel, Eric R. Chan, Chao Liu, Zhiding Yu, Sameh Khamis, Manmohan Chandraker, Ravi Ramamoorthi, Koki Nagano

Abstract: We present a one-shot method to infer and render a photorealistic 3D representation from a single unposed image (e.g., face portrait) in real-time. Given a single RGB input, our image encoder directly predicts a canonical triplane representation of a neural radiance field for 3D-aware novel view synthesis via volume rendering. Our method is fast (24 fps) on consumer hardware, and produces higher q… ▽ More We present a one-shot method to infer and render a photorealistic 3D representation from a single unposed image (e.g., face portrait) in real-time. Given a single RGB input, our image encoder directly predicts a canonical triplane representation of a neural radiance field for 3D-aware novel view synthesis via volume rendering. Our method is fast (24 fps) on consumer hardware, and produces higher quality results than strong GAN-inversion baselines that require test-time optimization. To train our triplane encoder pipeline, we use only synthetic data, showing how to distill the knowledge from a pretrained 3D GAN into a feedforward encoder. Technical contributions include a Vision Transformer-based triplane encoder, a camera data augmentation strategy, and a well-designed loss function for synthetic data training. We benchmark against the state-of-the-art methods, demonstrating significant improvements in robustness and image quality in challenging real-world settings. We showcase our results on portraits of faces (FFHQ) and cats (AFHQ), but our algorithm can also be applied in the future to other categories with a 3D-aware image generator. △ Less

Submitted 3 May, 2023; originally announced May 2023.

Comments: Project page: https://research.nvidia.com/labs/nxp/lp3d/

Neutron scattering sum rules, symmetric exchanges, and helicoidal magnetism in MnSb$_2$O$_6$

Authors: E. Chan, H. Lane, J. Pásztorová, M. Songvilay, R. D. Johnson, R. Downie, J-W. G. Bos, J. A. Rodriguez-Rivera, S. -W. Cheong, R. A. Ewings, N. Qureshi, C. Stock

Abstract: MnSb$_{2}$O$_{6}$ is based on the noncentrosymmetric $P321$ space group with magnetic Mn$^{2+}$ ($S={5/2}$, $L\approx 0$) spins ordering below $T_{\mathrm{N}}=12$ K in a helicoidal structure. The ground state magnetic structure, expected to be built and originate from 7 Heisenberg exchange constants, has been shown to be coupled to the underlying crystallographic chirality with polar domain switch… ▽ More MnSb$_{2}$O$_{6}$ is based on the noncentrosymmetric $P321$ space group with magnetic Mn$^{2+}$ ($S={5/2}$, $L\approx 0$) spins ordering below $T_{\mathrm{N}}=12$ K in a helicoidal structure. The ground state magnetic structure, expected to be built and originate from 7 Heisenberg exchange constants, has been shown to be coupled to the underlying crystallographic chirality with polar domain switching being reported. We apply neutron spectroscopy to extract these symmetric exchange constants. Given the high complexity of the magnetic exchange network, crystallographic structure and complications fitting linear spin-wave models, we take advantage of multiplexed neutron instrumentation to use the first moment sum rule of neutron scattering to estimate the 7 exchange constants. We then use these parameters to calculate the low-energy spin-waves in the Néel state to reproduce the neutron response without strong antisymmetric coupling. Using Green's response functions, the stability of long-wavelength excitations in the context of proposed magnetic structures is then discussed. The results show the presence of strong exchange constants for the chiral exchange pathways and illustrate an underlying coupling between crystallographic and magnetic ``chirality" through predominantely symmetric exchange. △ Less

Submitted 12 April, 2023; originally announced April 2023.

Journal ref: Phys. Rev. B 107, 144420 (2023)

Quorum Subsumption for Heterogeneous Quorum Systems

Authors: Xiao Li, Eric Chan, Mohsen Lesani

Abstract: Byzantine quorum systems provide higher throughput than proof-of-work and incur modest energy consumption. Further, their modern incarnations incorporate personalized and heterogeneous trust. Thus, they are emerging as an appealing candidate for global financial infrastructure. However, since their quorums are not uniform across processes anymore, the properties that they should maintain to suppor… ▽ More Byzantine quorum systems provide higher throughput than proof-of-work and incur modest energy consumption. Further, their modern incarnations incorporate personalized and heterogeneous trust. Thus, they are emerging as an appealing candidate for global financial infrastructure. However, since their quorums are not uniform across processes anymore, the properties that they should maintain to support abstractions such as reliable broadcast and consensus are not well-understood. It has been shown that the two properties quorum intersection and availability are necessary. In this paper, we prove that they are not sufficient. We then define the notion of quorum subsumption, and show that the three conditions together are sufficient: we present reliable broadcast and consensus protocols, and prove their correctness for quorum systems that provide the three properties. △ Less

Submitted 10 August, 2023; v1 submitted 11 April, 2023; originally announced April 2023.

Generative Novel View Synthesis with 3D-Aware Diffusion Models

Authors: Eric R. Chan, Koki Nagano, Matthew A. Chan, Alexander W. Bergman, Jeong Joon Park, Axel Levy, Miika Aittala, Shalini De Mello, Tero Karras, Gordon Wetzstein

Abstract: We present a diffusion-based model for 3D-aware generative novel view synthesis from as few as a single input image. Our model samples from the distribution of possible renderings consistent with the input and, even in the presence of ambiguity, is capable of rendering diverse and plausible novel views. To achieve this, our method makes use of existing 2D diffusion backbones but, crucially, incorp… ▽ More We present a diffusion-based model for 3D-aware generative novel view synthesis from as few as a single input image. Our model samples from the distribution of possible renderings consistent with the input and, even in the presence of ambiguity, is capable of rendering diverse and plausible novel views. To achieve this, our method makes use of existing 2D diffusion backbones but, crucially, incorporates geometry priors in the form of a 3D feature volume. This latent feature field captures the distribution over possible scene representations and improves our method's ability to generate view-consistent novel renderings. In addition to generating novel views, our method has the ability to autoregressively synthesize 3D-consistent sequences. We demonstrate state-of-the-art results on synthetic renderings and room-scale scenes; we also show compelling results for challenging, real-world objects. △ Less

Submitted 5 April, 2023; originally announced April 2023.

Comments: Project page: https://nvlabs.github.io/genvs

Learning Object-Centric Neural Scattering Functions for Free-Viewpoint Relighting and Scene Composition

Authors: Hong-Xing Yu, Michelle Guo, Alireza Fathi, Yen-Yu Chang, Eric Ryan Chan, Ruohan Gao, Thomas Funkhouser, Jiajun Wu

Abstract: Photorealistic object appearance modeling from 2D images is a constant topic in vision and graphics. While neural implicit methods (such as Neural Radiance Fields) have shown high-fidelity view synthesis results, they cannot relight the captured objects. More recent neural inverse rendering approaches have enabled object relighting, but they represent surface properties as simple BRDFs, and theref… ▽ More Photorealistic object appearance modeling from 2D images is a constant topic in vision and graphics. While neural implicit methods (such as Neural Radiance Fields) have shown high-fidelity view synthesis results, they cannot relight the captured objects. More recent neural inverse rendering approaches have enabled object relighting, but they represent surface properties as simple BRDFs, and therefore cannot handle translucent objects. We propose Object-Centric Neural Scattering Functions (OSFs) for learning to reconstruct object appearance from only images. OSFs not only support free-viewpoint object relighting, but also can model both opaque and translucent objects. While accurately modeling subsurface light transport for translucent objects can be highly complex and even intractable for neural methods, OSFs learn to approximate the radiance transfer from a distant light to an outgoing direction at any spatial location. This approximation avoids explicitly modeling complex subsurface scattering, making learning a neural implicit model tractable. Experiments on real and synthetic data show that OSFs accurately reconstruct appearances for both opaque and translucent objects, allowing faithful free-viewpoint relighting as well as scene composition. △ Less

Submitted 3 October, 2023; v1 submitted 10 March, 2023; originally announced March 2023.

Comments: Journal extension of arXiv:2012.08503 (TMLR 2023). The first two authors contributed equally to this work. Project page: https://kovenyu.com/osf/

Journal ref: Transactions on Machine Learning Research (TMLR), 2023

Diffusion in the Dark: A Diffusion Model for Low-Light Text Recognition

Authors: Cindy M. Nguyen, Eric R. Chan, Alexander W. Bergman, Gordon Wetzstein

Abstract: Capturing images is a key part of automation for high-level tasks such as scene text recognition. Low-light conditions pose a challenge for high-level perception stacks, which are often optimized on well-lit, artifact-free images. Reconstruction methods for low-light images can produce well-lit counterparts, but typically at the cost of high-frequency details critical for downstream tasks. We prop… ▽ More Capturing images is a key part of automation for high-level tasks such as scene text recognition. Low-light conditions pose a challenge for high-level perception stacks, which are often optimized on well-lit, artifact-free images. Reconstruction methods for low-light images can produce well-lit counterparts, but typically at the cost of high-frequency details critical for downstream tasks. We propose Diffusion in the Dark (DiD), a diffusion model for low-light image reconstruction for text recognition. DiD provides qualitatively competitive reconstructions with that of state-of-the-art (SOTA), while preserving high-frequency details even in extremely noisy, dark conditions. We demonstrate that DiD, without any task-specific optimization, can outperform SOTA low-light methods in low-light text recognition on real images, bolstering the potential of diffusion models to solve ill-posed inverse problems. △ Less

Submitted 30 October, 2023; v1 submitted 7 March, 2023; originally announced March 2023.

Comments: WACV 2024. Project website: https://ccnguyen.github.io/diffusion-in-the-dark/

KG-Hub -- Building and Exchanging Biological Knowledge Graphs

Authors: J Harry Caufield, Tim Putman, Kevin Schaper, Deepak R Unni, Harshad Hegde, Tiffany J Callahan, Luca Cappelletti, Sierra AT Moxon, Vida Ravanmehr, Seth Carbon, Lauren E Chan, Katherina Cortes, Kent A Shefchek, Glass Elsarboukh, James P Balhoff, Tommaso Fontana, Nicolas Matentzoglu, Richard M Bruskiewich, Anne E Thessen, Nomi L Harris, Monica C Munoz-Torres, Melissa A Haendel, Peter N Robinson, Marcin P Joachimiak, Christopher J Mungall , et al. (1 additional authors not shown)

Abstract: Knowledge graphs (KGs) are a powerful approach for integrating heterogeneous data and making inferences in biology and many other domains, but a coherent solution for constructing, exchanging, and facilitating the downstream use of knowledge graphs is lacking. Here we present KG-Hub, a platform that enables standardized construction, exchange, and reuse of knowledge graphs. Features include a simp… ▽ More Knowledge graphs (KGs) are a powerful approach for integrating heterogeneous data and making inferences in biology and many other domains, but a coherent solution for constructing, exchanging, and facilitating the downstream use of knowledge graphs is lacking. Here we present KG-Hub, a platform that enables standardized construction, exchange, and reuse of knowledge graphs. Features include a simple, modular extract-transform-load (ETL) pattern for producing graphs compliant with Biolink Model (a high-level data model for standardizing biological data), easy integration of any OBO (Open Biological and Biomedical Ontologies) ontology, cached downloads of upstream data sources, versioned and automatically updated builds with stable URLs, web-browsable storage of KG artifacts on cloud infrastructure, and easy reuse of transformed subgraphs across projects. Current KG-Hub projects span use cases including COVID-19 research, drug repurposing, microbial-environmental interactions, and rare disease research. KG-Hub is equipped with tooling to easily analyze and manipulate knowledge graphs. KG-Hub is also tightly integrated with graph machine learning (ML) tools which allow automated graph machine learning, including node embeddings and training of models for link prediction and node classification. △ Less

Submitted 31 January, 2023; originally announced February 2023.

Deep Offline Reinforcement Learning for Real-world Treatment Optimization Applications

Authors: Milashini Nambiar, Supriyo Ghosh, Priscilla Ong, Yu En Chan, Yong Mong Bee, Pavitra Krishnaswamy

Abstract: There is increasing interest in data-driven approaches for recommending optimal treatment strategies in many chronic disease management and critical care applications. Reinforcement learning methods are well-suited to this sequential decision-making problem, but must be trained and evaluated exclusively on retrospective medical record datasets as direct online exploration is unsafe and infeasible.… ▽ More There is increasing interest in data-driven approaches for recommending optimal treatment strategies in many chronic disease management and critical care applications. Reinforcement learning methods are well-suited to this sequential decision-making problem, but must be trained and evaluated exclusively on retrospective medical record datasets as direct online exploration is unsafe and infeasible. Despite this requirement, the vast majority of treatment optimization studies use off-policy RL methods (e.g., Double Deep Q Networks (DDQN) or its variants) that are known to perform poorly in purely offline settings. Recent advances in offline RL, such as Conservative Q-Learning (CQL), offer a suitable alternative. But there remain challenges in adapting these approaches to real-world applications where suboptimal examples dominate the retrospective dataset and strict safety constraints need to be satisfied. In this work, we introduce a practical and theoretically grounded transition sampling approach to address action imbalance during offline RL training. We perform extensive experiments on two real-world tasks for diabetes and sepsis treatment optimization to compare performance of the proposed approach against prominent off-policy and offline RL baselines (DDQN and CQL). Across a range of principled and clinically relevant metrics, we show that our proposed approach enables substantial improvements in expected health outcomes and in accordance with relevant practice and safety guidelines. △ Less

Submitted 13 June, 2023; v1 submitted 15 February, 2023; originally announced February 2023.

Addressing Deep Learning Model Calibration Using Evidential Neural Networks and Uncertainty-Aware Training

Authors: Tareen Dawood, Emily Chan, Reza Razavi, Andrew P. King, Esther Puyol-Anton

Abstract: In terms of accuracy, deep learning (DL) models have had considerable success in classification problems for medical imaging applications. However, it is well-known that the outputs of such models, which typically utilise the SoftMax function in the final classification layer can be over-confident, i.e. they are poorly calibrated. Two competing solutions to this problem have been proposed: uncerta… ▽ More In terms of accuracy, deep learning (DL) models have had considerable success in classification problems for medical imaging applications. However, it is well-known that the outputs of such models, which typically utilise the SoftMax function in the final classification layer can be over-confident, i.e. they are poorly calibrated. Two competing solutions to this problem have been proposed: uncertainty-aware training and evidential neural networks (ENNs). In this paper, we perform an investigation into the improvements to model calibration that can be achieved by each of these approaches individually, and their combination. We perform experiments on two classification tasks: a simpler MNIST digit classification task and a more complex and realistic medical imaging artefact detection task using Phase Contrast Cardiac Magnetic Resonance images. The experimental results demonstrate that model calibration can suffer when the task becomes challenging enough to require a higher-capacity model. However, in our complex artefact detection task, we saw an improvement in calibration for both a low and higher-capacity model when implementing both the ENN and uncertainty-aware training together, indicating that this approach can offer a promising way to improve calibration in such settings. The findings highlight the potential use of these approaches to improve model calibration in a complex application, which would in turn improve clinician trust in DL models. △ Less

Submitted 27 February, 2023; v1 submitted 30 January, 2023; originally announced January 2023.

3D Neural Field Generation using Triplane Diffusion

Authors: J. Ryan Shue, Eric Ryan Chan, Ryan Po, Zachary Ankner, Jiajun Wu, Gordon Wetzstein

Abstract: Diffusion models have emerged as the state-of-the-art for image generation, among other tasks. Here, we present an efficient diffusion-based model for 3D-aware generation of neural fields. Our approach pre-processes training data, such as ShapeNet meshes, by converting them to continuous occupancy fields and factoring them into a set of axis-aligned triplane feature representations. Thus, our 3D t… ▽ More Diffusion models have emerged as the state-of-the-art for image generation, among other tasks. Here, we present an efficient diffusion-based model for 3D-aware generation of neural fields. Our approach pre-processes training data, such as ShapeNet meshes, by converting them to continuous occupancy fields and factoring them into a set of axis-aligned triplane feature representations. Thus, our 3D training scenes are all represented by 2D feature planes, and we can directly train existing 2D diffusion models on these representations to generate 3D neural fields with high quality and diversity, outperforming alternative approaches to 3D-aware generation. Our approach requires essential modifications to existing triplane factorization pipelines to make the resulting features easy to learn for the diffusion model. We demonstrate state-of-the-art results on 3D generation on several object classes from ShapeNet. △ Less

Submitted 29 November, 2022; originally announced November 2022.

Comments: Project page: https://jryanshue.com/nfd

DiffDreamer: Towards Consistent Unsupervised Single-view Scene Extrapolation with Conditional Diffusion Models

Authors: Shengqu Cai, Eric Ryan Chan, Songyou Peng, Mohamad Shahbazi, Anton Obukhov, Luc Van Gool, Gordon Wetzstein

Abstract: Scene extrapolation -- the idea of generating novel views by flying into a given image -- is a promising, yet challenging task. For each predicted frame, a joint inpainting and 3D refinement problem has to be solved, which is ill posed and includes a high level of ambiguity. Moreover, training data for long-range scenes is difficult to obtain and usually lacks sufficient views to infer accurate ca… ▽ More Scene extrapolation -- the idea of generating novel views by flying into a given image -- is a promising, yet challenging task. For each predicted frame, a joint inpainting and 3D refinement problem has to be solved, which is ill posed and includes a high level of ambiguity. Moreover, training data for long-range scenes is difficult to obtain and usually lacks sufficient views to infer accurate camera poses. We introduce DiffDreamer, an unsupervised framework capable of synthesizing novel views depicting a long camera trajectory while training solely on internet-collected images of nature scenes. Utilizing the stochastic nature of the guided denoising steps, we train the diffusion models to refine projected RGBD images but condition the denoising steps on multiple past and future frames for inference. We demonstrate that image-conditioned diffusion models can effectively perform long-range scene extrapolation while preserving consistency significantly better than prior GAN-based methods. DiffDreamer is a powerful and efficient solution for scene extrapolation, producing impressive results despite limited supervision. Project page: https://primecai.github.io/diffdreamer. △ Less

Submitted 18 March, 2023; v1 submitted 22 November, 2022; originally announced November 2022.

Cross-chain Swaps with Preferences

Authors: Eric Chan, Marek Chrobak, Mohsen Lesani

Abstract: Extreme valuation and volatility of cryptocurrencies require investors to diversify often which demands secure exchange protocols. A cross-chain swap protocol allows distrusting parties to securely exchange their assets. However, the current models and protocols assume predefined user preferences for acceptable outcomes. This paper presents a generalized model of swaps that allows each party to sp… ▽ More Extreme valuation and volatility of cryptocurrencies require investors to diversify often which demands secure exchange protocols. A cross-chain swap protocol allows distrusting parties to securely exchange their assets. However, the current models and protocols assume predefined user preferences for acceptable outcomes. This paper presents a generalized model of swaps that allows each party to specify its preferences on the subsets of its incoming and outgoing assets. It shows that the existing swap protocols are not necessarily a strong Nash equilibrium in this model. It characterizes the class of swap graphs that have protocols that are safe, live and a strong Nash equilibrium, and presents such a protocol for this class. Further, it shows that deciding whether a swap is in this class is NP-hard through a reduction from 3SAT, and further is $Σ_2^{\mathsf{P}}$-complete through a reduction from $\exists\forall\mathsf{DNF}$. △ Less

Submitted 23 May, 2023; v1 submitted 21 October, 2022; originally announced October 2022.

Comments: Full version; conference version to appear in CSF2023

Automated Quality Controlled Analysis of 2D Phase Contrast Cardiovascular Magnetic Resonance Imaging

Authors: Emily Chan, Ciaran O'Hanlon, Carlota Asegurado Marquez, Marwenie Petalcorin, Jorge Mariscal-Harana, Haotian Gu, Raymond J. Kim, Robert M. Judd, Phil Chowienczyk, Julia A. Schnabel, Reza Razavi, Andrew P. King, Bram Ruijsink, Esther Puyol-Antón

Abstract: Flow analysis carried out using phase contrast cardiac magnetic resonance imaging (PC-CMR) enables the quantification of important parameters that are used in the assessment of cardiovascular function. An essential part of this analysis is the identification of the correct CMR views and quality control (QC) to detect artefacts that could affect the flow quantification. We propose a novel deep lear… ▽ More Flow analysis carried out using phase contrast cardiac magnetic resonance imaging (PC-CMR) enables the quantification of important parameters that are used in the assessment of cardiovascular function. An essential part of this analysis is the identification of the correct CMR views and quality control (QC) to detect artefacts that could affect the flow quantification. We propose a novel deep learning based framework for the fully-automated analysis of flow from full CMR scans that first carries out these view selection and QC steps using two sequential convolutional neural networks, followed by automatic aorta and pulmonary artery segmentation to enable the quantification of key flow parameters. Accuracy values of 0.958 and 0.914 were obtained for view classification and QC, respectively. For segmentation, Dice scores were $>$0.969 and the Bland-Altman plots indicated excellent agreement between manual and automatic peak flow values. In addition, we tested our pipeline on an external validation data set, with results indicating good robustness of the pipeline. This work was carried out using multivendor clinical data consisting of 986 cases, indicating the potential for the use of this pipeline in a clinical setting. △ Less

Submitted 28 September, 2022; originally announced September 2022.

Comments: STACOM 2022 workshop

Indefinite and Bidirectional Near Infrared Nanocrystal Photoswitching

Authors: Changhwan Lee, Emma Z. Xu, Kevin W. C. Kwock, Ayelet Teitelboim, Yawei Liu, Natalie Fardian-Melamed, Cassio C. S. Pedroso, Hye Sun Park, Jongwoo Kim, Stefanie D. Pritzl, Sang Hwan Nam, Theobald Lohmueller, Peter Ercius, Yung Doug Suh, Bruce E Cohen, Emory M Chan, P. James Schuck

Abstract: Materials whose luminescence can be switched by optical stimulation drive technologies ranging from superresolution imaging1-4, nanophotonics5, and optical data storage6-8, to targeted pharmacology, optogenetics, and chemical reactivity9. These photoswitchable probes, including organic fluorophores and proteins, are prone to photodegradation, and often require phototoxic doses of ultraviolet (UV)… ▽ More Materials whose luminescence can be switched by optical stimulation drive technologies ranging from superresolution imaging1-4, nanophotonics5, and optical data storage6-8, to targeted pharmacology, optogenetics, and chemical reactivity9. These photoswitchable probes, including organic fluorophores and proteins, are prone to photodegradation, and often require phototoxic doses of ultraviolet (UV) or visible light. Colloidal inorganic nanoparticles have significant stability advantages over existing photoswitchable materials, but the ability to switch emission bidirectionally, particularly with NIR light, has not been reported with nanoparticles. Here, we present 2-way, near-infrared (NIR) photoswitching of avalanching nanoparticles (ANPs), showing full optical control of upconverted emission using phototriggers in the NIR-I and NIR-II spectral regions useful for subsurface imaging. Employing single-step photodarkening10-13 and photobrightening12,14-18, we demonstrate indefinite photoswitching of individual nanoparticles (>1000 cycles over 7 h) in ambient or aqueous conditions without measurable photodegradation. Critical steps of the photoswitching mechanism are elucidated by modeling and by measuring the photon avalanche properties of single ANPs in both bright and dark states. Unlimited, reversible photoswitching of ANPs enables indefinitely rewritable 2D and 3D multi-level optical patterning of ANPs, as well as optical nanoscopy with sub-Å localization superresolution that allows us to distinguish individual ANPs within tightly packed clusters. △ Less

Submitted 13 September, 2022; originally announced September 2022.

Comments: 15 pages, 5 figures

Colorful Optical Vortices with White Light Illumination

Authors: Hongtao Wang, Hao Wang, Qifeng Ruan, John You En Chan, Wang Zhang, Hailong Liu, Soroosh Daqiqeh Rezaei, Jonathan Trisno, Cheng-Wei Qiu, Min Gu, Joel K. W. Yang

Abstract: The orbital angular momentum (OAM) of light holds great promise for applications in optical communication, super-resolution imaging, and high-dimensional quantum computing. However, the spatio-temporal coherence of the light source has been essential for generating OAM beams, as incoherent ambient light would result in polychromatic and obscured OAM beams in the visible spectrum. Here, we extend t… ▽ More The orbital angular momentum (OAM) of light holds great promise for applications in optical communication, super-resolution imaging, and high-dimensional quantum computing. However, the spatio-temporal coherence of the light source has been essential for generating OAM beams, as incoherent ambient light would result in polychromatic and obscured OAM beams in the visible spectrum. Here, we extend the applications of OAM to ambient lighting conditions. By miniaturizing spiral phase plates and integrating them with structural color filters, we achieve spatio-temporal coherence using only an incoherent white light source. These optical elements act as building blocks that encode both color and OAM information in the form of colorful optical vortices. Thus, pairs of transparent substrates that contain matching positions of these vortices constitute a reciprocal optical lock and key system. Due to the multiple helical eigenstates of OAM, the pairwise coupling can be further extended to form a one-to-many matching and validation scheme. Generating and decoding colorful optical vortices with broadband white light could find potential applications in anti-counterfeiting, optical metrology, high-capacity optical encryption, and on-chip 3D photonic devices. △ Less

Submitted 27 July, 2022; originally announced July 2022.

Neutron diffraction in MnSb2O6: Magnetic and structural domains in a helicoidal polar magnet with coupled chiralities

Authors: E. Chan, J. Pásztorová, R. D. Johnson, M. Songvilay, R. A. Downie, J-W. G. Bos, O. Fabelo, C. Ritter, K. Beauvois, Ch. Niedermayer, S. -W. Cheong, N. Qureshi, C. Stock

Abstract: MnSb$_{2}$O$_{6}$ is based on the structural chiral $P$321 space group #150 where the magnetic Mn$^{2+}$ moments ($S=5/2$, $L\approx 0$) order antiferromagnetically at $T_\mathrm{N}=12$ K. Unlike the related iron based langasite (Ba$_3$NbFe$_3$Si$_2$O$_{14}$) where the low temperature magnetism is based on a proper helix characterized by a time-even pseudoscalar `magnetic' chirality, the Mn… ▽ More MnSb$_{2}$O$_{6}$ is based on the structural chiral $P$321 space group #150 where the magnetic Mn$^{2+}$ moments ($S=5/2$, $L\approx 0$) order antiferromagnetically at $T_\mathrm{N}=12$ K. Unlike the related iron based langasite (Ba$_3$NbFe$_3$Si$_2$O$_{14}$) where the low temperature magnetism is based on a proper helix characterized by a time-even pseudoscalar `magnetic' chirality, the Mn$^{2+}$ ions in MnSb$_{2}$O$_{6}$ order with a cycloidal structure at low temperatures, described instead by a time-even vector `magnetic' polarity. A tilted cycloidal structure has been found [M. Kinoshita et al. Phys. Rev. Lett. 117, 047201 (2016)] to facilitate ferroelectric switching under an applied magnetic field. In this work, we apply polarized and unpolarized neutron diffraction analyzing the magnetic and nuclear structures in MnSb$_{2}$O$_{6}$ with the aim of understanding this magnetoelectric coupling. We find no evidence for a helicoidal magnetic structure with one of the spin envelope axes tilted away from the cycloidal $c$-axis. However, on application of a magnetic field $\parallel$ $\vec{c}$ the spin rotation plane can be tilted, giving rise to a cycloid-helix admixture that evolves towards a distorted helix (zero cycloidal component) for fields great than $\approx$ 2 T. We propose a mechanism for the previously reported ferroelectric switching based on coupled structural and magnetic chiralities requiring only an imbalance of structural chiral domains. △ Less

Submitted 3 August, 2022; v1 submitted 22 July, 2022; originally announced July 2022.

Journal ref: Phys. Rev. B 106, 064403 (2022)

Nonparametric Estimation of the Potential Impact Fraction and Population Attributable Fraction with Individual-Level and Aggregated Data

Authors: Colleen E. Chan, Rodrigo Zepeda-Tello, Dalia Camacho-García-Formentí, Frederick Cudhea, Rafael Meza, Eliane Rodrigues, Donna Spiegelman, Tonatiuh Barrientos-Gutierrez, Xin Zhou

Abstract: The estimation of the potential impact fraction (including the population attributable fraction) with continuous exposure data frequently relies on strong distributional assumptions. However, these assumptions are often violated if the underlying exposure distribution is unknown or if the same distribution is assumed across time or space. Nonparametric methods to estimate the potential impact frac… ▽ More The estimation of the potential impact fraction (including the population attributable fraction) with continuous exposure data frequently relies on strong distributional assumptions. However, these assumptions are often violated if the underlying exposure distribution is unknown or if the same distribution is assumed across time or space. Nonparametric methods to estimate the potential impact fraction are available for cohort data, but no alternatives exist for cross-sectional data. In this article, we discuss the impact of distributional assumptions in the estimation of the population impact fraction, showing that under an infinite set of possibilities, distributional violations lead to biased estimates. We propose nonparametric methods to estimate the potential impact fraction for aggregated (mean and standard deviation) or individual data (e.g. observations from a cross-sectional population survey), and develop simulation scenarios to compare their performance against standard parametric procedures. We illustrate our methodology on an application of sugar-sweetened beverage consumption on incidence of type 2 diabetes. We also present an R package pifpaf to implement these methods. △ Less

Submitted 24 January, 2023; v1 submitted 7 July, 2022; originally announced July 2022.

Generative Neural Articulated Radiance Fields

Authors: Alexander W. Bergman, Petr Kellnhofer, Wang Yifan, Eric R. Chan, David B. Lindell, Gordon Wetzstein

Abstract: Unsupervised learning of 3D-aware generative adversarial networks (GANs) using only collections of single-view 2D photographs has very recently made much progress. These 3D GANs, however, have not been demonstrated for human bodies and the generated radiance fields of existing frameworks are not directly editable, limiting their applicability in downstream tasks. We propose a solution to these cha… ▽ More Unsupervised learning of 3D-aware generative adversarial networks (GANs) using only collections of single-view 2D photographs has very recently made much progress. These 3D GANs, however, have not been demonstrated for human bodies and the generated radiance fields of existing frameworks are not directly editable, limiting their applicability in downstream tasks. We propose a solution to these challenges by develo** a 3D GAN framework that learns to generate radiance fields of human bodies or faces in a canonical pose and warp them using an explicit deformation field into a desired body pose or facial expression. Using our framework, we demonstrate the first high-quality radiance field generation results for human bodies. Moreover, we show that our deformation-aware training procedure significantly improves the quality of generated bodies or faces when editing their poses or facial expressions compared to a 3D GAN that is not trained with explicit deformations. △ Less

Submitted 9 January, 2023; v1 submitted 28 June, 2022; originally announced June 2022.

Comments: Project website: http://www.computationalimaging.org/publications/gnarf/

A method for comparing multiple imputation techniques: a case study on the U.S. National COVID Cohort Collaborative

Authors: Elena Casiraghi, Rachel Wong, Margaret Hall, Ben Coleman, Marco Notaro, Michael D. Evans, Jena S. Tronieri, Hannah Blau, Bryan Laraway, Tiffany J. Callahan, Lauren E. Chan, Carolyn T. Bramante, John B. Buse, Richard A. Moffitt, Til Sturmer, Steven G. Johnson, Yu Raymond Shao, Justin Reese, Peter N. Robinson, Alberto Paccanaro, Giorgio Valentini, Jared D. Huling, Kenneth Wilkins, :, Tell Bennet , et al. (12 additional authors not shown)

Abstract: Healthcare datasets obtained from Electronic Health Records have proven to be extremely useful to assess associations between patients' predictors and outcomes of interest. However, these datasets often suffer from missing values in a high proportion of cases and the simple removal of these cases may introduce severe bias. For these reasons, several multiple imputation algorithms have been propose… ▽ More Healthcare datasets obtained from Electronic Health Records have proven to be extremely useful to assess associations between patients' predictors and outcomes of interest. However, these datasets often suffer from missing values in a high proportion of cases and the simple removal of these cases may introduce severe bias. For these reasons, several multiple imputation algorithms have been proposed to attempt to recover the missing information. Each algorithm presents strengths and weaknesses, and there is currently no consensus on which multiple imputation algorithms works best in a given scenario. Furthermore, the selection of each algorithm parameters and data-related modelling choices are also both crucial and challenging. In this paper, we propose a novel framework to numerically evaluate strategies for handling missing data in the context of statistical analysis, with a particular focus on multiple imputation techniques. We demonstrate the feasibility of our approach on a large cohort of type-2 diabetes patients provided by the National COVID Cohort Collaborative (N3C) Enclave, where we explored the influence of various patient characteristics on outcomes related to COVID-19. Our analysis included classic multiple imputation techniques as well as simple complete-case Inverse Probability Weighted models. The experiments presented here show that our approach could effectively highlight the most valid and performant missing-data handling strategy for our case study. Moreover, our methodology allowed us to gain an understanding of the behavior of the different models and of how it changed as we modified their parameters. Our method is general and can be applied to different research fields and on datasets containing heterogeneous types. △ Less

Submitted 25 September, 2022; v1 submitted 13 June, 2022; originally announced June 2022.

Picophotonics -- Subatomic Optical Localization Beyond Thermal Fluctuations

Authors: Tongjun Liu, Cheng-Hung Chi, Jun-Yu Ou, Jie Xu, Eng Aik Chan, Kevin F. MacDonald, Nikolay I. Zheludev

Abstract: Despite recent tremendous progress in optical imaging and metrology, the resolution gap between atomic scale transmission electron microscopy and optical techniques has not been closed. Is optical imaging and metrology of nanostructures exhibiting Brownian motion possible with resolution beyond thermal fluctuations? Here we report on an experiment in which the average position of a nanowire with a… ▽ More Despite recent tremendous progress in optical imaging and metrology, the resolution gap between atomic scale transmission electron microscopy and optical techniques has not been closed. Is optical imaging and metrology of nanostructures exhibiting Brownian motion possible with resolution beyond thermal fluctuations? Here we report on an experiment in which the average position of a nanowire with a thermal oscillation amplitude of ~150 pm is resolved in single-shot measurements with precision of 92 pm using light at a wavelength of λ = 488 nm, providing the first example of such sub-Brownian metrology with ~λ/5,300 precision. To localize the nanowire, we employ a deep learning analysis of the scattering of topologically structured light, which is highly sensitive to the nanowire's position. As a non-invasive optical metrology with sub-Brownian absolute errors, down to a fraction of the typical size of an atom (Si: 220 pm diameter), it opens the exciting field of picophotonics. △ Less

Submitted 30 January, 2023; v1 submitted 3 May, 2022; originally announced May 2022.

Comments: 11 pages, 4 figures

Journal ref: Nature Materials, 22, 844 (2023)

Full Geometric Control of Hidden Color Information in Diffraction Gratings under Angled White Light Illumination

Authors: John You En Chan, Qifeng Ruan, Hongtao Wang, Hao Wang, Hailong Liu, Zhiyuan Yan, Cheng-Wei Qiu, Joel K. W. Yang

Abstract: Under white light illumination, gratings produce an angular distribution of wavelengths dependent on the diffraction order and geometric parameters. However, previous studies of gratings are limited to at least one geometric parameter (height, periodicity, orientation, angle of incidence) kept constant. Here, we vary all geometric parameters in the gratings using a versatile nanofabrication techni… ▽ More Under white light illumination, gratings produce an angular distribution of wavelengths dependent on the diffraction order and geometric parameters. However, previous studies of gratings are limited to at least one geometric parameter (height, periodicity, orientation, angle of incidence) kept constant. Here, we vary all geometric parameters in the gratings using a versatile nanofabrication technique, two-photon polymerization lithography, to encode hidden color information through 2 design approaches. The first approach hides color information by decoupling the effects of grating height and periodicity under normal and oblique incidence. The second approach hides multiple sets of color information by arranging gratings in sectors around semi-circular pixels. Different images are revealed with negligible crosstalk under oblique incidence and varying sample rotation angles. Our analysis shows that an angular separation >= 10° between adjacent sectors is required to suppress crosstalk. This work has potential applications in information storage and security watermarks. △ Less

Submitted 6 September, 2022; v1 submitted 13 April, 2022; originally announced April 2022.

3D GAN Inversion for Controllable Portrait Image Animation

Authors: Connor Z. Lin, David B. Lindell, Eric R. Chan, Gordon Wetzstein

Abstract: Millions of images of human faces are captured every single day; but these photographs portray the likeness of an individual with a fixed pose, expression, and appearance. Portrait image animation enables the post-capture adjustment of these attributes from a single image while maintaining a photorealistic reconstruction of the subject's likeness or identity. Still, current methods for portrait im… ▽ More Millions of images of human faces are captured every single day; but these photographs portray the likeness of an individual with a fixed pose, expression, and appearance. Portrait image animation enables the post-capture adjustment of these attributes from a single image while maintaining a photorealistic reconstruction of the subject's likeness or identity. Still, current methods for portrait image animation are typically based on 2D war** operations or manipulations of a 2D generative adversarial network (GAN) and lack explicit mechanisms to enforce multi-view consistency. Thus these methods may significantly alter the identity of the subject, especially when the viewpoint relative to the camera is changed. In this work, we leverage newly developed 3D GANs, which allow explicit control over the pose of the image subject with multi-view consistency. We propose a supervision strategy to flexibly manipulate expressions with 3D morphable models, and we show that the proposed method also supports editing appearance attributes, such as age or hairstyle, by interpolating within the latent space of the GAN. The proposed technique for portrait image animation outperforms previous methods in terms of image quality, identity preservation, and pose transfer while also supporting attribute editing. △ Less

Submitted 25 March, 2022; originally announced March 2022.

Comments: Project page: https://www.computationalimaging.org/publications/3dganinversion/

Overcoming Van der Waals Forces in reconfigurable nanostructures

Authors: Wang Zhang, Hao Wang, Alvin T. L. Tan, Anupama Sargur Ranganath, Biao Zhang, Hongtao Wang, John You En Chan, Qifeng Ruan, Hailong Liu, Son Tung Ha, Dong Wang, Venkat K. Ravikumar, Hong Yee Low, Joel K. W. Yang

Abstract: Reconfigurable metamaterials require constituent nanostructures to demonstrate switching of shapes with external stimuli. For generality, such nanostructures would touch and stick to other surfaces in one of its configurations. Yet, a longstanding challenge is in overcoming this stiction caused by Van der Waals forces, which impedes shape recovery. Here, we introduce a stiff yet self-recovering ma… ▽ More Reconfigurable metamaterials require constituent nanostructures to demonstrate switching of shapes with external stimuli. For generality, such nanostructures would touch and stick to other surfaces in one of its configurations. Yet, a longstanding challenge is in overcoming this stiction caused by Van der Waals forces, which impedes shape recovery. Here, we introduce a stiff yet self-recovering material system based on acrylic acid, and tested it in high-aspect ratio structures, where recovery is weak. This designer material has a storage modulus of ~5.2 GPa at room temperature and ~90 MPa in the rubbery state at 150 Celsius, an order of magnitude higher than previous reports. A high-resolution resin for two-photon lithography was developed based on this polymer system, enabling 3D printing of nanopillars with diameters of ~400 nm and aspect ratio as high as ~10. Experimentally, we observed self-recovery as collapsed and touching structures overcome stiction to stand back up. We developed a theoretical model to explain the recoverability of these sub-micron structures. Reconfigurable structural colour prints and holograms were demonstrated, indicating potential applications of the material system as a shape memory polymer suitable for sub-micron reconfigurable metamaterials. △ Less

Submitted 22 February, 2022; v1 submitted 21 February, 2022; originally announced February 2022.

Efficient Geometry-aware 3D Generative Adversarial Networks

Authors: Eric R. Chan, Connor Z. Lin, Matthew A. Chan, Koki Nagano, Boxiao Pan, Shalini De Mello, Orazio Gallo, Leonidas Guibas, Jonathan Tremblay, Sameh Khamis, Tero Karras, Gordon Wetzstein

Abstract: Unsupervised generation of high-quality multi-view-consistent images and 3D shapes using only collections of single-view 2D photographs has been a long-standing challenge. Existing 3D GANs are either compute-intensive or make approximations that are not 3D-consistent; the former limits quality and resolution of the generated images and the latter adversely affects multi-view consistency and shape… ▽ More Unsupervised generation of high-quality multi-view-consistent images and 3D shapes using only collections of single-view 2D photographs has been a long-standing challenge. Existing 3D GANs are either compute-intensive or make approximations that are not 3D-consistent; the former limits quality and resolution of the generated images and the latter adversely affects multi-view consistency and shape quality. In this work, we improve the computational efficiency and image quality of 3D GANs without overly relying on these approximations. We introduce an expressive hybrid explicit-implicit network architecture that, together with other design choices, synthesizes not only high-resolution multi-view-consistent images in real time but also produces high-quality 3D geometry. By decoupling feature generation and neural rendering, our framework is able to leverage state-of-the-art 2D CNN generators, such as StyleGAN2, and inherit their efficiency and expressiveness. We demonstrate state-of-the-art 3D-aware synthesis with FFHQ and AFHQ Cats, among other experiments. △ Less

Submitted 27 April, 2022; v1 submitted 15 December, 2021; originally announced December 2021.

Comments: Project page: https://matthew-a-chan.github.io/EG3D

Mechanism of Spin-Orbit Torques in Platinum Oxide Systems

Authors: Jayshankar Nath, Alexandru Vladimir Trifu, Mihai Sebastian Gabor, Ali Hallal, Stephane Auffret, Sebastien Labau, Aymen Mahjoub, Edmond Chan, Avinash Kumar Chaurasiya, Amrit Kumar Mondal, Haozhe Yang, Eva Schmoranzerova, Mohamed Ali Nsibi, Isabelle Joumard, Anjan Barman, Bernard Pelissier, Mairbek Chshiev, Gilles Gaudin, Ioan Mihai Miron

Abstract: Spin-Orbit Torque (SOT) Magnetic Random-Access Memories (MRAM) have shown promising results towards the realization of fast, non-volatile memory systems. Oxidation of the heavy-metal (HM) layer of the SOT-MRAM has been proposed as a method to increase its energy efficiency. But the results are widely divergent due to the difficulty in controlling the HM oxidation because of its low enthalpy of for… ▽ More Spin-Orbit Torque (SOT) Magnetic Random-Access Memories (MRAM) have shown promising results towards the realization of fast, non-volatile memory systems. Oxidation of the heavy-metal (HM) layer of the SOT-MRAM has been proposed as a method to increase its energy efficiency. But the results are widely divergent due to the difficulty in controlling the HM oxidation because of its low enthalpy of formation. Here, we reconcile these differences by performing a gradual oxidation procedure, which allows correlating the chemical structure to the physical properties of the stack. As an HM layer, we chose Pt because of the strong SOT and the low enthalpy of formation of its oxides. We find evidence of an oxide inversion layer at the FM/HM interface: the oxygen is drawn into the FM, while the HM remains metallic near the interface. We further demonstrate that the oxygen migrates in the volume of the FM layer rather than being concentrated at the interface. Consequently, we find that the intrinsic magnitude of the SOT is unchanged compared to the fully metallic structure. The previously reported apparent increase of SOTs is not intrinsic to platinum oxide and instead arises from systemic changes produced by oxidation. △ Less

Submitted 13 December, 2021; originally announced December 2021.

Comments: 13 pages, 3 figures + Supplementary Information (15 pages, 16 figures)

The Effect of Model Size on Worst-Group Generalization

Authors: Alan Pham, Eunice Chan, Vikranth Srivatsa, Dhruba Ghosh, Yaoqing Yang, Yaodong Yu, Ruiqi Zhong, Joseph E. Gonzalez, Jacob Steinhardt

Abstract: Overparameterization is shown to result in poor test accuracy on rare subgroups under a variety of settings where subgroup information is known. To gain a more complete picture, we consider the case where subgroup information is unknown. We investigate the effect of model size on worst-group generalization under empirical risk minimization (ERM) across a wide range of settings, varying: 1) archite… ▽ More Overparameterization is shown to result in poor test accuracy on rare subgroups under a variety of settings where subgroup information is known. To gain a more complete picture, we consider the case where subgroup information is unknown. We investigate the effect of model size on worst-group generalization under empirical risk minimization (ERM) across a wide range of settings, varying: 1) architectures (ResNet, VGG, or BERT), 2) domains (vision or natural language processing), 3) model size (width or depth), and 4) initialization (with pre-trained or random weights). Our systematic evaluation reveals that increasing model size does not hurt, and may help, worst-group test performance under ERM across all setups. In particular, increasing pre-trained model size consistently improves performance on Waterbirds and MultiNLI. We advise practitioners to use larger pre-trained models when subgroup labels are unknown. △ Less

Submitted 7 December, 2021; originally announced December 2021.

Comments: The first four authors contributed equally to the work

Relating spin-polarized STM imaging and inelastic neutron scattering in the van-der-Waals ferromagnet Fe3GeTe2

Authors: Christopher Trainer, Olivia R. Armitage, Harry Lane, Luke C. Rhodes, Edmond Chan, Izidor Benedičič, Jose A. Rodriguez-Rivera, Oscar Fabelo, Chris Stock, Peter Wahl

Abstract: Van-der-Waals (vdW) ferromagnets have enabled the development of heterostructures assembled from exfoliated monolayers with spintronics functionalities, making it important to understand and ultimately tune their magnetic properties at the microscopic level. Information about the magnetic properties of these systems comes so far largely from macroscopic techniques, with little being known about th… ▽ More Van-der-Waals (vdW) ferromagnets have enabled the development of heterostructures assembled from exfoliated monolayers with spintronics functionalities, making it important to understand and ultimately tune their magnetic properties at the microscopic level. Information about the magnetic properties of these systems comes so far largely from macroscopic techniques, with little being known about the microscopic magnetic properties. Here, we combine spin-polarized scanning tunneling microscopy and quasi-particle interference imaging with neutron scattering to establish the magnetic and electronic properties of the metallic vdW ferromagnet Fe3GeTe2. By imaging domain walls at the atomic scale, we can relate the domain wall width to the exchange interaction and magnetic anisotropy extracted from the magnon dispersion as measured in inelastic neutron scattering, with excellent agreement between the two techniques. From comparison with Density Functional Theory calculations we can assign the quasi-particle interference to be dominated by spin-majority bands. We find a dimensional dichotomy of the bands at the Fermi energy: bands of minority character are predominantly two-dimensional in character, whereas the bands of majority character are three-dimensional. We expect that this will enable new design principles for spintronics devices. △ Less

Submitted 15 November, 2021; originally announced November 2021.

Comments: 18 pages main manuscript with 5 figures plus 12 pages supplementary

Journal ref: Phys. Rev. B. 106, L081405 (2022)