Showing 1–8 of 8 results for author: Power, B

GPT-4 Technical Report

Authors: OpenAI, Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, Red Avila, Igor Babuschkin, Suchir Balaji, Valerie Balcom, Paul Baltescu, Haiming Bao, Mohammad Bavarian, Jeff Belgum, Irwan Bello, Jake Berdine, Gabriel Bernadett-Shapiro, Christopher Berner, Lenny Bogdonoff, Oleg Boiko , et al. (256 additional authors not shown)

Abstract: We report the development of GPT-4, a large-scale, multimodal model which can accept image and text inputs and produce text outputs. While less capable than humans in many real-world scenarios, GPT-4 exhibits human-level performance on various professional and academic benchmarks, including passing a simulated bar exam with a score around the top 10% of test takers. GPT-4 is a Transformer-based mo… ▽ More We report the development of GPT-4, a large-scale, multimodal model which can accept image and text inputs and produce text outputs. While less capable than humans in many real-world scenarios, GPT-4 exhibits human-level performance on various professional and academic benchmarks, including passing a simulated bar exam with a score around the top 10% of test takers. GPT-4 is a Transformer-based model pre-trained to predict the next token in a document. The post-training alignment process results in improved performance on measures of factuality and adherence to desired behavior. A core component of this project was develo** infrastructure and optimization methods that behave predictably across a wide range of scales. This allowed us to accurately predict some aspects of GPT-4's performance based on models trained with no more than 1/1,000th the compute of GPT-4. △ Less

Submitted 4 March, 2024; v1 submitted 15 March, 2023; originally announced March 2023.

Comments: 100 pages; updated authors list; fixed author names and added citation

Text and Code Embeddings by Contrastive Pre-Training

Authors: Arvind Neelakantan, Tao Xu, Raul Puri, Alec Radford, Jesse Michael Han, Jerry Tworek, Qiming Yuan, Nikolas Tezak, Jong Wook Kim, Chris Hallacy, Johannes Heidecke, Pranav Shyam, Boris Power, Tyna Eloundou Nekoul, Girish Sastry, Gretchen Krueger, David Schnurr, Felipe Petroski Such, Kenny Hsu, Madeleine Thompson, Tabarak Khan, Toki Sherbakov, Joanne Jang, Peter Welinder, Lilian Weng

Abstract: Text embeddings are useful features in many applications such as semantic search and computing text similarity. Previous work typically trains models customized for different use cases, varying in dataset choice, training objective and model architecture. In this work, we show that contrastive pre-training on unsupervised data at scale leads to high quality vector representations of text and code.… ▽ More Text embeddings are useful features in many applications such as semantic search and computing text similarity. Previous work typically trains models customized for different use cases, varying in dataset choice, training objective and model architecture. In this work, we show that contrastive pre-training on unsupervised data at scale leads to high quality vector representations of text and code. The same unsupervised text embeddings that achieve new state-of-the-art results in linear-probe classification also display impressive semantic search capabilities and sometimes even perform competitively with fine-tuned models. On linear-probe classification accuracy averaging over 7 tasks, our best unsupervised model achieves a relative improvement of 4% and 1.8% over previous best unsupervised and supervised text embedding models respectively. The same text embeddings when evaluated on large-scale semantic search attains a relative improvement of 23.4%, 14.7%, and 10.6% over previous best unsupervised methods on MSMARCO, Natural Questions and TriviaQA benchmarks, respectively. Similarly to text embeddings, we train code embedding models on (text, code) pairs, obtaining a 20.8% relative improvement over prior best work on code search. △ Less

Submitted 24 January, 2022; originally announced January 2022.

Opportunistic Emulation of Computationally Expensive Simulations via Deep Learning

Authors: Conrad Sanderson, Dan Pagendam, Brendan Power, Frederick Bennett, Ross Darnell

Abstract: With the underlying aim of increasing efficiency of computational modelling pertinent for managing & protecting the Great Barrier Reef, we perform a preliminary investigation on the use of deep neural networks for opportunistic model emulation of APSIM models by repurposing an existing large dataset containing outputs of APSIM model runs. The dataset has not been specifically tailored for the mode… ▽ More With the underlying aim of increasing efficiency of computational modelling pertinent for managing & protecting the Great Barrier Reef, we perform a preliminary investigation on the use of deep neural networks for opportunistic model emulation of APSIM models by repurposing an existing large dataset containing outputs of APSIM model runs. The dataset has not been specifically tailored for the model emulation task. We employ two neural network architectures for the emulation task: densely connected feed-forward neural network (FFNN), and gated recurrent unit feeding into FFNN (GRU-FFNN), a type of a recurrent neural network. Various configurations of the architectures are trialled. A minimum correlation statistic is used to identify clusters of APSIM scenarios that can be aggregated to form training sets for model emulation. We focus on emulating 4 important outputs of the APSIM model: runoff, soil_loss, DINrunoff, Nleached. The GRU-FFNN architecture with three hidden layers and 128 units per layer provides good emulation of runoff and DINrunoff. However, soil_loss and Nleached were emulated relatively poorly under a wide range of the considered architectures; the emulators failed to capture variability at higher values of these two outputs. While the opportunistic data available from past modelling activities provides a large and useful dataset for exploring APSIM emulation, it may not be sufficiently rich enough for successful deep learning of more complex model dynamics. Design of Computer Experiments may be required to generate more informative data to emulate all output variables of interest. We also suggest the use of synthetic meteorology settings to allow the model to be fed a wide range of inputs. These need not all be representative of normal conditions, but can provide a denser, more informative dataset from which complex relationships between input and outputs can be learned. △ Less

Submitted 16 December, 2021; v1 submitted 25 August, 2021; originally announced August 2021.

Journal ref: International Congress on Modelling and Simulation, 2021

Conformal Invariance of the One-Loop All-Plus Helicity Scattering Amplitudes

Authors: Johannes Henn, Bláithín Power, Simone Zoia

Abstract: The massless QCD Lagrangian is conformally invariant and, as a consequence, so are the tree-level scattering amplitudes. However, the implications of this powerful symmetry at loop level are only beginning to be explored systematically. Even for finite loop amplitudes, the way conformal symmetry manifests itself may be subtle, e.g. in the form of anomalous conformal Ward identities. As they are fi… ▽ More The massless QCD Lagrangian is conformally invariant and, as a consequence, so are the tree-level scattering amplitudes. However, the implications of this powerful symmetry at loop level are only beginning to be explored systematically. Even for finite loop amplitudes, the way conformal symmetry manifests itself may be subtle, e.g. in the form of anomalous conformal Ward identities. As they are finite and rational, the one-loop all-plus and single-minus amplitudes are a natural first step towards understanding the conformal properties of Yang-Mills theory at loop level. Remarkably, we find that the one-loop all-plus amplitudes are conformally invariant, whereas the single-minus are not. Moreover, we present a formula for the one-loop all-plus amplitudes where the symmetry is manifest term by term. Surprisingly, each term transforms covariantly under directional dual conformal variations. We prove the formula directly using recursive techniques, and check that it has the correct physical factorisations. △ Less

Submitted 11 December, 2019; v1 submitted 27 November, 2019; originally announced November 2019.

Comments: 23 pages, 2 figures; v2: minor changes, added references

3D Conditional Generative Adversarial Networks to enable large-scale seismic image enhancement

Authors: Praneet Dutta, Bruce Power, Adam Halpert, Carlos Ezequiel, Aravind Subramanian, Chanchal Chatterjee, Sindhu Hari, Kenton Prindle, Vishal Vaddina, Andrew Leach, Raj Domala, Laura Bandura, Massimo Mascaro

Abstract: We propose GAN-based image enhancement models for frequency enhancement of 2D and 3D seismic images. Seismic imagery is used to understand and characterize the Earth's subsurface for energy exploration. Because these images often suffer from resolution limitations and noise contamination, our proposed method performs large-scale seismic volume frequency enhancement and denoising. The enhanced imag… ▽ More We propose GAN-based image enhancement models for frequency enhancement of 2D and 3D seismic images. Seismic imagery is used to understand and characterize the Earth's subsurface for energy exploration. Because these images often suffer from resolution limitations and noise contamination, our proposed method performs large-scale seismic volume frequency enhancement and denoising. The enhanced images reduce uncertainty and improve decisions about issues, such as optimal well placement, that often rely on low signal-to-noise ratio (SNR) seismic volumes. We explored the impact of adding lithology class information to the models, resulting in improved performance on PSNR and SSIM metrics over a baseline model with no conditional information. △ Less

Submitted 15 November, 2019; originally announced November 2019.

Comments: To be Presented at the NeurIPS 2019, Second Workshop on Machine Learning and the Physicial Sciences, Vancouver, Canada

A Non-Invasive Method for the Safe Interaction of Cities and Electric Vehicle Fleets

Authors: Bill Power, Brian Mulkeene, Anthony D. Fagan, Robert Shorten

Abstract: Electric and hybrid vehicles are growing in popularity. While these vehicles produce less pollution, they also produce less audible noise, especially at lower speeds. This makes it harder for pedestrians and cyclists to detect an approaching vehicle. Thus, an additional system is required to detect electric and hybrid vehicles and alert pedestrians and cyclists of their whereabouts, especially whi… ▽ More Electric and hybrid vehicles are growing in popularity. While these vehicles produce less pollution, they also produce less audible noise, especially at lower speeds. This makes it harder for pedestrians and cyclists to detect an approaching vehicle. Thus, an additional system is required to detect electric and hybrid vehicles and alert pedestrians and cyclists of their whereabouts, especially while these vehicles are driving at low speeds in cities. This paper introduces one such method based on high frequency audio emissions that are present in EVs, which arise, for example, from the process of magnetostriction. Our method is tested experimentally using 4 different tests vehicles, and a preliminary EV detection algorithm is also presented. △ Less

Submitted 24 April, 2018; originally announced April 2018.

Comments: Submitted to ITSC 2018

Galaxy Cluster Mass Reconstruction Project: III. The impact of dynamical substructure on cluster mass estimates

Authors: L. Old, R. Wojtak, F. R. Pearce, M. E. Gray, G. A. Mamon, C. Sifón, E. Tempel, A. Biviano, H. K. C. Yee, R. de Carvalho, V. Müller, T. Sepp, R. A. Skibba, D. Croton, S. P. Bamford C. Power, A. von der Linden, A. Saro

Abstract: With the advent of wide-field cosmological surveys, we are approaching samples of hundreds of thousands of galaxy clusters. While such large numbers will help reduce statistical uncertainties, the control of systematics in cluster masses becomes ever more crucial. Here we examine the effects of an important source of systematic uncertainty in galaxy-based cluster mass estimation techniques: the pr… ▽ More With the advent of wide-field cosmological surveys, we are approaching samples of hundreds of thousands of galaxy clusters. While such large numbers will help reduce statistical uncertainties, the control of systematics in cluster masses becomes ever more crucial. Here we examine the effects of an important source of systematic uncertainty in galaxy-based cluster mass estimation techniques: the presence of significant dynamical substructure. Dynamical substructure manifests as dynamically distinct subgroups in phase-space, indicating an 'unrelaxed' state. This issue affects around a quarter of clusters in a generally selected sample. We employ a set of mock clusters whose masses have been measured homogeneously with commonly-used galaxy-based mass estimation techniques (kinematic, richness, caustic, radial methods). We use these to study how the relation between observationally estimated and true cluster mass depends on the presence of substructure, as identified by various popular diagnostics. We find that the scatter for an ensemble of clusters does not increase dramatically for clusters with dynamical substructure. However, we find a systematic bias for all methods, such that clusters with significant substructure have higher measured masses than their relaxed counterparts. This bias depends on cluster mass: the most massive clusters are largely unaffected by the presence of significant substructure, but masses are significantly overestimated for lower mass clusters, by $\sim10\%$ at $10^{14}$ and $\geq20\%$ for $\leq10^{13.5}$. The use of cluster samples with different levels of substructure can, therefore, bias certain cosmological parameters up to a level comparable to the typical uncertainties in current cosmological studies. △ Less

Submitted 28 September, 2017; originally announced September 2017.

Comments: 14 pages, 8 figures, 3 tables. Submitted to MNRAS

Temperature dependence of the magnetic Casimir-Polder interaction

Authors: H. Haakh, F. Intravaia, C. Henkel, S. Spagnolo, R. Passante, B. Power, F. Sols

Abstract: We analyze the magnetic dipole contribution to atom-surface dispersion forces. Unlike its electrical counterpart, it involves small transition frequencies that are comparable to thermal energy scales. A significant temperature dependence is found near surfaces with a nonzero DC conductivity, leading to a strong suppression of the dispersion force at T > 0. We use thermal response theory for the… ▽ More We analyze the magnetic dipole contribution to atom-surface dispersion forces. Unlike its electrical counterpart, it involves small transition frequencies that are comparable to thermal energy scales. A significant temperature dependence is found near surfaces with a nonzero DC conductivity, leading to a strong suppression of the dispersion force at T > 0. We use thermal response theory for the surface material and discuss both normal metals and superconductors. The asymptotes of the free energy of interaction and of the entropy are calculated analytically over a large range of distances. Near a superconductor, the onset of dissipation at the phase transition strongly changes the interaction, including a discontinuous entropy. We discuss the similarities with the Casimir interaction beween two surfaces and suggest that precision measurements of the atom-surface interaction may shed new light upon open questions around the temperature dependence of dispersion forces between lossy media. △ Less

Submitted 20 January, 2010; v1 submitted 16 October, 2009; originally announced October 2009.

Comments: 11 figures

Journal ref: Phys. Rev. A 80 (2009) 062905