-
Considerations for health care institutions training large language models on electronic health records
Authors:
Weipeng Zhou,
Danielle Bitterman,
Majid Afshar,
Timothy A. Miller
Abstract:
Large language models (LLMs) like ChatGPT have excited scientists across fields; in medicine, one source of excitement is the potential applications of LLMs trained on electronic health record (EHR) data. But there are tough questions we must first answer if health care institutions are interested in having LLMs trained on their own data; should they train an LLM from scratch or fine-tune it from…
▽ More
Large language models (LLMs) like ChatGPT have excited scientists across fields; in medicine, one source of excitement is the potential applications of LLMs trained on electronic health record (EHR) data. But there are tough questions we must first answer if health care institutions are interested in having LLMs trained on their own data; should they train an LLM from scratch or fine-tune it from an open-source model? For healthcare institutions with a predefined budget, what are the biggest LLMs they can afford? In this study, we take steps towards answering these questions with an analysis on dataset sizes, model sizes, and costs for LLM training using EHR data. This analysis provides a framework for thinking about these questions in terms of data scale, compute scale, and training budgets.
△ Less
Submitted 23 August, 2023;
originally announced September 2023.
-
Natural language processing to automatically extract the presence and severity of esophagitis in notes of patients undergoing radiotherapy
Authors:
Shan Chen,
Marco Guevara,
Nicolas Ramirez,
Arpi Murray,
Jeremy L. Warner,
Hugo JWL Aerts,
Timothy A. Miller,
Guergana K. Savova,
Raymond H. Mak,
Danielle S. Bitterman
Abstract:
Radiotherapy (RT) toxicities can impair survival and quality-of-life, yet remain under-studied. Real-world evidence holds potential to improve our understanding of toxicities, but toxicity information is often only in clinical notes. We developed natural language processing (NLP) models to identify the presence and severity of esophagitis from notes of patients treated with thoracic RT. We fine-tu…
▽ More
Radiotherapy (RT) toxicities can impair survival and quality-of-life, yet remain under-studied. Real-world evidence holds potential to improve our understanding of toxicities, but toxicity information is often only in clinical notes. We developed natural language processing (NLP) models to identify the presence and severity of esophagitis from notes of patients treated with thoracic RT. We fine-tuned statistical and pre-trained BERT-based models for three esophagitis classification tasks: Task 1) presence of esophagitis, Task 2) severe esophagitis or not, and Task 3) no esophagitis vs. grade 1 vs. grade 2-3. Transferability was tested on 345 notes from patients with esophageal cancer undergoing RT.
Fine-tuning PubmedBERT yielded the best performance. The best macro-F1 was 0.92, 0.82, and 0.74 for Task 1, 2, and 3, respectively. Selecting the most informative note sections during fine-tuning improved macro-F1 by over 2% for all tasks. Silver-labeled data improved the macro-F1 by over 3% across all tasks. For the esophageal cancer notes, the best macro-F1 was 0.73, 0.74, and 0.65 for Task 1, 2, and 3, respectively, without additional fine-tuning.
To our knowledge, this is the first effort to automatically extract esophagitis toxicity severity according to CTCAE guidelines from clinic notes. The promising performance provides proof-of-concept for NLP-based automated detailed toxicity monitoring in expanded domains.
△ Less
Submitted 23 March, 2023;
originally announced March 2023.
-
Turbulence explains the accelerations of an eagle in natural flight
Authors:
Kasey M. Laurent,
Bob Fogg,
Tobias Ginsburg,
Casey Halverson,
Michael Lanzone,
Tricia A. Miller,
David W. Winkler,
Gregory P. Bewley
Abstract:
Turbulent winds and gusts fluctuate on a wide range of timescales from milliseconds to minutes and longer, a range that overlaps the timescales of avian flight behavior, yet the importance of turbulence to avian behavior is unclear. By combining wind speed data with the measured accelerations of a golden eagle (Aquila chrysaetos) flying in the wild, we show that the eagle's accelerations can be ex…
▽ More
Turbulent winds and gusts fluctuate on a wide range of timescales from milliseconds to minutes and longer, a range that overlaps the timescales of avian flight behavior, yet the importance of turbulence to avian behavior is unclear. By combining wind speed data with the measured accelerations of a golden eagle (Aquila chrysaetos) flying in the wild, we show that the eagle's accelerations can be explained by a linear interaction with turbulence for timescales between about 1/2 and 10 s. These timescales are comparable to those of typical eagle behaviors, corresponding to between about 1 and 25 wingbeats, and to those of turbulent gusts both larger than the eagle's wingspan and smaller than large-scale atmospheric phenomena such as convection cells. The eagle's accelerations exhibit power spectra and intermittent activity characteristic of turbulence, and increase in proportion to the turbulence intensity. Intermittency results in accelerations that are occasionally several times stronger than gravity, and much larger than the ones we experience while driving, for instance. These imprints of turbulence on the bird's movements need to be further explored to understand the energetics of birds and other volant lifeforms, to improve our own methods of flying through ceaselessly turbulent environments, and to engage airborne wildlife as distributed probes of the changing conditions in the atmosphere.
△ Less
Submitted 19 May, 2021; v1 submitted 19 February, 2021;
originally announced February 2021.
-
Simplified Neural Unsupervised Domain Adaptation
Authors:
Timothy A Miller
Abstract:
Unsupervised domain adaptation (UDA) is the task of modifying a statistical model trained on labeled data from a source domain to achieve better performance on data from a target domain, with access to only unlabeled data in the target domain. Existing state-of-the-art UDA approaches use neural networks to learn representations that can predict the values of subset of important features called "pi…
▽ More
Unsupervised domain adaptation (UDA) is the task of modifying a statistical model trained on labeled data from a source domain to achieve better performance on data from a target domain, with access to only unlabeled data in the target domain. Existing state-of-the-art UDA approaches use neural networks to learn representations that can predict the values of subset of important features called "pivot features." In this work, we show that it is possible to improve on these methods by jointly training the representation learner with the task learner, and examine the importance of existing pivot selection methods.
△ Less
Submitted 22 May, 2019;
originally announced May 2019.
-
Nanoscale spectroscopic imaging of phase separation in a correlated material by resonant X-ray holography
Authors:
Luciana Vidas,
Christian M. Günther,
Timothy A. Miller,
Bastian. Pfau,
Michael Schneider,
Erik Guehrs,
Robert E. Marvel,
Kent A. Hallman,
Richard F. Haglund Jr,
Stefan Eisebitt,
Simon Wall
Abstract:
Nanoscale phase coexistence and inhomogeneity are ubiquitous in correlated electron materials, existing in doped Mott insulators, manganites, and high-temperature superconductors. The small length scales and lack of contrast mechanisms make it extremely challenging to measure real-space images of the phase coexistence with high resolution. Yet, images aid our understanding of how phase coexistence…
▽ More
Nanoscale phase coexistence and inhomogeneity are ubiquitous in correlated electron materials, existing in doped Mott insulators, manganites, and high-temperature superconductors. The small length scales and lack of contrast mechanisms make it extremely challenging to measure real-space images of the phase coexistence with high resolution. Yet, images aid our understanding of how phase coexistence and domain boundaries dictate the exotic material properties. Here we show that resonant soft-X-ray holography, previously employed to image magnetic domains, can be used to image phase separation on the nanoscale. We observe nucleation and growth of nanometre-sized metallic domains out of the insulating phase of the prototypical correlated material VO2, using linearly polarized coherent synchrotron radiation. By spectrally resolving the holograms, we extract differential soft X-ray absorption spectra with 50 nm spatial resolution. Furthermore, when combined with ultra-bright and ultra-short X-ray sources, X-ray holography could capture both nanoscale spatial variations and temporal fluctuations that occur close to the transition temperatures or are induced by femtosecond light pulses that cannot be observed with alternative imaging methods.
△ Less
Submitted 30 November, 2017; v1 submitted 23 December, 2016;
originally announced December 2016.
-
The Ultrafast Optical Response of the Amorphous and Crystalline States of the Phase Change Material Ge$_2$Sb$_2$Te$_5$
Authors:
Timothy A. Miller,
Miquel Rudé,
Valerio Pruneri,
Simon Wall
Abstract:
We examine the ultrafast optical response of the crystalline and amorphous phases of the phase change material Ge$_2$Sb$_2$Te$_5$ below the phase transformation threshold. Simultaneous measurement of the transmissivity and reflectivity of thin film samples yields the time-dependent evolution of the dielectric function for both phases. We then identify how lattice motion and electronic excitation m…
▽ More
We examine the ultrafast optical response of the crystalline and amorphous phases of the phase change material Ge$_2$Sb$_2$Te$_5$ below the phase transformation threshold. Simultaneous measurement of the transmissivity and reflectivity of thin film samples yields the time-dependent evolution of the dielectric function for both phases. We then identify how lattice motion and electronic excitation manifest in the dielectric response. The dielectric response of both phases is large but markedly different. At 800 nm, the changes in amorphous GST are well described by the Drude response of the generated photo-carriers, whereas the crystalline phase is better described by the depopulation of resonant bonds. We find that the generated coherent phonons have a greater influence in the amorphous phase than the crystalline phase. Furthermore, coherent phonons do not influence resonant bonding. For fluences up to 50% of the transformation threshold, the structure does not exhibit bond softening in either phase, enabling large changes of the optical properties without structural modification.
△ Less
Submitted 18 July, 2016; v1 submitted 22 January, 2016;
originally announced January 2016.
-
Resonant optical control of the structural distortions that drive ultrafast demagnetization in Cr$_2$O$_3$
Authors:
Vera G. Sala,
Stefano Dal Conte,
Timothy A. Miller,
Daniele Viola,
Elenora Luppi,
Valérie Véniard,
Giulio Cerullo,
Simon Wall
Abstract:
We study how the color and polarization of ultrashort pulses of visible light can be used to control the demagnetization processes of the antiferromagnetic insulator Cr$_2$O$_3$. We utilize time-resolved second harmonic generation (SHG) to probe how changes in the magnetic and structural state evolve in time. We show that, varying the pump photon-energy to excite either localized transitions withi…
▽ More
We study how the color and polarization of ultrashort pulses of visible light can be used to control the demagnetization processes of the antiferromagnetic insulator Cr$_2$O$_3$. We utilize time-resolved second harmonic generation (SHG) to probe how changes in the magnetic and structural state evolve in time. We show that, varying the pump photon-energy to excite either localized transitions within the Cr or charge transfer states, leads to markedly different dynamics. Through a full polarization analysis of the SHG signal, symmetry considerations and density functional theory calculations, we show that, in the non-equilibrium state, SHG is sensitive to {\em both} lattice displacements and changes to the magnetic order, which allows us to conclude that different excited states couple to phonon modes of different symmetries. Furthermore, the spin-scattering rate depends on the induced distortion, enabling us to control the timescale for the demagnetization process. Our results suggest that selective photoexcitation of antiferromagnetic insulators allows fast and efficient manipulation of their magnetic state.
△ Less
Submitted 20 July, 2016; v1 submitted 30 September, 2015;
originally announced September 2015.
-
Ultrafast broadband tuning of resonant optical nanostructures using phase change materials
Authors:
Miquel Rudé,
Vahagn Mkhitaryan,
Arif E. Cetin,
Timothy A. Miller,
Albert Carrilero,
Simon Wall,
F. Javier García de Abajo,
Hatice Altug,
Valerio Pruneri
Abstract:
The functionalities of a wide range of optical and opto-electronic devices are based on resonance effects and active tuning of the amplitude and wavelength response is often essential. Plasmonic nanostructures are an efficient way to create optical resonances, a prominent example is the extraordinary optical transmission (EOT) through arrays of nanoholes patterned in a metallic film. Tuning of res…
▽ More
The functionalities of a wide range of optical and opto-electronic devices are based on resonance effects and active tuning of the amplitude and wavelength response is often essential. Plasmonic nanostructures are an efficient way to create optical resonances, a prominent example is the extraordinary optical transmission (EOT) through arrays of nanoholes patterned in a metallic film. Tuning of resonances by heating, applying electrical or optical signals has proven to be more elusive, due to the lack of materials that can induce modulation over a broad spectral range and/or at high speeds. Here we show that nanopatterned metals combined with phase change materials (PCMs) can overcome this limitation due to the large change in optical constants which can be induced thermally or on an ultrafast timescale. We demonstrate resonance wavelength shifts as large as 385 nm - an order of magnitude higher than previously reported - by combining properly designed Au EOT nanostructures with Ge2Sb2Te5 (GST). Moreover, we show, through pump probe measurements, repeatable and reversible, large amplitude modulations in the resonances, especially at telecommunication wavelengths, over ps time scales and at powers far below those needed to produce a permanent phase transition. Our findings open a pathway to the design of hybrid metal PCM nanostructures with ultrafast and widely tuneable resonance responses, which hold potential impact on active nanophotonic devices such as tuneable optical filters, smart windows, biosensors and reconfigurable memories.
△ Less
Submitted 29 October, 2015; v1 submitted 11 June, 2015;
originally announced June 2015.
-
THz Field Control of In-Plane Orbital Order in La0.5Sr1.5MnO4
Authors:
Timothy A. Miller,
Ravindra W. Chhajlany,
Luca Tagliacozzo,
Bertram Green,
Sergey Kovalev,
Dharmalingam Prabhakaran,
Maciej Lewenstein,
Michael Gensch,
Simon Wall
Abstract:
In-plane anisotropic ground states are ubiquitous in correlated solids such as pnictides, cuprates and manganites. They can arise from do** Mott insulators and compete with phases such as superconductivity, however their origins are debated. Strong coupling between lattice, charge, orbital and spin degrees of freedom results in simultaneous ordering of multiple parameters, masking the mechanism…
▽ More
In-plane anisotropic ground states are ubiquitous in correlated solids such as pnictides, cuprates and manganites. They can arise from do** Mott insulators and compete with phases such as superconductivity, however their origins are debated. Strong coupling between lattice, charge, orbital and spin degrees of freedom results in simultaneous ordering of multiple parameters, masking the mechanism that drives the transition. We demonstrate that the anisotropic orbital domains in a manganite can be oriented by the polarization of a pulsed THz light field. Through the application of the Hubbard model, we show that domain control can be achieved either through field assisted hop** of charges or a field-induced modification of bond angles. Both routes enhance the local Coulomb repulsions which drive domain reorientation and the dominant mechanism is dictated by the equilibrium Mn-O-Mn bond angle. Our results highlight the key role played by the Coulomb interaction in driving orbital order in manganites and demonstrate how THz can be utilized in new ways to understand and manipulate anisotropic phases in a broad range of correlated materials.
△ Less
Submitted 23 September, 2015; v1 submitted 4 June, 2015;
originally announced June 2015.
-
Time-Domain Separation of Optical Properties From Structural Transitions in Resonantly Bonded Materials
Authors:
Lutz Waldecker,
Timothy A. Miller,
Miquel Rude,
Roman Bertoni,
Johann Osmond,
Valerio Pruneri,
Robert Simpson,
Ralph Ernstorfer,
Simon Wall
Abstract:
The extreme electro-optical contrast between crystalline and amorphous states in phase change materials is routinely exploited in optical data storage and future applications include universal memories, flexible displays, reconfigurable optical circuits, and logic devices. Optical contrast is believed to arise due to a change in crystallinity. Here we show that the connection between optical prope…
▽ More
The extreme electro-optical contrast between crystalline and amorphous states in phase change materials is routinely exploited in optical data storage and future applications include universal memories, flexible displays, reconfigurable optical circuits, and logic devices. Optical contrast is believed to arise due to a change in crystallinity. Here we show that the connection between optical properties and structure can be broken. Using a unique combination of single-shot femtosecond electron diffraction and optical spectroscopy, we simultaneously follow the lattice dynamics and dielectric function in the phase change material Ge2Sb2Te5 during an irreversible state transformation. The dielectric function changes by 30% within 100 femtoseconds due to a rapid depletion of electrons from resonantly-bonded states. This occurs without perturbing the crystallinity of the lattice, which heats with a 2 ps time constant. The optical changes are an order-of-magnitude larger than those achievable with silicon and present new routes to manipulate light on an ultrafast timescale without structural changes.
△ Less
Submitted 10 August, 2015; v1 submitted 2 December, 2014;
originally announced December 2014.
-
Status of head-on beam-beam compensation in RHIC
Authors:
W. Fischer,
Z. Altinbas,
M. Anerella,
M. Blaskiewicz,
D. Bruno,
M. Costanzo,
W. C. Dawson,
D. M. Gassner,
X. Gu,
R. C. Gupta,
K. Hamdi,
J. Hock,
L. T. Hoff,
R. Hulsart,
A. K. Jain,
R. Lambiase,
Y. Luo,
M. Mapes,
A. Marone,
R. Michnoff,
T. A. Miller,
M. Minty,
C. Montag,
J. Muratore,
S. Nemesure
, et al. (12 additional authors not shown)
Abstract:
In polarized proton operation, the performance of the Relativistic Heavy Ion Collider (RHIC) is limited by the head-on beam-beam effect. To overcome this limitation, two electron lenses are under commissioning. We give an overview of head-on beam-beam compensation in general and in the specific design for RHIC, which is based on electron lenses. The status of installation and commissioning are pre…
▽ More
In polarized proton operation, the performance of the Relativistic Heavy Ion Collider (RHIC) is limited by the head-on beam-beam effect. To overcome this limitation, two electron lenses are under commissioning. We give an overview of head-on beam-beam compensation in general and in the specific design for RHIC, which is based on electron lenses. The status of installation and commissioning are presented along with plans for the future.
△ Less
Submitted 20 October, 2014;
originally announced October 2014.
