Iterated Decomposition: Improving Science Q&A by Supervising Reasoning Processes
Authors:
Justin Reppert,
Ben Rachbach,
Charlie George,
Luke Stebbing,
Jungwon Byun,
Maggie Appleton,
Andreas Stuhlmüller
Abstract:
Language models (LMs) can perform complex reasoning either end-to-end, with hidden latent state, or compositionally, with transparent intermediate state. Composition offers benefits for interpretability and safety, but may need workflow support and infrastructure to remain competitive. We describe iterated decomposition, a human-in-the-loop workflow for develo** and refining compositional LM pro…
▽ More
Language models (LMs) can perform complex reasoning either end-to-end, with hidden latent state, or compositionally, with transparent intermediate state. Composition offers benefits for interpretability and safety, but may need workflow support and infrastructure to remain competitive. We describe iterated decomposition, a human-in-the-loop workflow for develo** and refining compositional LM programs. We improve the performance of compositions by zooming in on failing components and refining them through decomposition, additional context, chain of thought, etc. To support this workflow, we develop ICE, an open-source tool for visualizing the execution traces of LM programs. We apply iterated decomposition to three real-world tasks and improve the accuracy of LM programs over less compositional baselines: describing the placebo used in a randomized controlled trial (25% to 65%), evaluating participant adherence to a medical intervention (53% to 70%), and answering NLP questions on the Qasper dataset (38% to 69%). These applications serve as case studies for a workflow that, if automated, could keep ML systems interpretable and safe even as they scale to increasingly complex tasks.
△ Less
Submitted 4 January, 2023; v1 submitted 4 January, 2023;
originally announced January 2023.
Double resonance Raman study of disorder in CVD-grown single-walled carbon nanotubes
Authors:
Rahul Rao,
Jason Reppert,
Xianfeng Zhang,
Ramakrishna Podila,
Apparao M. Rao,
Saikat Talapatra,
Benji Maruyama
Abstract:
Single-walled carbon nanotubes (SWNTs) with varying degrees of disorder were investigated using multiple-excitation Raman spectroscopy. The lattice disorder was imparted into the nanotubes by the addition of varying amounts of sulfur to the iron catalyst in a thermal chemical vapor deposition process. Changes in the intensities of peaks occurring due to a double resonance Raman process were studie…
▽ More
Single-walled carbon nanotubes (SWNTs) with varying degrees of disorder were investigated using multiple-excitation Raman spectroscopy. The lattice disorder was imparted into the nanotubes by the addition of varying amounts of sulfur to the iron catalyst in a thermal chemical vapor deposition process. Changes in the intensities of peaks occurring due to a double resonance Raman process were studied. The intensity of the disorder-induced D band increased with a decrease in the sulfur content. Upon post-synthesis heat treatment, the double resonance process got quenched due to defect healing. The second order G' band and iTOLA bands exhibited a two-peak structure, of which one of the peaks is relatively more sensitive to defects and decreased in intensity with heat treatment.
△ Less
Submitted 29 November, 2010; v1 submitted 22 October, 2010;
originally announced October 2010.
