-
Reinforcement Learning Tutor Better Supported Lower Performers in a Math Task
Authors:
Sherry Ruan,
Allen Nie,
William Steenbergen,
Jiayu He,
JQ Zhang,
Meng Guo,
Yao Liu,
Kyle Dang Nguyen,
Catherine Y Wang,
Rui Ying,
James A Landay,
Emma Brunskill
Abstract:
Resource limitations make it hard to provide all students with one of the most effective educational interventions: personalized instruction. Reinforcement learning could be a key tool to reduce the development cost and improve the effectiveness of intelligent tutoring software that aims to provide the right support, at the right time, to a student. Here we illustrate that deep reinforcement learn…
▽ More
Resource limitations make it hard to provide all students with one of the most effective educational interventions: personalized instruction. Reinforcement learning could be a key tool to reduce the development cost and improve the effectiveness of intelligent tutoring software that aims to provide the right support, at the right time, to a student. Here we illustrate that deep reinforcement learning can be used to provide adaptive pedagogical support to students learning about the concept of volume in a narrative storyline software. Using explainable artificial intelligence tools, we extracted interpretable insights about the pedagogical policy learned and demonstrated that the resulting policy had similar performance in a different student population. Most importantly, in both studies, the reinforcement-learning narrative system had the largest benefit for those students with the lowest initial pretest scores, suggesting the opportunity for AI to adapt and provide support for those most in need.
△ Less
Submitted 13 April, 2023; v1 submitted 10 April, 2023;
originally announced April 2023.
-
Data-Efficient Pipeline for Offline Reinforcement Learning with Limited Data
Authors:
Allen Nie,
Yannis Flet-Berliac,
Deon R. Jordan,
William Steenbergen,
Emma Brunskill
Abstract:
Offline reinforcement learning (RL) can be used to improve future performance by leveraging historical data. There exist many different algorithms for offline RL, and it is well recognized that these algorithms, and their hyperparameter settings, can lead to decision policies with substantially differing performance. This prompts the need for pipelines that allow practitioners to systematically pe…
▽ More
Offline reinforcement learning (RL) can be used to improve future performance by leveraging historical data. There exist many different algorithms for offline RL, and it is well recognized that these algorithms, and their hyperparameter settings, can lead to decision policies with substantially differing performance. This prompts the need for pipelines that allow practitioners to systematically perform algorithm-hyperparameter selection for their setting. Critically, in most real-world settings, this pipeline must only involve the use of historical data. Inspired by statistical model selection methods for supervised learning, we introduce a task- and method-agnostic pipeline for automatically training, comparing, selecting, and deploying the best policy when the provided dataset is limited in size. In particular, our work highlights the importance of performing multiple data splits to produce more reliable algorithm-hyperparameter selection. While this is a common approach in supervised learning, to our knowledge, this has not been discussed in detail in the offline RL setting. We show it can have substantial impacts when the dataset is small. Compared to alternate approaches, our proposed pipeline outputs higher-performing deployed policies from a broad range of offline policy learning algorithms and across various simulation domains in healthcare, education, and robotics. This work contributes toward the development of a general-purpose meta-algorithm for automatic algorithm-hyperparameter selection for offline RL.
△ Less
Submitted 12 January, 2023; v1 submitted 16 October, 2022;
originally announced October 2022.
-
A Framework for Directional and Higher-Order Reconstruction in Photoacoustic Tomography
Authors:
Yoeri E. Boink,
Marinus J. Lagerwerf,
Wiendelt Steenbergen,
Stephan A. van Gils,
Srirang Manohar,
Christoph Brune
Abstract:
Photoacoustic tomography is a hybrid imaging technique that combines high optical tissue contrast with high ultrasound resolution. Direct reconstruction methods such as filtered backprojection, time reversal and least squares suffer from curved line artefacts and blurring, especially in case of limited angles or strong noise. In recent years, there has been great interest in regularised iterative…
▽ More
Photoacoustic tomography is a hybrid imaging technique that combines high optical tissue contrast with high ultrasound resolution. Direct reconstruction methods such as filtered backprojection, time reversal and least squares suffer from curved line artefacts and blurring, especially in case of limited angles or strong noise. In recent years, there has been great interest in regularised iterative methods. These methods employ prior knowledge on the image to provide higher quality reconstructions. However, easy comparisons between regularisers and their properties are limited, since many tomography implementations heavily rely on the specific regulariser chosen. To overcome this bottleneck, we present a modular reconstruction framework for photoacoustic tomography. It enables easy comparisons between regularisers with different properties, e.g. nonlinear, higher-order or directional. We solve the underlying minimisation problem with an efficient first-order primal-dual algorithm. Convergence rates are optimised by choosing an operator dependent preconditioning strategy. Our reconstruction methods are tested on challenging 2D synthetic and experimental data sets. They outperform direct reconstruction approaches for strong noise levels and limited angle measurements, offering immediate benefits in terms of acquisition time and quality. This work provides a basic platform for the investigation of future advanced regularisation methods in photoacoustic tomography.
△ Less
Submitted 28 September, 2017; v1 submitted 7 July, 2017;
originally announced July 2017.
-
A method for delineation of bone surfaces in photoacoustic computed tomography of the finger
Authors:
Samir Biswas,
Peter van Es,
Wiendelt Steenbergen,
Srirang Manohar
Abstract:
Photoacoustic imaging of interphalangeal peripheral joints is of interest in the context of using the synovial membrane as a surrogate marker of rheumatoid arthritis. Previous work has shown that ultrasound produced by absorption of light at the epidermis reflects on the bone surfaces within the finger. When the reflected signals are backprojected in the region of interest, artifacts are produced,…
▽ More
Photoacoustic imaging of interphalangeal peripheral joints is of interest in the context of using the synovial membrane as a surrogate marker of rheumatoid arthritis. Previous work has shown that ultrasound produced by absorption of light at the epidermis reflects on the bone surfaces within the finger. When the reflected signals are backprojected in the region of interest, artifacts are produced, confounding interpretation of the images. In this work, we present an approach where the photoacoustic signals known to originate from the epidermis, are treated as virtual ultrasound transmitters, and a separate reconstruction is performed as in ultrasound reflection imaging. This allows us to identify the bone surfaces. Further, the identification of the joint space is important as this provides a landmark to localize a region-of-interest in seeking the inflamed synovial membrane. The ability to delineate bone surfaces allows us not only to identify the artifacts, but also to identify the interphalangeal joint space without recourse to new US hardware or a new measurement. We test the approach on phantoms and on a healthy human finger.
△ Less
Submitted 6 June, 2015;
originally announced June 2015.
-
Initial results of finger imaging using Photoacoustic Computed Tomography
Authors:
Peter van Es,
Samir K. Biswas,
Hein J. Bernelot Moens,
Wiendelt Steenbergen,
Srirang Manohar
Abstract:
We present a photoacoustic computed tomography investigation on a healthy human finger, to image blood vessels with a focus on vascularity across the interphalangeal joints. The cross-sectional images were acquired using an imager specifically developed for this purpose. The images show rich detail of the digital blood vessels with diameters between 100 $μ$m and 1.5 mm in various orientations and…
▽ More
We present a photoacoustic computed tomography investigation on a healthy human finger, to image blood vessels with a focus on vascularity across the interphalangeal joints. The cross-sectional images were acquired using an imager specifically developed for this purpose. The images show rich detail of the digital blood vessels with diameters between 100 $μ$m and 1.5 mm in various orientations and at various depths. Different vascular layers in the skin including the subpapillary plexus could also be visualized. Acoustic reflections on the finger bone of photoacoustic signals from skin were visible in sequential slice images along the finger except at the location of the joint gaps. Not unexpectedly, the healthy synovial membrane at the joint gaps was not detected due to its small size and normal vascularization. Future research will concentrate on studying digits afflicted with rheumatoid arthritis to detect the inflamed synovium with its heightened vascularization, whose characteristics are potential markers for disease activity.
△ Less
Submitted 20 June, 2014;
originally announced June 2014.
-
A new acoustic lens material for large area detectors in photoacoustic breast tomography
Authors:
Wenfeng Xia,
Daniele Piras,
Johan C. G. van Hespen,
Wiendelt Steenbergen,
Srirang Manohar
Abstract:
Acoustic lenses made of acrylic plastic (PMMA) have been used to enlarge the acceptance angle of sensitive large surface area detectors and improve lateral resolution. However, PMMA lenses introduce image artifacts due to ultrasound internal reflections within the lenses. In this work we investigated this issue proposing a new lens material Stycast 1090SI. We characterized the acoustic properties…
▽ More
Acoustic lenses made of acrylic plastic (PMMA) have been used to enlarge the acceptance angle of sensitive large surface area detectors and improve lateral resolution. However, PMMA lenses introduce image artifacts due to ultrasound internal reflections within the lenses. In this work we investigated this issue proposing a new lens material Stycast 1090SI. We characterized the acoustic properties of the proposed material in comparison with PMMA. Detector performance using negative lenses with the two materials, was tested using finite element simulation and experiment. Further the image quality of a photoacoustic tomography system was studied using k-Wave simulation and experiment. Our acoustic characterization showed that Stycast 1090SI has tissue-like acoustic impedance, high speed of sound and low acoustic attenuation. Both acoustic lenses show significant enlargement of detector acceptance angle and lateral resolution improvement. However, image artifacts induced by acoustic lenses are reduced using the proposed lens compared to PMMA lens.
△ Less
Submitted 17 May, 2013;
originally announced May 2013.
-
An optimized ultrasound detector for photoacoustic breast tomography
Authors:
Wenfeng Xia,
Daniele Piras,
Johan Van Hespen,
Spiridon Van Veldhoven,
Christian Prins,
Ton Van Leeuwen,
Wiendelt Steenbergen,
Srirang Manohar
Abstract:
Photoacoustic imaging has proven to be able to detect vascularization-driven optical absorption contrast associated with tumors. In order to detect breast tumors located a few centimeter deep in tissue, a sensitive ultrasound detector is of crucial importance for photoacoustic mammography. Further, because the expected photoacoustic frequency bandwidth (a few MHz to tens of kHz) is inversely propo…
▽ More
Photoacoustic imaging has proven to be able to detect vascularization-driven optical absorption contrast associated with tumors. In order to detect breast tumors located a few centimeter deep in tissue, a sensitive ultrasound detector is of crucial importance for photoacoustic mammography. Further, because the expected photoacoustic frequency bandwidth (a few MHz to tens of kHz) is inversely proportional to the dimensions of light absorbing structures (0.5 to 10+ mm), proper choices of materials and their geometries, and proper considerations in design have to be made for optimal photoacoustic detectors. In this study, we design and evaluate a specialized ultrasound detector for photoacoustic mammography. Based on the required detector sensitivity and its frequency response, a selection of active material and matching layers and their geometries is made leading to a functional detector models. By iteration between simulation of detector performances, fabrication and experimental characterization of functional models an optimized implementation is made and evaluated. The experimental results of the designed first and second functional detectors matched with the simulations. In subsequent bare piezoelectric samples the effect of lateral resonances was addressed and their influence minimized by sub-dicing the samples. Consequently, using simulations, the final optimized detector could be designed, with a center frequency of 1 MHz and a -6 dB bandwidth of ~80%. The minimum detectable pressure was measured to be 0.5 Pa, which will facilitate deeper imaging compared to the currrent systems. The detector should be capable of detecting vascularized tumors with resolution of 1-2 mm. Further improvements by proper electrical grounding and shielding and implementation of this design into an arrayed detector will pave the way for clinical applications of photoacoustic mammography.
△ Less
Submitted 13 February, 2013; v1 submitted 11 December, 2012;
originally announced December 2012.
-
Speed-of-sound compensated photoacoustic tomography for accurate imaging
Authors:
Jithin Jose,
Rene G. H. Willemink,
Wiendelt Steenbergen,
Cornelis H. Slump,
Ton G. van Leeuwen,
Srirang Manohar
Abstract:
In most photoacoustic (PA) measurements, variations in speed-of-sound (SOS) of the subject are neglected under the assumption of acoustic homogeneity. Biological tissue with spatially heterogeneous SOS cannot be accurately reconstructed under this assumption. We present experimental and image reconstruction methods with which 2-D SOS distributions can be accurately acquired and reconstructed, and…
▽ More
In most photoacoustic (PA) measurements, variations in speed-of-sound (SOS) of the subject are neglected under the assumption of acoustic homogeneity. Biological tissue with spatially heterogeneous SOS cannot be accurately reconstructed under this assumption. We present experimental and image reconstruction methods with which 2-D SOS distributions can be accurately acquired and reconstructed, and with which the SOS map can be used subsequently to reconstruct highly accurate PA tomograms. We begin with a 2-D iterative reconstruction approach in an ultrasound transmission tomography (UTT) setting, which uses ray refracted paths instead of straight ray paths to recover accurate SOS images of the subject. Subsequently, we use the SOS distribution in a new 2-D iterative approach, where refraction of rays originating from PA sources are accounted for in accurately retrieving the distribution of these sources. Both the SOS reconstruction and SOS-compensated PA reconstruction methods utilize the Eikonal equation to model acoustic wavefront propagation, which is solved using a high accuracy fast marching method (HAFMM). We validate the new reconstruction algorithms using numerical phantoms. For experiments we use the PER-PACT method which can be used to simultaneously acquire SOS and PA data from subjects. We test the reconstruction algorithms using experimental data acquired with the PER-PACT setup from challenging physical phantoms. The results show that it is important to take SOS inhomogeneities into account. The iterative reconstruction algorithms, that model acoustic refractive effects, yield artifact-free highly accurate images. Our approach of using the hybrid measurement method and the new reconstruction algorithms, is successful in substantially improving the quality of PA images with a minimization of blurring and artefacts.
△ Less
Submitted 30 October, 2012; v1 submitted 6 June, 2012;
originally announced June 2012.
-
Towards quantitative tissue absorption imaging by combining photoacoustics and acousto-optics
Authors:
Khalid Daoudi,
Wiendelt Steenbergen
Abstract:
We propose a strategy for quantitative photoacoustic map** of chromophore concentrations that can be performed purely experimentally. We exploit the possibility of acousto-optic modulation using focused ultrasound, and the principle that photons follow trajectories through a turbid medium in two directions with equal probability. A theory is presented that expresses the local absorption coeffici…
▽ More
We propose a strategy for quantitative photoacoustic map** of chromophore concentrations that can be performed purely experimentally. We exploit the possibility of acousto-optic modulation using focused ultrasound, and the principle that photons follow trajectories through a turbid medium in two directions with equal probability. A theory is presented that expresses the local absorption coefficient inside a medium in terms of noninvasively measured quantities and experimental parameters. Proof of the validity of the theory is given with Monte Carlo simulations.
△ Less
Submitted 11 January, 2012;
originally announced January 2012.
