-
Roleplay-doh: Enabling Domain-Experts to Create LLM-simulated Patients via Eliciting and Adhering to Principles
Authors:
Ryan Louie,
Ananjan Nandi,
William Fang,
Cheng Chang,
Emma Brunskill,
Diyi Yang
Abstract:
Recent works leverage LLMs to roleplay realistic social scenarios, aiding novices in practicing their social skills. However, simulating sensitive interactions, such as in mental health, is challenging. Privacy concerns restrict data access, and collecting expert feedback, although vital, is laborious. To address this, we develop Roleplay-doh, a novel human-LLM collaboration pipeline that elicits…
▽ More
Recent works leverage LLMs to roleplay realistic social scenarios, aiding novices in practicing their social skills. However, simulating sensitive interactions, such as in mental health, is challenging. Privacy concerns restrict data access, and collecting expert feedback, although vital, is laborious. To address this, we develop Roleplay-doh, a novel human-LLM collaboration pipeline that elicits qualitative feedback from a domain-expert, which is transformed into a set of principles, or natural language rules, that govern an LLM-prompted roleplay. We apply this pipeline to enable senior mental health supporters to create customized AI patients for simulated practice partners for novice counselors. After uncovering issues in GPT-4 simulations not adhering to expert-defined principles, we also introduce a novel principle-adherence prompting pipeline which shows 30\% improvements in response quality and principle following for the downstream task. Via a user study with 25 counseling experts, we demonstrate that the pipeline makes it easy and effective to create AI patients that more faithfully resemble real patients, as judged by creators and third-party counselors.
△ Less
Submitted 30 June, 2024;
originally announced July 2024.
-
Multi-Level Feedback Generation with Large Language Models for Empowering Novice Peer Counselors
Authors:
Alicja Chaszczewicz,
Raj Sanjay Shah,
Ryan Louie,
Bruce A Arnow,
Robert Kraut,
Diyi Yang
Abstract:
Realistic practice and tailored feedback are key processes for training peer counselors with clinical skills. However, existing mechanisms of providing feedback largely rely on human supervision. Peer counselors often lack mechanisms to receive detailed feedback from experienced mentors, making it difficult for them to support the large number of people with mental health issues who use peer couns…
▽ More
Realistic practice and tailored feedback are key processes for training peer counselors with clinical skills. However, existing mechanisms of providing feedback largely rely on human supervision. Peer counselors often lack mechanisms to receive detailed feedback from experienced mentors, making it difficult for them to support the large number of people with mental health issues who use peer counseling. Our work aims to leverage large language models to provide contextualized and multi-level feedback to empower peer counselors, especially novices, at scale. To achieve this, we co-design with a group of senior psychotherapy supervisors to develop a multi-level feedback taxonomy, and then construct a publicly available dataset with comprehensive feedback annotations of 400 emotional support conversations. We further design a self-improvement method on top of large language models to enhance the automatic generation of feedback. Via qualitative and quantitative evaluation with domain experts, we demonstrate that our method minimizes the risk of potentially harmful and low-quality feedback generation which is desirable in such high-stakes scenarios.
△ Less
Submitted 21 March, 2024;
originally announced March 2024.
-
Expressive Communication: A Common Framework for Evaluating Developments in Generative Models and Steering Interfaces
Authors:
Ryan Louie,
Jesse Engel,
Anna Huang
Abstract:
There is an increasing interest from ML and HCI communities in empowering creators with better generative models and more intuitive interfaces with which to control them. In music, ML researchers have focused on training models capable of generating pieces with increasing long-range structure and musical coherence, while HCI researchers have separately focused on designing steering interfaces that…
▽ More
There is an increasing interest from ML and HCI communities in empowering creators with better generative models and more intuitive interfaces with which to control them. In music, ML researchers have focused on training models capable of generating pieces with increasing long-range structure and musical coherence, while HCI researchers have separately focused on designing steering interfaces that support user control and ownership. In this study, we investigate through a common framework how developments in both models and user interfaces are important for empowering co-creation where the goal is to create music that communicates particular imagery or ideas (e.g., as is common for other purposeful tasks in music creation like establishing mood or creating accompanying music for another media). Our study is distinguished in that it measures communication through both composer's self-reported experiences, and how listeners evaluate this communication through the music. In an evaluation study with 26 composers creating 100+ pieces of music and listeners providing 1000+ head-to-head comparisons, we find that more expressive models and more steerable interfaces are important and complementary ways to make a difference in composers communicating through music and supporting their creative empowerment.
△ Less
Submitted 29 November, 2021;
originally announced November 2021.
-
Analysis and Design of Multiple-Antenna Cognitive Radios with Multiple Primary User Signals
Authors:
David Morales-Jimenez,
Raymond H. Y. Louie,
Matthew R. McKay,
Yang Chen
Abstract:
We consider multiple-antenna signal detection of primary user transmission signals by a secondary user receiver in cognitive radio networks. The optimal detector is analyzed for the scenario where the number of primary user signals is no less than the number of receive antennas at the secondary user. We first derive exact expressions for the moments of the generalized likelihood ratio test (GLRT)…
▽ More
We consider multiple-antenna signal detection of primary user transmission signals by a secondary user receiver in cognitive radio networks. The optimal detector is analyzed for the scenario where the number of primary user signals is no less than the number of receive antennas at the secondary user. We first derive exact expressions for the moments of the generalized likelihood ratio test (GLRT) statistic, yielding approximations for the false alarm and detection probabilities. We then show that the normalized GLRT statistic converges in distribution to a Gaussian random variable when the number of antennas and observations grow large at the same rate. Further, using results from large random matrix theory, we derive expressions to compute the detection probability without explicit knowledge of the channel, and then particularize these expressions for two scenarios of practical interest: 1) a single primary user sending spatially multiplexed signals, and 2) multiple spatially distributed primary users. Our analytical results are finally used to obtain simple design rules for the signal detection threshold.
△ Less
Submitted 23 April, 2015; v1 submitted 25 May, 2014;
originally announced May 2014.
-
Dynamic Decentralized Algorithms for Cognitive Radio Relay Networks
Authors:
Siavash Bayat,
Raymond H. Y. Louie,
Branka Vucetic,
Yonghui Li
Abstract:
We propose a distributed spectrum access algorithm for cognitive radio relay networks with multiple primary users (PU) and multiple secondary users (SU). The key idea behind the proposed algorithm is that the PUs negotiate with the SUs on both the amount of monetary compensation, and the amount of time the SUs are either (i) allowed spectrum access, or (ii) cooperatively relaying the PU's data, su…
▽ More
We propose a distributed spectrum access algorithm for cognitive radio relay networks with multiple primary users (PU) and multiple secondary users (SU). The key idea behind the proposed algorithm is that the PUs negotiate with the SUs on both the amount of monetary compensation, and the amount of time the SUs are either (i) allowed spectrum access, or (ii) cooperatively relaying the PU's data, such that both the PUs' and the SUs' minimum rate requirement are satisfied. The proposed algorithm is shown to be flexible in prioritizing either the primary or the secondary users. We prove that the proposed algorithm will result in the best possible stable matching and is weak Pareto optimal. Numerical analysis also reveal that the distributed algorithm can achieve a performance comparable to an optimal centralized solution, but with significantly less overhead and complexity.
△ Less
Submitted 2 July, 2013; v1 submitted 13 September, 2011;
originally announced September 2011.
-
Relay Selection with Network Coding in Two-Way Relay Channels
Authors:
Yonghui Li,
Raymond H. Y. Louie,
Branka Vucetic
Abstract:
In this paper, we consider the design of joint network coding (NC)and relay selection (RS) in two-way relay channels. In the proposed schemes, two users first sequentially broadcast their respective information to all the relays. We propose two RS schemes, a single relay selection with NC and a dual relay selection with NC. For both schemes, the selected relay(s) perform NC on the received signals…
▽ More
In this paper, we consider the design of joint network coding (NC)and relay selection (RS) in two-way relay channels. In the proposed schemes, two users first sequentially broadcast their respective information to all the relays. We propose two RS schemes, a single relay selection with NC and a dual relay selection with NC. For both schemes, the selected relay(s) perform NC on the received signals sent from the two users and forward them to both users. The proposed schemes are analyzed and the exact bit error rate (BER) expressions are derived and verified through Monte Carlo simulations. It is shown that the dual relay selection with NC outperforms other considered relay selection schemes in two-way relay channels. The results also reveal that the proposed NC relay selection schemes provide a selection gain compared to a NC scheme with no relay selection, and a network coding gain relative to a conventional relay selection scheme with no NC.
△ Less
Submitted 26 September, 2010;
originally announced September 2010.
-
Open-Loop Spatial Multiplexing and Diversity Communications in Ad Hoc Networks
Authors:
Raymond H. Y. Louie,
Matthew R. McKay,
Iain B. Collings
Abstract:
This paper investigates the performance of open-loop multi-antenna point-to-point links in ad hoc networks with slotted ALOHA medium access control (MAC). We consider spatial multiplexing transmission with linear maximum ratio combining and zero forcing receivers, as well as orthogonal space time block coded transmission. New closed-form expressions are derived for the outage probability, throughp…
▽ More
This paper investigates the performance of open-loop multi-antenna point-to-point links in ad hoc networks with slotted ALOHA medium access control (MAC). We consider spatial multiplexing transmission with linear maximum ratio combining and zero forcing receivers, as well as orthogonal space time block coded transmission. New closed-form expressions are derived for the outage probability, throughput and transmission capacity. Our results demonstrate that both the best performing scheme and the optimum number of transmit antennas depend on different network parameters, such as the node intensity and the signal-to-interference-and-noise ratio operating value. We then compare the performance to a network consisting of single-antenna devices and an idealized fully centrally coordinated MAC. These results show that multi-antenna schemes with a simple decentralized slotted ALOHA MAC can outperform even idealized single-antenna networks in various practical scenarios.
△ Less
Submitted 17 September, 2010; v1 submitted 16 September, 2010;
originally announced September 2010.
-
Spatial multiplexing with MMSE receivers: Single-stream optimality in ad hoc networks
Authors:
Raymond H. Y. Louie,
Matthew R. McKay,
Nihar **dal,
Iain B. Collings
Abstract:
The performance of spatial multiplexing systems with linear minimum-mean-squared-error receivers is investigated in ad hoc networks. It is shown that single-stream transmission is preferable over multi-stream transmission, due to the weaker interference powers from the strongest interferers remaining after interference-cancelation. This result is obtained by new exact closed-form expressions we de…
▽ More
The performance of spatial multiplexing systems with linear minimum-mean-squared-error receivers is investigated in ad hoc networks. It is shown that single-stream transmission is preferable over multi-stream transmission, due to the weaker interference powers from the strongest interferers remaining after interference-cancelation. This result is obtained by new exact closed-form expressions we derive for the outage probability and transmission capacity.
△ Less
Submitted 15 March, 2010;
originally announced March 2010.
-
Maximum Sum-Rate of MIMO Multiuser Scheduling with Linear Receivers
Authors:
Raymond H. Y. Louie,
Matthew R. McKay,
Iain B. Collings
Abstract:
We analyze scheduling algorithms for multiuser communication systems with users having multiple antennas and linear receivers. When there is no feedback of channel information, we consider a common round robin scheduling algorithm, and derive new exact and high signal-to-noise ratio (SNR) maximum sum-rate results for the maximum ratio combining (MRC) and minimum mean squared error (MMSE) receive…
▽ More
We analyze scheduling algorithms for multiuser communication systems with users having multiple antennas and linear receivers. When there is no feedback of channel information, we consider a common round robin scheduling algorithm, and derive new exact and high signal-to-noise ratio (SNR) maximum sum-rate results for the maximum ratio combining (MRC) and minimum mean squared error (MMSE) receivers. We also present new analysis of MRC, zero forcing (ZF) and MMSE receivers in the low SNR regime. When there are limited feedback capabilities in the system, we consider a common practical scheduling scheme based on signal-to-interference-and-noise ratio (SINR) feedback at the transmitter. We derive new accurate approximations for the maximum sum-rate, for the cases of MRC, ZF and MMSE receivers. We also derive maximum sum-rate scaling laws, which reveal that the maximum sum-rate of all three linear receivers converge to the same value for a large number of users, but at different rates.
△ Less
Submitted 18 December, 2008; v1 submitted 17 December, 2008;
originally announced December 2008.
