-
VADER: Visual Affordance Detection and Error Recovery for Multi Robot Human Collaboration
Authors:
Michael Ahn,
Montserrat Gonzalez Arenas,
Matthew Bennice,
Noah Brown,
Christine Chan,
Byron David,
Anthony Francis,
Gavin Gonzalez,
Rainer Hessmer,
Tomas Jackson,
Nikhil J Joshi,
Daniel Lam,
Tsang-Wei Edward Lee,
Alex Luong,
Sharath Maddineni,
Harsh Patel,
Jodilyn Peralta,
Jornell Quiambao,
Diego Reyes,
Rosario M Jauregui Ruano,
Dorsa Sadigh,
Pannag Sanketi,
Leila Takayama,
Pavel Vodenski,
Fei Xia
Abstract:
Robots today can exploit the rich world knowledge of large language models to chain simple behavioral skills into long-horizon tasks. However, robots often get interrupted during long-horizon tasks due to primitive skill failures and dynamic environments. We propose VADER, a plan, execute, detect framework with seeking help as a new skill that enables robots to recover and complete long-horizon ta…
▽ More
Robots today can exploit the rich world knowledge of large language models to chain simple behavioral skills into long-horizon tasks. However, robots often get interrupted during long-horizon tasks due to primitive skill failures and dynamic environments. We propose VADER, a plan, execute, detect framework with seeking help as a new skill that enables robots to recover and complete long-horizon tasks with the help of humans or other robots. VADER leverages visual question answering (VQA) modules to detect visual affordances and recognize execution errors. It then generates prompts for a language model planner (LMP) which decides when to seek help from another robot or human to recover from errors in long-horizon task execution. We show the effectiveness of VADER with two long-horizon robotic tasks. Our pilot study showed that VADER is capable of performing complex long-horizon tasks by asking for help from another robot to clear a table. Our user study showed that VADER is capable of performing complex long-horizon tasks by asking for help from a human to clear a path. We gathered feedback from people (N=19) about the performance of the VADER performance vs. a robot that did not ask for help. https://google-vader.github.io/
△ Less
Submitted 30 May, 2024; v1 submitted 24 May, 2024;
originally announced May 2024.
-
VACO: a Multi-perspective Development of a Therapeutic and Motivational Virtual Robotic Agent for Concentration for children with ADHD
Authors:
Birte Richter,
Ira-Katharina Petras,
Anna-Lisa Vollmer,
Ayla Luong,
Michael Siniatchkin,
Britta Wrede
Abstract:
In this work, we present (i) a novel approach how artificial intelligence can support in the therapy for better concentration of children with Attention Deficit Hyperactivity Disorder (ADHD) through motivational attention training with a virtual robotic agent and (ii) a development process in which different stakeholders are included with their perspectives. Therefore, we present three participati…
▽ More
In this work, we present (i) a novel approach how artificial intelligence can support in the therapy for better concentration of children with Attention Deficit Hyperactivity Disorder (ADHD) through motivational attention training with a virtual robotic agent and (ii) a development process in which different stakeholders are included with their perspectives. Therefore, we present three participative approaches to include the perspectives of different stakeholders. An online survey (Study I) was conducted with parents in Germany with the aim of ascertaining whether they would use software to promote their children's attention, what influences their attitude towards using it, and what requirements it would have to meet. About half of the parents would be willing to use software to promote attention. To develop the software as close to practice as possible, one of the developers took part in an intensive training for ADHD with the aim of testing which of the elements are technically feasible. Afterward, a first prototype was presented to clinicians (Study II) to make further adjustments. A first feasibility test (Study III) was conducted with the end users to check if the system works and if children and adolescents can use it. Attentional performance software offers multiple opportunities in the treatment of ADHD if the system is adapted to the needs of the practitioner and end user. This development process requires a lot of time and close interdisciplinary collaboration.
△ Less
Submitted 6 May, 2024;
originally announced May 2024.
-
Streaming Active Deep Forest for Evolving Data Stream Classification
Authors:
Anh Vu Luong,
Tien Thanh Nguyen,
Alan Wee-Chung Liew
Abstract:
In recent years, Deep Neural Networks (DNNs) have gained progressive momentum in many areas of machine learning. The layer-by-layer process of DNNs has inspired the development of many deep models, including deep ensembles. The most notable deep ensemble-based model is Deep Forest, which can achieve highly competitive performance while having much fewer hyper-parameters comparing to DNNs. In spite…
▽ More
In recent years, Deep Neural Networks (DNNs) have gained progressive momentum in many areas of machine learning. The layer-by-layer process of DNNs has inspired the development of many deep models, including deep ensembles. The most notable deep ensemble-based model is Deep Forest, which can achieve highly competitive performance while having much fewer hyper-parameters comparing to DNNs. In spite of its huge success in the batch learning setting, no effort has been made to adapt Deep Forest to the context of evolving data streams. In this work, we introduce the Streaming Deep Forest (SDF) algorithm, a high-performance deep ensemble method specially adapted to stream classification. We also present the Augmented Variable Uncertainty (AVU) active learning strategy to reduce the labeling cost in the streaming context. We compare the proposed methods to state-of-the-art streaming algorithms in a wide range of datasets. The results show that by following the AVU active learning strategy, SDF with only 70\% of labeling budget significantly outperforms other methods trained with all instances.
△ Less
Submitted 26 February, 2020;
originally announced February 2020.
-
Sitara: Spectrum Measurement Goes Mobile Through Crowd-sourcing
Authors:
Phillip Smith,
Anh Luong,
Shamik Sarkar,
Harsimran Singh,
Neal Patwari,
Sneha Kasera,
Kurt Derr,
Samuel Ramirez
Abstract:
Software-defined radios (SDRs) are often used in the experimental evaluation of next-generation wireless technologies. While crowd-sourced spectrum monitoring is an important component of future spectrum-agile technologies, there is no clear way to test it in the real world, i.e., with hundreds of users each with an SDR in their pocket participating in RF experiments controlled by, and data upload…
▽ More
Software-defined radios (SDRs) are often used in the experimental evaluation of next-generation wireless technologies. While crowd-sourced spectrum monitoring is an important component of future spectrum-agile technologies, there is no clear way to test it in the real world, i.e., with hundreds of users each with an SDR in their pocket participating in RF experiments controlled by, and data uploaded to, the cloud. Current fully functional SDRs are bulky, with components connected via wires, and last at most hours on a single battery charge. To address the needs of such experiments, we design and develop a compact, portable, untethered, and inexpensive SDR we call Sitara. Our SDR interfaces with a mobile device over Bluetooth 5 and can function standalone or as a client to a central command and control server. The Sitara offers true portability: it operates up to one week on battery power, requires no external wired connections and occupies a footprint smaller than a credit card. It transmits and receives common waveforms, uploads IQ samples or processed receiver data through a mobile device to a server for remote processing and performs spectrum sensing functions. Multiple Sitaras form a distributed system capable of conducting experiments in wireless networking and communication in addition to RF monitoring and sensing activities. In this paper, we describe our design, evaluate our solution, present experimental results from multi-sensor deployments and discuss the value of this system in future experimentation.
△ Less
Submitted 30 May, 2019;
originally announced May 2019.
-
Learning Deep Representations from Clinical Data for Chronic Kidney Disease
Authors:
Duc Thanh Anh Luong,
Varun Chandola
Abstract:
We study the behavior of a Time-Aware Long Short-Term Memory Autoencoder, a state-of-the-art method, in the context of learning latent representations from irregularly sampled patient data. We identify a key issue in the way such recurrent neural network models are being currently used and show that the solution of the issue leads to significant improvements in the learnt representations on both s…
▽ More
We study the behavior of a Time-Aware Long Short-Term Memory Autoencoder, a state-of-the-art method, in the context of learning latent representations from irregularly sampled patient data. We identify a key issue in the way such recurrent neural network models are being currently used and show that the solution of the issue leads to significant improvements in the learnt representations on both synthetic and real datasets. A detailed analysis of the improved methodology for representing patients suffering from Chronic Kidney Disease (CKD) using clinical data is provided. Experimental results show that the proposed T-LSTM model is able to capture the long-term trends in the data, while effectively handling the noise in the signal. Finally, we show that by using the latent representations of the CKD patients obtained from the T-LSTM autoencoder, one can identify unusual patient profiles from the target population.
△ Less
Submitted 9 February, 2019; v1 submitted 30 September, 2018;
originally announced October 2018.
-
dynamicMF: A Matrix Factorization Approach to Monitor Resource Usage in High Performance Computing Systems
Authors:
Niyazi Sorkunlu,
Duc Thanh Anh Luong,
Varun Chandola
Abstract:
High performance computing (HPC) facilities consist of a large number of interconnected computing units (or nodes) that execute highly complex scientific simulations to support scientific research. Monitoring such facilities, in real-time, is essential to ensure that the system operates at peak efficiency. Such systems are typically monitored using a variety of measurement and log data which captu…
▽ More
High performance computing (HPC) facilities consist of a large number of interconnected computing units (or nodes) that execute highly complex scientific simulations to support scientific research. Monitoring such facilities, in real-time, is essential to ensure that the system operates at peak efficiency. Such systems are typically monitored using a variety of measurement and log data which capture the state of the various components within the system at regular intervals of time. As modern HPC systems grow in capacity and complexity, the data produced by current resource monitoring tools is at a scale that it is no longer feasible to be visually monitored by analysts. We propose a method that transforms the multi-dimensional output of resource monitoring tools to a low dimensional representation that facilitates the understanding of the behavior of a High Performance Computing (HPC) system. The proposed method automatically extracts the low-dimensional signal in the data which can be used to track the system efficiency and identify performance anomalies. The method models the resource usage data as a three dimensional tensor (capturing resource usage of all compute nodes for difference resources over time). A dynamic matrix factorization algorithm, called dynamicMF, is proposed to extract a low-dimensional temporal signal for each node, which is subsequently fed into an anomaly detector. Results on resource usage data collected from the Lonestar 4 system at the Texas Advanced Computing Center show that the identified anomalies are correlated with actual anomalous events reported in the system log messages.
△ Less
Submitted 26 September, 2018;
originally announced September 2018.
-
Dial It In: Rotating RF Sensors to Enhance Radio Tomography
Authors:
Maurizio Bocca,
Anh Luong,
Neal Patwari,
Thomas Schmid
Abstract:
A radio tomographic imaging (RTI) system uses the received signal strength (RSS) measured by RF sensors in a static wireless network to localize people in the deployment area, without having them to carry or wear an electronic device. This paper addresses the fact that small-scale changes in the position and orientation of the antenna of each RF sensor can dramatically affect imaging and localizat…
▽ More
A radio tomographic imaging (RTI) system uses the received signal strength (RSS) measured by RF sensors in a static wireless network to localize people in the deployment area, without having them to carry or wear an electronic device. This paper addresses the fact that small-scale changes in the position and orientation of the antenna of each RF sensor can dramatically affect imaging and localization performance of an RTI system. However, the best placement for a sensor is unknown at the time of deployment. Improving performance in a deployed RTI system requires the deployer to iteratively "guess-and-retest", i.e., pick a sensor to move and then re-run a calibration experiment to determine if the localization performance had improved or degraded. We present an RTI system of servo-nodes, RF sensors equipped with servo motors which autonomously "dial it in", i.e., change position and orientation to optimize the RSS on links of the network. By doing so, the localization accuracy of the RTI system is quickly improved, without requiring any calibration experiment from the deployer. Experiments conducted in three indoor environments demonstrate that the servo-nodes system reduces localization error on average by 32% compared to a standard RTI system composed of static RF sensors.
△ Less
Submitted 19 December, 2013;
originally announced December 2013.
