Showing 1–49 of 49 results for author: Duarte, F

Expressing and Inferring Action Carefulness in Human-to-Robot Handovers

Authors: Linda Lastrico, Nuno Ferreira Duarte, Alessandro Carfì, Francesco Rea, Alessandra Sciutti, Fulvio Mastrogiovanni, José Santos-Victor

Abstract: Implicit communication plays such a crucial role during social exchanges that it must be considered for a good experience in human-robot interaction. This work addresses implicit communication associated with the detection of physical properties, transport, and manipulation of objects. We propose an ecological approach to infer object characteristics from subtle modulations of the natural kinemati… ▽ More Implicit communication plays such a crucial role during social exchanges that it must be considered for a good experience in human-robot interaction. This work addresses implicit communication associated with the detection of physical properties, transport, and manipulation of objects. We propose an ecological approach to infer object characteristics from subtle modulations of the natural kinematics occurring during human object manipulation. Similarly, we take inspiration from human strategies to shape robot movements to be communicative of the object properties while pursuing the action goals. In a realistic HRI scenario, participants handed over cups - filled with water or empty - to a robotic manipulator that sorted them. We implemented an online classifier to differentiate careful/not careful human movements, associated with the cups' content. We compared our proposed "expressive" controller, which modulates the movements according to the cup filling, against a neutral motion controller. Results show that human kinematics is adjusted during the task, as a function of the cup content, even in reach-to-grasp motion. Moreover, the carefulness during the handover of full cups can be reliably inferred online, well before action completion. Finally, although questionnaires did not reveal explicit preferences from participants, the expressive robot condition improved task efficiency. △ Less

Submitted 30 September, 2023; originally announced October 2023.

Comments: PREPRINT VERSION Accepted for publication at the 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS2023)

Journal ref: 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

Understanding Place Identity with Generative AI

Authors: Kee Moon Jang, Junda Chen, Yuhao Kang, Junghwan Kim, **hyung Lee, Fábio Duarte

Abstract: Researchers are constantly leveraging new forms of data with the goal of understanding how people perceive the built environment and build the collective place identity of cities. Latest advancements in generative artificial intelligence (AI) models have enabled the production of realistic representations learned from vast amounts of data. In this study, we aim to test the potential of generative… ▽ More Researchers are constantly leveraging new forms of data with the goal of understanding how people perceive the built environment and build the collective place identity of cities. Latest advancements in generative artificial intelligence (AI) models have enabled the production of realistic representations learned from vast amounts of data. In this study, we aim to test the potential of generative AI as the source of textual and visual information in capturing the place identity of cities assessed by filtered descriptions and images. We asked questions on the place identity of a set of 31 global cities to two generative AI models, ChatGPT and DALL-E2. Since generative AI has raised ethical concerns regarding its trustworthiness, we performed cross-validation to examine whether the results show similar patterns to real urban settings. In particular, we compared the outputs with Wikipedia data for text and images searched from Google for image. Our results indicate that generative AI models have the potential to capture the collective image of cities that can make them distinguishable. This study is among the first attempts to explore the capabilities of generative AI in understanding human perceptions of the built environment. It contributes to urban design literature by discussing future research opportunities and potential limitations. △ Less

Submitted 6 June, 2023; originally announced June 2023.

Comments: 6 pages, 3 figures, GIScience 2023

3DSGrasp: 3D Shape-Completion for Robotic Grasp

Authors: Seyed S. Mohammadi, Nuno F. Duarte, Dimitris Dimou, Yiming Wang, Matteo Taiana, Pietro Morerio, Atabak Dehban, Plinio Moreno, Alexandre Bernardino, Alessio Del Bue, Jose Santos-Victor

Abstract: Real-world robotic gras** can be done robustly if a complete 3D Point Cloud Data (PCD) of an object is available. However, in practice, PCDs are often incomplete when objects are viewed from few and sparse viewpoints before the gras** action, leading to the generation of wrong or inaccurate grasp poses. We propose a novel gras** strategy, named 3DSGrasp, that predicts the missing geometry fr… ▽ More Real-world robotic gras** can be done robustly if a complete 3D Point Cloud Data (PCD) of an object is available. However, in practice, PCDs are often incomplete when objects are viewed from few and sparse viewpoints before the gras** action, leading to the generation of wrong or inaccurate grasp poses. We propose a novel gras** strategy, named 3DSGrasp, that predicts the missing geometry from the partial PCD to produce reliable grasp poses. Our proposed PCD completion network is a Transformer-based encoder-decoder network with an Offset-Attention layer. Our network is inherently invariant to the object pose and point's permutation, which generates PCDs that are geometrically consistent and completed properly. Experiments on a wide range of partial PCD show that 3DSGrasp outperforms the best state-of-the-art method on PCD completion tasks and largely improves the gras** success rate in real-world scenarios. The code and dataset will be made available upon acceptance. △ Less

Submitted 2 January, 2023; originally announced January 2023.

Urban Visual Intelligence: Studying Cities with AI and Street-level Imagery

Authors: Fan Zhang, Arianna Salazar Miranda, Fábio Duarte, Lawrence Vale, Gary Hack, Min Chen, Yu Liu, Michael Batty, Carlo Ratti

Abstract: The visual dimension of cities has been a fundamental subject in urban studies, since the pioneering work of scholars such as Sitte, Lynch, Arnheim, and Jacobs. Several decades later, big data and artificial intelligence (AI) are revolutionizing how people move, sense, and interact with cities. This paper reviews the literature on the appearance and function of cities to illustrate how visual info… ▽ More The visual dimension of cities has been a fundamental subject in urban studies, since the pioneering work of scholars such as Sitte, Lynch, Arnheim, and Jacobs. Several decades later, big data and artificial intelligence (AI) are revolutionizing how people move, sense, and interact with cities. This paper reviews the literature on the appearance and function of cities to illustrate how visual information has been used to understand them. A conceptual framework, Urban Visual Intelligence, is introduced to systematically elaborate on how new image data sources and AI techniques are resha** the way researchers perceive and measure cities, enabling the study of the physical environment and its interactions with socioeconomic environments at various scales. The paper argues that these new approaches enable researchers to revisit the classic urban theories and themes, and potentially help cities create environments that are more in line with human behaviors and aspirations in the digital age. △ Less

Submitted 17 June, 2023; v1 submitted 2 January, 2023; originally announced January 2023.

If You Are Careful, So Am I! How Robot Communicative Motions Can Influence Human Approach in a Joint Task

Authors: Linda Lastrico, Nuno Ferreira Duarte, Alessandro Carfì, Francesco Rea, Fulvio Mastrogiovanni, Alessandra Sciutti, José Santos-Victor

Abstract: As humans, we have a remarkable capacity for reading the characteristics of objects only by observing how another person carries them. Indeed, how we perform our actions naturally embeds information on the item features. Collaborative robots can achieve the same ability by modulating the strategy used to transport objects with their end-effector. A contribution in this sense would promote spontane… ▽ More As humans, we have a remarkable capacity for reading the characteristics of objects only by observing how another person carries them. Indeed, how we perform our actions naturally embeds information on the item features. Collaborative robots can achieve the same ability by modulating the strategy used to transport objects with their end-effector. A contribution in this sense would promote spontaneous interactions by making an implicit yet effective communication channel available. This work investigates if humans correctly perceive the implicit information shared by a robotic manipulator through its movements during a dyadic collaboration task. Exploiting a generative approach, we designed robot actions to convey virtual properties of the transported objects, particularly to inform the partner if any caution is required to handle the carried item. We found that carefulness is correctly interpreted when observed through the robot movements. In the experiment, we used identical empty plastic cups; nevertheless, participants approached them differently depending on the attitude shown by the robot: humans change how they reach for the object, being more careful whenever the robot does the same. This emerging form of motor contagion is entirely spontaneous and happens even if the task does not require it. △ Less

Submitted 24 October, 2022; originally announced October 2022.

Comments: Accepted to ICSR 2022, 14th International Conference on Social Robotics, preprint version

Survey of Deep Learning for Autonomous Surface Vehicles in the Marine Environment

Authors: Yuanyuan Qiao, Jiaxin Yin, Wei Wang, Fábio Duarte, Jie Yang, Carlo Ratti

Abstract: Within the next several years, there will be a high level of autonomous technology that will be available for widespread use, which will reduce labor costs, increase safety, save energy, enable difficult unmanned tasks in harsh environments, and eliminate human error. Compared to software development for other autonomous vehicles, maritime software development, especially on aging but still functi… ▽ More Within the next several years, there will be a high level of autonomous technology that will be available for widespread use, which will reduce labor costs, increase safety, save energy, enable difficult unmanned tasks in harsh environments, and eliminate human error. Compared to software development for other autonomous vehicles, maritime software development, especially on aging but still functional fleets, is described as being in a very early and emerging phase. This introduces very large challenges and opportunities for researchers and engineers to develop maritime autonomous systems. Recent progress in sensor and communication technology has introduced the use of autonomous surface vehicles (ASVs) in applications such as coastline surveillance, oceanographic observation, multi-vehicle cooperation, and search and rescue missions. Advanced artificial intelligence technology, especially deep learning (DL) methods that conduct nonlinear map** with self-learning representations, has brought the concept of full autonomy one step closer to reality. This paper surveys the existing work regarding the implementation of DL methods in ASV-related fields. First, the scope of this work is described after reviewing surveys on ASV developments and technologies, which draws attention to the research gap between DL and maritime operations. Then, DL-based navigation, guidance, control (NGC) systems and cooperative operations, are presented. Finally, this survey is completed by highlighting the current challenges and future research directions. △ Less

Submitted 10 January, 2023; v1 submitted 16 October, 2022; originally announced October 2022.

Robotic Learning the Sequence of Packing Irregular Objects from Human Demonstrations

Authors: André Santos, Nuno Ferreira Duarte, Atabak Dehban, José Santos-Victor

Abstract: We tackle the challenge of robotic bin packing with irregular objects, such as groceries. Given the diverse physical attributes of these objects and the complex constraints governing their placement and manipulation, employing preprogrammed strategies becomes unfeasible. Our approach is to learn directly from expert demonstrations in order to extract implicit task knowledge and strategies to ensur… ▽ More We tackle the challenge of robotic bin packing with irregular objects, such as groceries. Given the diverse physical attributes of these objects and the complex constraints governing their placement and manipulation, employing preprogrammed strategies becomes unfeasible. Our approach is to learn directly from expert demonstrations in order to extract implicit task knowledge and strategies to ensure safe object positioning, efficient use of space, and the generation of human-like behaviors that enhance human-robot trust. We rely on human demonstrations to learn a Markov chain for predicting the object packing sequence for a given set of items and then compare it with human performance. Our experimental results show that the model outperforms human performance by generating sequence predictions that humans classify as human-like more frequently than human-generated sequences. The human demonstrations were collected using our proposed VR platform, BoxED, which is a box packaging environment for simulating real-world objects and scenarios for fast and streamlined data collection with the purpose of teaching robots. We collected data from 43 participants packing a total of 263 boxes with supermarket-like objects, yielding 4644 object manipulations. Our VR platform can be easily adapted to new scenarios and objects, and is publicly available, alongside our dataset, at https://github.com/andrejfsantos4/BoxED. △ Less

Submitted 8 November, 2023; v1 submitted 4 October, 2022; originally announced October 2022.

Comments: 7 pages, 7 figures

Wanderlust: 3D Impressionism in Human Journeys

Authors: Guangyu Du, Lei Dong, Fabio Duarte, Carlo Ratti

Abstract: The movements of individuals are fundamental to building and maintaining social connections. This pictorial presents Wanderlust, an experimental three-dimensional data visualization on the universal visitation pattern revealed from large-scale mobile phone tracking data. It explores ways of visualizing recurrent flows and the attractive places they implied. Inspired by the 19th-century art movemen… ▽ More The movements of individuals are fundamental to building and maintaining social connections. This pictorial presents Wanderlust, an experimental three-dimensional data visualization on the universal visitation pattern revealed from large-scale mobile phone tracking data. It explores ways of visualizing recurrent flows and the attractive places they implied. Inspired by the 19th-century art movement Impressionism, we develop a multi-layered effect, an impression, of mountains emerging from consolidated flows, to capture the essence of human journeys and urban spatial structure. △ Less

Submitted 28 November, 2021; v1 submitted 31 August, 2021; originally announced September 2021.

Comments: IEEE VIS Arts Program (VISAP) 2021, pictorial, 14 pages

Favelas 4D: Scalable methods for morphology analysis of informal settlements using terrestrial laser scanning data

Authors: Arianna Salazar Miranda, Guangyu Du, Claire Gorman, Fabio Duarte, Washington Fajardo, Carlo Ratti

Abstract: One billion people live in informal settlements worldwide. The complex and multilayered spaces that characterize this unplanned form of urbanization pose a challenge to traditional approaches to map** and morphological analysis. This study proposes a methodology to study the morphological properties of informal settlements based on terrestrial LiDAR (Light Detection and Ranging) data collected i… ▽ More One billion people live in informal settlements worldwide. The complex and multilayered spaces that characterize this unplanned form of urbanization pose a challenge to traditional approaches to map** and morphological analysis. This study proposes a methodology to study the morphological properties of informal settlements based on terrestrial LiDAR (Light Detection and Ranging) data collected in Rocinha, the largest favela in Rio de Janeiro, Brazil. Our analysis operates at two resolutions, including a \emph{global} analysis focused on comparing different streets of the favela to one another, and a \emph{local} analysis unpacking the variation of morphological metrics within streets. We show that our methodology reveals meaningful differences and commonalities both in terms of the global morphological characteristics across streets and their local distributions. Finally, we create morphological maps at high spatial resolution from LiDAR data, which can inform urban planning assessments of concerns related to crowding, structural safety, air quality, and accessibility in the favela. The methods for this study are automated and can be easily scaled to analyze entire informal settlements, leveraging the increasing availability of inexpensive LiDAR scanners on portable devices such as cellphones. △ Less

Submitted 23 April, 2021; originally announced May 2021.

Modeling the geospatial evolution of COVID-19 using spatio-temporal convolutional sequence-to-sequence neural networks

Authors: Mário Cardoso, André Cavalheiro, Alexandre Borges, Ana F. Duarte, Amílcar Soares, Maria João Pereira, Nuno J. Nunes, Leonardo Azevedo, Arlindo L. Oliveira

Abstract: Europe was hit hard by the COVID-19 pandemic and Portugal was one of the most affected countries, having suffered three waves in the first twelve months. Approximately between Jan 19th and Feb 5th 2021 Portugal was the country in the world with the largest incidence rate, with 14-days incidence rates per 100,000 inhabitants in excess of 1000. Despite its importance, accurate prediction of the geos… ▽ More Europe was hit hard by the COVID-19 pandemic and Portugal was one of the most affected countries, having suffered three waves in the first twelve months. Approximately between Jan 19th and Feb 5th 2021 Portugal was the country in the world with the largest incidence rate, with 14-days incidence rates per 100,000 inhabitants in excess of 1000. Despite its importance, accurate prediction of the geospatial evolution of COVID-19 remains a challenge, since existing analytical methods fail to capture the complex dynamics that result from both the contagion within a region and the spreading of the infection from infected neighboring regions. We use a previously developed methodology and official municipality level data from the Portuguese Directorate-General for Health (DGS), relative to the first twelve months of the pandemic, to compute an estimate of the incidence rate in each location of mainland Portugal. The resulting sequence of incidence rate maps was then used as a gold standard to test the effectiveness of different approaches in the prediction of the spatial-temporal evolution of the incidence rate. Four different methods were tested: a simple cell level autoregressive moving average (ARMA) model, a cell level vector autoregressive (VAR) model, a municipality-by-municipality compartmental SIRD model followed by direct block sequential simulation and a convolutional sequence-to-sequence neural network model based on the STConvS2S architecture. We conclude that the convolutional sequence-to-sequence neural network is the best performing method, when predicting the medium-term future incidence rate, using the available information. △ Less

Submitted 6 May, 2021; originally announced May 2021.

Comments: 10 pages, 8 figures

MSC Class: 92-10 ACM Class: I.2.6

Robust Place Recognition using an Imaging Lidar

Authors: Tixiao Shan, Brendan Englot, Fabio Duarte, Carlo Ratti, Daniela Rus

Abstract: We propose a methodology for robust, real-time place recognition using an imaging lidar, which yields image-quality high-resolution 3D point clouds. Utilizing the intensity readings of an imaging lidar, we project the point cloud and obtain an intensity image. ORB feature descriptors are extracted from the image and encoded into a bag-of-words vector. The vector, used to identify the point cloud,… ▽ More We propose a methodology for robust, real-time place recognition using an imaging lidar, which yields image-quality high-resolution 3D point clouds. Utilizing the intensity readings of an imaging lidar, we project the point cloud and obtain an intensity image. ORB feature descriptors are extracted from the image and encoded into a bag-of-words vector. The vector, used to identify the point cloud, is inserted into a database that is maintained by DBoW for fast place recognition queries. The returned candidate is further validated by matching visual feature descriptors. To reject matching outliers, we apply PnP, which minimizes the reprojection error of visual features' positions in Euclidean space with their correspondences in 2D image space, using RANSAC. Combining the advantages from both camera and lidar-based place recognition approaches, our method is truly rotation-invariant, and can tackle reverse revisiting and upside down revisiting. The proposed method is evaluated on datasets gathered from a variety of platforms over different scales and environments. Our implementation and datasets are available at https://git.io/image-lidar △ Less

Submitted 21 April, 2021; v1 submitted 2 March, 2021; originally announced March 2021.

Comments: ICRA 2021

Leveraging Artificial Intelligence to Analyze Citizens' Opinions on Urban Green Space

Authors: Mohammadhossein Ghahramani, Nadina J. Galle, Fabio Duarte, Carlo Ratti, Francesco Pilla

Abstract: Continued population growth and urbanization is shifting research to consider the quality of urban green space over the quantity of these parks, woods, and wetlands. The quality of urban green space has been hitherto measured by expert assessments, including in-situ observations, surveys, and remote sensing analyses. Location data platforms, such as TripAdvisor, can provide people's opinion on man… ▽ More Continued population growth and urbanization is shifting research to consider the quality of urban green space over the quantity of these parks, woods, and wetlands. The quality of urban green space has been hitherto measured by expert assessments, including in-situ observations, surveys, and remote sensing analyses. Location data platforms, such as TripAdvisor, can provide people's opinion on many destinations and experiences, including UGS. This paper leverages Artificial Intelligence techniques for opinion mining and text classification using such platform's reviews as a novel approach to urban green space quality assessments. Natural Language Processing is used to analyze contextual information given supervised scores of words by implementing computational analysis. Such an application can support local authorities and stakeholders in their understanding of and justification for future investments in urban green space. △ Less

Submitted 12 February, 2021; originally announced February 2021.

Cluster-Head-Driven UAV Relaying with Recursive Maximum Minimum Distance using CRANs

Authors: F. L. Duarte, R. C. de Lamare

Abstract: In this letter, a C-RAN-type cluster-head-driven uplink model for multiple-antenna Unmanned Aerial Vehicles (UAV) relaying schemes, which enables joint Maximum Likelihood (ML) symbol detection in the UAV cluster-head and the selection of UAV sources to communicate with each other aided by UAV-based relays, {is presented. In this context,} a relay selection technique, named Cluster-Head-Driven Best… ▽ More In this letter, a C-RAN-type cluster-head-driven uplink model for multiple-antenna Unmanned Aerial Vehicles (UAV) relaying schemes, which enables joint Maximum Likelihood (ML) symbol detection in the UAV cluster-head and the selection of UAV sources to communicate with each other aided by UAV-based relays, {is presented. In this context,} a relay selection technique, named Cluster-Head-Driven Best-Link (CHD-Best-Link), that employs cluster-head buffers and physical-layer network coding, {is devised}. Then, a recursive maximum minimum distance relay selection strategy that exploits time-correlated channels and equips the CHD-Best-Link scheme is developed. Simulations illustrate that CHD-Best-Link has superior average delay and bit error rate performances to that of previous schemes. △ Less

Submitted 20 July, 2020; originally announced July 2020.

Comments: 8 pages, 4 figures

Study of Cloud-Aided Multi-Way Multiple-Antenna Relaying with Best-User Link Selection and Joint ML Detection

Authors: F. L. Duarte, R. C. de Lamare

Abstract: In this work, we present a cloud-aided uplink framework for multi-way multiple-antenna relay systems which facilitates joint linear Maximum Likelihood (ML) symbol detection in the cloud and where users are selected to simultaneously transmit to each other aided by relays. We also investigate relay selection techniques for the proposed cloud-aided uplink framework that uses cloud-based buffers and… ▽ More In this work, we present a cloud-aided uplink framework for multi-way multiple-antenna relay systems which facilitates joint linear Maximum Likelihood (ML) symbol detection in the cloud and where users are selected to simultaneously transmit to each other aided by relays. We also investigate relay selection techniques for the proposed cloud-aided uplink framework that uses cloud-based buffers and physical-layer network coding. In particular, we develop a novel multi-way relay selection protocol based on the selection of the best link, denoted as Multi-Way Cloud-Aided Best-User-Link (MWC-Best-User-Link). We then devise the maximum minimum distance relay selection criterion along with the algorithm that is incorporated into the proposed MWC-Best-User-Link protocol. Simulations show that MWC-Best-User-Link outperforms previous works in terms of average delay, sum-rate and bit error rate. △ Less

Submitted 2 February, 2020; originally announced February 2020.

Comments: 7 pages, 3 figures

Study of Switched Max-Link Relay Selection for Cooperative Multiple-Antenna Systems

Authors: F. L. Duarte, R. C. de Lamare

Abstract: In this work, we present a switched relaying framework for multiple-input multiple-output (MIMO) relay systems where a source node may transmit directly to a destination node or aided by relays. We also investigate relay selection techniques for the proposed switched relaying framework, whose relays are equipped with buffers. In particular, we develop a novel relay selection protocol based on swit… ▽ More In this work, we present a switched relaying framework for multiple-input multiple-output (MIMO) relay systems where a source node may transmit directly to a destination node or aided by relays. We also investigate relay selection techniques for the proposed switched relaying framework, whose relays are equipped with buffers. In particular, we develop a novel relay selection protocol based on switching and the selection of the best link, denoted as Switched Max-Link. We then propose the Maximum Minimum Distance (MMD) relay selection criterion for MIMO systems, which is based on the optimal Maximum Likelihood (ML) principle and can provide significant performance gains over other criteria, along with algorithms that are incorporated into the proposed Switched Max-Link protocol. An analysis of the proposed Switched Max-Link protocol and the MMD relay selection criterion in terms of computational cost, pairwise error probability, sum-rate and average delay is carried out. Simulations show that Switched Max-Link using the MMD criterion outperforms previous works in terms of sum-rate, pairwise error probability, average delay and bit error rate. △ Less

Submitted 26 December, 2019; originally announced December 2019.

Comments: 15 pages, 13 figures

Study of Max-Link Relay Selection with Buffers for Multi-Way Cooperative Multi-Antenna Systems

Authors: F. L. Duarte, R. C. de Lamare

Abstract: In this paper, we present a relay-selection strategy for multi-way cooperative multi-antenna systems that are aided by a central processor node, where a cluster formed by two users is selected to simultaneously transmit to each other with the help of relays. In particular, we present a novel multi-way relay selection strategy based on the selection of the best link, exploiting the use of buffers a… ▽ More In this paper, we present a relay-selection strategy for multi-way cooperative multi-antenna systems that are aided by a central processor node, where a cluster formed by two users is selected to simultaneously transmit to each other with the help of relays. In particular, we present a novel multi-way relay selection strategy based on the selection of the best link, exploiting the use of buffers and physical-layer network coding, that is called Multi-Way Buffer-Aided Max-Link (MW-Max-Link). We compare the proposed MW-Max-Link to existing techniques in terms of bit error rate, pairwise error probability, sum rate and computational complexity. Simulations are then employed to evaluate the performance of the proposed and existing techniques. △ Less

Submitted 15 July, 2019; originally announced July 2019.

Comments: 5 pages, 3 figures

Explainable Machine Learning for Scientific Insights and Discoveries

Authors: Ribana Roscher, Bastian Bohn, Marco F. Duarte, Jochen Garcke

Abstract: Machine learning methods have been remarkably successful for a wide range of application areas in the extraction of essential information from data. An exciting and relatively recent development is the uptake of machine learning in the natural sciences, where the major goal is to obtain novel scientific insights and discoveries from observational or simulated data. A prerequisite for obtaining a s… ▽ More Machine learning methods have been remarkably successful for a wide range of application areas in the extraction of essential information from data. An exciting and relatively recent development is the uptake of machine learning in the natural sciences, where the major goal is to obtain novel scientific insights and discoveries from observational or simulated data. A prerequisite for obtaining a scientific outcome is domain knowledge, which is needed to gain explainability, but also to enhance scientific consistency. In this article we review explainable machine learning in view of applications in the natural sciences and discuss three core elements which we identified as relevant in this context: transparency, interpretability, and explainability. With respect to these core elements, we provide a survey of recent scientific works that incorporate machine learning and the way that explainable machine learning is used in combination with domain knowledge from the application areas. △ Less

Submitted 12 January, 2020; v1 submitted 21 May, 2019; originally announced May 2019.

Journal ref: IEEE Access, vol. 8, pp. 42200-42216, 2020

Learning Motor Resonance in Human-Human and Human-Robot Interaction with Coupled Dynamical System

Authors: Nuno Ferreira Duarte, Mirko Raković, José Santos-Victor

Abstract: Human interaction involves very sophisticated non-verbal communication skills like understanding the goals and actions of others and coordinating our own actions accordingly. Neuroscience refers to this mechanism as motor resonance, in the sense that the perception of another person's actions and sensory experiences activates the observer's brain as if (s)he would be performing the same actions an… ▽ More Human interaction involves very sophisticated non-verbal communication skills like understanding the goals and actions of others and coordinating our own actions accordingly. Neuroscience refers to this mechanism as motor resonance, in the sense that the perception of another person's actions and sensory experiences activates the observer's brain as if (s)he would be performing the same actions and having the same experiences. We analyze and model non-verbal cues (arm movements) exchanged between two humans that interact and execute handover actions. The contributions of this paper are the following: (i) computational models, using recorded motion data, describing the motor behaviour of each actor in action-in-interaction situations, (ii) a computational model that captures the behaviour if the "giver" and "receiver" during an object handover action, by coupling the arm motion of both actors, and (iii) embedded these models in the iCub robot for both action execution and recognition. Our results show that: (i) the robot can interpret the human arm motion and recognize handover actions; and (ii) behave in a "human-like" manner to receive the object of the recognized handover action. △ Less

Submitted 10 May, 2019; originally announced May 2019.

Comments: 7 pages, 6 figures

Switched Max-Link Relay Selection Algorithms for Buffer-Aided Relay Systems

Authors: F. L. Duarte, R. C. de Lamare

Abstract: In this paper, we investigate relay selection for cooperative multiple-antenna systems that are equipped with buffers, which increase the reliability of wireless links. In particular, we present a novel relay selection technique based on switching and the selection of the best link, that is named Switched Max-Link. We also introduce a novel relay selection criterion based on the Maximum Likelihood… ▽ More In this paper, we investigate relay selection for cooperative multiple-antenna systems that are equipped with buffers, which increase the reliability of wireless links. In particular, we present a novel relay selection technique based on switching and the selection of the best link, that is named Switched Max-Link. We also introduce a novel relay selection criterion based on the Maximum Likelihood (ML) principle and the Pairwise Error Probability (PEP) denoted Maximum Minimum Distance (MMD) that is incorporated into the proposed Switched Max-Link protocol. We compare the proposed MMD to the existing Quadratic Norm (QN), in terms of PEP and computational complexity. Simulations are then employed to evaluate the performance of the proposed and existing techniques. △ Less

Submitted 22 March, 2019; originally announced March 2019.

Comments: 8 pages, 6 figures. arXiv admin note: substantial text overlap with arXiv:1807.03642

Few-Shot Learning-Based Human Activity Recognition

Authors: Siwei Feng, Marco F. Duarte

Abstract: Few-shot learning is a technique to learn a model with a very small amount of labeled training data by transferring knowledge from relevant tasks. In this paper, we propose a few-shot learning method for wearable sensor based human activity recognition, a technique that seeks high-level human activity knowledge from low-level sensor inputs. Due to the high costs to obtain human generated activity… ▽ More Few-shot learning is a technique to learn a model with a very small amount of labeled training data by transferring knowledge from relevant tasks. In this paper, we propose a few-shot learning method for wearable sensor based human activity recognition, a technique that seeks high-level human activity knowledge from low-level sensor inputs. Due to the high costs to obtain human generated activity data and the ubiquitous similarities between activity modes, it can be more efficient to borrow information from existing activity recognition models than to collect more data to train a new model from scratch when only a few data are available for model training. The proposed few-shot human activity recognition method leverages a deep learning model for feature extraction and classification while knowledge transfer is performed in the manner of model parameter transfer. In order to alleviate negative transfer, we propose a metric to measure cross-domain class-wise relevance so that knowledge of higher relevance is assigned larger weights during knowledge transfer. Promising results in extensive experiments show the advantages of the proposed approach. △ Less

Submitted 25 March, 2019; originally announced March 2019.

Deep Learning Based Video System for Accurate and Real-Time Parking Measurement

Authors: Bill Yang Cai, Ricardo Alvarez, Michelle Sit, Fábio Duarte, Carlo Ratti

Abstract: Parking spaces are costly to build, parking payments are difficult to enforce, and drivers waste an excessive amount of time searching for empty lots. Accurate quantification would inform developers and municipalities in space allocation and design, while real-time measurements would provide drivers and parking enforcement with information that saves time and resources. In this paper, we propose a… ▽ More Parking spaces are costly to build, parking payments are difficult to enforce, and drivers waste an excessive amount of time searching for empty lots. Accurate quantification would inform developers and municipalities in space allocation and design, while real-time measurements would provide drivers and parking enforcement with information that saves time and resources. In this paper, we propose an accurate and real-time video system for future Internet of Things (IoT) and smart cities applications. Using recent developments in deep convolutional neural networks (DCNNs) and a novel vehicle tracking filter, we combine information across multiple image frames in a video sequence to remove noise introduced by occlusions and detection failures. We demonstrate that our system achieves higher accuracy than pure image-based instance segmentation, and is comparable in performance to industry benchmark systems that utilize more expensive sensors such as radar. Furthermore, our system shows significant potential in its scalability to a city-wide scale and also in the richness of its output that goes beyond traditional binary occupancy statistics. △ Less

Submitted 19 February, 2019; originally announced February 2019.

Comments: Accepted for publication in IEEE Internet of Things Journal, Special Issue on Enabling a Smart City: IoT Meets AI

Quantifying Legibility of Indoor Spaces Using Deep Convolutional Neural Networks: Case Studies in Train Stations

Authors: Zhoutong Wang, Qianhui Liang, Fabio Duarte, Fan Zhang, Louis Charron, Lenna Johnsen, Bill Cai, Carlo Ratti

Abstract: Legibility is the extent to which a space can be easily recognized. Evaluating legibility is particularly desirable in indoor spaces, since it has a large impact on human behavior and the efficiency of space utilization. However, indoor space legibility has only been studied through survey and trivial simulations and lacks reliable quantitative measurement. We utilized a Deep Convolutional Neural… ▽ More Legibility is the extent to which a space can be easily recognized. Evaluating legibility is particularly desirable in indoor spaces, since it has a large impact on human behavior and the efficiency of space utilization. However, indoor space legibility has only been studied through survey and trivial simulations and lacks reliable quantitative measurement. We utilized a Deep Convolutional Neural Network (DCNN), which is structurally similar to a human perception system, to model legibility in indoor spaces. To implement the modeling of legibility for any indoor spaces, we designed an end-to-end processing pipeline from indoor data retrieving to model training to spatial legibility analysis. Although the model performed very well (98% top-1 accuracy) overall, there are still discrepancies in accuracy among different spaces, reflecting legibility differences. To prove the validity of the pipeline, we deployed a survey on Amazon Mechanical Turk, collecting 4,015 samples. The human samples showed a similar behavior pattern and mechanism as the DCNN models. Further, we used model results to visually explain legibility in different architectural programs, building age, building style, visual clusterings of spaces and visual explanations for building age and architectural functions. △ Less

Submitted 22 January, 2019; originally announced January 2019.

Comments: 20 pages, 19 figures, 7 tables

Binary Sequence Set Design for Interferer Rejection in Multi-Branch Modulation

Authors: Dian Mo, Marco F. Duarte

Abstract: Wideband communication is often expected to deal with a very wide spectrum, which in many environments of interest includes strong interferers. Thus receivers for the wideband communication systems often need to mitigate interferers to reduce the distortion caused by the amplifier nonlinearity and noise. Recently, a new architecture for communication receivers known as random modulation mixes a si… ▽ More Wideband communication is often expected to deal with a very wide spectrum, which in many environments of interest includes strong interferers. Thus receivers for the wideband communication systems often need to mitigate interferers to reduce the distortion caused by the amplifier nonlinearity and noise. Recently, a new architecture for communication receivers known as random modulation mixes a signal with different pseudorandom sequences using multiple branches of channels before sampling. While random modulation is used in these receivers to acquire the signal at low sampling rates, the modulation sequences used lack the ability to suppress interferers due to their flat spectra. In previous work, we introduced the design of a single spectrally shaped binary sequence that mitigates interferers to replace the pseudorandom sequence in a channel. However, the designed sequences cannot provide the stable recovery achieved by pseudorandom sequence approaches. In this paper, we extend our previous sequence design to guarantee stable recovery by designing a set of sequences to be orthogonal to each other. We show that it is difficult to find the necessary number of sequences featuring mutual orthogonality and introduce oversampling to the sequence set design to improve the recovery performance. We propose an algorithm for multi-branch sequence design as a binary optimization problem, which is solved using a semidefinite program relaxation and randomized projection. While it is common to model narrowband interferers as a subspace spanned by a subset of elements from the Fourier basis, we show that the Slepian basis provides an alternative and more suitable compact representation for signals with components contained in narrow spectrum bands. Numerical experiments using the proposed sequence sets show their advantages against pseudorandom sequences and our previous work. △ Less

Submitted 3 June, 2020; v1 submitted 29 November, 2018; originally announced November 2018.

Comments: 11 pages, 6 figures. To appear in IEEE Transactions on Signal Processing

Design of Spectrally Shaped Binary Sequences via Randomized Convex Relaxation

Authors: Dian Mo, Marco F. Duarte

Abstract: Wideband communication receivers often deal with the problems of detecting weak signals from distant sources received together with strong nearby interferers. When the techniques of random modulation are used in communication system receivers, one can design a spectrally shaped sequence that mitigates interferer bands while preserving message bands. Common implementation constraints require sequen… ▽ More Wideband communication receivers often deal with the problems of detecting weak signals from distant sources received together with strong nearby interferers. When the techniques of random modulation are used in communication system receivers, one can design a spectrally shaped sequence that mitigates interferer bands while preserving message bands. Common implementation constraints require sequence quantization, which turns the design problem formulation to an integer optimization problem solved using a semidefinite program on a matrix that is restricted to have rank one. Common approximation schemes for this problem are not amenable due to the distortion to the spectrum caused by the required quantization. We propose a method that leverages a randomized projection and quantization of the solution of a semidefinite program, an approach that has been previously used for related integer programs. We provide a theoretical and numerical analysis on the feasibility and quality of the approximation provided by the proposed approach. Furthermore, numerical simulations show that our proposed approach returns the same sequence as an exhaustive search (when feasible), showcasing its accuracy and efficiency. Furthermore, our proposed method succeeds in finding suitable spectrally shaped sequences for cases where exhaustive search is not feasible, achieving better performance than existing alternatives. △ Less

Submitted 14 November, 2018; originally announced November 2018.

Comments: 27 Pages, 7 figures

Autoencoder Based Sample Selection for Self-Taught Learning

Authors: Siwei Feng, Han Yu, Marco F. Duarte

Abstract: Self-taught learning is a technique that uses a large number of unlabeled data as source samples to improve the task performance on target samples. Compared with other transfer learning techniques, self-taught learning can be applied to a broader set of scenarios due to the loose restrictions on the source data. However, knowledge transferred from source samples that are not sufficiently related t… ▽ More Self-taught learning is a technique that uses a large number of unlabeled data as source samples to improve the task performance on target samples. Compared with other transfer learning techniques, self-taught learning can be applied to a broader set of scenarios due to the loose restrictions on the source data. However, knowledge transferred from source samples that are not sufficiently related to the target domain may negatively influence the target learner, which is referred to as negative transfer. In this paper, we propose a metric for the relevance between a source sample and the target samples. To be more specific, both source and target samples are reconstructed through a single-layer autoencoder with a linear relationship between source samples and reconstructed target samples being simultaneously enforced. An $\ell_{2,1}$-norm sparsity constraint is imposed on the transformation matrix to identify source samples relevant to the target domain. Source domain samples that are deemed relevant are assigned pseudo-labels reflecting their relevance to target domain samples, and are combined with target samples in order to provide an expanded training set for classifier training. Local data structures are also preserved during source sample selection through spectral graph analysis. Promising results in extensive experiments show the advantages of the proposed approach. △ Less

Submitted 1 December, 2019; v1 submitted 5 August, 2018; originally announced August 2018.

Comments: 38 pages, 4 figures, to appear in Elsevier Knowledge-Based Systems

Study of Switched Max-Link Buffer-Aided Relay Selection for Cooperative MIMO Systems

Authors: F. L. Duarte, R. C. de Lamare

Abstract: In this paper, we investigate relay selection for cooperative multiple-antenna systems that are equipped with buffers, which increase the reliability of wireless links. In particular, we present a novel relay selection technique based on switching and the Max-Link protocol that is named Switched Max-Link. We also introduce a novel relay selection criterion based on the maximum likelihood (ML) prin… ▽ More In this paper, we investigate relay selection for cooperative multiple-antenna systems that are equipped with buffers, which increase the reliability of wireless links. In particular, we present a novel relay selection technique based on switching and the Max-Link protocol that is named Switched Max-Link. We also introduce a novel relay selection criterion based on the maximum likelihood (ML) principle denoted maximum minimum distance that is incorporated into. Simulations are then employed to evaluate the performance of the proposed and existing techniques. △ Less

Submitted 16 July, 2018; v1 submitted 8 July, 2018; originally announced July 2018.

Comments: 8 pages, 3 figures. arXiv admin note: text overlap with arXiv:1707.00953

Action Anticipation: Reading the Intentions of Humans and Robots

Authors: Nuno Ferreira Duarte, Jovica Tasevski, Moreno Coco, Mirko Raković, Aude Billard, José Santos-Victor

Abstract: Humans have the fascinating capacity of processing non-verbal visual cues to understand and anticipate the actions of other humans. This "intention reading" ability is underpinned by shared motor-repertoires and action-models, which we use to interpret the intentions of others as if they were our own. We investigate how the different cues contribute to the legibility of human actions during inte… ▽ More Humans have the fascinating capacity of processing non-verbal visual cues to understand and anticipate the actions of other humans. This "intention reading" ability is underpinned by shared motor-repertoires and action-models, which we use to interpret the intentions of others as if they were our own. We investigate how the different cues contribute to the legibility of human actions during interpersonal interactions. Our first contribution is a publicly available dataset with recordings of human body-motion and eye-gaze, acquired in an experimental scenario with an actor interacting with three subjects. From these data, we conducted a human study to analyse the importance of the different non-verbal cues for action perception. As our second contribution, we used the motion/gaze recordings to build a computational model describing the interaction between two persons. As a third contribution, we embedded this model in the controller of an iCub humanoid robot and conducted a second human study, in the same scenario with the robot as an actor, to validate the model's "intention reading" capability. Our results show that it is possible to model (non-verbal) signals exchanged by humans during interaction, and how to incorporate such a mechanism in robotic systems with the twin goal of : (i) being able to "read" human action intentions, and (ii) acting in a way that is legible by humans. △ Less

Submitted 10 August, 2018; v1 submitted 8 February, 2018; originally announced February 2018.

Comments: 8 pages, 7 Figures, IEEE Robotics and Automation Letters 2018

Graph Autoencoder-Based Unsupervised Feature Selection with Broad and Local Data Structure Preservation

Authors: Siwei Feng, Marco F. Duarte

Abstract: Feature selection is a dimensionality reduction technique that selects a subset of representative features from high dimensional data by eliminating irrelevant and redundant features. Recently, feature selection combined with sparse learning has attracted significant attention due to its outstanding performance compared with traditional feature selection methods that ignores correlation between fe… ▽ More Feature selection is a dimensionality reduction technique that selects a subset of representative features from high dimensional data by eliminating irrelevant and redundant features. Recently, feature selection combined with sparse learning has attracted significant attention due to its outstanding performance compared with traditional feature selection methods that ignores correlation between features. These works first map data onto a low-dimensional subspace and then select features by posing a sparsity constraint on the transformation matrix. However, they are restricted by design to linear data transformation, a potential drawback given that the underlying correlation structures of data are often non-linear. To leverage a more sophisticated embedding, we propose an autoencoder-based unsupervised feature selection approach that leverages a single-layer autoencoder for a joint framework of feature selection and manifold learning. More specifically, we enforce column sparsity on the weight matrix connecting the input layer and the hidden layer, as in previous work. Additionally, we include spectral graph analysis on the projected data into the learning process to achieve local data geometry preservation from the original data space to the low-dimensional feature space. Extensive experiments are conducted on image, audio, text, and biological data. The promising experimental results validate the superiority of the proposed method. △ Less

Submitted 20 April, 2018; v1 submitted 7 January, 2018; originally announced January 2018.

Semi-Supervised Endmember Identification In Nonlinear Spectral Mixtures Via Semantic Representation

Authors: Yuki Itoh, Siwei Feng, Marco F. Duarte, Mario Parente

Abstract: This paper proposes a new hyperspectral unmixing method for nonlinearly mixed hyperspectral data using a semantic representation in a semi-supervised fashion, assuming the availability of a spectral reference library. Existing semi-supervised unmixing algorithms select members from an endmember library that are present at each of the pixels; most such methods assume a linear mixing model. However,… ▽ More This paper proposes a new hyperspectral unmixing method for nonlinearly mixed hyperspectral data using a semantic representation in a semi-supervised fashion, assuming the availability of a spectral reference library. Existing semi-supervised unmixing algorithms select members from an endmember library that are present at each of the pixels; most such methods assume a linear mixing model. However, those methods will fail in the presence of nonlinear mixing among the observed spectra. To address this issue, we develop an endmember selection method using a recently proposed semantic spectral representation obtained via non-homogeneous hidden Markov chain (NHMC) model for a wavelet transform of the spectra. The semantic representation can encode spectrally discriminative features for any observed spectrum and, therefore, our proposed method can perform endmember selection without any assumption on the mixing model. Experimental results show that in the presence of sufficiently nonlinear mixing our proposed method outperforms dictionary-based sparse unmixing approaches based on linear models. △ Less

Submitted 3 January, 2017; originally announced January 2017.

Comments: 15 pages, 11 figures, 4 tables

Indoor Space Recognition using Deep Convolutional Neural Network: A Case Study at MIT Campus

Authors: Fan Zhang, Fabio Duarte, Ruixian Ma, Dimitrios Milioris, Hui Lin, Carlo Ratti

Abstract: In this paper, we propose a robust and parsimonious approach using Deep Convolutional Neural Network (DCNN) to recognize and interpret interior space. DCNN has achieved incredible success in object and scene recognition. In this study we design and train a DCNN to classify a pre-zoning indoor space, and from a single phone photo to recognize the learned space features, with no need of additional a… ▽ More In this paper, we propose a robust and parsimonious approach using Deep Convolutional Neural Network (DCNN) to recognize and interpret interior space. DCNN has achieved incredible success in object and scene recognition. In this study we design and train a DCNN to classify a pre-zoning indoor space, and from a single phone photo to recognize the learned space features, with no need of additional assistive technology. We collect more than 600,000 images inside MIT campus buildings to train our DCNN model, and achieved 97.9% accuracy in validation dataset and 81.7% accuracy in test dataset based on spatial-scale fixed model. Furthermore, the recognition accuracy and spatial resolution can be potentially improved through multiscale classification model. We identify the discriminative image regions through Class Activating Map** (CAM) technique, to observe the model's behavior in how to recognize space and interpret it in an abstract way. By evaluating the results with misclassification matrix, we investigate the visual spatial feature of interior space by looking into its visual similarity and visual distinctiveness, giving insights into interior design and human indoor perception and wayfinding research. The contribution of this paper is threefold. First, we propose a robust and parsimonious approach for indoor navigation using DCNN. Second, we demonstrate that DCNN also has a potential capability in space feature learning and recognition, even under severe appearance changes. Third, we introduce a DCNN based approach to look into the visual similarity and visual distinctiveness of interior space. △ Less

Submitted 7 October, 2016; originally announced October 2016.

Comments: 22 pages; 14 figures

Out-of-Sample Extension for Dimensionality Reduction of Noisy Time Series

Authors: Hamid Dadkhahi, Marco F. Duarte, Benjamin Marlin

Abstract: This paper proposes an out-of-sample extension framework for a global manifold learning algorithm (Isomap) that uses temporal information in out-of-sample points in order to make the embedding more robust to noise and artifacts. Given a set of noise-free training data and its embedding, the proposed framework extends the embedding for a noisy time series. This is achieved by adding a spatio-tempor… ▽ More This paper proposes an out-of-sample extension framework for a global manifold learning algorithm (Isomap) that uses temporal information in out-of-sample points in order to make the embedding more robust to noise and artifacts. Given a set of noise-free training data and its embedding, the proposed framework extends the embedding for a noisy time series. This is achieved by adding a spatio-temporal compactness term to the optimization objective of the embedding. To the best of our knowledge, this is the first method for out-of-sample extension of manifold embeddings that leverages timing information available for the extension set. Experimental results demonstrate that our out-of-sample extension algorithm renders a more robust and accurate embedding of sequentially ordered image data in the presence of various noise and artifacts when compared to other timing-aware embeddings. Additionally, we show that an out-of-sample extension framework based on the proposed algorithm outperforms the state of the art in eye-gaze estimation. △ Less

Submitted 28 July, 2017; v1 submitted 27 June, 2016; originally announced June 2016.

Masking Strategies for Image Manifolds

Authors: Hamid Dadkhahi, Marco F. Duarte

Abstract: We consider the problem of selecting an optimal mask for an image manifold, i.e., choosing a subset of the pixels of the image that preserves the manifold's geometric structure present in the original data. Such masking implements a form of compressive sensing through emerging imaging sensor platforms for which the power expense grows with the number of pixels acquired. Our goal is for the manifol… ▽ More We consider the problem of selecting an optimal mask for an image manifold, i.e., choosing a subset of the pixels of the image that preserves the manifold's geometric structure present in the original data. Such masking implements a form of compressive sensing through emerging imaging sensor platforms for which the power expense grows with the number of pixels acquired. Our goal is for the manifold learned from masked images to resemble its full image counterpart as closely as possible. More precisely, we show that one can indeed accurately learn an image manifold without having to consider a large majority of the image pixels. In doing so, we consider two masking methods that preserve the local and global geometric structure of the manifold, respectively. In each case, the process of finding the optimal masking pattern can be cast as a binary integer program, which is computationally expensive but can be approximated by a fast greedy algorithm. Numerical experiments show that the relevant manifold structure is preserved through the data-dependent masking process, even for modest mask sizes. △ Less

Submitted 14 June, 2016; originally announced June 2016.

Wavelet-Based Semantic Features for Hyperspectral Signature Discrimination

Authors: Siwei Feng, Yuki Itoh, Mario Parente, Marco F. Duarte

Abstract: Hyperspectral signature classification is a quantitative analysis approach for hyperspectral imagery which performs detection and classification of the constituent materials at the pixel level in the scene. The classification procedure can be operated directly on hyperspectral data or performed by using some features extracted from the corresponding hyperspectral signatures containing information… ▽ More Hyperspectral signature classification is a quantitative analysis approach for hyperspectral imagery which performs detection and classification of the constituent materials at the pixel level in the scene. The classification procedure can be operated directly on hyperspectral data or performed by using some features extracted from the corresponding hyperspectral signatures containing information like the signature's energy or shape. In this paper, we describe a technique that applies non-homogeneous hidden Markov chain (NHMC) models to hyperspectral signature classification. The basic idea is to use statistical models (such as NHMC) to characterize wavelet coefficients which capture the spectrum semantics (i.e., structural information) at multiple levels. Experimental results show that the approach based on NHMC models can outperform existing approaches relevant in classification tasks. △ Less

Submitted 8 April, 2016; v1 submitted 11 February, 2016; originally announced February 2016.

Comments: 21 pages, 8 figures, 4 tables, preprint, revised April 8 2016

Perfect Recovery Conditions For Non-Negative Sparse Modeling

Authors: Yuki Itoh, Marco F. Duarte, Mario Parente

Abstract: Sparse modeling has been widely and successfully used in many applications such as computer vision, machine learning, and pattern recognition. Accompanied with those applications, significant research has studied the theoretical limits and algorithm design for convex relaxations in sparse modeling. However, theoretical analyses on non-negative versions of sparse modeling are limited in the literat… ▽ More Sparse modeling has been widely and successfully used in many applications such as computer vision, machine learning, and pattern recognition. Accompanied with those applications, significant research has studied the theoretical limits and algorithm design for convex relaxations in sparse modeling. However, theoretical analyses on non-negative versions of sparse modeling are limited in the literature either to a noiseless setting or a scenario with a specific statistical noise model such as Gaussian noise. This paper studies the performance of non-negative sparse modeling in a more general scenario where the observed signals have an unknown arbitrary distortion, especially focusing on non-negativity constrained and L1-penalized least squares, and gives an exact bound for which this problem can recover the correct signal elements. We pose two conditions to guarantee the correct signal recovery: minimum coefficient condition (MCC) and nonlinearity vs. subset coherence condition (NSCC). The former defines the minimum weight for each of the correct atoms present in the signal and the latter defines the tolerable deviation from the linear model relative to the positive subset coherence (PSC), a novel type of "coherence" metric. We provide rigorous performance guarantees based on these conditions and experimentally verify their precise predictive power in a hyperspectral data unmixing application. △ Less

Submitted 19 September, 2016; v1 submitted 8 December, 2015; originally announced December 2015.

Comments: 12 pages, 8 figures

Performance of Compressive Parameter Estimation via K-Median Clustering

Authors: Dian Mo, Marco F. Duarte

Abstract: Compressive sensing (CS) has attracted significant attention in parameter estimation tasks, where parametric dictionaries (PDs) collect signal observations for a sampling of the parameter space and yield sparse representations for signals of interest when the sampling is dense. While this sampling also leads to high dictionary coherence, one can leverage structured sparsity models to prevent highl… ▽ More Compressive sensing (CS) has attracted significant attention in parameter estimation tasks, where parametric dictionaries (PDs) collect signal observations for a sampling of the parameter space and yield sparse representations for signals of interest when the sampling is dense. While this sampling also leads to high dictionary coherence, one can leverage structured sparsity models to prevent highly coherent dictionary elements from appearing simultaneously in the recovered signal. However, the resulting approaches depend heavily on the careful setting of the maximum allowable coherence; furthermore, their guarantees are not concerned with general parameter estimation performance. We propose the use of earth mover's distance (EMD), as applied to a pair of true and estimated PD coefficient vectors, to measure the parameter estimation error. We formally analyze the connection between the EMD and the parameter estimation error and show that the EMD provides a better-suited metric for parameter estimation performance than the Euclidean distance. Additionally, we analyze the previously described relationship between K-median clustering and EMD-optimal sparse approximation and leverage it to develop improved PD-based parameter estimation algorithms. Finally, our numerical experiments verify our theoretical results and show that the proposed compressive parameter estimation algorithms have improved performance over existing approaches. △ Less

Submitted 11 May, 2017; v1 submitted 21 December, 2014; originally announced December 2014.

Comments: 41 pages, 8 figures; revision includes additional discussions and experiment

Conditioning of Random Block Subdictionaries with Applications to Block-Sparse Recovery and Regression

Authors: Waheed U. Bajwa, Marco F. Duarte, Robert Calderbank

Abstract: The linear model, in which a set of observations is assumed to be given by a linear combination of columns of a matrix, has long been the mainstay of the statistics and signal processing literature. One particular challenge for inference under linear models is understanding the conditions on the dictionary under which reliable inference is possible. This challenge has attracted renewed attention i… ▽ More The linear model, in which a set of observations is assumed to be given by a linear combination of columns of a matrix, has long been the mainstay of the statistics and signal processing literature. One particular challenge for inference under linear models is understanding the conditions on the dictionary under which reliable inference is possible. This challenge has attracted renewed attention in recent years since many modern inference problems deal with the "underdetermined" setting, in which the number of observations is much smaller than the number of columns in the dictionary. This paper makes several contributions for this setting when the set of observations is given by a linear combination of a small number of groups of columns of the dictionary, termed the "block-sparse" case. First, it specifies conditions on the dictionary under which most block subdictionaries are well conditioned. This result is fundamentally different from prior work on block-sparse inference because (i) it provides conditions that can be explicitly computed in polynomial time, (ii) the given conditions translate into near-optimal scaling of the number of columns of the block subdictionaries as a function of the number of observations for a large class of dictionaries, and (iii) it suggests that the spectral norm and the quadratic-mean block coherence of the dictionary (rather than the worst-case coherences) fundamentally limit the scaling of dimensions of the well-conditioned block subdictionaries. Second, this paper investigates the problems of block-sparse recovery and block-sparse regression in underdetermined settings. Near-optimal block-sparse recovery and regression are possible for certain dictionaries as long as the dictionary satisfies easily computable conditions and the coefficients describing the linear combination of groups of columns can be modeled through a mild statistical prior. △ Less

Submitted 28 August, 2016; v1 submitted 20 September, 2013; originally announced September 2013.

Comments: 39 pages, 3 figures. A revised and expanded version of the paper published in IEEE Transactions on Information Theory (DOI: 10.1109/TIT.2015.2429632); this revision includes corrections in the proofs of some of the results

Journal ref: IEEE Trans. Information Theory, vol. 61, no. 7, pp. 4060-4079, Jul. 2015

Compressive Time Delay Estimation Using Interpolation

Authors: Karsten Fyhn, Marco F. Duarte, Søren Holdt Jensen

Abstract: Time delay estimation has long been an active area of research. In this work, we show that compressive sensing with interpolation may be used to achieve good estimation precision while lowering the sampling frequency. We propose an Interpolating Band-Excluded Orthogonal Matching Pursuit algorithm that uses one of two interpolation functions to estimate the time delay parameter. The numerical resul… ▽ More Time delay estimation has long been an active area of research. In this work, we show that compressive sensing with interpolation may be used to achieve good estimation precision while lowering the sampling frequency. We propose an Interpolating Band-Excluded Orthogonal Matching Pursuit algorithm that uses one of two interpolation functions to estimate the time delay parameter. The numerical results show that interpolation improves estimation precision and that compressive sensing provides an elegant tradeoff that may lower the required sampling frequency while still attaining a desired estimation performance. △ Less

Submitted 11 June, 2013; originally announced June 2013.

Comments: 5 pages, 2 figures, technical report supporting 1 page submission for GlobalSIP 2013

Compressive Parameter Estimation for Sparse Translation-Invariant Signals Using Polar Interpolation

Authors: Karsten Fyhn, Marco F. Duarte, Søren Holdt Jensen

Abstract: We propose new compressive parameter estimation algorithms that make use of polar interpolation to improve the estimator precision. Our work extends previous approaches involving polar interpolation for compressive parameter estimation in two aspects: (i) we extend the formulation from real non-negative amplitude parameters to arbitrary complex ones, and (ii) we allow for mismatch between the mani… ▽ More We propose new compressive parameter estimation algorithms that make use of polar interpolation to improve the estimator precision. Our work extends previous approaches involving polar interpolation for compressive parameter estimation in two aspects: (i) we extend the formulation from real non-negative amplitude parameters to arbitrary complex ones, and (ii) we allow for mismatch between the manifold described by the parameters and its polar approximation. To quantify the improvements afforded by the proposed extensions, we evaluate six algorithms for estimation of parameters in sparse translation-invariant signals, exemplified with the time delay estimation problem. The evaluation is based on three performance metrics: estimator precision, sampling rate and computational complexity. We use compressive sensing with all the algorithms to lower the necessary sampling rate and show that it is still possible to attain good estimation precision and keep the computational complexity low. Our numerical experiments show that the proposed algorithms outperform existing approaches that either leverage polynomial interpolation or are based on a conversion to a frequency-estimation problem followed by a super-resolution algorithm. The algorithms studied here provide various tradeoffs between computational complexity, estimation precision, and necessary sampling rate. The work shows that compressive sensing for the class of sparse translation-invariant signals allows for a decrease in sampling rate and that the use of polar interpolation increases the estimation precision. △ Less

Submitted 6 January, 2015; v1 submitted 15 May, 2013; originally announced May 2013.

Comments: 13 pages, 5 figures, to appear in IEEE Transactions on Signal Processing; minor edits and corrections

Spectral Compressive Sensing with Polar Interpolation

Authors: Karsten Fyhn, Hamid Dadkhahi, Marco F. Duarte

Abstract: Existing approaches to compressive sensing of frequency-sparse signals focuses on signal recovery rather than spectral estimation. Furthermore, the recovery performance is limited by the coherence of the required sparsity dictionaries and by the discretization of the frequency parameter space. In this paper, we introduce a greedy recovery algorithm that leverages a band-exclusion function and a po… ▽ More Existing approaches to compressive sensing of frequency-sparse signals focuses on signal recovery rather than spectral estimation. Furthermore, the recovery performance is limited by the coherence of the required sparsity dictionaries and by the discretization of the frequency parameter space. In this paper, we introduce a greedy recovery algorithm that leverages a band-exclusion function and a polar interpolation function to address these two issues in spectral compressive sensing. Our algorithm is geared towards line spectral estimation from compressive measurements and outperforms most existing approaches in fidelity and tolerance to noise. △ Less

Submitted 21 May, 2013; v1 submitted 12 March, 2013; originally announced March 2013.

Comments: v1: 5 pages, 2 figures, accepted for publication at ICASSP 2013. v2,v3: This version corrects minor typos in Algorithm 1 from the published version

Performance Bounds for Grouped Incoherent Measurements in Compressive Sensing

Authors: Adam C. Polak, Marco F. Duarte, Dennis L. Goeckel

Abstract: Compressive sensing (CS) allows for acquisition of sparse signals at sampling rates significantly lower than the Nyquist rate required for bandlimited signals. Recovery guarantees for CS are generally derived based on the assumption that measurement projections are selected independently at random. However, for many practical signal acquisition applications, including medical imaging and remote se… ▽ More Compressive sensing (CS) allows for acquisition of sparse signals at sampling rates significantly lower than the Nyquist rate required for bandlimited signals. Recovery guarantees for CS are generally derived based on the assumption that measurement projections are selected independently at random. However, for many practical signal acquisition applications, including medical imaging and remote sensing, this assumption is violated as the projections must be taken in groups. In this paper, we consider such applications and derive requirements on the number of measurements needed for successful recovery of signals when groups of dependent projections are taken at random. We find a penalty factor on the number of required measurements with respect to the standard CS scheme that employs conventional independent measurement selection and evaluate the accuracy of the predicted penalty through simulations. △ Less

Submitted 20 October, 2014; v1 submitted 9 May, 2012; originally announced May 2012.

Comments: Revised for publication. 21 pages, 10 figures

Recovery from Linear Measurements with Complexity-Matching Universal Signal Estimation

Authors: Junan Zhu, Dror Baron, Marco F. Duarte

Abstract: We study the compressed sensing (CS) signal estimation problem where an input signal is measured via a linear matrix multiplication under additive noise. While this setup usually assumes sparsity or compressibility in the input signal during recovery, the signal structure that can be leveraged is often not known a priori. In this paper, we consider universal CS recovery, where the statistics of a… ▽ More We study the compressed sensing (CS) signal estimation problem where an input signal is measured via a linear matrix multiplication under additive noise. While this setup usually assumes sparsity or compressibility in the input signal during recovery, the signal structure that can be leveraged is often not known a priori. In this paper, we consider universal CS recovery, where the statistics of a stationary ergodic signal source are estimated simultaneously with the signal itself. Inspired by Kolmogorov complexity and minimum description length, we focus on a maximum a posteriori (MAP) estimation framework that leverages universal priors to match the complexity of the source. Our framework can also be applied to general linear inverse problems where more measurements than in CS might be needed. We provide theoretical results that support the algorithmic feasibility of universal MAP estimation using a Markov chain Monte Carlo implementation, which is computationally challenging. We incorporate some techniques to accelerate the algorithm while providing comparable and in many cases better reconstruction quality than existing algorithms. Experimental results show the promise of universality in CS, particularly for low-complexity sources that do not exhibit standard sparsity or compressibility. △ Less

Submitted 21 December, 2014; v1 submitted 12 April, 2012; originally announced April 2012.

Comments: 29 pages, 8 figures

Structured Compressed Sensing: From Theory to Applications

Authors: Marco F. Duarte, Yonina C. Eldar

Abstract: Compressed sensing (CS) is an emerging field that has attracted considerable research interest over the past few years. Previous review articles in CS limit their scope to standard discrete-to-discrete measurement architectures using matrices of randomized nature and signal models based on standard sparsity. In recent years, CS has worked its way into several new application areas. This, in turn,… ▽ More Compressed sensing (CS) is an emerging field that has attracted considerable research interest over the past few years. Previous review articles in CS limit their scope to standard discrete-to-discrete measurement architectures using matrices of randomized nature and signal models based on standard sparsity. In recent years, CS has worked its way into several new application areas. This, in turn, necessitates a fresh look on many of the basics of CS. The random matrix measurement operator must be replaced by more structured sensing architectures that correspond to the characteristics of feasible acquisition hardware. The standard sparsity prior has to be extended to include a much richer class of signals and to encode broader data models, including continuous-time signals. In our overview, the theme is exploiting signal and measurement structure in compressive sensing. The prime focus is bridging theory and practice; that is, to pinpoint the potential of structured CS strategies to emerge from the math to the hardware. Our summary highlights new directions as well as relations to more traditional CS, with the hope of serving both as a review to practitioners wanting to join this emerging field, and as a reference for researchers that attempts to put some of the existing ideas in perspective of practical applications. △ Less

Submitted 28 July, 2011; v1 submitted 30 June, 2011; originally announced June 2011.

Comments: To appear as an overview paper in IEEE Transactions on Signal Processing

Measurement Bounds for Sparse Signal Ensembles via Graphical Models

Authors: Marco F. Duarte, Michael B. Wakin, Dror Baron, Shriram Sarvotham, Richard G. Baraniuk

Abstract: In compressive sensing, a small collection of linear projections of a sparse signal contains enough information to permit signal recovery. Distributed compressive sensing (DCS) extends this framework by defining ensemble sparsity models, allowing a correlated ensemble of sparse signals to be jointly recovered from a collection of separately acquired compressive measurements. In this paper, we intr… ▽ More In compressive sensing, a small collection of linear projections of a sparse signal contains enough information to permit signal recovery. Distributed compressive sensing (DCS) extends this framework by defining ensemble sparsity models, allowing a correlated ensemble of sparse signals to be jointly recovered from a collection of separately acquired compressive measurements. In this paper, we introduce a framework for modeling sparse signal ensembles that quantifies the intra- and inter-signal dependencies within and among the signals. This framework is based on a novel bipartite graph representation that links the sparse signal coefficients with the measurements obtained for each signal. Using our framework, we provide fundamental bounds on the number of noiseless measurements that each sensor must collect to ensure that the signals are jointly recoverable. △ Less

Submitted 27 March, 2013; v1 submitted 14 February, 2011; originally announced February 2011.

Comments: 11 pages, 2 figures

Journal ref: IEEE Transactions on Information Theory, 2013

Asynchronous Code-Division Random Access Using Convex Optimization

Authors: Lorne Applebaum, Waheed U. Bajwa, Marco F. Duarte, Robert Calderbank

Abstract: Many applications in cellular systems and sensor networks involve a random subset of a large number of users asynchronously reporting activity to a base station. This paper examines the problem of multiuser detection (MUD) in random access channels for such applications. Traditional orthogonal signaling ignores the random nature of user activity in this problem and limits the total number of users… ▽ More Many applications in cellular systems and sensor networks involve a random subset of a large number of users asynchronously reporting activity to a base station. This paper examines the problem of multiuser detection (MUD) in random access channels for such applications. Traditional orthogonal signaling ignores the random nature of user activity in this problem and limits the total number of users to be on the order of the number of signal space dimensions. Contention-based schemes, on the other hand, suffer from delays caused by colliding transmissions and the hidden node problem. In contrast, this paper presents a novel pairing of an asynchronous non-orthogonal code-division random access scheme with a convex optimization-based MUD algorithm that overcomes the issues associated with orthogonal signaling and contention-based methods. Two key distinguishing features of the proposed MUD algorithm are that it does not require knowledge of the delay or channel state information of every user and it has polynomial-time computational complexity. The main analytical contribution of this paper is the relationship between the performance of the proposed MUD algorithm in the presence of arbitrary or random delays and two simple metrics of the set of user codewords. The study of these metrics is then focused on two specific sets of codewords, random binary codewords and specially constructed algebraic codewords, for asynchronous random access. The ensuing analysis confirms that the proposed scheme together with either of these two codeword sets significantly outperforms the orthogonal signaling-based random access in terms of the total number of users in the system. △ Less

Submitted 20 June, 2011; v1 submitted 7 January, 2011; originally announced January 2011.

Comments: Journal version of work presented at 2010 Allerton Conference on Communication, Control and Computing. Version 2 includes additional analysis of randomly distributed user delays as well as a comparison with a matched filter receiver

Journal ref: Elsevier Phy. Commun., vol. 5, no. 2, pp. 129-147, Jun. 2012

Beyond Nyquist: Efficient Sampling of Sparse Bandlimited Signals

Authors: Joel A. Tropp, Jason N. Laska, Marco F. Duarte, Justin K. Romberg, Richard G. Baraniuk

Abstract: Wideband analog signals push contemporary analog-to-digital conversion systems to their performance limits. In many applications, however, sampling at the Nyquist rate is inefficient because the signals of interest contain only a small number of significant frequencies relative to the bandlimit, although the locations of the frequencies may not be known a priori. For this type of sparse signal,… ▽ More Wideband analog signals push contemporary analog-to-digital conversion systems to their performance limits. In many applications, however, sampling at the Nyquist rate is inefficient because the signals of interest contain only a small number of significant frequencies relative to the bandlimit, although the locations of the frequencies may not be known a priori. For this type of sparse signal, other sampling strategies are possible. This paper describes a new type of data acquisition system, called a random demodulator, that is constructed from robust, readily available components. Let K denote the total number of frequencies in the signal, and let W denote its bandlimit in Hz. Simulations suggest that the random demodulator requires just O(K log(W/K)) samples per second to stably reconstruct the signal. This sampling rate is exponentially lower than the Nyquist rate of W Hz. In contrast with Nyquist sampling, one must use nonlinear methods, such as convex programming, to recover the signal from the samples taken by the random demodulator. This paper provides a detailed theoretical analysis of the system's performance that supports the empirical observations. △ Less

Submitted 21 September, 2009; v1 submitted 31 January, 2009; originally announced February 2009.

Comments: 24 pages, 8 figures

Journal ref: IEEE Trans. Inform. Theory, Vol.56, num. 1, pp. 520-544, Jan. 2010

Distributed Compressive Sensing

Authors: Dror Baron, Marco F. Duarte, Michael B. Wakin, Shriram Sarvotham, Richard G. Baraniuk

Abstract: Compressive sensing is a signal acquisition framework based on the revelation that a small collection of linear projections of a sparse signal contains enough information for stable recovery. In this paper we introduce a new theory for distributed compressive sensing (DCS) that enables new distributed coding algorithms for multi-signal ensembles that exploit both intra- and inter-signal correlat… ▽ More Compressive sensing is a signal acquisition framework based on the revelation that a small collection of linear projections of a sparse signal contains enough information for stable recovery. In this paper we introduce a new theory for distributed compressive sensing (DCS) that enables new distributed coding algorithms for multi-signal ensembles that exploit both intra- and inter-signal correlation structures. The DCS theory rests on a new concept that we term the joint sparsity of a signal ensemble. Our theoretical contribution is to characterize the fundamental performance limits of DCS recovery for jointly sparse signal ensembles in the noiseless measurement setting; our result connects single-signal, joint, and distributed (multi-encoder) compressive sensing. To demonstrate the efficacy of our framework and to show that additional challenges such as computational tractability can be addressed, we study in detail three example models for jointly sparse signals. For these models, we develop practical algorithms for joint recovery of multiple signals from incoherent projections. In two of our three models, the results are asymptotically best-possible, meaning that both the upper and lower bounds match the performance of our practical algorithms. Moreover, simulations indicate that the asymptotics take effect with just a moderate number of signals. DCS is immediately applicable to a range of problems in sensor arrays and networks. △ Less

Submitted 21 January, 2009; originally announced January 2009.

Comments: 42 pages, 6 figures. Submitted November 27, 2005; Revised January 21, 2009

A Theoretical Analysis of Joint Manifolds

Authors: Mark A. Davenport, Chinmay Hegde, Marco F. Duarte, Richard G. Baraniuk

Abstract: The emergence of low-cost sensor architectures for diverse modalities has made it possible to deploy sensor arrays that capture a single event from a large number of vantage points and using multiple modalities. In many scenarios, these sensors acquire very high-dimensional data such as audio signals, images, and video. To cope with such high-dimensional data, we typically rely on low-dimensiona… ▽ More The emergence of low-cost sensor architectures for diverse modalities has made it possible to deploy sensor arrays that capture a single event from a large number of vantage points and using multiple modalities. In many scenarios, these sensors acquire very high-dimensional data such as audio signals, images, and video. To cope with such high-dimensional data, we typically rely on low-dimensional models. Manifold models provide a particularly powerful model that captures the structure of high-dimensional data when it is governed by a low-dimensional set of parameters. However, these models do not typically take into account dependencies among multiple sensors. We thus propose a new joint manifold framework for data ensembles that exploits such dependencies. We show that simple algorithms can exploit the joint manifold structure to improve their performance on standard signal processing applications. Additionally, recent results concerning dimensionality reduction for manifolds enable us to formulate a network-scalable data compression scheme that uses random projections of the sensed data. This scheme efficiently fuses the data from all sensors through the addition of such projections, regardless of the data modalities and dimensions. △ Less

Submitted 9 December, 2009; v1 submitted 7 January, 2009; originally announced January 2009.

Comments: 24 pages, 4 figures. Corrected typo on grant number in acknowledgements, page 1

Report number: TREE0901, Department of Electrical and Computer Engineering, Rice University

Model-Based Compressive Sensing

Authors: Richard G. Baraniuk, Volkan Cevher, Marco F. Duarte, Chinmay Hegde

Abstract: Compressive sensing (CS) is an alternative to Shannon/Nyquist sampling for the acquisition of sparse or compressible signals that can be well approximated by just K << N elements from an N-dimensional basis. Instead of taking periodic samples, CS measures inner products with M < N random vectors and then recovers the signal via a sparsity-seeking optimization or greedy algorithm. Standard CS dic… ▽ More Compressive sensing (CS) is an alternative to Shannon/Nyquist sampling for the acquisition of sparse or compressible signals that can be well approximated by just K << N elements from an N-dimensional basis. Instead of taking periodic samples, CS measures inner products with M < N random vectors and then recovers the signal via a sparsity-seeking optimization or greedy algorithm. Standard CS dictates that robust signal recovery is possible from M = O(K log(N/K)) measurements. It is possible to substantially decrease M without sacrificing robustness by leveraging more realistic signal models that go beyond simple sparsity and compressibility by including structural dependencies between the values and locations of the signal coefficients. This paper introduces a model-based CS theory that parallels the conventional theory and provides concrete guidelines on how to create model-based recovery algorithms with provable performance guarantees. A highlight is the introduction of a new class of structured compressible signals along with a new sufficient condition for robust structured compressible signal recovery that we dub the restricted amplification property, which is the natural counterpart to the restricted isometry property of conventional CS. Two examples integrate two relevant signal models - wavelet trees and block sparsity - into two state-of-the-art CS recovery algorithms and prove that they offer robust recovery from just M=O(K) measurements. Extensive numerical simulations demonstrate the validity and applicability of our new theory and algorithms. △ Less

Submitted 9 December, 2009; v1 submitted 26 August, 2008; originally announced August 2008.

Comments: 20 pages, 10 figures. Typo corrected in grant number. To appear in IEEE Transactions on Information Theory

Characterizing Video Responses in Social Networks

Authors: Fabricio Benevenuto, Fernando Duarte, Tiago Rodrigues, Virgilio Almeida, Jussara Almeida, Keith Ross

Abstract: Video sharing sites, such as YouTube, use video responses to enhance the social interactions among their users. The video response feature allows users to interact and converse through video, by creating a video sequence that begins with an opening video and followed by video responses from other users. Our characterization is over 3.4 million videos and 400,000 video responses collected from Yo… ▽ More Video sharing sites, such as YouTube, use video responses to enhance the social interactions among their users. The video response feature allows users to interact and converse through video, by creating a video sequence that begins with an opening video and followed by video responses from other users. Our characterization is over 3.4 million videos and 400,000 video responses collected from YouTube during a 7-day period. We first analyze the characteristics of the video responses, such as popularity, duration, and geography. We then examine the social networks that emerge from the video response interactions. △ Less

Submitted 30 April, 2008; originally announced April 2008.

ACM Class: J.4; H.3.5