-
Combining Game Design and Data Visualization to Inform Plastics Policy: Fostering Collaboration between Science, Decision-Makers, and Artificial Intelligence
Authors:
A Samuel Pottinger,
Nivedita Biyani,
Roland Geyer,
Douglas J McCauley,
Magali de Bruyn,
Molly R Morse,
Neil Nathan,
Kevin Koy,
Ciera Martinez
Abstract:
This multi-disciplinary case study details how a public web application combines information and game design to visualize effects of user-defined policies intended to reduce plastic waste. Contextualizing this open source software within a broader lineage of digital media research, this user experience exploration outlines potential directions for facilitating conversation between artificial intel…
▽ More
This multi-disciplinary case study details how a public web application combines information and game design to visualize effects of user-defined policies intended to reduce plastic waste. Contextualizing this open source software within a broader lineage of digital media research, this user experience exploration outlines potential directions for facilitating conversation between artificial intelligence, scientists, and decision makers during an iterative policy building process. Furthermore, this system dissection reveals how this interactive science effort considers the practicalities of a treaty's shifting priorities and proposals in its designs. Specifically, this historically situated investigation of the tool's approach highlights options for centering human decision making where artificial intelligence helps reason about interventions but does not prescribe them. Finally, analysis summarizes this application's specific game design-inspired mechanics and their efforts to: enable users' agency to explore solution possibilities freely, invite deep engagement with scientific findings, and simultaneously serve multiple audiences with divergent objectives and expertise.
△ Less
Submitted 19 December, 2023; v1 submitted 18 December, 2023;
originally announced December 2023.
-
Performance of Integrated IoT Network with Hybrid mmWave/FSO/THz Backhaul Link
Authors:
Pranay Bhardwaj,
Vedant Bansal,
Nikhil Biyani,
Shiv Shukla,
S. M. Zafaruddin
Abstract:
Establishing end-to-end connectivity of Internet of Things (IoT) network with the core for collecting sensing data from remote and hard-to-reach terrains is a challenging task. In this article, we analyze the performance of an IoT network integrated with wireless backhaul link for data collection. We propose a solution that involves a self-configuring protocol for aggregate node (AN) selection in…
▽ More
Establishing end-to-end connectivity of Internet of Things (IoT) network with the core for collecting sensing data from remote and hard-to-reach terrains is a challenging task. In this article, we analyze the performance of an IoT network integrated with wireless backhaul link for data collection. We propose a solution that involves a self-configuring protocol for aggregate node (AN) selection in an IoT network, which sends the data packet to an unmanned aerial vehicle (UAV) over radio frequency (RF) channels. We adopt a novel hybrid transmission technique for wireless backhaul employing opportunistic selections combining (OSC) and maximal ratio combining (MRC) that simultaneously transmits the data packet on mmWave (mW), free space optical (FSO), and terahertz (THz) technologies to take advantage of their complementary characteristics. We employ the decode-and-forward (DF) protocol to integrate the IoT and backhaul links and provide physical layer performance assessment using outage probability and average bit-error-rate (BER) under diverse channel conditions. We also develop simplified expressions to gain a better understanding of the system's performance at high signal-to-noise ratio (SNR). We provide computer simulations to compare different wireless backhaul technologies under various channel and SNR scenarios and demonstrate the performance of the data collection using the integrated link.
△ Less
Submitted 10 June, 2023; v1 submitted 3 April, 2023;
originally announced April 2023.
-
Large-scale Robustness Analysis of Video Action Recognition Models
Authors:
Madeline Chantry Schiappa,
Naman Biyani,
Prudvi Kamtam,
Shruti Vyas,
Hamid Palangi,
Vibhav Vineet,
Yogesh Rawat
Abstract:
We have seen a great progress in video action recognition in recent years. There are several models based on convolutional neural network (CNN) and some recent transformer based approaches which provide top performance on existing benchmarks. In this work, we perform a large-scale robustness analysis of these existing models for video action recognition. We focus on robustness against real-world d…
▽ More
We have seen a great progress in video action recognition in recent years. There are several models based on convolutional neural network (CNN) and some recent transformer based approaches which provide top performance on existing benchmarks. In this work, we perform a large-scale robustness analysis of these existing models for video action recognition. We focus on robustness against real-world distribution shift perturbations instead of adversarial perturbations. We propose four different benchmark datasets, HMDB51-P, UCF101-P, Kinetics400-P, and SSv2-P to perform this analysis. We study robustness of six state-of-the-art action recognition models against 90 different perturbations. The study reveals some interesting findings, 1) transformer based models are consistently more robust compared to CNN based models, 2) Pretraining improves robustness for Transformer based models more than CNN based models, and 3) All of the studied models are robust to temporal perturbations for all datasets but SSv2; suggesting the importance of temporal information for action recognition varies based on the dataset and activities. Next, we study the role of augmentations in model robustness and present a real-world dataset, UCF101-DS, which contains realistic distribution shifts, to further validate some of these findings. We believe this study will serve as a benchmark for future research in robust video action recognition.
△ Less
Submitted 7 April, 2023; v1 submitted 4 July, 2022;
originally announced July 2022.
-
LARNet: Latent Action Representation for Human Action Synthesis
Authors:
Naman Biyani,
Aayush J Rana,
Shruti Vyas,
Yogesh S Rawat
Abstract:
We present LARNet, a novel end-to-end approach for generating human action videos. A joint generative modeling of appearance and dynamics to synthesize a video is very challenging and therefore recent works in video synthesis have proposed to decompose these two factors. However, these methods require a driving video to model the video dynamics. In this work, we propose a generative approach inste…
▽ More
We present LARNet, a novel end-to-end approach for generating human action videos. A joint generative modeling of appearance and dynamics to synthesize a video is very challenging and therefore recent works in video synthesis have proposed to decompose these two factors. However, these methods require a driving video to model the video dynamics. In this work, we propose a generative approach instead, which explicitly learns action dynamics in latent space avoiding the need of a driving video during inference. The generated action dynamics is integrated with the appearance using a recurrent hierarchical structure which induces motion at different scales to focus on both coarse as well as fine level action details. In addition, we propose a novel mix-adversarial loss function which aims at improving the temporal coherency of synthesized videos. We evaluate the proposed approach on four real-world human action datasets demonstrating the effectiveness of the proposed approach in generating human actions. Code available at https://github.com/aayushjr/larnet.
△ Less
Submitted 26 October, 2021; v1 submitted 21 October, 2021;
originally announced October 2021.
-
"Knights": First Place Submission for VIPriors21 Action Recognition Challenge at ICCV 2021
Authors:
Ishan Dave,
Naman Biyani,
Brandon Clark,
Rohit Gupta,
Yogesh Rawat,
Mubarak Shah
Abstract:
This technical report presents our approach "Knights" to solve the action recognition task on a small subset of Kinetics-400 i.e. Kinetics400ViPriors without using any extra-data. Our approach has 3 main components: state-of-the-art Temporal Contrastive self-supervised pretraining, video transformer models, and optical flow modality. Along with the use of standard test-time augmentation, our propo…
▽ More
This technical report presents our approach "Knights" to solve the action recognition task on a small subset of Kinetics-400 i.e. Kinetics400ViPriors without using any extra-data. Our approach has 3 main components: state-of-the-art Temporal Contrastive self-supervised pretraining, video transformer models, and optical flow modality. Along with the use of standard test-time augmentation, our proposed solution achieves 73% on Kinetics400ViPriors test set, which is the best among all of the other entries Visual Inductive Priors for Data-Efficient Computer Vision's Action Recognition Challenge, ICCV 2021.
△ Less
Submitted 14 October, 2021;
originally announced October 2021.
-
Structured Prediction in NLP -- A survey
Authors:
Chauhan Dev,
Naman Biyani,
Nirmal P. Suthar,
Prashant Kumar,
Priyanshu Agarwal
Abstract:
Over the last several years, the field of Structured prediction in NLP has had seen huge advancements with sophisticated probabilistic graphical models, energy-based networks, and its combination with deep learning-based approaches. This survey provides a brief of major techniques in structured prediction and its applications in the NLP domains like parsing, sequence labeling, text generation, and…
▽ More
Over the last several years, the field of Structured prediction in NLP has had seen huge advancements with sophisticated probabilistic graphical models, energy-based networks, and its combination with deep learning-based approaches. This survey provides a brief of major techniques in structured prediction and its applications in the NLP domains like parsing, sequence labeling, text generation, and sequence to sequence tasks. We also deep-dived into energy-based and attention-based techniques in structured prediction, identified some relevant open issues and gaps in the current state-of-the-art research, and have come up with some detailed ideas for future research in these fields.
△ Less
Submitted 31 August, 2021;
originally announced October 2021.
