-
MARBLER: An Open Platform for Standardized Evaluation of Multi-Robot Reinforcement Learning Algorithms
Authors:
Reza Torbati,
Shubham Lohiya,
Shivika Singh,
Meher Shashwat Nigam,
Harish Ravichandar
Abstract:
Multi-Agent Reinforcement Learning (MARL) has enjoyed significant recent progress thanks, in part, to the integration of deep learning techniques for modeling interactions in complex environments. This is naturally starting to benefit multi-robot systems (MRS) in the form of multi-robot RL (MRRL). However, existing infrastructure to train and evaluate policies predominantly focus on the challenges…
▽ More
Multi-Agent Reinforcement Learning (MARL) has enjoyed significant recent progress thanks, in part, to the integration of deep learning techniques for modeling interactions in complex environments. This is naturally starting to benefit multi-robot systems (MRS) in the form of multi-robot RL (MRRL). However, existing infrastructure to train and evaluate policies predominantly focus on the challenges of coordinating virtual agents, and ignore characteristics important to robotic systems. Few platforms support realistic robot dynamics, and fewer still can evaluate Sim2Real performance of learned behavior. To address these issues, we contribute MARBLER: Multi-Agent RL Benchmark and Learning Environment for the Robotarium. MARBLER offers a robust and comprehensive evaluation platform for MRRL by marrying Georgia Tech's Robotarium (which enables rapid deployment on physical MRS) and OpenAI's Gym interface (which facilitates standardized use of modern learning algorithms). MARBLER offers a highly controllable environment with realistic dynamics, including barrier certificate-based obstacle avoidance. It allows anyone across the world to train and deploy MRRL algorithms on a physical testbed with reproducibility. Further, we introduce five novel scenarios inspired by common challenges in MRS and provide support for new custom scenarios. Finally, we use MARBLER to evaluate popular MARL algorithms and provide insights into their suitability for MRRL. In summary, MARBLER can be a valuable tool to the MRS research community by facilitating comprehensive and standardized evaluation of learning algorithms on realistic simulations and physical hardware. Links to our open-source framework and videos of real-world experiments can be found at https://shubhlohiya.github.io/MARBLER/.
△ Less
Submitted 21 October, 2023; v1 submitted 7 July, 2023;
originally announced July 2023.
-
MVRackLay: Monocular Multi-View Layout Estimation for Warehouse Racks and Shelves
Authors:
Pranjali Pathre,
Anurag Sahu,
Ashwin Rao,
Avinash Prabhu,
Meher Shashwat Nigam,
Tanvi Karandikar,
Harit Pandya,
K. Madhava Krishna
Abstract:
In this paper, we propose and showcase, for the first time, monocular multi-view layout estimation for warehouse racks and shelves. Unlike typical layout estimation methods, MVRackLay estimates multi-layered layouts, wherein each layer corresponds to the layout of a shelf within a rack. Given a sequence of images of a warehouse scene, a dual-headed Convolutional-LSTM architecture outputs segmented…
▽ More
In this paper, we propose and showcase, for the first time, monocular multi-view layout estimation for warehouse racks and shelves. Unlike typical layout estimation methods, MVRackLay estimates multi-layered layouts, wherein each layer corresponds to the layout of a shelf within a rack. Given a sequence of images of a warehouse scene, a dual-headed Convolutional-LSTM architecture outputs segmented racks, the front and the top view layout of each shelf within a rack. With minimal effort, such an output is transformed into a 3D rendering of all racks, shelves and objects on the shelves, giving an accurate 3D depiction of the entire warehouse scene in terms of racks, shelves and the number of objects on each shelf. MVRackLay generalizes to a diverse set of warehouse scenes with varying number of objects on each shelf, number of shelves and in the presence of other such racks in the background. Further, MVRackLay shows superior performance vis-a-vis its single view counterpart, RackLay, in layout accuracy, quantized in terms of the mean IoU and mAP metrics. We also showcase a multi-view stitching of the 3D layouts resulting in a representation of the warehouse scene with respect to a global reference frame akin to a rendering of the scene from a SLAM pipeline. To the best of our knowledge, this is the first such work to portray a 3D rendering of a warehouse scene in terms of its semantic components - Racks, Shelves and Objects - all from a single monocular camera.
△ Less
Submitted 30 November, 2022;
originally announced November 2022.
-
Is Twitter Enough? Investigating Situational Awareness in Social and Print Media during the Second COVID-19 Wave in India
Authors:
Ishita Vohra,
Meher Shashwat Nigam,
Aryan Sakaria,
Amey Kudari,
Nimmi Rangaswamy
Abstract:
The pandemic required efficient allocation of public resources and transforming existing ways of societal functions. To manage any crisis, governments and public health researchers exploit the information available to them in order to make informed decisions, also defined as situational awareness. Gathering situational awareness using social media has been functional to manage epidemics. Previous…
▽ More
The pandemic required efficient allocation of public resources and transforming existing ways of societal functions. To manage any crisis, governments and public health researchers exploit the information available to them in order to make informed decisions, also defined as situational awareness. Gathering situational awareness using social media has been functional to manage epidemics. Previous research focused on using discussions during periods of epidemic crises on social media platforms like Twitter, Reddit, or Facebook and develo** NLP techniques to filter out relevant discussions from a huge corpus of messages and posts. Social media usage varies with internet penetration and other socioeconomic factors, which might induce disparity in analyzing discussions across different geographies. However, print media is a ubiquitous information source, irrespective of geography. Further, topics discussed in news articles are already newsworthy, while on social media newsworthiness is a product of techno-social processes. Develo** this fundamental difference, we study Twitter data during the second wave in India focused on six high-population cities with varied macroeconomic factors. Through a mixture of qualitative and quantitative methods, we further analyze two Indian newspapers during the same period and compare topics from both Twitter and the newspapers to evaluate situational awareness around the second phase of COVID on each of these platforms. We conclude that factors like internet penetration and GDP in a specific city influence the discourse surrounding situational updates on social media. Thus, augmenting information from newspapers with information extracted from social media would provide a more comprehensive perspective in resource deficit cities.
△ Less
Submitted 29 November, 2022;
originally announced November 2022.
-
Synthetic 3D Data Generation Pipeline for Geometric Deep Learning in Architecture
Authors:
Stanislava Fedorova,
Alberto Tono,
Meher Shashwat Nigam,
Jiayao Zhang,
Amirhossein Ahmadnia,
Cecilia Bolognesi,
Dominik L. Michels
Abstract:
With the growing interest in deep learning algorithms and computational design in the architectural field, the need for large, accessible and diverse architectural datasets increases. We decided to tackle this problem by constructing a field-specific synthetic data generation pipeline that generates an arbitrary amount of 3D data along with the associated 2D and 3D annotations. The variety of anno…
▽ More
With the growing interest in deep learning algorithms and computational design in the architectural field, the need for large, accessible and diverse architectural datasets increases. We decided to tackle this problem by constructing a field-specific synthetic data generation pipeline that generates an arbitrary amount of 3D data along with the associated 2D and 3D annotations. The variety of annotations, the flexibility to customize the generated building and dataset parameters make this framework suitable for multiple deep learning tasks, including geometric deep learning that requires direct 3D supervision. Creating our building data generation pipeline we leveraged architectural knowledge from experts in order to construct a framework that would be modular, extendable and would provide a sufficient amount of class-balanced data samples. Moreover, we purposefully involve the researcher in the dataset customization allowing the introduction of additional building components, material textures, building classes, number and type of annotations as well as the number of views per 3D model sample. In this way, the framework would satisfy different research requirements and would be adaptable to a large variety of tasks. All code and data are made publicly available.
△ Less
Submitted 26 April, 2021;
originally announced April 2021.
-
Monocular Multi-Layer Layout Estimation for Warehouse Racks
Authors:
Meher Shashwat Nigam,
Avinash Prabhu,
Anurag Sahu,
Puru Gupta,
Tanvi Karandikar,
N. Sai Shankar,
Ravi Kiran Sarvadevabhatla,
K. Madhava Krishna
Abstract:
Given a monocular colour image of a warehouse rack, we aim to predict the bird's-eye view layout for each shelf in the rack, which we term as multi-layer layout prediction. To this end, we present RackLay, a deep neural network for real-time shelf layout estimation from a single image. Unlike previous layout estimation methods, which provide a single layout for the dominant ground plane alone, Rac…
▽ More
Given a monocular colour image of a warehouse rack, we aim to predict the bird's-eye view layout for each shelf in the rack, which we term as multi-layer layout prediction. To this end, we present RackLay, a deep neural network for real-time shelf layout estimation from a single image. Unlike previous layout estimation methods, which provide a single layout for the dominant ground plane alone, RackLay estimates the top-view and front-view layout for each shelf in the considered rack populated with objects. RackLay's architecture and its variants are versatile and estimate accurate layouts for diverse scenes characterized by varying number of visible shelves in an image, large range in shelf occupancy factor and varied background clutter. Given the extreme paucity of datasets in this space and the difficulty involved in acquiring real data from warehouses, we additionally release a flexible synthetic dataset generation pipeline WareSynth which allows users to control the generation process and tailor the dataset according to contingent application. The ablations across architectural variants and comparison with strong prior baselines vindicate the efficacy of RackLay as an apt architecture for the novel problem of multi-layered layout estimation. We also show that fusing the top-view and front-view enables 3D reasoning applications such as metric free space estimation for the considered rack.
△ Less
Submitted 28 October, 2021; v1 submitted 16 March, 2021;
originally announced March 2021.
