-
DRAMA: Joint Risk Localization and Captioning in Driving
Authors:
Srikanth Malla,
Chiho Choi,
Isht Dwivedi,
Joon Hee Choi,
Jiachen Li
Abstract:
Considering the functionality of situational awareness in safety-critical automation systems, the perception of risk in driving scenes and its explainability is of particular importance for autonomous and cooperative driving. Toward this goal, this paper proposes a new research direction of joint risk localization in driving scenes and its risk explanation as a natural language description. Due to…
▽ More
Considering the functionality of situational awareness in safety-critical automation systems, the perception of risk in driving scenes and its explainability is of particular importance for autonomous and cooperative driving. Toward this goal, this paper proposes a new research direction of joint risk localization in driving scenes and its risk explanation as a natural language description. Due to the lack of standard benchmarks, we collected a large-scale dataset, DRAMA (Driving Risk Assessment Mechanism with A captioning module), which consists of 17,785 interactive driving scenarios collected in Tokyo, Japan. Our DRAMA dataset accommodates video- and object-level questions on driving risks with associated important objects to achieve the goal of visual captioning as a free-form language description utilizing closed and open-ended responses for multi-level questions, which can be used to evaluate a range of visual captioning capabilities in driving scenarios. We make this data available to the community for further research. Using DRAMA, we explore multiple facets of joint risk localization and captioning in interactive driving scenarios. In particular, we benchmark various multi-task prediction architectures and provide a detailed analysis of joint risk localization and risk captioning. The data set is available at https://usa.honda-ri.com/drama
△ Less
Submitted 5 October, 2022; v1 submitted 21 September, 2022;
originally announced September 2022.
-
Weakly-Supervised Online Action Segmentation in Multi-View Instructional Videos
Authors:
Reza Ghoddoosian,
Isht Dwivedi,
Nakul Agarwal,
Chiho Choi,
Behzad Dariush
Abstract:
This paper addresses a new problem of weakly-supervised online action segmentation in instructional videos. We present a framework to segment streaming videos online at test time using Dynamic Programming and show its advantages over greedy sliding window approach. We improve our framework by introducing the Online-Offline Discrepancy Loss (OODL) to encourage the segmentation results to have a hig…
▽ More
This paper addresses a new problem of weakly-supervised online action segmentation in instructional videos. We present a framework to segment streaming videos online at test time using Dynamic Programming and show its advantages over greedy sliding window approach. We improve our framework by introducing the Online-Offline Discrepancy Loss (OODL) to encourage the segmentation results to have a higher temporal consistency. Furthermore, only during training, we exploit frame-wise correspondence between multiple views as supervision for training weakly-labeled instructional videos. In particular, we investigate three different multi-view inference techniques to generate more accurate frame-wise pseudo ground-truth with no additional annotation cost. We present results and ablation studies on two benchmark multi-view datasets, Breakfast and IKEA ASM. Experimental results show efficacy of the proposed methods both qualitatively and quantitatively in two domains of cooking and assembly.
△ Less
Submitted 24 March, 2022;
originally announced March 2022.
-
SSP: Single Shot Future Trajectory Prediction
Authors:
Isht Dwivedi,
Srikanth Malla,
Behzad Dariush,
Chiho Choi
Abstract:
We propose a robust solution to future trajectory forecast, which can be practically applicable to autonomous agents in highly crowded environments. For this, three aspects are particularly addressed in this paper. First, we use composite fields to predict future locations of all road agents in a single-shot, which results in a constant time complexity, regardless of the number of agents in the sc…
▽ More
We propose a robust solution to future trajectory forecast, which can be practically applicable to autonomous agents in highly crowded environments. For this, three aspects are particularly addressed in this paper. First, we use composite fields to predict future locations of all road agents in a single-shot, which results in a constant time complexity, regardless of the number of agents in the scene. Second, interactions between agents are modeled as a non-local response, enabling spatial relationships between different locations to be captured temporally as well (i.e., in spatio-temporal interactions). Third, the semantic context of the scene are modeled and take into account the environmental constraints that potentially influence the future motion. To this end, we validate the robustness of the proposed approach using the ETH, UCY, and SDD datasets and highlight its practical functionality compared to the current state-of-the-art methods.
△ Less
Submitted 8 November, 2020; v1 submitted 13 April, 2020;
originally announced April 2020.
-
NEMO: Future Object Localization Using Noisy Ego Priors
Authors:
Srikanth Malla,
Isht Dwivedi,
Behzad Dariush,
Chiho Choi
Abstract:
Predicting the future trajectory of agents from visual observations is an important problem for realization of safe and effective navigation of autonomous systems in dynamic environments. This paper focuses on two important aspects of future trajectory forecast which are particularly relevant for mobile platforms: 1) modeling uncertainty of the predictions, particularly from egocentric views, wher…
▽ More
Predicting the future trajectory of agents from visual observations is an important problem for realization of safe and effective navigation of autonomous systems in dynamic environments. This paper focuses on two important aspects of future trajectory forecast which are particularly relevant for mobile platforms: 1) modeling uncertainty of the predictions, particularly from egocentric views, where uncertainty in the interactive reactions and behaviors of other agents must consider the uncertainty in the ego-motion, and 2) modeling multi-modality nature of the problem, which are particularly prevalent at junctions in urban traffic scenes. To address these problems in a unified approach, we propose NEMO (Noisy Ego MOtion priors for future object localization) for future forecast of agents in the egocentric view. In the proposed approach, a predictive distribution of future forecast is jointly modeled with the uncertainty of predictions. For this, we divide the problem into two tasks: future ego-motion prediction and future object localization. We first model the multi-modal distribution of future ego-motion with uncertainty estimates. The resulting distribution of ego-behavior is used to sample multiple modes of future ego-motion. Then, each modality is used as a prior to understand the interactions between the ego-vehicle and target agent. We predict the multi-modal future locations of the target from individual modes of the ego-vehicle while modeling the uncertainty of the target's behavior. To this end, we extensively evaluate the proposed framework using the publicly available benchmark dataset (HEV-I) supplemented with odometry data from an Inertial Measurement Unit (IMU).
△ Less
Submitted 22 July, 2020; v1 submitted 17 September, 2019;
originally announced September 2019.
-
Dynamic Traffic Scene Classification with Space-Time Coherence
Authors:
Athma Narayanan,
Isht Dwivedi,
Behzad Dariush
Abstract:
This paper examines the problem of dynamic traffic scene classification under space-time variations in viewpoint that arise from video captured on-board a moving vehicle. Solutions to this problem are important for realization of effective driving assistance technologies required to interpret or predict road user behavior. Currently, dynamic traffic scene classification has not been adequately add…
▽ More
This paper examines the problem of dynamic traffic scene classification under space-time variations in viewpoint that arise from video captured on-board a moving vehicle. Solutions to this problem are important for realization of effective driving assistance technologies required to interpret or predict road user behavior. Currently, dynamic traffic scene classification has not been adequately addressed due to a lack of benchmark datasets that consider spatiotemporal evolution of traffic scenes resulting from a vehicle's ego-motion. This paper has three main contributions. First, an annotated dataset is released to enable dynamic scene classification that includes 80 hours of diverse high quality driving video data clips collected in the San Francisco Bay area. The dataset includes temporal annotations for road places, road types, weather, and road surface conditions. Second, we introduce novel and baseline algorithms that utilize semantic context and temporal nature of the dataset for dynamic classification of road scenes. Finally, we showcase algorithms and experimental results that highlight how extracted features from scene classification serve as strong priors and help with tactical driver behavior understanding. The results show significant improvement from previously reported driving behavior detection baselines in the literature.
△ Less
Submitted 29 May, 2019;
originally announced May 2019.
-
High Quality Prediction of Protein Q8 Secondary Structure by Diverse Neural Network Architectures
Authors:
Iddo Drori,
Isht Dwivedi,
Pranav Shrestha,
Jeffrey Wan,
Yueqi Wang,
Yunchu He,
Anthony Mazza,
Hugh Krogh-Freeman,
Dimitri Leggas,
Kendal Sandridge,
Linyong Nan,
Kaveri Thakoor,
Chinmay Joshi,
Sonam Goenka,
Chen Keasar,
Itsik Pe'er
Abstract:
We tackle the problem of protein secondary structure prediction using a common task framework. This lead to the introduction of multiple ideas for neural architectures based on state of the art building blocks, used in this task for the first time. We take a principled machine learning approach, which provides genuine, unbiased performance measures, correcting longstanding errors in the applicatio…
▽ More
We tackle the problem of protein secondary structure prediction using a common task framework. This lead to the introduction of multiple ideas for neural architectures based on state of the art building blocks, used in this task for the first time. We take a principled machine learning approach, which provides genuine, unbiased performance measures, correcting longstanding errors in the application domain. We focus on the Q8 resolution of secondary structure, an active area for continuously improving methods. We use an ensemble of strong predictors to achieve accuracy of 70.7% (on the CB513 test set using the CB6133filtered training set). These results are statistically indistinguishable from those of the top existing predictors. In the spirit of reproducible research we make our data, models and code available, aiming to set a gold standard for purity of training and testing sets. Such good practices lower entry barriers to this domain and facilitate reproducible, extendable research.
△ Less
Submitted 17 November, 2018;
originally announced November 2018.
-
SketchParse : Towards Rich Descriptions for Poorly Drawn Sketches using Multi-Task Hierarchical Deep Networks
Authors:
Ravi Kiran Sarvadevabhatla,
Isht Dwivedi,
Abhijat Biswas,
Sahil Manocha,
R. Venkatesh Babu
Abstract:
The ability to semantically interpret hand-drawn line sketches, although very challenging, can pave way for novel applications in multimedia. We propose SketchParse, the first deep-network architecture for fully automatic parsing of freehand object sketches. SketchParse is configured as a two-level fully convolutional network. The first level contains shared layers common to all object categories.…
▽ More
The ability to semantically interpret hand-drawn line sketches, although very challenging, can pave way for novel applications in multimedia. We propose SketchParse, the first deep-network architecture for fully automatic parsing of freehand object sketches. SketchParse is configured as a two-level fully convolutional network. The first level contains shared layers common to all object categories. The second level contains a number of expert sub-networks. Each expert specializes in parsing sketches from object categories which contain structurally similar parts. Effectively, the two-level configuration enables our architecture to scale up efficiently as additional categories are added. We introduce a router layer which (i) relays sketch features from shared layers to the correct expert (ii) eliminates the need to manually specify object category during inference. To bypass laborious part-level annotation, we sketchify photos from semantic object-part image datasets and use them for training. Our architecture also incorporates object pose prediction as a novel auxiliary task which boosts overall performance while providing supplementary information regarding the sketch. We demonstrate SketchParse's abilities (i) on two challenging large-scale sketch datasets (ii) in parsing unseen, semantically related object categories (iii) in improving fine-grained sketch-based image retrieval. As a novel application, we also outline how SketchParse's output can be used to generate caption-style descriptions for hand-drawn sketches.
△ Less
Submitted 5 September, 2017;
originally announced September 2017.
-
Depletion of energy from Naked Singular regions during gravitational collapse
Authors:
S. Jhingan,
I. H. Dwivedi,
S. Barve
Abstract:
A distinguishable and observable physical property of Naked Singular Regions of the spacetime formed during a gravitational collapse has important implications for both experimental and theoretical relativity. We examine here whether energy can escape physically from naked singular regions to reach either a local or a distant observer within the framework of general relativity. We find that in cas…
▽ More
A distinguishable and observable physical property of Naked Singular Regions of the spacetime formed during a gravitational collapse has important implications for both experimental and theoretical relativity. We examine here whether energy can escape physically from naked singular regions to reach either a local or a distant observer within the framework of general relativity. We find that in case of imploding null dust collapse scenarios field outgoing singular null geodesics including the cauchy horizon can be immersed between two Vaidya spacetimes as null boundary layers with non vanishing positive energy density. Thus energy can transported from the naked singularity to either a local or a distant observer. And example illustrating that similar considerations can be applied to dust models is given.
△ Less
Submitted 7 June, 2011; v1 submitted 18 April, 2010;
originally announced April 2010.
-
Appearance of the central singularity in spherical collapse
Authors:
S. S. Deshingkar,
P. S. Joshi,
I. H. Dwivedi
Abstract:
We analyze here the structure of non-radial nonspacelike geodesics terminating in the past at a naked singularity formed as the end state of inhomogeneous dust collapse. The spectrum of outgoing nonspacelike null geodesics is examined analytically. The local and global visibility of the singularity is also examined by integrating numerically the null geodesics equations. The possible implication…
▽ More
We analyze here the structure of non-radial nonspacelike geodesics terminating in the past at a naked singularity formed as the end state of inhomogeneous dust collapse. The spectrum of outgoing nonspacelike null geodesics is examined analytically. The local and global visibility of the singularity is also examined by integrating numerically the null geodesics equations. The possible implications of existence of such families towards the appearance of the star in late stages of gravitational collapse are considered. It is seen that the outgoing non-radial geodesics give an appearance to the naked central singularity as that of an expanding ball whose radius reaches a maximum before the star goes within its apparent horizon. The radiated energy (along the null geodesics) is shown to decay very sharply in the neighbourhood of the singularity. Thus the total energy esca** via non-radial null geodesics from the naked central singularity vanishes in the scenario considered here.
△ Less
Submitted 16 November, 2001;
originally announced November 2001.
-
Naked Singularity, Black-hole and the Mass Loss in a Spherically Symmetric Gravitational Collapse
Authors:
I. H. Dwivedi
Abstract:
A distinguishable physical property between a naked singularity and a black-hole, formed during a gravitational collapse has important implications for both experimental and theoretical relativity. We examine the energy radiated during the spherically symmetric Gravitational collapse in this context within the framework of general relativity. It is shown that total energy radiated(Mass Loss) dur…
▽ More
A distinguishable physical property between a naked singularity and a black-hole, formed during a gravitational collapse has important implications for both experimental and theoretical relativity. We examine the energy radiated during the spherically symmetric Gravitational collapse in this context within the framework of general relativity. It is shown that total energy radiated(Mass Loss) during the collapse ending in a naked singularity scenario cannot be more than the case when a collapse scenario ends in a black-hole. In cases of interest(for example stars having same mass, size, and internal composition) considered here the total energy released in the collapse ending in a black hole can be considerably more than the case otherwise.
△ Less
Submitted 7 November, 2001;
originally announced November 2001.
-
A Tolman-Bondi-Lemaitre Cell-Model for the Universe and Gravitational Collapse
Authors:
A. Chamorro,
S. S. Deshingkar,
I. H. Dwivedi,
P. S. Joshi
Abstract:
A piecewise Tolman-Bondi-Lemaitre (TBL) cell-model for the universe incorporating local collapsing and expanding inhomogeneities is presented here. The cell-model is made up of TBL underdense and overdense spherical regions surrounded by an intermediate region of TBL shells embedded in an expanding universe. The cell-model generalizes the Friedmann as well as Einstein-Straus swiss-cheese models…
▽ More
A piecewise Tolman-Bondi-Lemaitre (TBL) cell-model for the universe incorporating local collapsing and expanding inhomogeneities is presented here. The cell-model is made up of TBL underdense and overdense spherical regions surrounded by an intermediate region of TBL shells embedded in an expanding universe. The cell-model generalizes the Friedmann as well as Einstein-Straus swiss-cheese models and presents a number of advantages over other models, and in particular the time evolution of the cosmological inhomogeneities is now incorporated within the scheme. Important problem of gravitational collapse of a massive dust cloud, such as a cluster of galaxies or even a massive star, in such a cosmological background is examined. It is shown that the collapsing local inhomogeneities in an expanding universe could result in either a black hole, or a naked singularity, depending on the nature of the set of initial data which consists of the matter distribution and the velocities of the collapsing shells in the cloud at the initial epoch from which the collapse commences.
△ Less
Submitted 18 October, 2000;
originally announced October 2000.
-
Physical nature of the central singularity in spherical collapse
Authors:
S. S. Deshingkar,
P. S. Joshi,
I. H. Dwivedi
Abstract:
We examine here the nature of the central singularity forming in the spherically symmetric collapse of a dust cloud and it is shown that this is always a strong curvature singularity where gravitational tidal forces diverge powerfully. An important consequence is that the nature of the naked singularity forming in the dust collapse turns out to be stable against the perturbations in the initial…
▽ More
We examine here the nature of the central singularity forming in the spherically symmetric collapse of a dust cloud and it is shown that this is always a strong curvature singularity where gravitational tidal forces diverge powerfully. An important consequence is that the nature of the naked singularity forming in the dust collapse turns out to be stable against the perturbations in the initial data from which the collapse commences.
△ Less
Submitted 5 November, 2001; v1 submitted 14 May, 1998;
originally announced May 1998.
-
Photon redshift and the appearance of a naked singularity
Authors:
I. H. Dwivedi
Abstract:
In this paper we analyze the redshift as observed by an external observer receiving photons which terminate in the past at the naked singularity formed in a Tolman-Bondi dust collapse. Within the context of models considered here it is shown that photons emitted from a weak curvature naked singularity are always finitely redshifted to an external observer. Certain cases of strong curvature naked…
▽ More
In this paper we analyze the redshift as observed by an external observer receiving photons which terminate in the past at the naked singularity formed in a Tolman-Bondi dust collapse. Within the context of models considered here it is shown that photons emitted from a weak curvature naked singularity are always finitely redshifted to an external observer. Certain cases of strong curvature naked singularities, including the self-similar one, where the photons are infinitely redshifted are also pointed out.
△ Less
Submitted 28 May, 1998; v1 submitted 9 May, 1998;
originally announced May 1998.
-
Initial data and the end state of spherically symmetric gravitational collapse
Authors:
P. S. Joshi,
I. H. Dwivedi
Abstract:
Generalizing earlier results on the initial data and the final fate of dust collapse, we study here the relevance of the initial state of a spherically symmetric matter cloud towards determining its end state in the course of a continuing gravitational collapse. It is shown that given an arbitrary regular distribution of matter at the initial epoch, there always exists an evolution from this ini…
▽ More
Generalizing earlier results on the initial data and the final fate of dust collapse, we study here the relevance of the initial state of a spherically symmetric matter cloud towards determining its end state in the course of a continuing gravitational collapse. It is shown that given an arbitrary regular distribution of matter at the initial epoch, there always exists an evolution from this initial data which would result either in a black hole or a naked singularity depending on the allowed choice of free functions available in the solution. It follows that given any initial density and pressure profiles for the cloud, there is a non-zero measure set of configurations leading either to black holes or naked singularities, subject to the usual energy conditions ensuring the positivity of energy density. We also characterize here wide new families of black hole solutions resulting from spherically symmetric collapse without requiring the cosmic censorship assumption.
△ Less
Submitted 28 April, 1998;
originally announced April 1998.
-
Field of a Radiation Distributuion
Authors:
I. H. Dwivedi
Abstract:
General relativistic spherically symmetric matter field with a vanishing stress energy scalar is analyzed. Procedure for generating exact solutions of the field equations for such matter distributions is given. It is further pointed out that all such type I spherically symmetric fields with distinct eignvalues in the radial two space can be treated as a mixture of isotropic and directed radiatio…
▽ More
General relativistic spherically symmetric matter field with a vanishing stress energy scalar is analyzed. Procedure for generating exact solutions of the field equations for such matter distributions is given. It is further pointed out that all such type I spherically symmetric fields with distinct eignvalues in the radial two space can be treated as a mixture of isotropic and directed radiations. Various classes of exact solutions are given. Junction conditions for such a matter field to the possible exterior solutions are also discussed.
△ Less
Submitted 20 June, 1998; v1 submitted 12 October, 1997;
originally announced October 1997.
-
Initial Data and the Final Fate of Inhomogeneous Dust Collapse
Authors:
I. H. Dwivedi,
P. S. Joshi
Abstract:
We examine here the relevance of the initial state of a collapsing dust cloud towards determining it's final fate in the course of a continuing gravitational collapse. It is shown that given any arbitrary matter distribution $M(r)$ for the cloud at the initial epoch, there is always a freedom to choose rest of the initial data, namely the initial velocities of the collapsing spherical shells, so…
▽ More
We examine here the relevance of the initial state of a collapsing dust cloud towards determining it's final fate in the course of a continuing gravitational collapse. It is shown that given any arbitrary matter distribution $M(r)$ for the cloud at the initial epoch, there is always a freedom to choose rest of the initial data, namely the initial velocities of the collapsing spherical shells, so that the collapse could result either in a black hole or a naked singularity depending on this choice. Thus, given the initial density profile, to achieve the desired end state of the gravitational collapse one has to give a suitable initial velocity to the cloud. We also characterize here a wide new family of black hole solutions resulting from inhomogeneous dust collapse. These configurations obey the usual energy conditions demanding the positivity of energy density.
△ Less
Submitted 10 December, 1996;
originally announced December 1996.
-
On the Occurrence of Naked Singularity in Spherically Symmetric Gravitational Collapse
Authors:
I. H. Dwivedi,
P. S. Joshi
Abstract:
Generalizing earlier results of Joshi and Dwivedi (Commun. Math. Phys. 146, 333 (1992); Lett. Math. Phys. 27, 235 (1993)), we analyze here the spherically symmetric gravitational collapse of a matter cloud with a general form of matter for the formation of a naked singularity. It is shown that this is related basically to the choice of initial data to the Einstein field equations, and would ther…
▽ More
Generalizing earlier results of Joshi and Dwivedi (Commun. Math. Phys. 146, 333 (1992); Lett. Math. Phys. 27, 235 (1993)), we analyze here the spherically symmetric gravitational collapse of a matter cloud with a general form of matter for the formation of a naked singularity. It is shown that this is related basically to the choice of initial data to the Einstein field equations, and would therefore occur in generic situations from regular initial data within the general context considered here, subject to the matter satisfying the weak energy condition. The condition on initial data which leads to the formation of black hole is also characterized.
△ Less
Submitted 23 May, 1994;
originally announced May 1994.
-
Naked Singularities in Spherically Symmetric Inhomogeneous Tolman-Bondi Dust Cloud Collapse
Authors:
P. S. Joshi,
I. H. Dwivedi
Abstract:
We investigate the occurrence and nature of naked singularity for the inhomogeneous gravitational collapse of Tolman-Bondi dust clouds.It is shown that the naked singularities form at the center of the collapsing cloud in a wide class of collapse models which includes the earlier cases considered by Eardley and Smarr and Christodoulou. This class also contains self-similar as well as non-self-si…
▽ More
We investigate the occurrence and nature of naked singularity for the inhomogeneous gravitational collapse of Tolman-Bondi dust clouds.It is shown that the naked singularities form at the center of the collapsing cloud in a wide class of collapse models which includes the earlier cases considered by Eardley and Smarr and Christodoulou. This class also contains self-similar as well as non-self-similar models. The structure and strength of this singularity is examined and the question is investigated as to when a non-zero measure set of non-spacelike trajectories could be emitted from the singularity as opposed to isolated trajectories coming out. It is seen that the weak energy condition and positivity of energy density ensures that the families of non-spacelike trajectories come out of the singularity. The curvature strength of the naked singularity is examined which provides an important test for its physical significance and powerful curvature growth near the naked singularity is pointed out for several subclasses considered. The conditions are discussed for the naked singularity to be globally naked. Implications for the basic issue of the final fate of gravitational collapse are considered once the inhomogeneities in the matter distribution are taken into account. It is argued that a physical formulation for the cosmic censorship may be evolved which avoids the features above. Possibilities in this direction are discussed while indicating that the analysis presented here should be useful for any possible rigorous formulation of the cosmic censorship hypothesis.
△ Less
Submitted 31 March, 1993;
originally announced March 1993.
-
The Structure of Naked Singularity in Self-similar Gravitational Collapse II
Authors:
P. S. Joshi,
I. H. Dwivedi
Abstract:
Generalizing the results of Joshi and Dwivedi in Commun.Math.Phys. 146, p.333 (1992), it is pointed out that strong curvature naked singularities could occur in the self-similar gravitational collapse of any form of matter satisfying the weak energy condition for the positivity of mass-energy density.
Generalizing the results of Joshi and Dwivedi in Commun.Math.Phys. 146, p.333 (1992), it is pointed out that strong curvature naked singularities could occur in the self-similar gravitational collapse of any form of matter satisfying the weak energy condition for the positivity of mass-energy density.
△ Less
Submitted 9 February, 1993;
originally announced February 1993.
