-
Keypoint Communities
Authors:
Duncan Zauss,
Sven Kreiss,
Alexandre Alahi
Abstract:
We present a fast bottom-up method that jointly detects over 100 keypoints on humans or objects, also referred to as human/object pose estimation. We model all keypoints belonging to a human or an object -- the pose -- as a graph and leverage insights from community detection to quantify the independence of keypoints. We use a graph centrality measure to assign training weights to different parts…
▽ More
We present a fast bottom-up method that jointly detects over 100 keypoints on humans or objects, also referred to as human/object pose estimation. We model all keypoints belonging to a human or an object -- the pose -- as a graph and leverage insights from community detection to quantify the independence of keypoints. We use a graph centrality measure to assign training weights to different parts of a pose. Our proposed measure quantifies how tightly a keypoint is connected to its neighborhood. Our experiments show that our method outperforms all previous methods for human pose estimation with fine-grained keypoint annotations on the face, the hands and the feet with a total of 133 keypoints. We also show that our method generalizes to car poses.
△ Less
Submitted 3 October, 2021;
originally announced October 2021.
-
Deep Social Force
Authors:
Sven Kreiss
Abstract:
The Social Force model introduced by Helbing and Molnar in 1995 is a cornerstone of pedestrian simulation. This paper introduces a differentiable simulation of the Social Force model where the assumptions on the shapes of interaction potentials are relaxed with the use of universal function approximators in the form of neural networks. Classical force-based pedestrian simulations suffer from unnat…
▽ More
The Social Force model introduced by Helbing and Molnar in 1995 is a cornerstone of pedestrian simulation. This paper introduces a differentiable simulation of the Social Force model where the assumptions on the shapes of interaction potentials are relaxed with the use of universal function approximators in the form of neural networks. Classical force-based pedestrian simulations suffer from unnatural locking behavior on head-on collision paths. In addition, they cannot model the bias of pedestrians to avoid each other on the right or left depending on the geographic region. My experiments with more general interaction potentials show that potentials with a sharp tip in the front avoid locking. In addition, asymmetric interaction potentials lead to a left or right bias when pedestrians avoid each other.
△ Less
Submitted 24 September, 2021;
originally announced September 2021.
-
OpenPifPaf: Composite Fields for Semantic Keypoint Detection and Spatio-Temporal Association
Authors:
Sven Kreiss,
Lorenzo Bertoni,
Alexandre Alahi
Abstract:
Many image-based perception tasks can be formulated as detecting, associating and tracking semantic keypoints, e.g., human body pose estimation and tracking. In this work, we present a general framework that jointly detects and forms spatio-temporal keypoint associations in a single stage, making this the first real-time pose detection and tracking algorithm. We present a generic neural network ar…
▽ More
Many image-based perception tasks can be formulated as detecting, associating and tracking semantic keypoints, e.g., human body pose estimation and tracking. In this work, we present a general framework that jointly detects and forms spatio-temporal keypoint associations in a single stage, making this the first real-time pose detection and tracking algorithm. We present a generic neural network architecture that uses Composite Fields to detect and construct a spatio-temporal pose which is a single, connected graph whose nodes are the semantic keypoints (e.g., a person's body joints) in multiple frames. For the temporal associations, we introduce the Temporal Composite Association Field (TCAF) which requires an extended network architecture and training method beyond previous Composite Fields. Our experiments show competitive accuracy while being an order of magnitude faster on multiple publicly available datasets such as COCO, CrowdPose and the PoseTrack 2017 and 2018 datasets. We also show that our method generalizes to any class of semantic keypoints such as car and animal parts to provide a holistic perception framework that is well suited for urban mobility such as self-driving cars and delivery robots.
△ Less
Submitted 21 September, 2021; v1 submitted 3 March, 2021;
originally announced March 2021.
-
Perceiving Traffic from Aerial Images
Authors:
George Adaimi,
Sven Kreiss,
Alexandre Alahi
Abstract:
Drones or UAVs, equipped with different sensors, have been deployed in many places especially for urban traffic monitoring or last-mile delivery. It provides the ability to control the different aspects of traffic given real-time obeservations, an important pillar for the future of transportation and smart cities. With the increasing use of such machines, many previous state-of-the-art object dete…
▽ More
Drones or UAVs, equipped with different sensors, have been deployed in many places especially for urban traffic monitoring or last-mile delivery. It provides the ability to control the different aspects of traffic given real-time obeservations, an important pillar for the future of transportation and smart cities. With the increasing use of such machines, many previous state-of-the-art object detectors, who have achieved high performance on front facing cameras, are being used on UAV datasets. When applied to high-resolution aerial images captured from such datasets, they fail to generalize to the wide range of objects' scales. In order to address this limitation, we propose an object detection method called Butterfly Detector that is tailored to detect objects in aerial images. We extend the concept of fields and introduce butterfly fields, a type of composite field that describes the spatial information of output features as well as the scale of the detected object. To overcome occlusion and viewing angle variations that can hinder the localization process, we employ a voting mechanism between related butterfly vectors pointing to the object center. We evaluate our Butterfly Detector on two publicly available UAV datasets (UAVDT and VisDrone2019) and show that it outperforms previous state-of-the-art methods while remaining real-time.
△ Less
Submitted 16 September, 2020;
originally announced September 2020.
-
Perceiving Humans: from Monocular 3D Localization to Social Distancing
Authors:
Lorenzo Bertoni,
Sven Kreiss,
Alexandre Alahi
Abstract:
Perceiving humans in the context of Intelligent Transportation Systems (ITS) often relies on multiple cameras or expensive LiDAR sensors. In this work, we present a new cost-effective vision-based method that perceives humans' locations in 3D and their body orientation from a single image. We address the challenges related to the ill-posed monocular 3D tasks by proposing a neural network architect…
▽ More
Perceiving humans in the context of Intelligent Transportation Systems (ITS) often relies on multiple cameras or expensive LiDAR sensors. In this work, we present a new cost-effective vision-based method that perceives humans' locations in 3D and their body orientation from a single image. We address the challenges related to the ill-posed monocular 3D tasks by proposing a neural network architecture that predicts confidence intervals in contrast to point estimates. Our neural network estimates human 3D body locations and their orientation with a measure of uncertainty. Our proposed solution (i) is privacy-safe, (ii) works with any fixed or moving cameras, and (iii) does not rely on ground plane estimation. We demonstrate the performance of our method with respect to three applications: locating humans in 3D, detecting social interactions, and verifying the compliance of recent safety measures due to the COVID-19 outbreak. We show that it is possible to rethink the concept of "social distancing" as a form of social interaction in contrast to a simple location-based rule. We publicly share the source code towards an open science mission.
△ Less
Submitted 24 March, 2021; v1 submitted 1 September, 2020;
originally announced September 2020.
-
MonStereo: When Monocular and Stereo Meet at the Tail of 3D Human Localization
Authors:
Lorenzo Bertoni,
Sven Kreiss,
Taylor Mordan,
Alexandre Alahi
Abstract:
Monocular and stereo visions are cost-effective solutions for 3D human localization in the context of self-driving cars or social robots. However, they are usually developed independently and have their respective strengths and limitations. We propose a novel unified learning framework that leverages the strengths of both monocular and stereo cues for 3D human localization. Our method jointly (i)…
▽ More
Monocular and stereo visions are cost-effective solutions for 3D human localization in the context of self-driving cars or social robots. However, they are usually developed independently and have their respective strengths and limitations. We propose a novel unified learning framework that leverages the strengths of both monocular and stereo cues for 3D human localization. Our method jointly (i) associates humans in left-right images, (ii) deals with occluded and distant cases in stereo settings by relying on the robustness of monocular cues, and (iii) tackles the intrinsic ambiguity of monocular perspective projection by exploiting prior knowledge of the human height distribution. We specifically evaluate outliers as well as challenging instances, such as occluded and far-away pedestrians, by analyzing the entire error distribution and by estimating calibrated confidence intervals. Finally, we critically review the official KITTI 3D metrics and propose a practical 3D localization metric tailored for humans.
△ Less
Submitted 22 March, 2021; v1 submitted 25 August, 2020;
originally announced August 2020.
-
Human Trajectory Forecasting in Crowds: A Deep Learning Perspective
Authors:
Parth Kothari,
Sven Kreiss,
Alexandre Alahi
Abstract:
Since the past few decades, human trajectory forecasting has been a field of active research owing to its numerous real-world applications: evacuation situation analysis, deployment of intelligent transport systems, traffic operations, to name a few. Early works handcrafted this representation based on domain knowledge. However, social interactions in crowded environments are not only diverse but…
▽ More
Since the past few decades, human trajectory forecasting has been a field of active research owing to its numerous real-world applications: evacuation situation analysis, deployment of intelligent transport systems, traffic operations, to name a few. Early works handcrafted this representation based on domain knowledge. However, social interactions in crowded environments are not only diverse but often subtle. Recently, deep learning methods have outperformed their handcrafted counterparts, as they learned about human-human interactions in a more generic data-driven fashion. In this work, we present an in-depth analysis of existing deep learning-based methods for modelling social interactions. We propose two knowledge-based data-driven methods to effectively capture these social interactions. To objectively compare the performance of these interaction-based forecasting models, we develop a large scale interaction-centric benchmark TrajNet++, a significant yet missing component in the field of human trajectory forecasting. We propose novel performance metrics that evaluate the ability of a model to output socially acceptable trajectories. Experiments on TrajNet++ validate the need for our proposed metrics, and our method outperforms competitive baselines on both real-world and synthetic datasets.
△ Less
Submitted 11 January, 2021; v1 submitted 7 July, 2020;
originally announced July 2020.
-
MonoLoco: Monocular 3D Pedestrian Localization and Uncertainty Estimation
Authors:
Lorenzo Bertoni,
Sven Kreiss,
Alexandre Alahi
Abstract:
We tackle the fundamentally ill-posed problem of 3D human localization from monocular RGB images. Driven by the limitation of neural networks outputting point estimates, we address the ambiguity in the task by predicting confidence intervals through a loss function based on the Laplace distribution. Our architecture is a light-weight feed-forward neural network that predicts 3D locations and corre…
▽ More
We tackle the fundamentally ill-posed problem of 3D human localization from monocular RGB images. Driven by the limitation of neural networks outputting point estimates, we address the ambiguity in the task by predicting confidence intervals through a loss function based on the Laplace distribution. Our architecture is a light-weight feed-forward neural network that predicts 3D locations and corresponding confidence intervals given 2D human poses. The design is particularly well suited for small training data, cross-dataset generalization, and real-time applications. Our experiments show that we (i) outperform state-of-the-art results on KITTI and nuScenes datasets, (ii) even outperform a stereo-based method for far-away pedestrians, and (iii) estimate meaningful confidence intervals. We further share insights on our model of uncertainty in cases of limited observations and out-of-distribution samples.
△ Less
Submitted 20 August, 2019; v1 submitted 14 June, 2019;
originally announced June 2019.
-
Deep Visual Re-Identification with Confidence
Authors:
George Adaimi,
Sven Kreiss,
Alexandre Alahi
Abstract:
Transportation systems often rely on understanding the flow of vehicles or pedestrian. From traffic monitoring at the city scale, to commuters in train terminals, recent progress in sensing technology make it possible to use cameras to better understand the demand, i.e., better track moving agents (e.g., vehicles and pedestrians). Whether the cameras are mounted on drones, vehicles, or fixed in th…
▽ More
Transportation systems often rely on understanding the flow of vehicles or pedestrian. From traffic monitoring at the city scale, to commuters in train terminals, recent progress in sensing technology make it possible to use cameras to better understand the demand, i.e., better track moving agents (e.g., vehicles and pedestrians). Whether the cameras are mounted on drones, vehicles, or fixed in the built environments, they inevitably remain scatter. We need to develop the technology to re-identify the same agents across images captured from non-overlap** field-of-views, referred to as the visual re-identification task. State-of-the-art methods learn a neural network based representation trained with the cross-entropy loss function. We argue that such loss function is not suited for the visual re-identification task hence propose to model confidence in the representation learning framework. We show the impact of our confidence-based learning framework with three methods: label smoothing, confidence penalty, and deep variational information bottleneck. They all show a boost in performance validating our claim. Our contribution is generic to any agent of interest, i.e., vehicles or pedestrians, and outperform highly specialized state-of-the-art methods across 5 datasets. The source code and models are shared towards an open science mission.
△ Less
Submitted 10 September, 2020; v1 submitted 11 June, 2019;
originally announced June 2019.
-
PifPaf: Composite Fields for Human Pose Estimation
Authors:
Sven Kreiss,
Lorenzo Bertoni,
Alexandre Alahi
Abstract:
We propose a new bottom-up method for multi-person 2D human pose estimation that is particularly well suited for urban mobility such as self-driving cars and delivery robots. The new method, PifPaf, uses a Part Intensity Field (PIF) to localize body parts and a Part Association Field (PAF) to associate body parts with each other to form full human poses. Our method outperforms previous methods at…
▽ More
We propose a new bottom-up method for multi-person 2D human pose estimation that is particularly well suited for urban mobility such as self-driving cars and delivery robots. The new method, PifPaf, uses a Part Intensity Field (PIF) to localize body parts and a Part Association Field (PAF) to associate body parts with each other to form full human poses. Our method outperforms previous methods at low resolution and in crowded, cluttered and occluded scenes thanks to (i) our new composite field PAF encoding fine-grained information and (ii) the choice of Laplace loss for regressions which incorporates a notion of uncertainty. Our architecture is based on a fully convolutional, single-shot, box-free design. We perform on par with the existing state-of-the-art bottom-up method on the standard COCO keypoint task and produce state-of-the-art results on a modified COCO keypoint task for the transportation domain.
△ Less
Submitted 5 April, 2019; v1 submitted 15 March, 2019;
originally announced March 2019.
-
Crowd-Robot Interaction: Crowd-aware Robot Navigation with Attention-based Deep Reinforcement Learning
Authors:
Changan Chen,
Yuejiang Liu,
Sven Kreiss,
Alexandre Alahi
Abstract:
Mobility in an effective and socially-compliant manner is an essential yet challenging task for robots operating in crowded spaces. Recent works have shown the power of deep reinforcement learning techniques to learn socially cooperative policies. However, their cooperation ability deteriorates as the crowd grows since they typically relax the problem as a one-way Human-Robot interaction problem.…
▽ More
Mobility in an effective and socially-compliant manner is an essential yet challenging task for robots operating in crowded spaces. Recent works have shown the power of deep reinforcement learning techniques to learn socially cooperative policies. However, their cooperation ability deteriorates as the crowd grows since they typically relax the problem as a one-way Human-Robot interaction problem. In this work, we want to go beyond first-order Human-Robot interaction and more explicitly model Crowd-Robot Interaction (CRI). We propose to (i) rethink pairwise interactions with a self-attention mechanism, and (ii) jointly model Human-Robot as well as Human-Human interactions in the deep reinforcement learning framework. Our model captures the Human-Human interactions occurring in dense crowds that indirectly affects the robot's anticipation capability. Our proposed attentive pooling mechanism learns the collective importance of neighboring humans with respect to their future states. Various experiments demonstrate that our model can anticipate human dynamics and navigate in crowds with time efficiency, outperforming state-of-the-art methods.
△ Less
Submitted 19 February, 2019; v1 submitted 24 September, 2018;
originally announced September 2018.
-
Decoupling Theoretical Uncertainties from Measurements of the Higgs Boson
Authors:
Kyle Cranmer,
Sven Kreiss,
David Lopez-Val,
Tilman Plehn
Abstract:
We develop a technique to present Higgs coupling measurements, which decouple the poorly defined theoretical uncertainties associated to inclusive and exclusive cross section predictions. The technique simplifies the combination of multiple measurements and can be used in a more general setting. We illustrate the approach with toy LHC Higgs coupling measurements and a collection of new physics mod…
▽ More
We develop a technique to present Higgs coupling measurements, which decouple the poorly defined theoretical uncertainties associated to inclusive and exclusive cross section predictions. The technique simplifies the combination of multiple measurements and can be used in a more general setting. We illustrate the approach with toy LHC Higgs coupling measurements and a collection of new physics models.
△ Less
Submitted 1 April, 2015; v1 submitted 30 December, 2013;
originally announced January 2014.
-
Les Houches 2011: Physics at TeV Colliders New Physics Working Group Report
Authors:
G. Brooijmans,
B. Gripaios,
F. Moortgat,
J. Santiago,
P. Skands,
D. Albornoz Vásquez,
B. C. Allanach,
A. Alloul,
A. Arbey,
A. Azatov,
H. Baer,
C. Balázs,
A. Barr,
L. Basso,
M. Battaglia,
P. Bechtle,
G. Bélanger,
A. Belyaev,
K. Benslama,
L. Bergström,
A. Bharucha,
C. Boehm,
M. Bondarenko,
O. Bondu,
E. Boos
, et al. (119 additional authors not shown)
Abstract:
We present the activities of the "New Physics" working group for the "Physics at TeV Colliders" workshop (Les Houches, France, 30 May-17 June, 2011). Our report includes new agreements on formats for interfaces between computational tools, new tool developments, important signatures for searches at the LHC, recommendations for presentation of LHC search results, as well as additional phenomenologi…
▽ More
We present the activities of the "New Physics" working group for the "Physics at TeV Colliders" workshop (Les Houches, France, 30 May-17 June, 2011). Our report includes new agreements on formats for interfaces between computational tools, new tool developments, important signatures for searches at the LHC, recommendations for presentation of LHC search results, as well as additional phenomenological studies.
△ Less
Submitted 20 April, 2012; v1 submitted 7 March, 2012;
originally announced March 2012.
-
The RooStats Project
Authors:
Lorenzo Moneta,
Kevin Belasco,
Kyle Cranmer,
Sven Kreiss,
Alfio Lazzaro,
Danilo Piparo,
Gregory Schott,
Wouter Verkerke,
Matthias Wolf
Abstract:
RooStats is a project to create advanced statistical tools required for the analysis of LHC data, with emphasis on discoveries, confidence intervals, and combined measurements. The idea is to provide the major statistical techniques as a set of C++ classes with coherent interfaces, so that can be used on arbitrary model and datasets in a common way. The classes are built on top of the RooFit packa…
▽ More
RooStats is a project to create advanced statistical tools required for the analysis of LHC data, with emphasis on discoveries, confidence intervals, and combined measurements. The idea is to provide the major statistical techniques as a set of C++ classes with coherent interfaces, so that can be used on arbitrary model and datasets in a common way. The classes are built on top of the RooFit package, which provides functionality for easily creating probability models, for analysis combinations and for digital publications of the results. We will present in detail the design and the implementation of the different statistical methods of RooStats. We will describe the various classes for interval estimation and for hypothesis test depending on different statistical techniques such as those based on the likelihood function, or on frequentists or bayesian statistics. These methods can be applied in complex problems, including cases with multiple parameters of interest and various nuisance parameters.
△ Less
Submitted 1 February, 2011; v1 submitted 6 September, 2010;
originally announced September 2010.
-
Physics Beyond the Standard Model: Supersymmetry
Authors:
M. M. Nojiri,
T. Plehn,
G. Polesello,
M. Alexander,
B. C. Allanach,
A. J. Barr,
K. Benakli,
F. Boudjema,
A. Freitas,
C. Gwenlan,
S. Jager,
S. Kraml,
S. Kreiss,
R. Lafaye,
C. G. Lester,
N. Kauer,
C. Milstene,
C. Moura,
G. S. Muanza,
A. R. Raklev,
M. Rauch,
M. Schmitt,
S. Sekmen,
P. Skands,
P. Slavich
, et al. (8 additional authors not shown)
Abstract:
This collection of studies on new physics at the LHC constitutes the report of the supersymmetry working group at the Workshop `Physics at TeV Colliders', Les Houches, France, 2007. They cover the wide spectrum of phenomenology in the LHC era, from alternative models and signatures to the extraction of relevant observables, the study of the MSSM parameter space and finally to the interplay of LH…
▽ More
This collection of studies on new physics at the LHC constitutes the report of the supersymmetry working group at the Workshop `Physics at TeV Colliders', Les Houches, France, 2007. They cover the wide spectrum of phenomenology in the LHC era, from alternative models and signatures to the extraction of relevant observables, the study of the MSSM parameter space and finally to the interplay of LHC observations with additional data expected on a similar time scale. The special feature of this collection is that while not each of the studies is explicitely performed together by theoretical and experimental LHC physicists, all of them were inspired by and discussed in this particular environment.
△ Less
Submitted 25 February, 2008;
originally announced February 2008.
-
SUSY Les Houches Accord 2
Authors:
B. C. Allanach,
C. Balazs,
G. Belanger,
M. Bernhardt,
F. Boudjema,
D. Choudhury,
K. Desch,
U. Ellwanger,
P. Gambino,
R. Godbole,
T. Goto,
J. Guasch,
M. Guchait,
T. Hahn,
S. Heinemeyer,
C. Hugonie,
T. Hurth,
S. Kraml S. Kreiss,
J. Lykken,
F. Moortgat,
S. Moretti,
S. Penaranda,
T. Plehn,
W. Porod,
A. Pukhov
, et al. (8 additional authors not shown)
Abstract:
The Supersymmetry Les Houches Accord (SLHA) provides a universal set of conventions for conveying spectral and decay information for supersymmetry analysis problems in high energy physics. Here, we propose extensions of the conventions of the first SLHA to include various generalisations: the minimal supersymmetric standard model with violation of CP, R-parity, and flavour, as well as the simple…
▽ More
The Supersymmetry Les Houches Accord (SLHA) provides a universal set of conventions for conveying spectral and decay information for supersymmetry analysis problems in high energy physics. Here, we propose extensions of the conventions of the first SLHA to include various generalisations: the minimal supersymmetric standard model with violation of CP, R-parity, and flavour, as well as the simplest next-to-minimal model.
△ Less
Submitted 22 November, 2009; v1 submitted 29 December, 2007;
originally announced January 2008.
