-
Quantifying Heterogeneous Ecosystem Services With Multi-Label Soft Classification
Authors:
Zhihui Tian,
John Upchurch,
G. Austin Simon,
José Dubeux,
Alina Zare,
Chang Zhao,
Joel B. Harley
Abstract:
Understanding and quantifying ecosystem services are crucial for sustainable environmental management, conservation efforts, and policy-making. The advancement of remote sensing technology and machine learning techniques has greatly facilitated this process. Yet, ground truth labels, such as biodiversity, are very difficult and expensive to measure. In addition, more easily obtainable proxy labels…
▽ More
Understanding and quantifying ecosystem services are crucial for sustainable environmental management, conservation efforts, and policy-making. The advancement of remote sensing technology and machine learning techniques has greatly facilitated this process. Yet, ground truth labels, such as biodiversity, are very difficult and expensive to measure. In addition, more easily obtainable proxy labels, such as land use, often fail to capture the complex heterogeneity of the ecosystem. In this paper, we demonstrate how land use proxy labels can be implemented with a soft, multi-label classifier to predict ecosystem services with complex heterogeneity.
△ Less
Submitted 24 June, 2024;
originally announced June 2024.
-
Convex Hull Thrackles
Authors:
Balázs Keszegh,
Dániel Simon
Abstract:
A \emph{thrackle} is a graph drawn in the plane so that every pair of its edges meet exactly once, either at a common end vertex or in a proper crossing. Conway's thrackle conjecture states that the number of edges is at most the number of vertices. It is known that this conjecture holds for linear thrackles, i.e., when the edges are drawn as straight line segments.
We consider \emph{convex hull…
▽ More
A \emph{thrackle} is a graph drawn in the plane so that every pair of its edges meet exactly once, either at a common end vertex or in a proper crossing. Conway's thrackle conjecture states that the number of edges is at most the number of vertices. It is known that this conjecture holds for linear thrackles, i.e., when the edges are drawn as straight line segments.
We consider \emph{convex hull thrackles}, a recent generalization of linear thrackles from segments to convex hulls of subsets of points. We prove that if the points are in convex position then the number of convex hulls is at most the number of vertices, but in general there is a construction with one more convex hull. On the other hand, we prove that the number of convex hulls is always at most twice the number of vertices.
△ Less
Submitted 6 July, 2023;
originally announced July 2023.
-
OA-SLAM: Leveraging Objects for Camera Relocalization in Visual SLAM
Authors:
Matthieu Zins,
Gilles Simon,
Marie-Odile Berger
Abstract:
In this work, we explore the use of objects in Simultaneous Localization and Map** in unseen worlds and propose an object-aided system (OA-SLAM). More precisely, we show that, compared to low-level points, the major benefit of objects lies in their higher-level semantic and discriminating power. Points, on the contrary, have a better spatial localization accuracy than the generic coarse models u…
▽ More
In this work, we explore the use of objects in Simultaneous Localization and Map** in unseen worlds and propose an object-aided system (OA-SLAM). More precisely, we show that, compared to low-level points, the major benefit of objects lies in their higher-level semantic and discriminating power. Points, on the contrary, have a better spatial localization accuracy than the generic coarse models used to represent objects (cuboid or ellipsoid). We show that combining points and objects is of great interest to address the problem of camera pose recovery. Our main contributions are: (1) we improve the relocalization ability of a SLAM system using high-level object landmarks; (2) we build an automatic system, capable of identifying, tracking and reconstructing objects with 3D ellipsoids; (3) we show that object-based localization can be used to reinitialize or resume camera tracking. Our fully automatic system allows on-the-fly object map** and enhanced pose tracking recovery, which we think, can significantly benefit to the AR community. Our experiments show that the camera can be relocalized from viewpoints where classical methods fail. We demonstrate that this localization allows a SLAM system to continue working despite a tracking loss, which can happen frequently with an uninitiated user. Our code and test data are released at gitlab.inria.fr/tangram/oa-slam.
△ Less
Submitted 17 September, 2022;
originally announced September 2022.
-
Perspective-1-Ellipsoid: Formulation, Analysis and Solutions of the Camera Pose Estimation Problem from One Ellipse-Ellipsoid Correspondence
Authors:
Vincent Gaudillière,
Gilles Simon,
Marie-Odile Berger
Abstract:
In computer vision, camera pose estimation from correspondences between 3D geometric entities and their projections into the image has been a widely investigated problem. Although most state-of-the-art methods exploit low-level primitives such as points or lines, the emergence of very effective CNN-based object detectors in the recent years has paved the way to the use of higher-level features car…
▽ More
In computer vision, camera pose estimation from correspondences between 3D geometric entities and their projections into the image has been a widely investigated problem. Although most state-of-the-art methods exploit low-level primitives such as points or lines, the emergence of very effective CNN-based object detectors in the recent years has paved the way to the use of higher-level features carrying semantically meaningful information. Pioneering works in that direction have shown that modelling 3D objects by ellipsoids and 2D detections by ellipses offers a convenient manner to link 2D and 3D data. However, the mathematical formalism most often used in the related litterature does not enable to easily distinguish ellipsoids and ellipses from other quadrics and conics, leading to a loss of specificity potentially detrimental in some developments. Moreover, the linearization process of the projection equation creates an over-representation of the camera parameters, also possibly causing an efficiency loss. In this paper, we therefore introduce an ellipsoid-specific theoretical framework and demonstrate its beneficial properties in the context of pose estimation. More precisely, we first show that the proposed formalism enables to reduce the pose estimation problem to a position or orientation-only estimation problem in which the remaining unknowns can be derived in closed-form. Then, we demonstrate that it can be further reduced to a 1 Degree-of-Freedom (1DoF) problem and provide the analytical derivations of the pose as a function of that unique scalar unknown. We illustrate our theoretical considerations by visual examples and include a discussion on the practical aspects. Finally, we release this paper along with the corresponding source code in order to contribute towards more efficient resolutions of ellipsoid-related pose estimation problems.
△ Less
Submitted 14 June, 2023; v1 submitted 26 August, 2022;
originally announced August 2022.
-
Level Set-Based Camera Pose Estimation From Multiple 2D/3D Ellipse-Ellipsoid Correspondences
Authors:
Matthieu Zins,
Gilles Simon,
Marie-Odile Berger
Abstract:
In this paper, we propose an object-based camera pose estimation from a single RGB image and a pre-built map of objects, represented with ellipsoidal models. We show that contrary to point correspondences, the definition of a cost function characterizing the projection of a 3D object onto a 2D object detection is not straightforward. We develop an ellipse-ellipse cost based on level sets sampling,…
▽ More
In this paper, we propose an object-based camera pose estimation from a single RGB image and a pre-built map of objects, represented with ellipsoidal models. We show that contrary to point correspondences, the definition of a cost function characterizing the projection of a 3D object onto a 2D object detection is not straightforward. We develop an ellipse-ellipse cost based on level sets sampling, demonstrate its nice properties for handling partially visible objects and compare its performance with other common metrics. Finally, we show that the use of a predictive uncertainty on the detected ellipses allows a fair weighting of the contribution of the correspondences which improves the computed pose. The code is released at https://gitlab.inria.fr/tangram/level-set-based-camera-pose-estimation.
△ Less
Submitted 19 August, 2022; v1 submitted 16 July, 2022;
originally announced July 2022.
-
Object-Based Visual Camera Pose Estimation From Ellipsoidal Model and 3D-Aware Ellipse Prediction
Authors:
Matthieu Zins,
Gilles Simon,
Marie-Odile Berger
Abstract:
In this paper, we propose a method for initial camera pose estimation from just a single image which is robust to viewing conditions and does not require a detailed model of the scene. This method meets the growing need of easy deployment of robotics or augmented reality applications in any environments, especially those for which no accurate 3D model nor huge amount of ground truth data are avail…
▽ More
In this paper, we propose a method for initial camera pose estimation from just a single image which is robust to viewing conditions and does not require a detailed model of the scene. This method meets the growing need of easy deployment of robotics or augmented reality applications in any environments, especially those for which no accurate 3D model nor huge amount of ground truth data are available. It exploits the ability of deep learning techniques to reliably detect objects regardless of viewing conditions. Previous works have also shown that abstracting the geometry of a scene of objects by an ellipsoid cloud allows to compute the camera pose accurately enough for various application needs. Though promising, these approaches use the ellipses fitted to the detection bounding boxes as an approximation of the imaged objects. In this paper, we go one step further and propose a learning-based method which detects improved elliptic approximations of objects which are coherent with the 3D ellipsoids in terms of perspective projection. Experiments prove that the accuracy of the computed pose significantly increases thanks to our method. This is achieved with very little effort in terms of training data acquisition - a few hundred calibrated images of which only three need manual object annotation. Code and models are released at https://gitlab.inria.fr/tangram/3d-aware-ellipses-for-visual-localization
△ Less
Submitted 9 March, 2022;
originally announced March 2022.
-
Predicting Cancer Treatments Induced Cardiotoxicity of Breast Cancer Patients
Authors:
Sicheng Zhou,
Rui Zhang,
Anne Blaes,
Chetan Shenoy,
Gyorgy Simon
Abstract:
Cardiotoxicity induced by the breast cancer treatments (i.e., chemotherapy, targeted therapy and radiation therapy) is a significant problem for breast cancer patients. The cardiotoxicity risk for breast cancer patients receiving different treatments remains unclear. We developed and evaluated risk predictive models for cardiotoxicity in breast cancer patients using EHR data. The AUC scores to pre…
▽ More
Cardiotoxicity induced by the breast cancer treatments (i.e., chemotherapy, targeted therapy and radiation therapy) is a significant problem for breast cancer patients. The cardiotoxicity risk for breast cancer patients receiving different treatments remains unclear. We developed and evaluated risk predictive models for cardiotoxicity in breast cancer patients using EHR data. The AUC scores to predict the CHF, CAD, CM and MI are 0.846, 0.857, 0.858 and 0.804 respectively. After adjusting for baseline differences in cardiovascular health, patients who received chemotherapy or targeted therapy appeared to have higher risk of cardiotoxicity than patients who received radiation therapy. Due to differences in baseline cardiac health across the different breast cancer treatment groups, caution is recommended in interpreting the cardiotoxic effect of these treatments.
△ Less
Submitted 31 January, 2022;
originally announced January 2022.
-
STraTA: Self-Training with Task Augmentation for Better Few-shot Learning
Authors:
Tu Vu,
Minh-Thang Luong,
Quoc V. Le,
Grady Simon,
Mohit Iyyer
Abstract:
Despite their recent successes in tackling many NLP tasks, large-scale pre-trained language models do not perform as well in few-shot settings where only a handful of training examples are available. To address this shortcoming, we propose STraTA, which stands for Self-Training with Task Augmentation, an approach that builds on two key ideas for effective leverage of unlabeled data. First, STraTA…
▽ More
Despite their recent successes in tackling many NLP tasks, large-scale pre-trained language models do not perform as well in few-shot settings where only a handful of training examples are available. To address this shortcoming, we propose STraTA, which stands for Self-Training with Task Augmentation, an approach that builds on two key ideas for effective leverage of unlabeled data. First, STraTA uses task augmentation, a novel technique that synthesizes a large amount of data for auxiliary-task fine-tuning from target-task unlabeled texts. Second, STraTA performs self-training by further fine-tuning the strong base model created by task augmentation on a broad distribution of pseudo-labeled data. Our experiments demonstrate that STraTA can substantially improve sample efficiency across 12 few-shot benchmarks. Remarkably, on the SST-2 sentiment dataset, STraTA, with only 8 training examples per class, achieves comparable results to standard fine-tuning with 67K training examples. Our analyses reveal that task augmentation and self-training are both complementary and independently effective.
△ Less
Submitted 12 April, 2022; v1 submitted 13 September, 2021;
originally announced September 2021.
-
FENXI: Deep-learning Traffic Analytics at the Edge
Authors:
Massimo Gallo,
Alessandro Finamore,
Gwendal Simon,
Dario Rossi
Abstract:
Live traffic analysis at the first aggregation point in the ISP network enables the implementation of complex traffic engineering policies but is limited by the scarce processing capabilities, especially for Deep Learning (DL) based analytics. The introduction of specialized hardware accelerators i.e., Tensor Processing Unit (TPU), offers the opportunity to enhance the processing capabilities of n…
▽ More
Live traffic analysis at the first aggregation point in the ISP network enables the implementation of complex traffic engineering policies but is limited by the scarce processing capabilities, especially for Deep Learning (DL) based analytics. The introduction of specialized hardware accelerators i.e., Tensor Processing Unit (TPU), offers the opportunity to enhance the processing capabilities of network devices at the edge. Yet, to date, no packet processing pipeline is capable of offering DL-based analysis capabilities in the data-plane, without interfering with network operations.
In this paper, we present FENXI, a system to run complex analytics by leveraging TPU. The design of FENXI decouples forwarding operations and traffic analytics which operates at different granularities i.e., packet and flow levels. We conceive two independent modules that asynchronously communicate to exchange network data and analytics results, and design data structures to extract flow level statistics without impacting per-packet processing. We prototyped and evaluated FENXI on general-purpose servers considering both adversarial and realistic network conditions. Our analysis shows that FENXI can sustain 100 Gbps line rate traffic processing requiring only limited resources, while also dynamically adapting to variable network conditions.
△ Less
Submitted 25 May, 2021;
originally announced May 2021.
-
3D-Aware Ellipse Prediction for Object-Based Camera Pose Estimation
Authors:
Matthieu Zins,
Gilles Simon,
Marie-Odile Berger
Abstract:
In this paper, we propose a method for coarse camera pose computation which is robust to viewing conditions and does not require a detailed model of the scene. This method meets the growing need of easy deployment of robotics or augmented reality applications in any environments, especially those for which no accurate 3D model nor huge amount of ground truth data are available. It exploits the abi…
▽ More
In this paper, we propose a method for coarse camera pose computation which is robust to viewing conditions and does not require a detailed model of the scene. This method meets the growing need of easy deployment of robotics or augmented reality applications in any environments, especially those for which no accurate 3D model nor huge amount of ground truth data are available. It exploits the ability of deep learning techniques to reliably detect objects regardless of viewing conditions. Previous works have also shown that abstracting the geometry of a scene of objects by an ellipsoid cloud allows to compute the camera pose accurately enough for various application needs. Though promising, these approaches use the ellipses fitted to the detection bounding boxes as an approximation of the imaged objects. In this paper, we go one step further and propose a learning-based method which detects improved elliptic approximations of objects which are coherent with the 3D ellipsoid in terms of perspective projection. Experiments prove that the accuracy of the computed pose significantly increases thanks to our method and is more robust to the variability of the boundaries of the detection boxes. This is achieved with very little effort in terms of training data acquisition -- a few hundred calibrated images of which only three need manual object annotation. Code and models are released at https://github.com/zinsmatt/3D-Aware-Ellipses-for-Visual-Localization.
△ Less
Submitted 24 May, 2021;
originally announced May 2021.
-
A novel method for Causal Structure Discovery from EHR data, a demonstration on type-2 diabetes mellitus
Authors:
Xinpeng Shen,
Sisi Ma,
Prashanthi Vemuri,
M. Regina Castro,
Pedro J. Caraballo,
Gyorgy J. Simon
Abstract:
Introduction: The discovery of causal mechanisms underlying diseases enables better diagnosis, prognosis and treatment selection. Clinical trials have been the gold standard for determining causality, but they are resource intensive, sometimes infeasible or unethical. Electronic Health Records (EHR) contain a wealth of real-world data that holds promise for the discovery of disease mechanisms, yet…
▽ More
Introduction: The discovery of causal mechanisms underlying diseases enables better diagnosis, prognosis and treatment selection. Clinical trials have been the gold standard for determining causality, but they are resource intensive, sometimes infeasible or unethical. Electronic Health Records (EHR) contain a wealth of real-world data that holds promise for the discovery of disease mechanisms, yet the existing causal structure discovery (CSD) methods fall short on leveraging them due to the special characteristics of the EHR data. We propose a new data transformation method and a novel CSD algorithm to overcome the challenges posed by these characteristics. Materials and methods: We demonstrated the proposed methods on an application to type-2 diabetes mellitus. We used a large EHR data set from Mayo Clinic to internally evaluate the proposed transformation and CSD methods and used another large data set from an independent health system, Fairview Health Services, as external validation. We compared the performance of our proposed method to Fast Greedy Equivalence Search (FGES), a state-of-the-art CSD method in terms of correctness, stability and completeness. We tested the generalizability of the proposed algorithm through external validation. Results and conclusions: The proposed method improved over the existing methods by successfully incorporating study design considerations, was robust in face of unreliable EHR timestamps and inferred causal effect directions more correctly and reliably. The proposed data transformation successfully improved the clinical correctness of the discovered graph and the consistency of edge orientation across bootstrap samples. It resulted in superior accuracy, stability, and completeness.
△ Less
Submitted 10 November, 2020;
originally announced November 2020.
-
Incorporating Causal Effects into Deep Learning Predictions on EHR Data
Authors:
Jia Li,
Haoyu Yang,
Xiaowei Jia,
Vipin Kumar,
Michael Steinbach,
Gyorgy Simon
Abstract:
Electronic Health Records (EHR) data analysis plays a crucial role in healthcare system quality. Because of its highly complex underlying causality and limited observable nature, causal inference on EHR is quite challenging. Deep Learning (DL) achieved great success among the advanced machine learning methodologies. Nevertheless, it is still obstructed by the inappropriately assumed causal conditi…
▽ More
Electronic Health Records (EHR) data analysis plays a crucial role in healthcare system quality. Because of its highly complex underlying causality and limited observable nature, causal inference on EHR is quite challenging. Deep Learning (DL) achieved great success among the advanced machine learning methodologies. Nevertheless, it is still obstructed by the inappropriately assumed causal conditions. This work proposed a novel method to quantify clinically well-defined causal effects as a generalized estimation vector that is simply utilizable for causal models. We incorporated it into DL models to achieve better predictive performance and result interpretation. Furthermore, we also proved the existence of causal information blink spots that regular DL models cannot reach.
△ Less
Submitted 26 October, 2022; v1 submitted 10 November, 2020;
originally announced November 2020.
-
High-Speed Train Cell-less Network Enabled by XGS-PON and Impacts on vRAN Split Interface Transmission
Authors:
Anas El Ankouri,
Luiz Neto,
Gaël Simon,
Hugues Le Bras,
Ali Sanhaji,
P. Chanclou
Abstract:
We successfully demonstrate a transmission of a high layer split mobile interface for cell-less, high-speed train network applications using a commercially available XGS-PON. Operation is also demonstrated for a GbE interface.
We successfully demonstrate a transmission of a high layer split mobile interface for cell-less, high-speed train network applications using a commercially available XGS-PON. Operation is also demonstrated for a GbE interface.
△ Less
Submitted 16 February, 2019;
originally announced February 2019.
-
Joint Facade Registration and Segmentation for Urban Localization
Authors:
Antoine Fond,
Marie-Odile Berger,
Gilles Simon
Abstract:
This paper presents an efficient approach for solving jointly facade registration and semantic segmentation. Progress in facade detection and recognition enable good initialization for the registration of a reference facade to a newly acquired target image. We propose here to rely on semantic segmentation to improve the accuracy of that initial registration. Simultaneously we aim to improve the qu…
▽ More
This paper presents an efficient approach for solving jointly facade registration and semantic segmentation. Progress in facade detection and recognition enable good initialization for the registration of a reference facade to a newly acquired target image. We propose here to rely on semantic segmentation to improve the accuracy of that initial registration. Simultaneously we aim to improve the quality of the semantic segmentation through the registration. These two problems are jointly solved in a Expectation-Maximization framework. We especially introduce a bayesian model that use prior semantic segmentation as well as geometric structure of the facade reference modeled by $L_p$ Gaussian Mixtures. We show the advantages of our method in term of robustness to clutter and change of illumination on urban images from various database.
△ Less
Submitted 21 February, 2019; v1 submitted 25 November, 2018;
originally announced November 2018.
-
Viewport-Driven Rate-Distortion Optimized 360° Video Streaming
Authors:
Jacob Chakareski,
Ridvan Aksu,
Xavier Corbillon,
Gwendal Simon,
Viswanathan Swaminathan
Abstract:
The growing popularity of virtual and augmented reality communications and 360° video streaming is moving video communication systems into much more dynamic and resource-limited operating settings. The enormous data volume of 360° videos requires an efficient use of network bandwidth to maintain the desired quality of experience for the end user. To this end, we propose a framework for viewport-dr…
▽ More
The growing popularity of virtual and augmented reality communications and 360° video streaming is moving video communication systems into much more dynamic and resource-limited operating settings. The enormous data volume of 360° videos requires an efficient use of network bandwidth to maintain the desired quality of experience for the end user. To this end, we propose a framework for viewport-driven rate-distortion optimized 360° video streaming that integrates the user view navigation pattern and the spatiotemporal rate-distortion characteristics of the 360° video content to maximize the delivered user quality of experience for the given network/system resources. The framework comprises a methodology for constructing dynamic heat maps that capture the likelihood of navigating different spatial segments of a 360° video over time by the user, an analysis and characterization of its spatiotemporal rate-distortion characteristics that leverage preprocessed spatial tilling of the 360° view sphere, and an optimization problem formulation that characterizes the delivered user quality of experience given the user navigation patterns, 360° video encoding decisions, and the available system/network resources. Our experimental results demonstrate the advantages of our framework over the conventional approach of streaming a monolithic uniformly encoded 360° video and a state-of-the-art reference method. Considerable video quality gains of 4 - 5 dB are demonstrated in the case of two popular 4K 360° videos.
△ Less
Submitted 21 March, 2018;
originally announced March 2018.
-
Mining Electronic Health Records: A Survey
Authors:
Pranjul Yadav,
Michael Steinbach,
Vipin Kumar,
Gyorgy Simon
Abstract:
The continuously increasing cost of the US healthcare system has received significant attention. Central to the ideas aimed at curbing this trend is the use of technology, in the form of the mandate to implement electronic health records (EHRs). EHRs consist of patient information such as demographics, medications, laboratory test results, diagnosis codes and procedures. Mining EHRs could lead to…
▽ More
The continuously increasing cost of the US healthcare system has received significant attention. Central to the ideas aimed at curbing this trend is the use of technology, in the form of the mandate to implement electronic health records (EHRs). EHRs consist of patient information such as demographics, medications, laboratory test results, diagnosis codes and procedures. Mining EHRs could lead to improvement in patient health management as EHRs contain detailed information related to disease prognosis for large patient populations. In this manuscript, we provide a structured and comprehensive overview of data mining techniques for modeling EHR data. We first provide a detailed understanding of the major application areas to which EHR mining has been applied and then discuss the nature of EHR data and its accompanying challenges. Next, we describe major approaches used for EHR mining, the metrics associated with EHRs, and the various study designs. With this foundation, we then provide a systematic and methodological organization of existing data mining techniques used to model EHRs and discuss ideas for future research. We conclude this survey with a comprehensive summary of clinical data mining applications of EHR data, as illustrated in the online supplement.
△ Less
Submitted 23 March, 2017; v1 submitted 9 February, 2017;
originally announced February 2017.
-
Causal Inference in Observational Data
Authors:
Pranjul Yadav,
Lisiane Prunelli,
Alexander Hoff,
Michael Steinbach,
Bonnie Westra,
Vipin Kumar,
Gyorgy Simon
Abstract:
Our aging population increasingly suffers from multiple chronic diseases simultaneously, necessitating the comprehensive treatment of these conditions. Finding the optimal set of drugs for a combinatorial set of diseases is a combinatorial pattern exploration problem. Association rule mining is a popular tool for such problems, but the requirement of health care for finding causal, rather than ass…
▽ More
Our aging population increasingly suffers from multiple chronic diseases simultaneously, necessitating the comprehensive treatment of these conditions. Finding the optimal set of drugs for a combinatorial set of diseases is a combinatorial pattern exploration problem. Association rule mining is a popular tool for such problems, but the requirement of health care for finding causal, rather than associative, patterns renders association rule mining unsuitable. To address this issue, we propose a novel framework based on the Rubin-Neyman causal model for extracting causal rules from observational data, correcting for a number of common biases. Specifically, given a set of interventions and a set of items that define subpopulations (e.g., diseases), we wish to find all subpopulations in which effective intervention combinations exist and in each such subpopulation, we wish to find all intervention combinations such that drop** any intervention from this combination will reduce the efficacy of the treatment. A key aspect of our framework is the concept of closed intervention sets which extend the concept of quantifying the effect of a single intervention to a set of concurrent interventions. We also evaluated our causal rule mining framework on the Electronic Health Records (EHR) data of a large cohort of patients from Mayo Clinic and showed that the patterns we extracted are sufficiently rich to explain the controversial findings in the medical literature regarding the effect of a class of cholesterol drugs on Type-II Diabetes Mellitus (T2DM).
△ Less
Submitted 14 November, 2016;
originally announced November 2016.
-
Viewport-Adaptive Navigable 360-Degree Video Delivery
Authors:
Xavier Corbillon,
Gwendal Simon,
Alisa Devlic,
Jacob Chakareski
Abstract:
The delivery and display of 360-degree videos on Head-Mounted Displays (HMDs) presents many technical challenges. 360-degree videos are ultra high resolution spherical videos, which contain an omnidirectional view of the scene. However only a portion of this scene is displayed on the HMD. Moreover, HMD need to respond in 10 ms to head movements, which prevents the server to send only the displayed…
▽ More
The delivery and display of 360-degree videos on Head-Mounted Displays (HMDs) presents many technical challenges. 360-degree videos are ultra high resolution spherical videos, which contain an omnidirectional view of the scene. However only a portion of this scene is displayed on the HMD. Moreover, HMD need to respond in 10 ms to head movements, which prevents the server to send only the displayed video part based on client feedback. To reduce the bandwidth waste, while still providing an immersive experience, a viewport-adaptive 360-degree video streaming system is proposed. The server prepares multiple video representations, which differ not only by their bit-rate, but also by the qualities of different scene regions. The client chooses a representation for the next segment such that its bit-rate fits the available throughput and a full quality region matches its viewing. We investigate the impact of various spherical-to-plane projections and quality arrangements on the video quality displayed to the user, showing that the cube map layout offers the best quality for the given bit-rate budget. An evaluation with a dataset of users navigating 360-degree videos demonstrates that segments need to be short enough to enable frequent view switches.
△ Less
Submitted 10 May, 2017; v1 submitted 26 September, 2016;
originally announced September 2016.
-
Optimized Adaptive Streaming Representations based on System Dynamics
Authors:
Laura Toni,
Ramon Aparicio-Pardo,
Karine Pires,
Gwendal Simon,
Alberto Blanc,
Pascal Frossard
Abstract:
Adaptive streaming addresses the increasing and heterogenous demand of multimedia content over the Internet by offering several encoded versions for each video sequence. Each version (or representation) has a different resolution and bit rate, aimed at a specific set of users, like TV or mobile phone clients. While most existing works on adaptive streaming deal with effective playout-control strat…
▽ More
Adaptive streaming addresses the increasing and heterogenous demand of multimedia content over the Internet by offering several encoded versions for each video sequence. Each version (or representation) has a different resolution and bit rate, aimed at a specific set of users, like TV or mobile phone clients. While most existing works on adaptive streaming deal with effective playout-control strategies at the client side, we take in this paper a providers' perspective and propose solutions to improve user satisfaction by optimizing the encoding rates of the video sequences. We formulate an integer linear program that maximizes users' average satisfaction, taking into account the network dynamics, the video content information, and the user population characteristics. The solution of the optimization is a set of encoding parameters that permit to create different streams to robustly satisfy users' requests over time. We simulate multiple adaptive streaming sessions characterized by realistic network connections models, where the proposed solution outperforms commonly used vendor recommendations, in terms of user satisfaction but also in terms of fairness and outage probability. The simulation results further show that video content information as well as network constraints and users' statistics play a crucial role in selecting proper encoding parameters to provide fairness a mong users and to reduce network resource usage. We finally propose a few practical guidelines that can be used to choose the encoding parameters based on the user base characteristics, the network capacity and the type of video content.
△ Less
Submitted 16 October, 2014; v1 submitted 12 June, 2014;
originally announced June 2014.
-
Maximum Bounded Rooted-Tree Packing Problem
Authors:
Herve Kerivin,
Jimmy Leblet,
Gwendal Simon,
Fen Zhou
Abstract:
Given a graph and a root, the Maximum Bounded Rooted-Tree Packing (MBRTP) problem aims at finding K rooted-trees that span the largest subset of vertices, when each vertex has a limited outdegree. This problem is motivated by peer-to-peer streaming overlays in under-provisioned systems. We prove that the MBRTP problem is NP-complete. We present two polynomial-time algorithms that computes an optim…
▽ More
Given a graph and a root, the Maximum Bounded Rooted-Tree Packing (MBRTP) problem aims at finding K rooted-trees that span the largest subset of vertices, when each vertex has a limited outdegree. This problem is motivated by peer-to-peer streaming overlays in under-provisioned systems. We prove that the MBRTP problem is NP-complete. We present two polynomial-time algorithms that computes an optimal solution on complete graphs and trees respectively.
△ Less
Submitted 2 November, 2011;
originally announced November 2011.
-
Large-Scale Time-Shifted Streaming Delivery
Authors:
Yaning Liu,
Gwendal Simon
Abstract:
An attractive new feature of connected TV systems consists in allowing users to access past portions of the TV channel. This feature, called time-shifted streaming, is now used by millions of TV viewers. We address in this paper the design of a large-scale delivery system for time-shifted streaming. We highlight the characteristics of time-shifted streaming that prevent known video delivery system…
▽ More
An attractive new feature of connected TV systems consists in allowing users to access past portions of the TV channel. This feature, called time-shifted streaming, is now used by millions of TV viewers. We address in this paper the design of a large-scale delivery system for time-shifted streaming. We highlight the characteristics of time-shifted streaming that prevent known video delivery systems to be used. Then, we present two proposals that meet the demand for two radically different types of TV operator. First, the Peer-Assisted Catch-Up Streaming system, namely PACUS, aims at reducing the load on the server of a large TV broadcasters without losing the control of the TV delivery. Second, the turntable structure, is an overlay of nodes that allow an independent content delivery network or a small independent TV broadcaster to ensure that all past TV programs are stored and as available as possible. We show through extensive simulations that our objectives are reached, with a reduction of up to three quarters of the traffic for PACUS and a 100\% guaranteed availability for the turntable structure. We also compare our proposals to the main previous works in the area.
△ Less
Submitted 9 November, 2011; v1 submitted 6 November, 2009;
originally announced November 2009.
-
A note on the data-driven capacity of P2P networks
Authors:
Jacob Chakareski,
Pascal Frossard,
Hervé Kerivin,
Jimmy Leblet,
Gwendal Simon
Abstract:
We consider two capacity problems in P2P networks. In the first one, the nodes have an infinite amount of data to send and the goal is to optimally allocate their uplink bandwidths such that the demands of every peer in terms of receiving data rate are met. We solve this problem through a map** from a node-weighted graph featuring two labels per node to a max flow problem on an edge-weighted b…
▽ More
We consider two capacity problems in P2P networks. In the first one, the nodes have an infinite amount of data to send and the goal is to optimally allocate their uplink bandwidths such that the demands of every peer in terms of receiving data rate are met. We solve this problem through a map** from a node-weighted graph featuring two labels per node to a max flow problem on an edge-weighted bipartite graph. In the second problem under consideration, the resource allocation is driven by the availability of the data resource that the peers are interested in sharing. That is a node cannot allocate its uplink resources unless it has data to transmit first. The problem of uplink bandwidth allocation is then equivalent to constructing a set of directed trees in the overlay such that the number of nodes receiving the data is maximized while the uplink capacities of the peers are not exceeded. We show that the problem is NP-complete, and provide a linear programming decomposition decoupling it into a master problem and multiple slave subproblems that can be resolved in polynomial time. We also design a heuristic algorithm in order to compute a suboptimal solution in a reasonable time. This algorithm requires only a local knowledge from nodes, so it should support distributed implementations.
We analyze both problems through a series of simulation experiments featuring different network sizes and network densities. On large networks, we compare our heuristic and its variants with a genetic algorithm and show that our heuristic computes the better resource allocation. On smaller networks, we contrast these performances to that of the exact algorithm and show that resource allocation fulfilling a large part of the peer can be found, even for hard configuration where no resources are in excess.
△ Less
Submitted 16 September, 2009;
originally announced September 2009.
-
Measuring the Dynamical State of the Internet: Large Scale Network Tomography via the ETOMIC Infrastructure
Authors:
Gabor Simon,
Jozsef Steger,
Peter Haga Istvan Csabai,
Gabor Vattay
Abstract:
In this paper we show how to go beyond the study of the topological properties of the Internet, by measuring its dynamical state using special active probing techniques and the methods of network tomography. We demonstrate this approach by measuring the key state parameters of Internet paths, the characteristics of queueing delay, in a part of the European Internet. In the paper we describe in d…
▽ More
In this paper we show how to go beyond the study of the topological properties of the Internet, by measuring its dynamical state using special active probing techniques and the methods of network tomography. We demonstrate this approach by measuring the key state parameters of Internet paths, the characteristics of queueing delay, in a part of the European Internet. In the paper we describe in detail the ETOMIC measurement platform that was used to conduct the experiments, and the applied method of queueing delay tomography. The main results of the paper are maps showing various spatial structure in the characteristics of queueing delay corresponding to the resolved part of the European Internet. These maps reveal that the average queueing delay of network segments spans more than two orders of magnitude, and that the distribution of this quantity is very well fitted by the log-normal distribution.
△ Less
Submitted 27 January, 2008;
originally announced January 2008.
-
Time Series Forecasting: Obtaining Long Term Trends with Self-Organizing Maps
Authors:
Geoffroy Simon,
Amaury Lendasse,
Marie Cottrell,
Jean-Claude Fort,
Michel Verleysen
Abstract:
Kohonen self-organisation maps are a well know classification tool, commonly used in a wide variety of problems, but with limited applications in time series forecasting context. In this paper, we propose a forecasting method specifically designed for multi-dimensional long-term trends prediction, with a double application of the Kohonen algorithm. Practical applications of the method are also p…
▽ More
Kohonen self-organisation maps are a well know classification tool, commonly used in a wide variety of problems, but with limited applications in time series forecasting context. In this paper, we propose a forecasting method specifically designed for multi-dimensional long-term trends prediction, with a double application of the Kohonen algorithm. Practical applications of the method are also presented.
△ Less
Submitted 8 January, 2007;
originally announced January 2007.
-
Self-generated Self-similar Traffic
Authors:
P. Haga,
P. Pollner,
G. Simon,
I. Csabai,
G. Vattay
Abstract:
Self-similarity in the network traffic has been studied from several aspects: both at the user side and at the network side there are many sources of the long range dependence. Recently some dynamical origins are also identified: the TCP adaptive congestion avoidance algorithm itself can produce chaotic and long range dependent throughput behavior, if the loss rate is very high. In this paper we…
▽ More
Self-similarity in the network traffic has been studied from several aspects: both at the user side and at the network side there are many sources of the long range dependence. Recently some dynamical origins are also identified: the TCP adaptive congestion avoidance algorithm itself can produce chaotic and long range dependent throughput behavior, if the loss rate is very high. In this paper we show that there is a close connection between the static and dynamic origins of self-similarity: parallel TCPs can generate the self-similarity themselves, they can introduce heavily fluctuations into the background traffic and produce high effective loss rate causing a long range dependent TCP flow, however, the dropped packet ratio is low.
△ Less
Submitted 17 February, 2004;
originally announced February 2004.
