-
Decision support system for Forest fire management using Ontology with Big Data and LLMs
Authors:
Ritesh Chandra,
Shashi Shekhar Kumar,
Rushil Patra,
Sonali Agarwal
Abstract:
Forests are crucial for ecological balance, but wildfires, a major cause of forest loss, pose significant risks. Fire weather indices, which assess wildfire risk and predict resource demands, are vital. With the rise of sensor networks in fields like healthcare and environmental monitoring, semantic sensor networks are increasingly used to gather climatic data such as wind speed, temperature, and…
▽ More
Forests are crucial for ecological balance, but wildfires, a major cause of forest loss, pose significant risks. Fire weather indices, which assess wildfire risk and predict resource demands, are vital. With the rise of sensor networks in fields like healthcare and environmental monitoring, semantic sensor networks are increasingly used to gather climatic data such as wind speed, temperature, and humidity. However, processing these data streams to determine fire weather indices presents challenges, underscoring the growing importance of effective forest fire detection. This paper discusses using Apache Spark for early forest fire detection, enhancing fire risk prediction with meteorological and geographical data. Building on our previous development of Semantic Sensor Network (SSN) ontologies and Semantic Web Rules Language (SWRL) for managing forest fires in Monesterial Natural Park, we expanded SWRL to improve a Decision Support System (DSS) using a Large Language Models (LLMs) and Spark framework. We implemented real-time alerts with Spark streaming, tailored to various fire scenarios, and validated our approach using ontology metrics, query-based evaluations, LLMs score precision, F1 score, and recall measures.
△ Less
Submitted 18 May, 2024;
originally announced May 2024.
-
A non-parametric approach for estimating consumer valuation distributions using second price auctions
Authors:
Sourav Mukherjee,
Rohit K Patra,
Kshitij Khare
Abstract:
We focus on online second price auctions, where bids are made sequentially, and the winning bidder pays the maximum of the second-highest bid and a seller specified reserve price. For many such auctions, the seller does not see all the bids or the total number of bidders accessing the auction, and only observes the current selling prices throughout the course of the auction. We develop a novel non…
▽ More
We focus on online second price auctions, where bids are made sequentially, and the winning bidder pays the maximum of the second-highest bid and a seller specified reserve price. For many such auctions, the seller does not see all the bids or the total number of bidders accessing the auction, and only observes the current selling prices throughout the course of the auction. We develop a novel non-parametric approach to estimate the underlying consumer valuation distribution based on this data. Previous non-parametric approaches in the literature only use the final selling price and assume knowledge of the total number of bidders. The resulting estimate, in particular, can be used by the seller to compute the optimal profit-maximizing price for the product. Our approach is free of tuning parameters, and we demonstrate its computational and statistical efficiency in a variety of simulation settings, and also on an Xbox 7-day auction dataset on eBay.
△ Less
Submitted 12 December, 2023;
originally announced December 2023.
-
Covariate-distance Weighted Regression (CWR): A Case Study for Estimation of House Prices
Authors:
Hone-Jay Chu,
Po-Hung Chen,
Sheng-Mao Chang,
Muhammad Zeeshan Ali,
Sumriti Ranjan Patra
Abstract:
Geographically weighted regression (GWR) is a popular tool for modeling spatial heterogeneity in a regression model. However, the current weighting function used in GWR only considers the geographical distance, while the attribute similarity is totally ignored. In this study, we proposed a covariate weighting function that combines the geographical distance and attribute distance. The covariate-di…
▽ More
Geographically weighted regression (GWR) is a popular tool for modeling spatial heterogeneity in a regression model. However, the current weighting function used in GWR only considers the geographical distance, while the attribute similarity is totally ignored. In this study, we proposed a covariate weighting function that combines the geographical distance and attribute distance. The covariate-distance weighted regression (CWR) is the extension of GWR including geographical distance and attribute distance. House prices are affected by numerous factors, such as house age, floor area, and land use. Prediction model is used to help understand the characteristics of regional house prices. The CWR was used to understand the relationship between the house price and controlling factors. The CWR can consider the geological and attribute distances, and produce accurate estimates of house price that preserve the weight matrix for geological and attribute distance functions. Results show that the house attributes/conditions and the characteristics of the house, such as floor area and house age, might affect the house price. After factor selection, in which only house age and floor area of a building are considered, the RMSE of the CWR model can be improved by 2.9%-26.3% for skyscrapers when compared to the GWR. CWR can effectively reduce estimation errors from traditional spatial regression models and provide novel and feasible models for spatial estimation.
△ Less
Submitted 14 May, 2023;
originally announced May 2023.
-
Calibrating Deep Neural Networks using Explicit Regularisation and Dynamic Data Pruning
Authors:
Ramya Hebbalaguppe,
Rishabh Patra,
Tirtharaj Dash,
Gautam Shroff,
Lovekesh Vig
Abstract:
Deep neural networks (DNN) are prone to miscalibrated predictions, often exhibiting a mismatch between the predicted output and the associated confidence scores. Contemporary model calibration techniques mitigate the problem of overconfident predictions by pushing down the confidence of the winning class while increasing the confidence of the remaining classes across all test samples. However, fro…
▽ More
Deep neural networks (DNN) are prone to miscalibrated predictions, often exhibiting a mismatch between the predicted output and the associated confidence scores. Contemporary model calibration techniques mitigate the problem of overconfident predictions by pushing down the confidence of the winning class while increasing the confidence of the remaining classes across all test samples. However, from a deployment perspective, an ideal model is desired to (i) generate well-calibrated predictions for high-confidence samples with predicted probability say >0.95, and (ii) generate a higher proportion of legitimate high-confidence samples. To this end, we propose a novel regularization technique that can be used with classification losses, leading to state-of-the-art calibrated predictions at test time; From a deployment standpoint in safety-critical applications, only high-confidence samples from a well-calibrated model are of interest, as the remaining samples have to undergo manual inspection. Predictive confidence reduction of these potentially ``high-confidence samples'' is a downside of existing calibration approaches. We mitigate this by proposing a dynamic train-time data pruning strategy that prunes low-confidence samples every few epochs, providing an increase in "confident yet calibrated samples". We demonstrate state-of-the-art calibration performance across image classification benchmarks, reducing training time without much compromise in accuracy. We provide insights into why our dynamic pruning strategy that prunes low-confidence training samples leads to an increase in high-confidence samples at test time.
△ Less
Submitted 20 December, 2022;
originally announced December 2022.
-
Most direct product of graphs are Type 1
Authors:
Diane Castonguay,
Celina M. H. de Figueiredo,
Luis Antonio Kowada,
Caroline Reis Patrão,
Diana Sasaki
Abstract:
A \textit{$k$-total coloring} of a graph $G$ is an assignment of $k$ colors to its elements (vertices and edges) so that adjacent or incident elements have different colors. The total chromatic number is the smallest integer $k$ for which the graph $G$ has a $k$-total coloring. Clearly, this number is at least $Δ(G)+1$, where $Δ(G)$ is the maximum degree of $G$. When the lower bound is reached, th…
▽ More
A \textit{$k$-total coloring} of a graph $G$ is an assignment of $k$ colors to its elements (vertices and edges) so that adjacent or incident elements have different colors. The total chromatic number is the smallest integer $k$ for which the graph $G$ has a $k$-total coloring. Clearly, this number is at least $Δ(G)+1$, where $Δ(G)$ is the maximum degree of $G$. When the lower bound is reached, the graph is said to be Type~1. The upper bound of $Δ(G)+2$ is a central problem that has been open for fifty years, is verified for graphs with maximum degree 4 but not for regular graphs.
Most classified direct product of graphs are Type~1. The particular cases of the direct product of cycle graphs $C_m \times C_n$, for $m =3p, 5\ell$ and $8\ell$ with $p \geq 2$ and $\ell \geq 1$, and arbitrary $n \geq 3$, were previously known to be Type 1 and motivated the conjecture that, except for $C_4 \times C_4$, all direct product of cycle graphs $C_m \times C_n$ with $m,n \geq 3$ are Type 1.
We give a general pattern proving that all $C_m \times C_n$ are Type 1, except for $C_4 \times C_4$. dditionally, we investigate sufficient conditions to ensure that the direct product reaches the lower bound for the total chromatic number.
△ Less
Submitted 27 October, 2021;
originally announced October 2021.
-
NoisyActions2M: A Multimedia Dataset for Video Understanding from Noisy Labels
Authors:
Mohit Sharma,
Raj Patra,
Harshal Desai,
Shruti Vyas,
Yogesh Rawat,
Rajiv Ratn Shah
Abstract:
Deep learning has shown remarkable progress in a wide range of problems. However, efficient training of such models requires large-scale datasets, and getting annotations for such datasets can be challenging and costly. In this work, we explore the use of user-generated freely available labels from web videos for video understanding. We create a benchmark dataset consisting of around 2 million vid…
▽ More
Deep learning has shown remarkable progress in a wide range of problems. However, efficient training of such models requires large-scale datasets, and getting annotations for such datasets can be challenging and costly. In this work, we explore the use of user-generated freely available labels from web videos for video understanding. We create a benchmark dataset consisting of around 2 million videos with associated user-generated annotations and other meta information. We utilize the collected dataset for action classification and demonstrate its usefulness with existing small-scale annotated datasets, UCF101 and HMDB51. We study different loss functions and two pretraining strategies, simple and self-supervised learning. We also show how a network pretrained on the proposed dataset can help against video corruption and label noise in downstream datasets. We present this as a benchmark dataset in noisy learning for video understanding. The dataset, code, and trained models will be publicly available for future research.
△ Less
Submitted 13 October, 2021;
originally announced October 2021.
-
FLeet: Online Federated Learning via Staleness Awareness and Performance Prediction
Authors:
Georgios Damaskinos,
Rachid Guerraoui,
Anne-Marie Kermarrec,
Vlad Nitu,
Rhicheek Patra,
Francois Taiani
Abstract:
Federated Learning (FL) is very appealing for its privacy benefits: essentially, a global model is trained with updates computed on mobile devices while kee** the data of users local. Standard FL infrastructures are however designed to have no energy or performance impact on mobile devices, and are therefore not suitable for applications that require frequent (online) model updates, such as news…
▽ More
Federated Learning (FL) is very appealing for its privacy benefits: essentially, a global model is trained with updates computed on mobile devices while kee** the data of users local. Standard FL infrastructures are however designed to have no energy or performance impact on mobile devices, and are therefore not suitable for applications that require frequent (online) model updates, such as news recommenders.
This paper presents FLeet, the first Online FL system, acting as a middleware between the Android OS and the machine learning application. FLeet combines the privacy of Standard FL with the precision of online learning thanks to two core components: (i) I-Prof, a new lightweight profiler that predicts and controls the impact of learning tasks on mobile devices, and (ii) AdaSGD, a new adaptive learning algorithm that is resilient to delayed updates.
Our extensive evaluation shows that Online FL, as implemented by FLeet, can deliver a 2.3x quality boost compared to Standard FL, while only consuming 0.036% of the battery per day. I-Prof can accurately control the impact of learning tasks by improving the prediction accuracy up to 3.6x (computation time) and up to 19x (energy). AdaSGD outperforms alternative FL approaches by 18.4% in terms of convergence speed on heterogeneous data.
△ Less
Submitted 3 December, 2020; v1 submitted 12 June, 2020;
originally announced June 2020.
-
On Least Squares Estimation under Heteroscedastic and Heavy-Tailed Errors
Authors:
Arun K. Kuchibhotla,
Rohit K. Patra
Abstract:
We consider least squares estimation in a general nonparametric regression model. The rate of convergence of the least squares estimator (LSE) for the unknown regression function is well studied when the errors are sub-Gaussian. We find upper bounds on the rates of convergence of the LSE when the errors have uniformly bounded conditional variance and have only finitely many moments. We show that t…
▽ More
We consider least squares estimation in a general nonparametric regression model. The rate of convergence of the least squares estimator (LSE) for the unknown regression function is well studied when the errors are sub-Gaussian. We find upper bounds on the rates of convergence of the LSE when the errors have uniformly bounded conditional variance and have only finitely many moments. We show that the interplay between the moment assumptions on the error, the metric entropy of the class of functions involved, and the "local" structure of the function class around the truth drives the rate of convergence of the LSE. We find sufficient conditions on the errors under which the rate of the LSE matches the rate of the LSE under sub-Gaussian error. Our results are finite sample and allow for heteroscedastic and heavy-tailed errors.
△ Less
Submitted 8 April, 2021; v1 submitted 4 September, 2019;
originally announced September 2019.
-
Asynchronous Byzantine Machine Learning (the case of SGD)
Authors:
Georgios Damaskinos,
El Mahdi El Mhamdi,
Rachid Guerraoui,
Rhicheek Patra,
Mahsa Taziki
Abstract:
Asynchronous distributed machine learning solutions have proven very effective so far, but always assuming perfectly functioning workers. In practice, some of the workers can however exhibit Byzantine behavior, caused by hardware failures, software bugs, corrupt data, or even malicious attacks. We introduce \emph{Kardam}, the first distributed asynchronous stochastic gradient descent (SGD) algorit…
▽ More
Asynchronous distributed machine learning solutions have proven very effective so far, but always assuming perfectly functioning workers. In practice, some of the workers can however exhibit Byzantine behavior, caused by hardware failures, software bugs, corrupt data, or even malicious attacks. We introduce \emph{Kardam}, the first distributed asynchronous stochastic gradient descent (SGD) algorithm that copes with Byzantine workers. Kardam consists of two complementary components: a filtering and a dampening component. The first is scalar-based and ensures resilience against $\frac{1}{3}$ Byzantine workers. Essentially, this filter leverages the Lipschitzness of cost functions and acts as a self-stabilizer against Byzantine workers that would attempt to corrupt the progress of SGD. The dampening component bounds the convergence rate by adjusting to stale information through a generic gradient weighting scheme. We prove that Kardam guarantees almost sure convergence in the presence of asynchrony and Byzantine behavior, and we derive its convergence rate. We evaluate Kardam on the CIFAR-100 and EMNIST datasets and measure its overhead with respect to non Byzantine-resilient solutions. We empirically show that Kardam does not introduce additional noise to the learning procedure but does induce a slowdown (the cost of Byzantine resilience) that we both theoretically and empirically show to be less than $f/n$, where $f$ is the number of Byzantine failures tolerated and $n$ the total number of workers. Interestingly, we also empirically observe that the dampening component is interesting in its own right for it enables to build an SGD algorithm that outperforms alternative staleness-aware asynchronous competitors in environments with honest workers.
△ Less
Submitted 9 July, 2018; v1 submitted 22 February, 2018;
originally announced February 2018.
-
Sequences, Items And Latent Links: Recommendation With Consumed Item Packs
Authors:
Rachid Guerraoui,
Erwan Le Merrer,
Rhicheek Patra,
Jean-Ronan Vigouroux
Abstract:
Recommenders personalize the web content by typically using collaborative filtering to relate users (or items) based on explicit feedback, e.g., ratings. The difficulty of collecting this feedback has recently motivated to consider implicit feedback (e.g., item consumption along with the corresponding time).
In this paper, we introduce the notion of consumed item pack (CIP) which enables to link…
▽ More
Recommenders personalize the web content by typically using collaborative filtering to relate users (or items) based on explicit feedback, e.g., ratings. The difficulty of collecting this feedback has recently motivated to consider implicit feedback (e.g., item consumption along with the corresponding time).
In this paper, we introduce the notion of consumed item pack (CIP) which enables to link users (or items) based on their implicit analogous consumption behavior. Our proposal is generic, and we show that it captures three novel implicit recommenders: a user-based (CIP-U), an item-based (CIP-I), and a word embedding-based (DEEPCIP), as well as a state-of-the-art technique using implicit feedback (FISM). We show that our recommenders handle incremental updates incorporating freshly consumed items. We demonstrate that all three recommenders provide a recommendation quality that is competitive with state-of-the-art ones, including one incorporating both explicit and implicit feedback.
△ Less
Submitted 7 December, 2017; v1 submitted 16 November, 2017;
originally announced November 2017.
-
BoostJet: Towards Combining Statistical Aggregates with Neural Embeddings for Recommendations
Authors:
Rhicheek Patra,
Egor Samosvat,
Michael Roizner,
Andrei Mishchenko
Abstract:
Recommenders have become widely popular in recent years because of their broader applicability in many e-commerce applications. These applications rely on recommenders for generating advertisements for various offers or providing content recommendations. However, the quality of the generated recommendations depends on user features (like demography, temporality), offer features (like popularity, p…
▽ More
Recommenders have become widely popular in recent years because of their broader applicability in many e-commerce applications. These applications rely on recommenders for generating advertisements for various offers or providing content recommendations. However, the quality of the generated recommendations depends on user features (like demography, temporality), offer features (like popularity, price), and user-offer features (like implicit or explicit feedback). Current state-of-the-art recommenders do not explore such diverse features concurrently while generating the recommendations.
In this paper, we first introduce the notion of Trackers which enables us to capture the above-mentioned features and thus incorporate users' online behaviour through statistical aggregates of different features (demography, temporality, popularity, price). We also show how to capture offer-to-offer relations, based on their consumption sequence, leveraging neural embeddings for offers in our Offer2Vec algorithm. We then introduce BoostJet, a novel recommender which integrates the Trackers along with the neural embeddings using MatrixNet, an efficient distributed implementation of gradient boosted decision tree, to improve the recommendation quality significantly. We provide an in-depth evaluation of BoostJet on Yandex's dataset, collecting online behaviour from tens of millions of online users, to demonstrate the practicality of BoostJet in terms of recommendation quality as well as scalability.
△ Less
Submitted 7 December, 2017; v1 submitted 15 November, 2017;
originally announced November 2017.
-
A Quantitative Analysis of WCAG 2.0 Compliance For Some Indian Web Portals
Authors:
Manas Ranjan Patra,
Amar Ranjan Dash,
Prasanna Kumar Mishra
Abstract:
Web portals have served as an excellent medium to facilitate user centric services for organizations irrespective of the type, size, and domain of operation. The objective of these portals has been to deliver a plethora of services such as information dissemination, transactional services, and customer feedback. Therefore, the design of a web portal is crucial in order that it is accessible to a w…
▽ More
Web portals have served as an excellent medium to facilitate user centric services for organizations irrespective of the type, size, and domain of operation. The objective of these portals has been to deliver a plethora of services such as information dissemination, transactional services, and customer feedback. Therefore, the design of a web portal is crucial in order that it is accessible to a wide range of user community irrespective of age group, physical abilities, and level of literacy. In this paper, we have studied the compliance of WCAG 2.0 by three different categories of Indian web sites which are most frequently accessed by a large section of user community. We have provided a quantitative evaluation of different aspects of accessibility which we believe can pave the way for better design of web sites by taking care of the deficiencies inherent in the web portals.
△ Less
Submitted 24 October, 2017;
originally announced October 2017.
-
Accessibility analysis of some Indian educational web portals
Authors:
Manas Ranjan Patra,
Amar Ranjan Dash
Abstract:
Web portals are being considered as excellent means for conducting teaching and learning activities electronically. The number of online services such as course enrollment, tutoring through online course materials, evaluation and even certification through web portals is increasing day by day. However, the effectiveness of an educational web portal depends on its accessibility to a wide range of s…
▽ More
Web portals are being considered as excellent means for conducting teaching and learning activities electronically. The number of online services such as course enrollment, tutoring through online course materials, evaluation and even certification through web portals is increasing day by day. However, the effectiveness of an educational web portal depends on its accessibility to a wide range of students irrespective of their age, and physical abilities. Accessibility of web portals largely depends on their userfriendliness in terms of design, contents, assistive features, and online support. In this paper, we have critically analyzed the web portals of thirty Indian Universities of different categories based on the WCAG 2.0 guidelines. The purpose of this study is to point out the deficiencies that are commonly observed in web portals and help web designers to remove such deficiencies from the academic web portals with a view to enhance their accessibility.
△ Less
Submitted 22 October, 2017;
originally announced October 2017.
