Showing 1–16 of 16 results for author: Ilic, V

ALWOD: Active Learning for Weakly-Supervised Object Detection

Authors: Yuting Wang, Velibor Ilic, Jiatong Li, Branislav Kisacanin, Vladimir Pavlovic

Abstract: Object detection (OD), a crucial vision task, remains challenged by the lack of large training datasets with precise object localization labels. In this work, we propose ALWOD, a new framework that addresses this problem by fusing active learning (AL) with weakly and semi-supervised object detection paradigms. Because the performance of AL critically depends on the model initialization, we propose… ▽ More Object detection (OD), a crucial vision task, remains challenged by the lack of large training datasets with precise object localization labels. In this work, we propose ALWOD, a new framework that addresses this problem by fusing active learning (AL) with weakly and semi-supervised object detection paradigms. Because the performance of AL critically depends on the model initialization, we propose a new auxiliary image generator strategy that utilizes an extremely small labeled set, coupled with a large weakly tagged set of images, as a warm-start for AL. We then propose a new AL acquisition function, another critical factor in AL success, that leverages the student-teacher OD pair disagreement and uncertainty to effectively propose the most informative images to annotate. Finally, to complete the AL loop, we introduce a new labeling task delegated to human annotators, based on selection and correction of model-proposed detections, which is both rapid and effective in labeling the informative images. We demonstrate, across several challenging benchmarks, that ALWOD significantly narrows the gap between the ODs trained on few partially labeled but strategically selected image instances and those that rely on the fully-labeled data. Our code is publicly available on https://github.com/seqam-lab/ALWOD. △ Less

Submitted 14 September, 2023; originally announced September 2023.

Comments: published in ICCV 2023

An overview of generalized entropic forms

Authors: V. Ilić, J. Korbel, S. Gupta, A. M. Scarfone

Abstract: The aim of this focus letter is to present a comprehensive classification of the main entropic forms introduced in the last fifty years in the framework of statistical physics and information theory. Most of them can be grouped into three families, characterized by two-deformation parameters, introduced respectively by Sharma, Taneja, and Mittal (entropies of degree $(α,\,β$)), by Sharma and Mitta… ▽ More The aim of this focus letter is to present a comprehensive classification of the main entropic forms introduced in the last fifty years in the framework of statistical physics and information theory. Most of them can be grouped into three families, characterized by two-deformation parameters, introduced respectively by Sharma, Taneja, and Mittal (entropies of degree $(α,\,β$)), by Sharma and Mittal (entropies of order $(α,\,β)$), and by Hanel and Thurner (entropies of class $(c,\,d)$). Many entropic forms examined will be characterized systematically by means of important concepts such as their axiomatic foundations {\em à la} Shannon-Khinchin and the consequent composability rule for statistically independent systems. Other critical aspects related to the Lesche stability of information measures and their consistency with the Shore-Johnson axioms will be briefly discussed on a general ground. △ Less

Submitted 19 February, 2021; originally announced February 2021.

Comments: 14 pages, 6 tables, no figures, to appear on EPL: Focus Issues "Progresses on Statistical Physics and Complexity"

On the equivalence between four versions of thermostatistics based on strongly pseudo-additive entropies

Authors: Velimir M. Ilic, Antonio Maria Scarfone, Tatsuaki Wada

Abstract: The class of SPA entropies, which can be represented as an increasing continuous transformation of Shannon and Rényi entropies, have intensively been studied in previous decades. Although their mathematical structure has thoroughly been explored and established by GSK axioms, the analysis of their thermostatistical properties have mostly been limited to special cases which belong to two parameter… ▽ More The class of SPA entropies, which can be represented as an increasing continuous transformation of Shannon and Rényi entropies, have intensively been studied in previous decades. Although their mathematical structure has thoroughly been explored and established by GSK axioms, the analysis of their thermostatistical properties have mostly been limited to special cases which belong to two parameter Sharma-Mittal entropy class, such as Tsallis, Renyi and Gaussian entropies. In this paper we present a general analysis of SPA entropies thermostatistics by taking into account both linear and escort constraints on internal energy. We develop two types of dualities between the thermostatistics formalisms. By the first one, the formalism of Rényi entropy is transformed in the formalism of SPA entropy under general energy constraint and, by the second one, the generalized thermostatistics which corresponds to the linear constraint is transformed into the one which corresponds to the escort constraint. Thus, we establish the equivalence between four different thermostatistics formalisms based on Rényi and SPA entropies coupled with linear and escort constraints and we provide the transformation formulas. In this way we obtain a general framework which is applicable to the wide class of entropies and constraints previously discussed in the literature. As an example, we rederive maximum entropy distributions for Sharma-Mittal entropy and we establish new relationships between the corresponding thermodynamic potentials. We obtain, as special cases, previously developed expressions for maximum entropy distributions and thermodynamic quantities for Tsallis, Rényi and Gaussian entropies. In addition, the results are applied for derivation of thermostatistical relationships for supra-extensive entropy, which has not previously been considered. △ Less

Submitted 1 June, 2019; v1 submitted 25 May, 2019; originally announced May 2019.

Comments: 15 pages, 1 figure

Journal ref: Phys. Rev. E 100, 062135 (2019)

Comments on "On q-non-extensive statistics with non-Tsallisian entropy"

Authors: Velimir M. Ilic, Miomir S. Stankovic

Abstract: Recently, in [P. Jizba and J. Korbel, Physica A 444, 2016, 808-827], four generalized Shannon-Khinchin [GSK] axioms have been proposed and a generalized entropy which uniquely satisfies the GSK axioms has been derived. In this comment, we show that the unique class of the entropies derived in the aforementioned paper is not correct, as it violates the fourth GSK axiom, and we derive the correct on… ▽ More Recently, in [P. Jizba and J. Korbel, Physica A 444, 2016, 808-827], four generalized Shannon-Khinchin [GSK] axioms have been proposed and a generalized entropy which uniquely satisfies the GSK axioms has been derived. In this comment, we show that the unique class of the entropies derived in the aforementioned paper is not correct, as it violates the fourth GSK axiom, and we derive the correct one. Nevertheless, the class of entropies proposed in the commented paper still can serve as a basis for generalized statistical mechanics. We propose a new axiomatic system which characterizes the class of entropies. △ Less

Submitted 1 September, 2016; originally announced September 2016.

An axiomatic characterization of generalized entropies under analyticity condition

Authors: Velimir M. Ilic, Miomir S. Stankovic

Abstract: We present the characterization of the Nath, Rényi and Havrda-Charvát-Tsallis entropies under the assumption that they are analytic function with respect to the distribution dimension, unlike the the previous characterizations, which supposes that they are expandable maximized for uniform distribution. We present the characterization of the Nath, Rényi and Havrda-Charvát-Tsallis entropies under the assumption that they are analytic function with respect to the distribution dimension, unlike the the previous characterizations, which supposes that they are expandable maximized for uniform distribution. △ Less

Submitted 1 November, 2013; originally announced November 2013.

Generalized Shannon-Khinchin Axioms and Uniqueness Theorem for Pseudo-additive Entropies

Authors: Velimir M. Ilic, Miomir S. Stankovic

Abstract: We consider the Shannon-Khinchin axiomatic systems for the characterization of generalized entropies such as Sharma-Mital and Frank-Daffertshofer entropy. We provide the generalization of Shannon-Khinchin axioms and give the corresponding uniqueness theorem. The previous attempts at such axiomatizations are also discussed. We consider the Shannon-Khinchin axiomatic systems for the characterization of generalized entropies such as Sharma-Mital and Frank-Daffertshofer entropy. We provide the generalization of Shannon-Khinchin axioms and give the corresponding uniqueness theorem. The previous attempts at such axiomatizations are also discussed. △ Less

Submitted 1 November, 2013; originally announced November 2013.

A unified characterization of generalized information and certainty measures

Authors: Velimir M. Ilic, Miomir S. Stankovic

Abstract: In this paper we consider the axiomatic characterization of information and certainty measures in a unified way. We present the general axiomatic system which captures the common properties of a large number of the measures previously considered by numerous authors. We provide the corresponding characterization theorems and define a new generalized measure called the Inforcer, which is the quasi-l… ▽ More In this paper we consider the axiomatic characterization of information and certainty measures in a unified way. We present the general axiomatic system which captures the common properties of a large number of the measures previously considered by numerous authors. We provide the corresponding characterization theorems and define a new generalized measure called the Inforcer, which is the quasi-linear mean of the function associated to the event probability following the general composition law. In particular, we pay attention to the polynomial composition and the corresponding polynomially composable Inforcer measure. The most common measures appearing in literature can be obtained by specific choice of parameters appearing in our generic measures and they are listed in tables. △ Less

Submitted 17 October, 2013; originally announced October 2013.

Comments on "A two-parameter generalization of Shannon-Khinchin Axioms and the uniqueness theorem"

Authors: Velimir M. Ilic, Edin H. Mulalic, Miomir S. Stankovic

Abstract: Wada and Suyari proposed a two-parameter generalization of Shannon-Khinchin axioms (TGSK axioms) [T. Wada and H. Suyari, Physics Letters A, 368(3)]. We derive a new class of entropies which differs from Wada-Suyari's class by fixing the incorrectness which occurs in the mentioned paper. Also, we consider a two-parameter class of entropies derived from the maxent principle proposed in [Kaniadakis,… ▽ More Wada and Suyari proposed a two-parameter generalization of Shannon-Khinchin axioms (TGSK axioms) [T. Wada and H. Suyari, Physics Letters A, 368(3)]. We derive a new class of entropies which differs from Wada-Suyari's class by fixing the incorrectness which occurs in the mentioned paper. Also, we consider a two-parameter class of entropies derived from the maxent principle proposed in [Kaniadakis, G. and Lissia, M. and Scarfone, AM, Physica A: Statistical Mechanics and its Applications, 340(1)]. We rederived this class by changing initial condition, obtaining the same class as our class derived from TGSK axioms. △ Less

Submitted 29 January, 2013; originally announced January 2013.

Comments: 4 pages

Comments on "Generalization of Shannon-Khinchin axioms to nonextensive systems and the uniqueness theorem for the nonextensive entropy"

Authors: Velimir M. Ilic, Miomir S. Stankovic, Edin H. Mulalic

Abstract: Recently, Suyari has proposed a generalization of Shannon-Khinchin axioms, which determines a class of entropies containing the well-known Tsalis and Havrda-Charvat entropies [H. Suyari, IEEE Trans. Inf. Theory, vol. 50, pp. 1783-1787, Aug. 2004]. In this comment we show that the class of entropy functions determined by Suyari's axioms is wider than the one proposed by Suyari and give a counterexa… ▽ More Recently, Suyari has proposed a generalization of Shannon-Khinchin axioms, which determines a class of entropies containing the well-known Tsalis and Havrda-Charvat entropies [H. Suyari, IEEE Trans. Inf. Theory, vol. 50, pp. 1783-1787, Aug. 2004]. In this comment we show that the class of entropy functions determined by Suyari's axioms is wider than the one proposed by Suyari and give a counterexample. Additionally, we generalize Suyari's axioms characterizing recently introduced class of entropies obtained by averaging pseudoadditive information content introduced in [V. Ilic and M. Stankovic, "Comments on "Nonextensive Entropies derived from Form Invariance of Pseudoadditivity"" Submited, 2012]. △ Less

Submitted 29 November, 2012; originally announced November 2012.

Comments: 4 pages

Comments on "Nonextensive Entropies derived from Form Invariance of Pseudoadditivity"

Authors: Velimir M. Ilic, Miomir S. Stankovic

Abstract: Recently, Suyari has defined nonextensive information content measure with unique class of functions which satisfies certain set of axioms. Nonextensive entropy is then defined as the appropriate expectation value of nonextensive information content [H. Suyari, Phys. Rev E 65 066118 (2002)]. In this comment we show that the class of functions determined by Suyari's axioms is actually wider than th… ▽ More Recently, Suyari has defined nonextensive information content measure with unique class of functions which satisfies certain set of axioms. Nonextensive entropy is then defined as the appropriate expectation value of nonextensive information content [H. Suyari, Phys. Rev E 65 066118 (2002)]. In this comment we show that the class of functions determined by Suyari's axioms is actually wider than the one given by Suyari and we determine the class. Particularly, an information content corresponding to Havrda-Charvat entropy satisfies Suyari's axioms and does not belong to the class given by Suyari but belongs to our class. Moreover, some of the conditions from Suyari's set of axioms are redundant, and some of them can be replaced with more intuitive weaker ones. We give a modification of Suyari's axiomatic system with these weaker assumptions and define the corresponding information content measure. △ Less

Submitted 28 October, 2012; originally announced October 2012.

The computation of first order moments on junction trees

Authors: Milos B. Djuric, Velimir M. Ilic, Miomir S. Stankovic

Abstract: We review some existing methods for the computation of first order moments on junction trees using Shafer-Shenoy algorithm. First, we consider the problem of first order moments computation as vertices problem in junction trees. In this way, the problem is solved using the memory space of an order of the junction tree edge-set cardinality. After that, we consider two algorithms, Lauritzen-Nilsson… ▽ More We review some existing methods for the computation of first order moments on junction trees using Shafer-Shenoy algorithm. First, we consider the problem of first order moments computation as vertices problem in junction trees. In this way, the problem is solved using the memory space of an order of the junction tree edge-set cardinality. After that, we consider two algorithms, Lauritzen-Nilsson algorithm, and Mauá et al. algorithm, which computes the first order moments as the normalization problem in junction tree, using the memory space of an order of the junction tree leaf-set cardinality. △ Less

Submitted 16 January, 2012; originally announced January 2012.

Comments: 9 pages, 1 figure

Cross-moments computation for stochastic context-free grammars

Authors: Velimir M. Ilic, Miroslav D. Ciric, Miomir S. Stankovic

Abstract: In this paper we consider the problem of efficient computation of cross-moments of a vector random variable represented by a stochastic context-free grammar. Two types of cross-moments are discussed. The sample space for the first one is the set of all derivations of the context-free grammar, and the sample space for the second one is the set of all derivations which generate a string belonging to… ▽ More In this paper we consider the problem of efficient computation of cross-moments of a vector random variable represented by a stochastic context-free grammar. Two types of cross-moments are discussed. The sample space for the first one is the set of all derivations of the context-free grammar, and the sample space for the second one is the set of all derivations which generate a string belonging to the language of the grammar. In the past, this problem was widely studied, but mainly for the cross-moments of scalar variables and up to the second order. This paper presents new algorithms for computing the cross-moments of an arbitrary order, and the previously developed ones are derived as special cases. △ Less

Submitted 12 October, 2013; v1 submitted 1 August, 2011; originally announced August 2011.

Entropy Semiring Forward-backward Algorithm for HMM Entropy Computation

Authors: Velimir M. Ilic

Abstract: The paper presents Entropy Semiring Forwardbackward algorithm (ESRFB) and its application for memory efficient computation of the subsequence constrained entropy and state sequence entropy of a Hidden Markov Model (HMM) when an observation sequence is given. ESRFB is based on forward-backward recursion over the entropy semiring, having the lower memory requirement than the algorithm developed by M… ▽ More The paper presents Entropy Semiring Forwardbackward algorithm (ESRFB) and its application for memory efficient computation of the subsequence constrained entropy and state sequence entropy of a Hidden Markov Model (HMM) when an observation sequence is given. ESRFB is based on forward-backward recursion over the entropy semiring, having the lower memory requirement than the algorithm developed by Mann and MacCallum, with the same time complexity. Furthermore, when it is used with forward pass only, it is applicable for the computation of HMM entropy for a given observation sequence, with the same time and memory complexity as the previously developed algorithm by Hernando et al △ Less

Submitted 1 August, 2011; originally announced August 2011.

Comments: submitted to Transactions on Advanced Research

Gradient Computation In Linear-Chain Conditional Random Fields Using The Entropy Message Passing Algorithm

Authors: Velimir M. Ilic, Dejan I. Mancev, Branimir T. Todorovic, Miomir S. Stankovic

Abstract: The paper proposes a numerically stable recursive algorithm for the exact computation of the linear-chain conditional random field gradient. It operates as a forward algorithm over the log-domain expectation semiring and has the purpose of enhancing memory efficiency when applied to long observation sequences. Unlike the traditional algorithm based on the forward-backward recursions, the memory co… ▽ More The paper proposes a numerically stable recursive algorithm for the exact computation of the linear-chain conditional random field gradient. It operates as a forward algorithm over the log-domain expectation semiring and has the purpose of enhancing memory efficiency when applied to long observation sequences. Unlike the traditional algorithm based on the forward-backward recursions, the memory complexity of our algorithm does not depend on the sequence length. The experiments on real data show that it can be useful for the problems which deal with long sequences. △ Less

Submitted 30 May, 2012; v1 submitted 5 November, 2010; originally announced November 2010.

Comments: 11 pages, 2 tables, 3 figures, 2 algorithms

Entropy Message Passing

Authors: Velimir M. Ilic, Miomir S. Stankovic, Branimir T. Todorovic

Abstract: The paper proposes a new message passing algorithm for cycle-free factor graphs. The proposed "entropy message passing" (EMP) algorithm may be viewed as sum-product message passing over the entropy semiring, which has previously appeared in automata theory. The primary use of EMP is to compute the entropy of a model. However, EMP can also be used to compute expressions that appear in expectation m… ▽ More The paper proposes a new message passing algorithm for cycle-free factor graphs. The proposed "entropy message passing" (EMP) algorithm may be viewed as sum-product message passing over the entropy semiring, which has previously appeared in automata theory. The primary use of EMP is to compute the entropy of a model. However, EMP can also be used to compute expressions that appear in expectation maximization and in gradient descent algorithms. △ Less

Submitted 7 November, 2010; v1 submitted 16 June, 2009; originally announced June 2009.

Comments: 5 pages, 1 figure, to appear in IEEE Transactions on Information Theory

Jet Energy Density in Hadron-Hadron Collisions at High Energies

Authors: J. V. Ilic, G. P. Skoro, M. V. Tokarev

Abstract: The average particle multiplicity density dN/deta is the dynamical quantity which reflects some regularities of particle production in low-pT range. The quantity is an important ingredient of z-scaling. Experimental results on charged particle density are available for pp, pA and AA collisions while experimental properties of the jet density are still an open question. The goal of this work is t… ▽ More The average particle multiplicity density dN/deta is the dynamical quantity which reflects some regularities of particle production in low-pT range. The quantity is an important ingredient of z-scaling. Experimental results on charged particle density are available for pp, pA and AA collisions while experimental properties of the jet density are still an open question. The goal of this work is to find the variable which will reflect the main features of the jet production in low transverse energy range and play the role of the scale factor for the scaling function psi(z) and variable z in data z-presentation. The appropriate candidate is the variable we called "scaled jet energy density". Scaled jet energy density is the probability to have a jet with defined ET in defined xT and pseudorapidity regions. The PYTHIA6.2 Monte Carlo generator is used for calculation of scaled jet energy density in proton-proton collisions over a high energy range (sqrt s = 200-14000 GeV) and at eta = 0. The properties of the new variable are discussed and sensitivity to "physical scenarios" applied in the standard Monte Carlo generator is noted. The results of scaled jet energy density at LHC energies are presented and compared with predictions based on z-scaling. △ Less

Submitted 5 December, 2004; originally announced December 2004.

Comments: 11 pages, LaTeX, 8 figures, Presented at the XVII International Baldin Seminar on High Energy Physics Problems "Relativistic Nuclear Physics & Quantum Chromodynamics", Dubna, Russia, September 27 - October 2, 2004