-
Matrix and graph representations of vine copula structures
Authors:
Dániel Pfeifer,
Edith Alice Kovács
Abstract:
Vine copulas can efficiently model multivariate probability distributions. This paper focuses on a more thorough understanding of their structures, since in the literature, vine copula representations are often ambiguous. The graph representations include the original, cherry and chordal graph sequence structures, which we show equivalence between. Importantly we also show a new result, namely tha…
▽ More
Vine copulas can efficiently model multivariate probability distributions. This paper focuses on a more thorough understanding of their structures, since in the literature, vine copula representations are often ambiguous. The graph representations include the original, cherry and chordal graph sequence structures, which we show equivalence between. Importantly we also show a new result, namely that when a perfect elimination ordering of a vine structure is given, then it can always be uniquely represented with a matrix. O. M. Nápoles has shown a way to represent vines in a matrix, and we algorithmify this previous approach, while also showing a new method for constructing such a matrix, through cherry tree sequences. We also calculate the runtime of these algorithms. Lastly, we prove that these two matrix-building algorithms are equivalent if the same perfect elimination ordering is being used.
△ Less
Submitted 10 March, 2023; v1 submitted 10 May, 2022;
originally announced May 2022.
-
LIoTS: League of IoT Sovereignties. A Scalable approach for a Transparent Privacy-safe Federation of Secured IoT Platforms
Authors:
Flavio Cirillo,
Nicola Capuano,
Simon Pietro Romano,
Ernö Kovacs
Abstract:
Internet-of-Things has entered all the fields where data are produced and processed, resulting in a plethora of IoT platforms, typically cloud-based, centralizing data and services management. This has brought to many disjoint IoT silos. Significant efforts have been devoted to integration, recurrently resulting into bigger centralized infrastructures. Such an approach often stumbles upon the relu…
▽ More
Internet-of-Things has entered all the fields where data are produced and processed, resulting in a plethora of IoT platforms, typically cloud-based, centralizing data and services management. This has brought to many disjoint IoT silos. Significant efforts have been devoted to integration, recurrently resulting into bigger centralized infrastructures. Such an approach often stumbles upon the reluctance of IoT system owners to loose the dominion over data. We introduce a secured and privacy-safe infrastructure where a federation overlay is distributed among parties and the data control is kept locally. This establishes a league of peers each sovereign of their IoT system and data: League of IoT Sovereignties (LIoTS). LIoTS is scalable by design, allowing iterative formation of domains levels due to the transparency of its federation. Tests show that the overhead is minimal when exchanged data is hefty, and that LIoTS performs better in large IoT deployments than centralized approaches.
△ Less
Submitted 13 May, 2020;
originally announced May 2020.
-
A Standard-based Open Source IoT Platform: FIWARE
Authors:
Flavio Cirillo,
Gürkan Solmaz,
Everton Luís Berz,
Martin Bauer,
Bin Cheng,
Ernoe Kovacs
Abstract:
The ever-increasing acceleration of technology evolution in all fields is rapidly changing the architectures of data-driven systems towards the Internet-of-Things concept. Many general and specific-purpose IoT platforms are already available. This article introduces the capabilities of the FIWARE framework that is transitioning from a research to a commercial level. We base our exposition on the a…
▽ More
The ever-increasing acceleration of technology evolution in all fields is rapidly changing the architectures of data-driven systems towards the Internet-of-Things concept. Many general and specific-purpose IoT platforms are already available. This article introduces the capabilities of the FIWARE framework that is transitioning from a research to a commercial level. We base our exposition on the analysis of three real-world use cases (global IoT market, analytics in smart cities, and IoT augmented autonomous driving) and their requirements that are addressed with the usage of FIWARE. We highlight the lessons learnt during the design, implementation and deployment phases for each of the use cases and their critical issues. Finally we give two examples showing that FIWARE still maintains openness to innovation: semantics and privacy.
△ Less
Submitted 6 May, 2020;
originally announced May 2020.
-
Copula-based anomaly scoring and localization for large-scale, high-dimensional continuous data
Authors:
Gábor Horváth,
Edith Kovács,
Roland Molontay,
Szabolcs Nováczki
Abstract:
The anomaly detection method presented by this paper has a special feature: it does not only indicate whether an observation is anomalous or not but also tells what exactly makes an anomalous observation unusual. Hence, it provides support to localize the reason of the anomaly.
The proposed approach is model-based; it relies on the multivariate probability distribution associated with the observ…
▽ More
The anomaly detection method presented by this paper has a special feature: it does not only indicate whether an observation is anomalous or not but also tells what exactly makes an anomalous observation unusual. Hence, it provides support to localize the reason of the anomaly.
The proposed approach is model-based; it relies on the multivariate probability distribution associated with the observations. Since the rare events are present in the tails of the probability distributions, we use copula functions, that are able to model the fat-tailed distributions well. The presented procedure scales well; it can cope with a large number of high-dimensional samples. Furthermore, our procedure can cope with missing values, too, which occur frequently in high-dimensional data sets.
In the second part of the paper, we demonstrate the usability of the method through a case study, where we analyze a large data set consisting of the performance counters of a real mobile telecommunication network. Since such networks are complex systems, the signs of sub-optimal operation can remain hidden for a potentially long time. With the proposed procedure, many such hidden issues can be isolated and indicated to the network operator.
△ Less
Submitted 4 December, 2019;
originally announced December 2019.
-
New stable method to solve heat conduction problems in extremely large systems
Authors:
Endre Kovács,
András Gilicz
Abstract:
We present a new explicit and stable numerical algorithm to solve the homogeneous heat equation. We illustrate the performance of the new method in the cases of two 2D systems with highly inhomogeneous random parameters. Spatial discretization of these problems results in huge and stiff ordinary differential equation systems, which can be solved by our novel method faster than by explicit or the c…
▽ More
We present a new explicit and stable numerical algorithm to solve the homogeneous heat equation. We illustrate the performance of the new method in the cases of two 2D systems with highly inhomogeneous random parameters. Spatial discretization of these problems results in huge and stiff ordinary differential equation systems, which can be solved by our novel method faster than by explicit or the commonly used implicit methods.
△ Less
Submitted 24 August, 2019;
originally announced August 2019.
-
Standards-Based Worldwide Semantic Interoperability for IoT
Authors:
Erno Kovacs,
Martin Bauer,
Jaeho Kim,
Jaeseok Yun,
Franck Le Gall,
Mengxuan Zhao
Abstract:
Global IoT services (GIoTS) are combining locally available IoT resources with Cloud-based services. They are targeting world-wide services. GIoTS require interoperability between the locally installed heterogeneous IoT systems. Semantic processing is an important technology to enable data mediation as well as knowledge-based processing. This paper explains a system architecture for achieving worl…
▽ More
Global IoT services (GIoTS) are combining locally available IoT resources with Cloud-based services. They are targeting world-wide services. GIoTS require interoperability between the locally installed heterogeneous IoT systems. Semantic processing is an important technology to enable data mediation as well as knowledge-based processing. This paper explains a system architecture for achieving world-wide semantic interoperability using international standards like oneM2M and the OMA NGSI-9/10 context interfaces (as used in the European Future Internet Platform FIWARE). Semantics also enables the use of Knowledge-based Semantic Processing Agents. Furthermore, we explain how semantic verification enables the testing of such complex systems.
△ Less
Submitted 1 August, 2018;
originally announced August 2018.
-
Discovering the Markov network structure
Authors:
Edith Kovács,
Tamás Szántai
Abstract:
In this paper a new proof is given for the supermodularity of information content. Using the decomposability of the information content an algorithm is given for discovering the Markov network graph structure endowed by the pairwise Markov property of a given probability distribution. A discrete probability distribution is given for which the equivalence of Hammersley-Clifford theorem is fulfilled…
▽ More
In this paper a new proof is given for the supermodularity of information content. Using the decomposability of the information content an algorithm is given for discovering the Markov network graph structure endowed by the pairwise Markov property of a given probability distribution. A discrete probability distribution is given for which the equivalence of Hammersley-Clifford theorem is fulfilled although some of the possible vector realizations are taken on with zero probability. Our algorithm for discovering the pairwise Markov network is illustrated on this example, too.
△ Less
Submitted 2 July, 2013;
originally announced July 2013.
-
Vine copulas as a mean for the construction of high dimensional probability distribution associated to a Markov Network
Authors:
Edith Kovacs,
Tamas Szantai
Abstract:
Building higher-dimensional copulas is generally recognized as a difficult problem. Regular-vines using bivariate copulas provide a flexible class of high-dimensional dependency models. In large dimensions, the drawback of the model is the exponentially increasing complexity. Recognizing some of the conditional independences is a possibility for reducing the number of levels of the pair-copula dec…
▽ More
Building higher-dimensional copulas is generally recognized as a difficult problem. Regular-vines using bivariate copulas provide a flexible class of high-dimensional dependency models. In large dimensions, the drawback of the model is the exponentially increasing complexity. Recognizing some of the conditional independences is a possibility for reducing the number of levels of the pair-copula decomposition, and hence to simplify its construction Aas et al (2009). The idea of using conditional independences was already performed under elliptical copula assumptions Hanea, Kurowicka and Cooke (2006), Kurowicka and Cooke (2002) and in the case of DAGs in a recent work Bauer, Czado and Klein (2011). We provide a method which uses some of the conditional independences encoded by the Markov network underlying the variables. We give a theorem which under some graph conditions makes possible to derive pair-copula decomposition of the probability density function associated to a Markov network. As the underlying Markov network is usually unknown, we first have to discover it from the sample data. Using our results published in Szantai and Kovacs (2008) and Kovacs and Szantai (2010a) we will show how to derive a multidimensional copula model exploiting the information on conditional independences hidden in the sample data.
△ Less
Submitted 6 June, 2012; v1 submitted 9 May, 2011;
originally announced May 2011.
-
Discovering a junction tree behind a Markov network by a greedy algorithm
Authors:
Tamas Szantai,
Edith Kovacs
Abstract:
In an earlier paper we introduced a special kind of k-width junction tree, called k-th order t-cherry junction tree in order to approximate a joint probability distribution. The approximation is the best if the Kullback-Leibler divergence between the true joint probability distribution and the approximating one is minimal. Finding the best approximating k-width junction tree is NP-complete if k>2.…
▽ More
In an earlier paper we introduced a special kind of k-width junction tree, called k-th order t-cherry junction tree in order to approximate a joint probability distribution. The approximation is the best if the Kullback-Leibler divergence between the true joint probability distribution and the approximating one is minimal. Finding the best approximating k-width junction tree is NP-complete if k>2. In our earlier paper we also proved that the best approximating k-width junction tree can be embedded into a k-th order t-cherry junction tree. We introduce a greedy algorithm resulting very good approximations in reasonable computing time.
In this paper we prove that if the Markov network underlying fullfills some requirements then our greedy algorithm is able to find the true probability distribution or its best approximation in the family of the k-th order t-cherry tree probability distributions. Our algorithm uses just the k-th order marginal probability distributions as input.
We compare the results of the greedy algorithm proposed in this paper with the greedy algorithm proposed by Malvestuto in 1991.
△ Less
Submitted 18 April, 2011; v1 submitted 14 April, 2011;
originally announced April 2011.
