-
Extracting Interpretable Local and Global Representations from Attention on Time Series
Authors:
Leonid Schwenke,
Martin Atzmueller
Abstract:
This paper targets two transformer attention based interpretability methods working with local abstraction and global representation, in the context of time series data. We distinguish local and global contexts, and provide a comprehensive framework for both general interpretation options. We discuss their specific instantiation via different methods in detail, also outlining their respective comp…
▽ More
This paper targets two transformer attention based interpretability methods working with local abstraction and global representation, in the context of time series data. We distinguish local and global contexts, and provide a comprehensive framework for both general interpretation options. We discuss their specific instantiation via different methods in detail, also outlining their respective computational implementation and abstraction variants. Furthermore, we provide extensive experimentation demonstrating the efficacy of the presented approaches. In particular, we perform our experiments using a selection of univariate datasets from the UCR UEA time series repository where we both assess the performance of the proposed approaches, as well as their impact on explainability and interpretability/complexity. Here, with an extensive analysis of hyperparameters, the presented approaches demonstrate an significant improvement in interpretability/complexity, while capturing many core decisions of and maintaining a similar performance to the baseline model. Finally, we draw general conclusions outlining and guiding the application of the presented methods.
△ Less
Submitted 15 September, 2023;
originally announced December 2023.
-
Graph Neural Networks for Multivariate Time Series Regression with Application to Seismic Data
Authors:
Stefan Bloemheuvel,
Jurgen van den Hoogen,
Dario Jozinović,
Alberto Michelini,
Martin Atzmueller
Abstract:
Machine learning, with its advances in deep learning has shown great potential in analyzing time series. In many scenarios, however, additional information that can potentially improve the predictions is available. This is crucial for data that arise from e.g., sensor networks that contain information about sensor locations. Then, such spatial information can be exploited by modeling it via graph…
▽ More
Machine learning, with its advances in deep learning has shown great potential in analyzing time series. In many scenarios, however, additional information that can potentially improve the predictions is available. This is crucial for data that arise from e.g., sensor networks that contain information about sensor locations. Then, such spatial information can be exploited by modeling it via graph structures, along with the sequential (time series) information. Recent advances in adapting deep learning to graphs have shown potential in various tasks. However, these methods have not been adapted for time series tasks to a great extent. Most attempts have essentially consolidated around time series forecasting with small sequence lengths. Generally, these architectures are not well suited for regression or classification tasks where the value to be predicted is not strictly depending on the most recent values, but rather on the whole length of the time series. We propose TISER-GCN, a novel graph neural network architecture for processing, in particular, these long time series in a multivariate regression task. Our proposed model is tested on two seismic datasets containing earthquake waveforms, where the goal is to predict maximum intensity measurements of ground shaking at each seismic station. Our findings demonstrate promising results of our approach -- with an average MSE reduction of 16.3% - compared to the best performing baselines. In addition, our approach matches the baseline scores by needing only half the input size. The results are discussed in depth with an additional ablation study.
△ Less
Submitted 31 October, 2022; v1 submitted 3 January, 2022;
originally announced January 2022.
-
Local Exceptionality Detection in Time Series Using Subgroup Discovery
Authors:
Dan Hudson,
Travis J. Wiltshire,
Martin Atzmueller
Abstract:
In this paper, we present a novel approach for local exceptionality detection on time series data. This method provides the ability to discover interpretable patterns in the data, which can be used to understand and predict the progression of a time series. This being an exploratory approach, the results can be used to generate hypotheses about the relationships between the variables describing a…
▽ More
In this paper, we present a novel approach for local exceptionality detection on time series data. This method provides the ability to discover interpretable patterns in the data, which can be used to understand and predict the progression of a time series. This being an exploratory approach, the results can be used to generate hypotheses about the relationships between the variables describing a specific process and its dynamics. We detail our approach in a concrete instantiation and exemplary implementation, specifically in the field of teamwork research. Using a real-world dataset of team interactions we include results from an example data analytics application of our proposed approach, showcase novel analysis options, and discuss possible implications of the results from the perspective of teamwork research.
△ Less
Submitted 5 August, 2021;
originally announced August 2021.
-
A Computational Framework for Modeling Complex Sensor Network Data Using Graph Signal Processing and Graph Neural Networks in Structural Health Monitoring
Authors:
Stefan Bloemheuvel,
Jurgen van den Hoogen,
Martin Atzmueller
Abstract:
Complex networks lend themselves to the modeling of multidimensional data, such as relational and/or temporal data. In particular, when such complex data and their inherent relationships need to be formalized, complex network modeling and its resulting graph representations enable a wide range of powerful options. In this paper, we target this - connected to specific machine learning approaches on…
▽ More
Complex networks lend themselves to the modeling of multidimensional data, such as relational and/or temporal data. In particular, when such complex data and their inherent relationships need to be formalized, complex network modeling and its resulting graph representations enable a wide range of powerful options. In this paper, we target this - connected to specific machine learning approaches on graphs for structural health monitoring on an analysis and predictive (maintenance) perspective. Specifically, we present a framework based on Complex Network Modeling, integrating Graph Signal Processing (GSP) and Graph Neural Network (GNN) approaches. We demonstrate this framework in our targeted application domain of Structural Health Monitoring (SHM). In particular, we focus on a prominent real-world structural health monitoring use case, i.e., modeling and analyzing sensor data (strain, vibration) of a large bridge in the Netherlands. In our experiments, we show that GSP enables the identification of the most important sensors, for which we investigate a set of search and optimization approaches. Furthermore, GSP enables the detection of specific graph signal patterns (mode shapes), capturing physical functional properties of the sensors in the applied complex network. In addition, we show the efficacy of applying GNNs for strain prediction on this kind of data.
△ Less
Submitted 1 May, 2021;
originally announced May 2021.
-
Social Analytics of Team Interaction using Dynamic Complexity Heat Maps and Network Visualizations
Authors:
Travis J. Wiltshire,
Dan Hudson,
Philia Lijdsman,
Stijn Wever,
Martin Atzmueller
Abstract:
Given the increasing complexity of many sociotechnical work domains, effective teamwork has become increasingly crucial. While there is evidence that face-to-face communication contributes to effective teamwork, methods for understanding the time-varying nature and structure of team communication are limited. In this work, we combine sensor-based social analytics of Sociometric badges (Rhythm Badg…
▽ More
Given the increasing complexity of many sociotechnical work domains, effective teamwork has become increasingly crucial. While there is evidence that face-to-face communication contributes to effective teamwork, methods for understanding the time-varying nature and structure of team communication are limited. In this work, we combine sensor-based social analytics of Sociometric badges (Rhythm Badge) with two visualization techniques (Dynamic Complexity Heat Maps and Network Visualizations) to advance an intuitive way of understanding the dynamics of team interaction. To demonstrate the utility of our approach, we provide a case study that examines one team's interaction for a Lost at Sea simulation. We were able to recover transitions in the task and team interaction as well as uncover structural changes in team member energy and engagement, which we visualize using networks. Taken together, this work represents an important first step at optimizing team effectiveness by identifying critical transitions/events in team communication and interaction patterns.
△ Less
Submitted 5 August, 2021; v1 submitted 7 September, 2020;
originally announced September 2020.
-
Feature-Set-Engineering for Detecting Freezing of Gait in Parkinson's Disease using Deep Recurrent Neural Networks
Authors:
Spyroula Masiala,
Willem Huijbers,
Martin Atzmueller
Abstract:
Freezing of gait (FoG) is a common gait disability in Parkinson's disease, that usually appears in its advanced stage. Freeze episodes are associated with falls, injuries, and psychological consequences, negatively affecting the patients' quality of life. For detecting FoG episodes automatically, a highly accurate detection method is necessary. This paper presents an approach for detecting FoG epi…
▽ More
Freezing of gait (FoG) is a common gait disability in Parkinson's disease, that usually appears in its advanced stage. Freeze episodes are associated with falls, injuries, and psychological consequences, negatively affecting the patients' quality of life. For detecting FoG episodes automatically, a highly accurate detection method is necessary. This paper presents an approach for detecting FoG episodes utilizing a deep recurrent neural network (RNN) on 3D-accelerometer measurements. We investigate suitable features and feature combinations extracted from the sensors' time series data. Specifically, for detecting FoG episodes, we apply a deep RNN with Long Short-Term Memory cells. In our experiments, we perform both user dependent and user independent experiments, to detect freeze episodes. Our experimental results show that the frequency domain features extracted from the trunk sensor are the most informative feature group in the subject independent method, achieving an average AUC score of 93%, Specificity of 90% and Sensitivity of 81%. Moreover, frequency and statistical features of all the sensors are identified as the best single input for the subject dependent method, achieving an average AUC score of 97%, Specificity of 96% and Sensitivity of 87%. Overall, in a comparison to state-of-the-art approaches from literature as baseline methods, our proposed approach outperforms these significantly.
△ Less
Submitted 8 September, 2019;
originally announced September 2019.
-
Towards Generating Explanations for ASP-Based Link Analysis using Declarative Program Transformations
Authors:
Martin Atzmueller,
Cicek Güven,
Dietmar Seipel
Abstract:
The explication and the generation of explanations are prominent topics in artificial intelligence and data science, in order to make methods and systems more transparent and understandable for humans. This paper investigates the problem of link analysis, specifically link prediction and anomalous link discovery in social networks using the declarative method of Answer set programming (ASP). Apply…
▽ More
The explication and the generation of explanations are prominent topics in artificial intelligence and data science, in order to make methods and systems more transparent and understandable for humans. This paper investigates the problem of link analysis, specifically link prediction and anomalous link discovery in social networks using the declarative method of Answer set programming (ASP). Applying ASP for link prediction provides a powerful declarative approach, e.g., for incorporating domain knowledge for explicative prediction. In this context, we propose a novel method for generating explanations - as offline justifications - using declarative program transformations. The method itself is purely based on syntactic transformations of declarative programs, e.g., in an ASP formalism, using rule instrumentation. We demonstrate the efficacy of the proposed approach, exemplifying it in an application on link analysis in social networks, also including domain knowledge.
△ Less
Submitted 8 September, 2019;
originally announced September 2019.
-
Finding Maximal Non-Redundant Association Rules in Tennis Data
Authors:
Daniel Weidner,
Martin Atzmueller,
Dietmar Seipel
Abstract:
The concept of association rules is well--known in data mining. But often redundancy and subsumption are not considered, and standard approaches produce thousands or even millions of resulting association rules. Without further information or post--mining approaches, this huge number of rules is typically useless for the domain specialist -- which is an instance of the infamous pattern explosion p…
▽ More
The concept of association rules is well--known in data mining. But often redundancy and subsumption are not considered, and standard approaches produce thousands or even millions of resulting association rules. Without further information or post--mining approaches, this huge number of rules is typically useless for the domain specialist -- which is an instance of the infamous pattern explosion problem. In this work, we present a new definition of redundancy and subsumption based on the confidence and the support of the rules and propose post-- mining to prune a set of association rules. In a case study, we apply our method to association rules mined from spatio--temporal data. The data represent the trajectories of the ball in tennis matches -- more precisely, the points/times the tennis ball hits the ground. The goal is to analyze the strategies of the players and to try to improve their performance by looking at the resulting association rules. The proposed approach is general, and can also be applied to other spatio--temporal data with a similar structure.
△ Less
Submitted 3 September, 2019;
originally announced September 2019.
-
Adaptive kNN using Expected Accuracy for Classification of Geo-Spatial Data
Authors:
Mark Kibanov,
Martin Becker,
Juergen Mueller,
Martin Atzmueller,
Andreas Hotho,
Gerd Stumme
Abstract:
The k-Nearest Neighbor (kNN) classification approach is conceptually simple - yet widely applied since it often performs well in practical applications. However, using a global constant k does not always provide an optimal solution, e.g., for datasets with an irregular density distribution of data points. This paper proposes an adaptive kNN classifier where k is chosen dynamically for each instanc…
▽ More
The k-Nearest Neighbor (kNN) classification approach is conceptually simple - yet widely applied since it often performs well in practical applications. However, using a global constant k does not always provide an optimal solution, e.g., for datasets with an irregular density distribution of data points. This paper proposes an adaptive kNN classifier where k is chosen dynamically for each instance (point) to be classified, such that the expected accuracy of classification is maximized. We define the expected accuracy as the accuracy of a set of structurally similar observations. An arbitrary similarity function can be used to find these observations. We introduce and evaluate different similarity functions. For the evaluation, we use five different classification tasks based on geo-spatial data. Each classification task consists of (tens of) thousands of items. We demonstrate, that the presented expected accuracy measures can be a good estimator for kNN performance, and the proposed adaptive kNN classifier outperforms common kNN and previously introduced adaptive kNN algorithms. Also, we show that the range of considered k can be significantly reduced to speed up the algorithm without negative influence on classification accuracy.
△ Less
Submitted 14 December, 2017;
originally announced January 2018.
-
Link Prediction and the Role of Stronger Ties in Networks of Face-to-Face Proximity
Authors:
Christoph Scholz,
Martin Atzmueller,
Gerd Stumme
Abstract:
Understanding the structures why links are formed is an important and prominent research topic. In this paper, we therefore consider the link prediction problem in face-to-face contact networks, and analyze the predictability of new and recurring links. Furthermore, we study additional influence factors, and the role of stronger ties in these networks. Specifically, we compare neighborhood-based a…
▽ More
Understanding the structures why links are formed is an important and prominent research topic. In this paper, we therefore consider the link prediction problem in face-to-face contact networks, and analyze the predictability of new and recurring links. Furthermore, we study additional influence factors, and the role of stronger ties in these networks. Specifically, we compare neighborhood-based and path-based network proximity measures in a threshold-based analysis for capturing temporal dynamics. The results and insights of the analysis are a first step onto predictability applications for human contact networks, for example, for improving recommendations.
△ Less
Submitted 8 July, 2014;
originally announced July 2014.
-
On the Predictability of Talk Attendance at Academic Conferences
Authors:
Christoph Scholz,
Jens Illig,
Martin Atzmueller,
Gerd Stumme
Abstract:
This paper focuses on the prediction of real-world talk attendances at academic conferences with respect to different influence factors. We study the predictability of talk attendances using real-world tracked face-to-face contacts. Furthermore, we investigate and discuss the predictive power of user interests extracted from the users' previous publications. We apply Hybrid Rooted PageRank, a stat…
▽ More
This paper focuses on the prediction of real-world talk attendances at academic conferences with respect to different influence factors. We study the predictability of talk attendances using real-world tracked face-to-face contacts. Furthermore, we investigate and discuss the predictive power of user interests extracted from the users' previous publications. We apply Hybrid Rooted PageRank, a state-of-the-art unsupervised machine learning method that combines information from different sources. Using this method, we analyze and discuss the predictive power of contact and interest networks separately and in combination. We find that contact and similarity networks achieve comparable results, and that combinations of different networks can only to a limited extend help to improve the prediction quality. For our experiments, we analyze the predictability of talk attendance at the ACM Conference on Hypertext and Hypermedia 2011 collected using the conference management system Conferator.
△ Less
Submitted 2 July, 2014;
originally announced July 2014.
-
Data Mining on Social Interaction Networks
Authors:
Martin Atzmueller
Abstract:
Social media and social networks have already woven themselves into the very fabric of everyday life. This results in a dramatic increase of social data capturing various relations between the users and their associated artifacts, both in online networks and the real world using ubiquitous devices. In this work, we consider social interaction networks from a data mining perspective - also with a s…
▽ More
Social media and social networks have already woven themselves into the very fabric of everyday life. This results in a dramatic increase of social data capturing various relations between the users and their associated artifacts, both in online networks and the real world using ubiquitous devices. In this work, we consider social interaction networks from a data mining perspective - also with a special focus on real-world face-to-face contact networks: We combine data mining and social network analysis techniques for examining the networks in order to improve our understanding of the data, the modeled behavior, and its underlying emergent processes. Furthermore, we adapt, extend and apply known predictive data mining algorithms on social interaction networks. Additionally, we present novel methods for descriptive data mining for uncovering and extracting relations and patterns for hypothesis generation and exploration, in order to provide characteristic information about the data and networks. The presented approaches and methods aim at extracting valuable knowledge for enhancing the understanding of the respective data, and for supporting the users of the respective systems. We consider data from several social systems, like the social bookmarking system BibSonomy, the social resource sharing system flickr, and ubiquitous social systems: Specifically, we focus on data from the social conference guidance system Conferator and the social group interaction system MyGroup. This work first gives a short introduction into social interaction networks, before we describe several analysis results in the context of online social networks and real-world face-to-face contact networks. Next, we present predictive data mining methods, i.e., for localization, recommendation and link prediction. After that, we present novel descriptive data mining methods for mining communities and patterns.
△ Less
Submitted 15 March, 2014; v1 submitted 23 December, 2013;
originally announced December 2013.
-
User-Relatedness and Community Structure in Social Interaction Networks
Authors:
Folke Mitzlaff,
Martin Atzmueller,
Dominik Benz,
Andreas Hotho,
Gerd Stumme
Abstract:
With social media and the according social and ubiquitous applications finding their way into everyday life, there is a rapidly growing amount of user generated content yielding explicit and implicit network structures. We consider social activities and phenomena as proxies for user relatedness. Such activities are represented in so-called social interaction networks or evidence networks, with dif…
▽ More
With social media and the according social and ubiquitous applications finding their way into everyday life, there is a rapidly growing amount of user generated content yielding explicit and implicit network structures. We consider social activities and phenomena as proxies for user relatedness. Such activities are represented in so-called social interaction networks or evidence networks, with different degrees of explicitness. We focus on evidence networks containing relations on users, which are represented by connections between individual nodes. Explicit interaction networks are then created by specific user actions, for example, when building a friend network. On the other hand, more implicit networks capture user traces or evidences of user actions as observed in Web portals, blogs, resource sharing systems, and many other social services. These implicit networks can be applied for a broad range of analysis methods instead of using expensive gold-standard information.
In this paper, we analyze different properties of a set of networks in social media. We show that there are dependencies and correlations between the networks. These allow for drawing reciprocal conclusions concerning pairs of networks, based on the assessment of structural correlations and ranking interchangeability. Additionally, we show how these inter-network correlations can be used for assessing the results of structural analysis techniques, e.g., community mining methods.
△ Less
Submitted 16 September, 2013;
originally announced September 2013.
