-
This Reads Like That: Deep Learning for Interpretable Natural Language Processing
Authors:
Claudio Fanconi,
Moritz Vandenhirtz,
Severin Husmann,
Julia E. Vogt
Abstract:
Prototype learning, a popular machine learning method designed for inherently interpretable decisions, leverages similarities to learned prototypes for classifying new data. While it is mainly applied in computer vision, in this work, we build upon prior research and further explore the extension of prototypical networks to natural language processing. We introduce a learned weighted similarity me…
▽ More
Prototype learning, a popular machine learning method designed for inherently interpretable decisions, leverages similarities to learned prototypes for classifying new data. While it is mainly applied in computer vision, in this work, we build upon prior research and further explore the extension of prototypical networks to natural language processing. We introduce a learned weighted similarity measure that enhances the similarity computation by focusing on informative dimensions of pre-trained sentence embeddings. Additionally, we propose a post-hoc explainability mechanism that extracts prediction-relevant words from both the prototype and input sentences. Finally, we empirically demonstrate that our proposed method not only improves predictive performance on the AG News and RT Polarity datasets over a previous prototype-based approach, but also improves the faithfulness of explanations compared to rationale-based recurrent convolutions.
△ Less
Submitted 25 October, 2023;
originally announced October 2023.
-
On the Importance of Clinical Notes in Multi-modal Learning for EHR Data
Authors:
Severin Husmann,
Hugo Yèche,
Gunnar Rätsch,
Rita Kuznetsova
Abstract:
Understanding deep learning model behavior is critical to accepting machine learning-based decision support systems in the medical community. Previous research has shown that jointly using clinical notes with electronic health record (EHR) data improved predictive performance for patient monitoring in the intensive care unit (ICU). In this work, we explore the underlying reasons for these improvem…
▽ More
Understanding deep learning model behavior is critical to accepting machine learning-based decision support systems in the medical community. Previous research has shown that jointly using clinical notes with electronic health record (EHR) data improved predictive performance for patient monitoring in the intensive care unit (ICU). In this work, we explore the underlying reasons for these improvements. While relying on a basic attention-based model to allow for interpretability, we first confirm that performance significantly improves over state-of-the-art EHR data models when combining EHR data and clinical notes. We then provide an analysis showing improvements arise almost exclusively from a subset of notes containing broader context on patient state rather than clinician notes. We believe such findings highlight deep learning models for EHR data to be more limited by partially-descriptive data than by modeling choice, motivating a more data-centric approach in the field.
△ Less
Submitted 6 December, 2022;
originally announced December 2022.
-
Company classification using machine learning
Authors:
Sven Husmann,
Antoniya Shivarova,
Rick Steinert
Abstract:
The recent advancements in computational power and machine learning algorithms have led to vast improvements in manifold areas of research. Especially in finance, the application of machine learning enables both researchers and practitioners to gain new insights into financial data and well-studied areas such as company classification. In our paper, we demonstrate that unsupervised machine learnin…
▽ More
The recent advancements in computational power and machine learning algorithms have led to vast improvements in manifold areas of research. Especially in finance, the application of machine learning enables both researchers and practitioners to gain new insights into financial data and well-studied areas such as company classification. In our paper, we demonstrate that unsupervised machine learning algorithms can be used to visualize and classify company data in an economically meaningful and effective way. In particular, we implement the data-driven dimension reduction and visualization tool t-distributed stochastic neighbor embedding (t-SNE) in combination with spectral clustering. The resulting company groups can then be utilized by experts in the field for empirical analysis and optimal decision making. By providing an exemplary out-of-sample study within a portfolio optimization framework, we show that the application of t-SNE and spectral clustering improves the overall portfolio performance. Therefore, we introduce our approach to the financial community as a valuable technique in the context of data analysis and company classification.
△ Less
Submitted 20 May, 2020; v1 submitted 31 March, 2020;
originally announced April 2020.
-
Cross-validated covariance estimators for high-dimensional minimum-variance portfolios
Authors:
Sven Husmann,
Antoniya Shivarova,
Rick Steinert
Abstract:
The global minimum-variance portfolio is a typical choice for investors because of its simplicity and broad applicability. Although it requires only one input, namely the covariance matrix of asset returns, estimating the optimal solution remains a challenge. In the presence of high-dimensionality in the data, the sample covariance estimator becomes ill-conditioned and leads to suboptimal portfoli…
▽ More
The global minimum-variance portfolio is a typical choice for investors because of its simplicity and broad applicability. Although it requires only one input, namely the covariance matrix of asset returns, estimating the optimal solution remains a challenge. In the presence of high-dimensionality in the data, the sample covariance estimator becomes ill-conditioned and leads to suboptimal portfolios out-of-sample. To address this issue, we review recently proposed efficient estimation methods for the covariance matrix and extend the literature by suggesting a multi-fold cross-validation technique for selecting the necessary tuning parameters within each method. Conducting an extensive empirical analysis with four datasets based on the S&P 500, we show that the data-driven choice of specific tuning parameters with the proposed cross-validation improves the out-of-sample performance of the global minimum-variance portfolio. In addition, we identify estimators that are strongly influenced by the choice of the tuning parameter and detect a clear relationship between the selection criterion within the cross-validation and the evaluated performance measure.
△ Less
Submitted 19 October, 2020; v1 submitted 30 October, 2019;
originally announced October 2019.
-
Sparsity and Stability for Minimum-Variance Portfolios
Authors:
Sven Husmann,
Antoniya Shivarova,
Rick Steinert
Abstract:
The popularity of modern portfolio theory has decreased among practitioners because of its unfavorable out-of-sample performance. Estimation errors tend to affect the optimal weight calculation noticeably, especially when a large number of assets is considered. To overcome these issues, many methods have been proposed in recent years, although most only address a small set of practically relevant…
▽ More
The popularity of modern portfolio theory has decreased among practitioners because of its unfavorable out-of-sample performance. Estimation errors tend to affect the optimal weight calculation noticeably, especially when a large number of assets is considered. To overcome these issues, many methods have been proposed in recent years, although most only address a small set of practically relevant questions related to portfolio allocation. This study therefore sheds light on different covariance estimation techniques, combines them with sparse model approaches, and includes a turnover constraint that induces stability. We use two datasets - comprising 319 and 100 companies of the S&P 500, respectively - to create a realistic and reproducible data foundation for our empirical study. To the best of our knowledge, this study is the first to show that it is possible to maintain the low-risk profile of efficient estimation methods while automatically selecting only a subset of assets and further inducing low portfolio turnover. Moreover, we provide evidence that using the LASSO as the sparsity-generating model is insufficient to lower turnover when the involved tuning parameter can change over time.
△ Less
Submitted 25 October, 2019;
originally announced October 2019.
-
Forecasting day ahead electricity spot prices: The impact of the EXAA to other European electricity markets
Authors:
Florian Ziel,
Rick Steinert,
Sven Husmann
Abstract:
In our paper we analyze the relationship between the day-ahead electricity price of the Energy Exchange Austria (EXAA) and other day-ahead electricity prices in Europe. We focus on markets, which settle their prices after the EXAA, which enables traders to include the EXAA price into their calculations. For each market we employ econometric models to incorporate the EXAA price and compare them wit…
▽ More
In our paper we analyze the relationship between the day-ahead electricity price of the Energy Exchange Austria (EXAA) and other day-ahead electricity prices in Europe. We focus on markets, which settle their prices after the EXAA, which enables traders to include the EXAA price into their calculations. For each market we employ econometric models to incorporate the EXAA price and compare them with their counterparts without the price of the Austrian exchange. By employing a forecasting study, we find that electricity price models can be improved when EXAA prices are considered.
△ Less
Submitted 1 December, 2015; v1 submitted 5 January, 2015;
originally announced January 2015.
-
Efficient Modeling and Forecasting of the Electricity Spot Price
Authors:
Florian Ziel,
Rick Steinert,
Sven Husmann
Abstract:
The increasing importance of renewable energy, especially solar and wind power, has led to new forces in the formation of electricity prices. Hence, this paper introduces an econometric model for the hourly time series of electricity prices of the European Power Exchange (EPEX) which incorporates specific features like renewable energy. The model consists of several sophisticated and established a…
▽ More
The increasing importance of renewable energy, especially solar and wind power, has led to new forces in the formation of electricity prices. Hence, this paper introduces an econometric model for the hourly time series of electricity prices of the European Power Exchange (EPEX) which incorporates specific features like renewable energy. The model consists of several sophisticated and established approaches and can be regarded as a periodic VAR-TARCH with wind power, solar power, and load as influences on the time series. It is able to map the distinct and well-known features of electricity prices in Germany. An efficient iteratively reweighted lasso approach is used for the estimation. Moreover, it is shown that several existing models are outperformed by the procedure developed in this paper.
△ Less
Submitted 13 October, 2014; v1 submitted 27 February, 2014;
originally announced February 2014.
