Showing 1–10 of 10 results for author: Barz, M

A look under the hood of the Interactive Deep Learning Enterprise (No-IDLE)

Authors: Daniel Sonntag, Michael Barz, Thiago Gouvêa

Abstract: This DFKI technical report presents the anatomy of the No-IDLE prototype system (funded by the German Federal Ministry of Education and Research) that provides not only basic and fundamental research in interactive machine learning, but also reveals deeper insights into users' behaviours, needs, and goals. Machine learning and deep learning should become accessible to millions of end users. No-IDL… ▽ More This DFKI technical report presents the anatomy of the No-IDLE prototype system (funded by the German Federal Ministry of Education and Research) that provides not only basic and fundamental research in interactive machine learning, but also reveals deeper insights into users' behaviours, needs, and goals. Machine learning and deep learning should become accessible to millions of end users. No-IDLE's goals and scienfific challenges centre around the desire to increase the reach of interactive deep learning solutions for non-experts in machine learning. One of the key innovations described in this technical report is a methodology for interactive machine learning combined with multimodal interaction which will become central when we start interacting with semi-intelligent machines in the upcoming area of neural networks and large language models. △ Less

Submitted 27 June, 2024; originally announced June 2024.

Comments: DFKI Technical Report

I-MPN: Inductive Message Passing Network for Effective and Efficient Human-in-the-Loop Annotation of Mobile Eye Tracking Data

Authors: Hoang H. Le, Duy M. H. Nguyen, Omair Shahzad Bhatti, Laszlo Kopacsi, Thinh P. Ngo, Binh T. Nguyen, Michael Barz, Daniel Sonntag

Abstract: Understanding human visual processing in dynamic environments is essential for psychology and human-centered interaction design. Mobile eye-tracking systems, combining egocentric video and gaze signals, offer valuable insights. However, manual analysis of these recordings is time-intensive. In this work, we present a novel human-centered learning algorithm designed for automated object recognition… ▽ More Understanding human visual processing in dynamic environments is essential for psychology and human-centered interaction design. Mobile eye-tracking systems, combining egocentric video and gaze signals, offer valuable insights. However, manual analysis of these recordings is time-intensive. In this work, we present a novel human-centered learning algorithm designed for automated object recognition within mobile eye-tracking settings. Our approach seamlessly integrates an object detector with an inductive message-passing network technique (I-MPN), harnessing node features such as node profile information and positions. This integration enables our algorithm to learn embedding functions capable of generalizing to new object angle views, thereby facilitating rapid adaptation and efficient reasoning in dynamic contexts as users navigate through their environment. Through experiments conducted on three distinct video sequences, our \textit{interactive-based method} showcases significant performance improvements over fixed training/testing algorithms, even when trained on considerably smaller annotated samples collected through user feedback. Furthermore, we showcase exceptional efficiency in data annotation processes, surpassing approaches that use complete object detectors, combine detectors with convolutional networks, or employ interactive video segmentation. △ Less

Submitted 10 June, 2024; originally announced June 2024.

Comments: First version

DRG-Net: Interactive Joint Learning of Multi-lesion Segmentation and Classification for Diabetic Retinopathy Grading

Authors: Hasan Md Tusfiqur, Duy M. H. Nguyen, Mai T. N. Truong, Triet A. Nguyen, Binh T. Nguyen, Michael Barz, Hans-Juergen Profitlich, Ngoc T. T. Than, Ngan Le, Pengtao Xie, Daniel Sonntag

Abstract: Diabetic Retinopathy (DR) is a leading cause of vision loss in the world, and early DR detection is necessary to prevent vision loss and support an appropriate treatment. In this work, we leverage interactive machine learning and introduce a joint learning framework, termed DRG-Net, to effectively learn both disease grading and multi-lesion segmentation. Our DRG-Net consists of two modules: (i) DR… ▽ More Diabetic Retinopathy (DR) is a leading cause of vision loss in the world, and early DR detection is necessary to prevent vision loss and support an appropriate treatment. In this work, we leverage interactive machine learning and introduce a joint learning framework, termed DRG-Net, to effectively learn both disease grading and multi-lesion segmentation. Our DRG-Net consists of two modules: (i) DRG-AI-System to classify DR Grading, localize lesion areas, and provide visual explanations; (ii) DRG-Expert-Interaction to receive feedback from user-expert and improve the DRG-AI-System. To deal with sparse data, we utilize transfer learning mechanisms to extract invariant feature representations by using Wasserstein distance and adversarial learning-based entropy minimization. Besides, we propose a novel attention strategy at both low- and high-level features to automatically select the most significant lesion information and provide explainable properties. In terms of human interaction, we further develop DRG-Net as a tool that enables expert users to correct the system's predictions, which may then be used to update the system as a whole. Moreover, thanks to the attention mechanism and loss functions constraint between lesion features and classification features, our approach can be robust given a certain level of noise in the feedback of users. We have benchmarked DRG-Net on the two largest DR datasets, i.e., IDRID and FGADR, and compared it to various state-of-the-art deep learning networks. In addition to outperforming other SOTA approaches, DRG-Net is effectively updated using user feedback, even in a weakly-supervised manner. △ Less

Submitted 30 December, 2022; originally announced December 2022.

Comments: First version

Incremental Improvement of a Question Answering System by Re-ranking Answer Candidates using Machine Learning

Authors: Michael Barz, Daniel Sonntag

Abstract: We implement a method for re-ranking top-10 results of a state-of-the-art question answering (QA) system. The goal of our re-ranking approach is to improve the answer selection given the user question and the top-10 candidates. We focus on improving deployed QA systems that do not allow re-training or re-training comes at a high cost. Our re-ranking approach learns a similarity function using n-gr… ▽ More We implement a method for re-ranking top-10 results of a state-of-the-art question answering (QA) system. The goal of our re-ranking approach is to improve the answer selection given the user question and the top-10 candidates. We focus on improving deployed QA systems that do not allow re-training or re-training comes at a high cost. Our re-ranking approach learns a similarity function using n-gram based features using the query, the answer and the initial system confidence as input. Our contributions are: (1) we generate a QA training corpus starting from 877 answers from the customer care domain of T-Mobile Austria, (2) we implement a state-of-the-art QA pipeline using neural sentence embeddings that encode queries in the same space than the answer index, and (3) we evaluate the QA pipeline and our re-ranking approach using a separately provided test set. The test set can be considered to be available after deployment of the system, e.g., based on feedback of users. Our results show that the system performance, in terms of top-n accuracy and the mean reciprocal rank, benefits from re-ranking using gradient boosted regression trees. On average, the mean reciprocal rank improves by 9.15%. △ Less

Submitted 27 August, 2019; originally announced August 2019.

Comments: Accepted for oral presentation at tenth International Workshop on Spoken Dialogue Systems Technology (IWSDS) 2019

How ill-defined constituents produce well-defined nanoparticles: Effect of polymer dispersity on the uniformity of copolymeric micelles

Authors: Sriteja Mantha, Shuanhu Qi, Matthias Barz, Friederike Schmid

Abstract: We investigate the effect of polymer length dispersity on the properties of self-assembled micelles in solution by self-consistent field calculations. Polydispersity stabilizes micelles by raising the free energy barriers of micelle formation and dissolution. Most importantly, it significantly reduces the size fluctuations of micelles: Block copolymers of moderate polydispersity form more uniform… ▽ More We investigate the effect of polymer length dispersity on the properties of self-assembled micelles in solution by self-consistent field calculations. Polydispersity stabilizes micelles by raising the free energy barriers of micelle formation and dissolution. Most importantly, it significantly reduces the size fluctuations of micelles: Block copolymers of moderate polydispersity form more uniform particles than their monodisperse counterparts. We attribute this to the fact that the packing of the solvophobic monomers in the core can be optimized if the constituent polymers have different length. △ Less

Submitted 28 January, 2019; originally announced January 2019.

Comments: 4 main figures and 4 supplementary figures. Manuscript accepted for publication in Physical Review Materials

Poly-Sarcosine and Poly(ethylene-glycol) interactions with proteins investigated using molecular dynamics simulations

Authors: Giovanni Settanni, Timo Schäfer, Christian Muhl, Matthias Barz, Friederike Schmid

Abstract: Nanoparticles coated with hydrophilic polymers often show a reduction in unspecific interactions with the biological environment, which improves their biocompatibility. The molecular determinants of this reduction are not very well understood yet, and their knowledge may help improving nanoparticle design. Here we address, using molecular dynamics simulations, the interactions of human serum album… ▽ More Nanoparticles coated with hydrophilic polymers often show a reduction in unspecific interactions with the biological environment, which improves their biocompatibility. The molecular determinants of this reduction are not very well understood yet, and their knowledge may help improving nanoparticle design. Here we address, using molecular dynamics simulations, the interactions of human serum albumin, the most abundant serum protein, with two promising hydrophilic polymers used for the coating of therapeutic nanoparticles, poly(ethylene-glycol) and poly-sarcosine. By simulating the protein immersed in a polymer-water mixture, we show that the two polymers have a very similar affinity for the protein surface, both in terms of the amount of polymer adsorbed and also in terms of the type of amino acids mainly involved in the interactions. We further analyze the kinetics of adsorption and how it affects the polymer conformations. Minor differences between the polymers are observed in the thickness of the adsorption layer, that are related to the different degree of flexibility of the two molecules. In comparison poly-alanine, an isomer of poly-sarcosine known to self-aggregate and induce protein aggregation, shows a significantly larger affinity for the protein surface than PEG and PSar, which we show to be related not to a different patterns of interactions with the protein surface, but to the different way the polymer interacts with water. △ Less

Submitted 29 October, 2018; originally announced October 2018.

A categorisation and implementation of digital pen features for behaviour characterisation

Authors: Alexander Prange, Michael Barz, Daniel Sonntag

Abstract: In this paper we provide a categorisation and implementation of digital ink features for behaviour characterisation. Based on four feature sets taken from literature, we provide a categorisation in different classes of syntactic and semantic features. We implemented a publicly available framework to calculate these features and show its deployment in the use case of analysing cognitive assessments… ▽ More In this paper we provide a categorisation and implementation of digital ink features for behaviour characterisation. Based on four feature sets taken from literature, we provide a categorisation in different classes of syntactic and semantic features. We implemented a publicly available framework to calculate these features and show its deployment in the use case of analysing cognitive assessments performed using a digital pen. △ Less

Submitted 1 October, 2018; originally announced October 2018.

A Survey on Deep Learning Toolkits and Libraries for Intelligent User Interfaces

Authors: Jan Zacharias, Michael Barz, Daniel Sonntag

Abstract: This paper provides an overview of prominent deep learning toolkits and, in particular, reports on recent publications that contributed open source software for implementing tasks that are common in intelligent user interfaces (IUI). We provide a scientific reference for researchers and software engineers who plan to utilise deep learning techniques within their IUI research and development projec… ▽ More This paper provides an overview of prominent deep learning toolkits and, in particular, reports on recent publications that contributed open source software for implementing tasks that are common in intelligent user interfaces (IUI). We provide a scientific reference for researchers and software engineers who plan to utilise deep learning techniques within their IUI research and development projects. △ Less

Submitted 14 March, 2018; v1 submitted 13 March, 2018; originally announced March 2018.

ACM Class: H.5.2

Fine-tuning deep CNN models on specific MS COCO categories

Authors: Daniel Sonntag, Michael Barz, Jan Zacharias, Sven Stauden, Vahid Rahmani, Áron Fóthi, András Lőrincz

Abstract: Fine-tuning of a deep convolutional neural network (CNN) is often desired. This paper provides an overview of our publicly available py-faster-rcnn-ft software library that can be used to fine-tune the VGG_CNN_M_1024 model on custom subsets of the Microsoft Common Objects in Context (MS COCO) dataset. For example, we improved the procedure so that the user does not have to look for suitable image… ▽ More Fine-tuning of a deep convolutional neural network (CNN) is often desired. This paper provides an overview of our publicly available py-faster-rcnn-ft software library that can be used to fine-tune the VGG_CNN_M_1024 model on custom subsets of the Microsoft Common Objects in Context (MS COCO) dataset. For example, we improved the procedure so that the user does not have to look for suitable image files in the dataset by hand which can then be used in the demo program. Our implementation randomly selects images that contain at least one object of the categories on which the model is fine-tuned. △ Less

Submitted 5 September, 2017; originally announced September 2017.

Complex Formation between Polyelectrolytes and Oppositely Charged Oligoelectrolytes

Authors: Jiajia Zhou, Matthias Barz, Friederike Schmid

Abstract: We study the complex formation between one long polyanion chain and many short oligocation chains by computer simulations. We employ a coarse-grained bead-spring model for the polyelectrolyte chains, and model explicitly the small salt ions. We systematically vary the concentration and the length of the oligocation, and examine how the oligocations affects the chain conformation, the static struct… ▽ More We study the complex formation between one long polyanion chain and many short oligocation chains by computer simulations. We employ a coarse-grained bead-spring model for the polyelectrolyte chains, and model explicitly the small salt ions. We systematically vary the concentration and the length of the oligocation, and examine how the oligocations affects the chain conformation, the static structure factor, the radial and axial distribution of various charged species, and the number of bound ions in the complex. At low oligocation concentration, the polyanion has an extended structure. Upon increasing the oligocation concentration, the polyanion chain collapses and forms a compact globule, but the complex still carries a net negative charge. Once the total charge of the oligocations is equal to that of the polyanion, the collapse stops and is replaced by a slow expansion. In this regime, the net charge on the complexes is positive or neutral, depending on the microion concentration in solution. The expansion can be explained by the reduction of the oligocation bridging. We find that the behavior and the structure of the complex are largely independent of the length of oligocations, and very similar to that observed when replacing the oligocations by multivalent salt cations, and conclude that the main driving force kee** the complex together is the release of monovalent counterions and coions. We speculate on the implications of this finding for the problem of controlled oligolyte release and oligolyte substitution. △ Less

Submitted 6 April, 2016; originally announced April 2016.

Comments: 13 pages, 11 figures, submitted to J. Chem. Phys

Journal ref: J. Chem. Phys. 144, 164902 (2016)