-
Can Github issues be solved with Tree Of Thoughts?
Authors:
Ricardo La Rosa,
Corey Hulse,
Bangdi Liu
Abstract:
While there have been extensive studies in code generation by large language models (LLM), where benchmarks like HumanEval have been surpassed with an impressive 96.3% success rate, these benchmarks predominantly judge a model's performance on basic function-level code generation and lack the critical thinking and concept of scope required of real-world scenarios such as solving GitHub issues. Thi…
▽ More
While there have been extensive studies in code generation by large language models (LLM), where benchmarks like HumanEval have been surpassed with an impressive 96.3% success rate, these benchmarks predominantly judge a model's performance on basic function-level code generation and lack the critical thinking and concept of scope required of real-world scenarios such as solving GitHub issues. This research introduces the application of the Tree of Thoughts (ToT) language model reasoning framework for enhancing the decision-making and problem-solving abilities of LLMs for this complex task. Compared to traditional input-output (IO) prompting and Retrieval Augmented Generation (RAG) techniques, ToT is designed to improve performance by facilitating a structured exploration of multiple reasoning trajectories and enabling self-assessment of potential solutions. We experimentally deploy ToT in tackling a Github issue contained within an instance of the SWE-bench. However, our results reveal that the ToT framework alone is not enough to give LLMs the critical reasoning capabilities to outperform existing methods. In this paper we analyze the potential causes of these shortcomings and identify key areas for improvement such as deepening the thought process and introducing agentic capabilities. The insights of this research are aimed at informing future directions for refining the application of ToT and better harnessing the potential of LLMs in real-world problem-solving scenarios.
△ Less
Submitted 20 May, 2024;
originally announced May 2024.
-
OOVs in the Spotlight: How to Inflect them?
Authors:
Tomáš Sourada,
Jana Straková,
Rudolf Rosa
Abstract:
We focus on morphological inflection in out-of-vocabulary (OOV) conditions, an under-researched subtask in which state-of-the-art systems usually are less effective. We developed three systems: a retrograde model and two sequence-to-sequence (seq2seq) models based on LSTM and Transformer. For testing in OOV conditions, we automatically extracted a large dataset of nouns in the morphologically rich…
▽ More
We focus on morphological inflection in out-of-vocabulary (OOV) conditions, an under-researched subtask in which state-of-the-art systems usually are less effective. We developed three systems: a retrograde model and two sequence-to-sequence (seq2seq) models based on LSTM and Transformer. For testing in OOV conditions, we automatically extracted a large dataset of nouns in the morphologically rich Czech language, with lemma-disjoint data splits, and we further manually annotated a real-world OOV dataset of neologisms. In the standard OOV conditions, Transformer achieves the best results, with increasing performance in ensemble with LSTM, the retrograde model and SIGMORPHON baselines. On the real-world OOV dataset of neologisms, the retrograde model outperforms all neural models. Finally, our seq2seq models achieve state-of-the-art results in 9 out of 16 languages from SIGMORPHON 2022 shared task data in the OOV evaluation (feature overlap) in the large data condition. We release the Czech OOV Inflection Dataset for rigorous evaluation in OOV conditions. Further, we release the inflection system with the seq2seq models as a ready-to-use Python library.
△ Less
Submitted 28 May, 2024; v1 submitted 13 April, 2024;
originally announced April 2024.
-
AsQM: Audio streaming Quality Metric based on Network Impairments and User Preferences
Authors:
Marcelo Rodrigo dos Santos,
Andreza Patrícia Batista,
Renata Lopes Rosa,
Muhammad Saadi,
Dick Carrillo Melgarejo,
Demóstenes Zegarra Rodríguez
Abstract:
There are many users of audio streaming services because of the proliferation of cloud-based audio streaming services for different content. The complex networks that support these services do not always guarantee an acceptable quality on the end-user side. In this paper, the impact of temporal interruptions on the reproduction of audio streaming and the users preference in relation to audio conte…
▽ More
There are many users of audio streaming services because of the proliferation of cloud-based audio streaming services for different content. The complex networks that support these services do not always guarantee an acceptable quality on the end-user side. In this paper, the impact of temporal interruptions on the reproduction of audio streaming and the users preference in relation to audio contents are studied. In order to determine the key parameters in the audio streaming service, subjective tests were conducted, and their results show that users Quality-of-Experience (QoE) is highly correlated with the following application parameters, the number of temporal interruptions or stalls, its frequency and length, and the temporal location in which they occur. However, most important, experimental results demonstrated that users preference for audio content plays an important role in users QoE. Thus, a Preference Factor (PF) function is defined and considered in the formulation of the proposed metric named Audio streaming Quality Metric (AsQM). Considering that multimedia service providers are based on web servers, a framework to obtain user information is proposed. Furthermore, results show that the AsQM implemented in the audio player of an end users device presents a low impact on energy, processing and memory consumption.
△ Less
Submitted 26 September, 2023;
originally announced September 2023.
-
$n$ Walks in the Fictional Woods
Authors:
Victor Schetinger,
Sara Di Bartolomeo,
Edirlei Soares de Lima,
Christofer Meinecke,
Rudolf Rosa
Abstract:
This paper presents a novel exploration of the interaction between generative AI models, visualization, and narrative generation processes, using OpenAI's GPT as a case study. We look at the question "Where Does Generativeness Comes From", which has a simple answer at the intersection of many domains. Drawing on Umberto Eco's "Six Walks in the Fictional Woods", we engender a speculative, transdisc…
▽ More
This paper presents a novel exploration of the interaction between generative AI models, visualization, and narrative generation processes, using OpenAI's GPT as a case study. We look at the question "Where Does Generativeness Comes From", which has a simple answer at the intersection of many domains. Drawing on Umberto Eco's "Six Walks in the Fictional Woods", we engender a speculative, transdisciplinary scientific narrative using ChatGPT in different roles: as an information repository, a ghost writer, a scientific coach, among others. The paper is written as a piling of plateaus where the titling of each (sub-)section, the "teaser" images, the headers, and a biblock of text are strata forming a narrative about narratives. To enrich our exposition, we present a visualization prototype to analyze storyboarded narratives, and extensive conversations with ChatGPT. Each link to a ChatGPT conversation is an experiment on writing where we try to use different plugins and techniques to investigate the topics that, ultimately form the content of this portable document file. Our visualization uses a dataset of stories with scene descriptions, textual descriptions of scenes (both generated by ChatGPT), and images (generated by Stable Diffusion using scene descriptions as prompts). We employ a simple graph-node diagram to try to make a "forest of narratives" visible, an example of a vis4gen application that can be used to analyze the output of Large Languange + Image Models.
△ Less
Submitted 23 August, 2023; v1 submitted 13 July, 2023;
originally announced August 2023.
-
Programming with Quantum Mechanics
Authors:
Evandro C. R. da Rosa,
Claudio Lima
Abstract:
Quantum computing is an emerging paradigm that opens a new era for exponential computational speedup. Still, quantum computers have yet to be ready for commercial use. However, it is essential to train and qualify today the workforce that will develop quantum acceleration solutions to get the quantum advantage in the future. This tutorial gives a broad view of quantum computing, abstracting most o…
▽ More
Quantum computing is an emerging paradigm that opens a new era for exponential computational speedup. Still, quantum computers have yet to be ready for commercial use. However, it is essential to train and qualify today the workforce that will develop quantum acceleration solutions to get the quantum advantage in the future. This tutorial gives a broad view of quantum computing, abstracting most of the mathematical formalism and proposing a hands-on with the quantum programming language Ket. The target audience is undergraduate and graduate students starting in quantum computing -- no prerequisites for following this tutorial.
△ Less
Submitted 27 October, 2022;
originally announced October 2022.
-
DialogueScript: Using Dialogue Agents to Produce a Script
Authors:
Patrícia Schmidtová,
Dávid Javorský,
Christián Mikláš,
Tomáš Musil,
Rudolf Rosa,
Ondřej Dušek
Abstract:
We present a novel approach to generating scripts by using agents with different personality types. To manage character interaction in the script, we employ simulated dramatic networks. Automatic and human evaluation on multiple criteria shows that our approach outperforms a vanilla-GPT2-based baseline. We further introduce a new metric to evaluate dialogue consistency based on natural language in…
▽ More
We present a novel approach to generating scripts by using agents with different personality types. To manage character interaction in the script, we employ simulated dramatic networks. Automatic and human evaluation on multiple criteria shows that our approach outperforms a vanilla-GPT2-based baseline. We further introduce a new metric to evaluate dialogue consistency based on natural language inference and demonstrate its validity.
△ Less
Submitted 16 June, 2022;
originally announced June 2022.
-
THEaiTRE 1.0: Interactive generation of theatre play scripts
Authors:
Rudolf Rosa,
Tomáš Musil,
Ondřej Dušek,
Dominik Jurko,
Patrícia Schmidtová,
David Mareček,
Ondřej Bojar,
Tom Kocmi,
Daniel Hrbek,
David Košťák,
Martina Kinská,
Marie Nováková,
Josef Doležal,
Klára Vosecká,
Tomáš Studeník,
Petr Žabka
Abstract:
We present the first version of a system for interactive generation of theatre play scripts. The system is based on a vanilla GPT-2 model with several adjustments, targeting specific issues we encountered in practice. We also list other issues we encountered but plan to only solve in a future version of the system. The presented system was used to generate a theatre play script planned for premier…
▽ More
We present the first version of a system for interactive generation of theatre play scripts. The system is based on a vanilla GPT-2 model with several adjustments, targeting specific issues we encountered in practice. We also list other issues we encountered but plan to only solve in a future version of the system. The presented system was used to generate a theatre play script planned for premiere in February 2021.
△ Less
Submitted 17 February, 2021;
originally announced February 2021.
-
Predicting Typological Features in WALS using Language Embeddings and Conditional Probabilities: ÚFAL Submission to the SIGTYP 2020 Shared Task
Authors:
Martin Vastl,
Daniel Zeman,
Rudolf Rosa
Abstract:
We present our submission to the SIGTYP 2020 Shared Task on the prediction of typological features. We submit a constrained system, predicting typological features only based on the WALS database. We investigate two approaches. The simpler of the two is a system based on estimating correlation of feature values within languages by computing conditional probabilities and mutual information. The sec…
▽ More
We present our submission to the SIGTYP 2020 Shared Task on the prediction of typological features. We submit a constrained system, predicting typological features only based on the WALS database. We investigate two approaches. The simpler of the two is a system based on estimating correlation of feature values within languages by computing conditional probabilities and mutual information. The second approach is to train a neural predictor operating on precomputed language embeddings based on WALS features. Our submitted system combines the two approaches based on their self-estimated confidence scores. We reach the accuracy of 70.7% on the test data and rank first in the shared task.
△ Less
Submitted 8 October, 2020;
originally announced October 2020.
-
Measuring Memorization Effect in Word-Level Neural Networks Probing
Authors:
Rudolf Rosa,
Tomáš Musil,
David Mareček
Abstract:
Multiple studies have probed representations emerging in neural networks trained for end-to-end NLP tasks and examined what word-level linguistic information may be encoded in the representations. In classical probing, a classifier is trained on the representations to extract the target linguistic information. However, there is a threat of the classifier simply memorizing the linguistic labels for…
▽ More
Multiple studies have probed representations emerging in neural networks trained for end-to-end NLP tasks and examined what word-level linguistic information may be encoded in the representations. In classical probing, a classifier is trained on the representations to extract the target linguistic information. However, there is a threat of the classifier simply memorizing the linguistic labels for individual words, instead of extracting the linguistic abstractions from the representations, thus reporting false positive results. While considerable efforts have been made to minimize the memorization problem, the task of actually measuring the amount of memorization happening in the classifier has been understudied so far. In our work, we propose a simple general method for measuring the memorization effect, based on a symmetric selection of comparable sets of test words seen versus unseen in training. Our method can be used to explicitly quantify the amount of memorization happening in a probing setup, so that an adequate setup can be chosen and the results of the probing can be interpreted with a reliability estimate. We exemplify this by showcasing our method on a case study of probing for part of speech in a trained neural machine translation encoder.
△ Less
Submitted 29 June, 2020;
originally announced June 2020.
-
THEaiTRE: Artificial Intelligence to Write a Theatre Play
Authors:
Rudolf Rosa,
Ondřej Dušek,
Tom Kocmi,
David Mareček,
Tomáš Musil,
Patrícia Schmidtová,
Dominik Jurko,
Ondřej Bojar,
Daniel Hrbek,
David Košťák,
Martina Kinská,
Josef Doležal,
Klára Vosecká
Abstract:
We present THEaiTRE, a starting project aimed at automatic generation of theatre play scripts. This paper reviews related work and drafts an approach we intend to follow. We plan to adopt generative neural language models and hierarchical generation approaches, supported by summarization and machine translation methods, and complemented with a human-in-the-loop approach.
We present THEaiTRE, a starting project aimed at automatic generation of theatre play scripts. This paper reviews related work and drafts an approach we intend to follow. We plan to adopt generative neural language models and hierarchical generation approaches, supported by summarization and machine translation methods, and complemented with a human-in-the-loop approach.
△ Less
Submitted 25 June, 2020;
originally announced June 2020.
-
Classical and Quantum Data Interaction in Programming Languages: A Runtime Architecture
Authors:
Evandro Chagas Ribeiro da Rosa,
Rafael de Santiago
Abstract:
We propose a runtime architecture that can be used in the development of a quantum programming language and its programming environment. The proposed runtime architecture enables dynamic interaction between classical and quantum data following the restriction that a quantum computer is available in the cloud as a batch computer, with no interaction with the classical computer during its execution.…
▽ More
We propose a runtime architecture that can be used in the development of a quantum programming language and its programming environment. The proposed runtime architecture enables dynamic interaction between classical and quantum data following the restriction that a quantum computer is available in the cloud as a batch computer, with no interaction with the classical computer during its execution. It is done by leaving the quantum code generation for the runtime and introducing the concept of futures for quantum measurements. When implemented in a quantum programming language, those strategies aim to facilitate the development of quantum applications, especially for beginning programmers and students. Being suitable for the current Noisy Intermediate-Scale Quantum (NISQ) Computers, the runtime architecture is also appropriate for simulation and future Fault-Tolerance Quantum Computers.
△ Less
Submitted 29 May, 2020;
originally announced June 2020.
-
Universal Dependencies according to BERT: both more specific and more general
Authors:
Tomasz Limisiewicz,
Rudolf Rosa,
David Mareček
Abstract:
This work focuses on analyzing the form and extent of syntactic abstraction captured by BERT by extracting labeled dependency trees from self-attentions.
Previous work showed that individual BERT heads tend to encode particular dependency relation types. We extend these findings by explicitly comparing BERT relations to Universal Dependencies (UD) annotations, showing that they often do not matc…
▽ More
This work focuses on analyzing the form and extent of syntactic abstraction captured by BERT by extracting labeled dependency trees from self-attentions.
Previous work showed that individual BERT heads tend to encode particular dependency relation types. We extend these findings by explicitly comparing BERT relations to Universal Dependencies (UD) annotations, showing that they often do not match one-to-one.
We suggest a method for relation identification and syntactic tree construction. Our approach produces significantly more consistent dependency trees than previous work, showing that it better explains the syntactic abstractions in BERT. At the same time, it can be successfully applied with only a minimal amount of supervision and generalizes well across languages.
△ Less
Submitted 6 October, 2020; v1 submitted 30 April, 2020;
originally announced April 2020.
-
On the Language Neutrality of Pre-trained Multilingual Representations
Authors:
**dřich Libovický,
Rudolf Rosa,
Alexander Fraser
Abstract:
Multilingual contextual embeddings, such as multilingual BERT and XLM-RoBERTa, have proved useful for many multi-lingual tasks. Previous work probed the cross-linguality of the representations indirectly using zero-shot transfer learning on morphological and syntactic tasks. We instead investigate the language-neutrality of multilingual contextual embeddings directly and with respect to lexical se…
▽ More
Multilingual contextual embeddings, such as multilingual BERT and XLM-RoBERTa, have proved useful for many multi-lingual tasks. Previous work probed the cross-linguality of the representations indirectly using zero-shot transfer learning on morphological and syntactic tasks. We instead investigate the language-neutrality of multilingual contextual embeddings directly and with respect to lexical semantics. Our results show that contextual embeddings are more language-neutral and, in general, more informative than aligned static word-type embeddings, which are explicitly trained for language neutrality. Contextual embeddings are still only moderately language-neutral by default, so we propose two simple methods for achieving stronger language neutrality: first, by unsupervised centering of the representation for each language and second, by fitting an explicit projection on small parallel data. Besides, we show how to reach state-of-the-art accuracy on language identification and match the performance of statistical methods for word alignment of parallel sentences without using parallel data.
△ Less
Submitted 29 September, 2020; v1 submitted 9 April, 2020;
originally announced April 2020.
-
How Language-Neutral is Multilingual BERT?
Authors:
**dřich Libovický,
Rudolf Rosa,
Alexander Fraser
Abstract:
Multilingual BERT (mBERT) provides sentence representations for 104 languages, which are useful for many multi-lingual tasks. Previous work probed the cross-linguality of mBERT using zero-shot transfer learning on morphological and syntactic tasks. We instead focus on the semantic properties of mBERT. We show that mBERT representations can be split into a language-specific component and a language…
▽ More
Multilingual BERT (mBERT) provides sentence representations for 104 languages, which are useful for many multi-lingual tasks. Previous work probed the cross-linguality of mBERT using zero-shot transfer learning on morphological and syntactic tasks. We instead focus on the semantic properties of mBERT. We show that mBERT representations can be split into a language-specific component and a language-neutral component, and that the language-neutral component is sufficiently general in terms of modeling semantics to allow high-accuracy word-alignment and sentence retrieval but is not yet good enough for the more difficult task of MT quality estimation. Our work presents interesting challenges which must be solved to build better language-neutral representations, particularly for tasks requiring linguistic transfer of semantics.
△ Less
Submitted 8 November, 2019;
originally announced November 2019.
-
Unsupervised Lemmatization as Embeddings-Based Word Clustering
Authors:
Rudolf Rosa,
Zdeněk Žabokrtský
Abstract:
We focus on the task of unsupervised lemmatization, i.e. grou** together inflected forms of one word under one label (a lemma) without the use of annotated training data. We propose to perform agglomerative clustering of word forms with a novel distance measure. Our distance measure is based on the observation that inflections of the same word tend to be similar both string-wise and in meaning.…
▽ More
We focus on the task of unsupervised lemmatization, i.e. grou** together inflected forms of one word under one label (a lemma) without the use of annotated training data. We propose to perform agglomerative clustering of word forms with a novel distance measure. Our distance measure is based on the observation that inflections of the same word tend to be similar both string-wise and in meaning. We therefore combine word embedding cosine similarity, serving as a proxy to the meaning similarity, with Jaro-Winkler edit distance. Our experiments on 23 languages show our approach to be promising, surpassing the baseline on 23 of the 28 evaluation datasets.
△ Less
Submitted 22 August, 2019;
originally announced August 2019.
-
Inducing Syntactic Trees from BERT Representations
Authors:
Rudolf Rosa,
David Mareček
Abstract:
We use the English model of BERT and explore how a deletion of one word in a sentence changes representations of other words. Our hypothesis is that removing a reducible word (e.g. an adjective) does not affect the representation of other words so much as removing e.g. the main verb, which makes the sentence ungrammatical and of "high surprise" for the language model. We estimate reducibilities of…
▽ More
We use the English model of BERT and explore how a deletion of one word in a sentence changes representations of other words. Our hypothesis is that removing a reducible word (e.g. an adjective) does not affect the representation of other words so much as removing e.g. the main verb, which makes the sentence ungrammatical and of "high surprise" for the language model. We estimate reducibilities of individual words and also of longer continuous phrases (word n-grams), study their syntax-related properties, and then also use them to induce full dependency trees.
△ Less
Submitted 27 June, 2019;
originally announced June 2019.
-
From Balustrades to Pierre Vinken: Looking for Syntax in Transformer Self-Attentions
Authors:
David Mareček,
Rudolf Rosa
Abstract:
We inspect the multi-head self-attention in Transformer NMT encoders for three source languages, looking for patterns that could have a syntactic interpretation. In many of the attention heads, we frequently find sequences of consecutive states attending to the same position, which resemble syntactic phrases. We propose a transparent deterministic method of quantifying the amount of syntactic info…
▽ More
We inspect the multi-head self-attention in Transformer NMT encoders for three source languages, looking for patterns that could have a syntactic interpretation. In many of the attention heads, we frequently find sequences of consecutive states attending to the same position, which resemble syntactic phrases. We propose a transparent deterministic method of quantifying the amount of syntactic information present in the self-attentions, based on automatically building and evaluating phrase-structure trees from the phrase-like sequences. We compare the resulting trees to existing constituency treebanks, both manually and by computing precision and recall.
△ Less
Submitted 5 June, 2019;
originally announced June 2019.
-
Network Service Orchestration: A Survey
Authors:
Nathan F. Saraiva de Sousa,
Danny A. Lachos Perez,
Raphael V. Rosa,
Mateus A. S. Santos,
Christian Esteve Rothenberg
Abstract:
Business models of network service providers are undergoing an evolving transformation fueled by vertical customer demands and technological advances such as 5G, Software Defined Networking~(SDN), and Network Function Virtualization~(NFV). Emerging scenarios call for agile network services consuming network, storage, and compute resources across heterogeneous infrastructures and administrative dom…
▽ More
Business models of network service providers are undergoing an evolving transformation fueled by vertical customer demands and technological advances such as 5G, Software Defined Networking~(SDN), and Network Function Virtualization~(NFV). Emerging scenarios call for agile network services consuming network, storage, and compute resources across heterogeneous infrastructures and administrative domains. Coordinating resource control and service creation across interconnected domains and diverse technologies becomes a grand challenge. Research and development efforts are being devoted to enabling orchestration processes to automate, coordinate, and manage the deployment and operation of network services. In this survey, we delve into the topic of Network Service Orchestration~(NSO) by reviewing the historical background, relevant research projects, enabling technologies, and standardization activities. We define key concepts and propose a taxonomy of NSO approaches and solutions to pave the way towards a common understanding of the various ongoing efforts around the realization of diverse NSO application scenarios. Based on the analysis of the state of affairs, we present a series of open challenges and research opportunities, altogether contributing to a timely and comprehensive survey on the vibrant and strategic topic of network service orchestration.
△ Less
Submitted 17 May, 2019; v1 submitted 17 March, 2018;
originally announced March 2018.
-
Confidence Decision Trees via Online and Active Learning for Streaming (BIG) Data
Authors:
Rocco De Rosa
Abstract:
Decision tree classifiers are a widely used tool in data stream mining. The use of confidence intervals to estimate the gain associated with each split leads to very effective methods, like the popular Hoeffding tree algorithm. From a statistical viewpoint, the analysis of decision tree classifiers in a streaming setting requires knowing when enough new information has been collected to justify sp…
▽ More
Decision tree classifiers are a widely used tool in data stream mining. The use of confidence intervals to estimate the gain associated with each split leads to very effective methods, like the popular Hoeffding tree algorithm. From a statistical viewpoint, the analysis of decision tree classifiers in a streaming setting requires knowing when enough new information has been collected to justify splitting a leaf. Although some of the issues in the statistical analysis of Hoeffding trees have been already clarified, a general and rigorous study of confidence intervals for splitting criteria is missing. We fill this gap by deriving accurate confidence intervals to estimate the splitting gain in decision tree learning with respect to three criteria: entropy, Gini index, and a third index proposed by Kearns and Mansour. Our confidence intervals depend in a more detailed way on the tree parameters. We also extend our confidence analysis to a selective sampling setting, in which the decision tree learner adaptively decides which labels to query in the stream. We furnish theoretical guarantee bounding the probability that the classification is non-optimal learning the decision tree via our selective sampling strategy. Experiments on real and synthetic data in a streaming setting show that our trees are indeed more accurate than trees with the same number of leaves generated by other techniques and our active learning module permits to save labeling cost. In addition, comparing our labeling strategy with recent methods, we show that our approach is more robust and consistent respect all the other techniques applied to incremental decision trees.
△ Less
Submitted 12 April, 2016;
originally announced April 2016.
-
Active Learning for Online Recognition of Human Activities from Streaming Videos
Authors:
Rocco De Rosa,
Ilaria Gori,
Fabio Cuzzolin,
Barbara Caputo,
Nicolò Cesa-Bianchi
Abstract:
Recognising human activities from streaming videos poses unique challenges to learning algorithms: predictive models need to be scalable, incrementally trainable, and must remain bounded in size even when the data stream is arbitrarily long. Furthermore, as parameter tuning is problematic in a streaming setting, suitable approaches should be parameterless, and make no assumptions on what class lab…
▽ More
Recognising human activities from streaming videos poses unique challenges to learning algorithms: predictive models need to be scalable, incrementally trainable, and must remain bounded in size even when the data stream is arbitrarily long. Furthermore, as parameter tuning is problematic in a streaming setting, suitable approaches should be parameterless, and make no assumptions on what class labels may occur in the stream. We present here an approach to the recognition of human actions from streaming data which meets all these requirements by: (1) incrementally learning a model which adaptively covers the feature space with simple local classifiers; (2) employing an active learning strategy to reduce annotation requests; (3) achieving promising accuracy within a fixed model size. Extensive experiments on standard benchmarks show that our approach is competitive with state-of-the-art non-incremental methods, and outperforms the existing active incremental baselines.
△ Less
Submitted 11 April, 2016;
originally announced April 2016.
-
Online Open World Recognition
Authors:
Rocco De Rosa,
Thomas Mensink,
Barbara Caputo
Abstract:
As we enter into the big data age and an avalanche of images have become readily available, recognition systems face the need to move from close, lab settings where the number of classes and training data are fixed, to dynamic scenarios where the number of categories to be recognized grows continuously over time, as well as new data providing useful information to update the system. Recent attempt…
▽ More
As we enter into the big data age and an avalanche of images have become readily available, recognition systems face the need to move from close, lab settings where the number of classes and training data are fixed, to dynamic scenarios where the number of categories to be recognized grows continuously over time, as well as new data providing useful information to update the system. Recent attempts, like the open world recognition framework, tried to inject dynamics into the system by detecting new unknown classes and adding them incrementally, while at the same time continuously updating the models for the known classes. incrementally adding new classes and detecting instances from unknown classes, while at the same time continuously updating the models for the known classes. In this paper we argue that to properly capture the intrinsic dynamic of open world recognition, it is necessary to add to these aspects (a) the incremental learning of the underlying metric, (b) the incremental estimate of confidence thresholds for the unknown classes, and (c) the use of local learning to precisely describe the space of classes. We extend three existing metric learning algorithms towards these goals by using online metric learning. Experimentally we validate our approach on two large-scale datasets in different learning scenarios. For all these scenarios our proposed methods outperform their non-online counterparts. We conclude that local and online learning is important to capture the full dynamics of open world recognition.
△ Less
Submitted 8 April, 2016;
originally announced April 2016.
-
The ABACOC Algorithm: a Novel Approach for Nonparametric Classification of Data Streams
Authors:
Rocco De Rosa,
Francesco Orabona,
Nicolò Cesa-Bianchi
Abstract:
Stream mining poses unique challenges to machine learning: predictive models are required to be scalable, incrementally trainable, must remain bounded in size (even when the data stream is arbitrarily long), and be nonparametric in order to achieve high accuracy even in complex and dynamic environments. Moreover, the learning system must be parameterless ---traditional tuning methods are problemat…
▽ More
Stream mining poses unique challenges to machine learning: predictive models are required to be scalable, incrementally trainable, must remain bounded in size (even when the data stream is arbitrarily long), and be nonparametric in order to achieve high accuracy even in complex and dynamic environments. Moreover, the learning system must be parameterless ---traditional tuning methods are problematic in streaming settings--- and avoid requiring prior knowledge of the number of distinct class labels occurring in the stream. In this paper, we introduce a new algorithmic approach for nonparametric learning in data streams. Our approach addresses all above mentioned challenges by learning a model that covers the input space using simple local classifiers. The distribution of these classifiers dynamically adapts to the local (unknown) complexity of the classification problem, thus achieving a good balance between model complexity and predictive accuracy. We design four variants of our approach of increasing adaptivity. By means of an extensive empirical evaluation against standard nonparametric baselines, we show state-of-the-art results in terms of accuracy versus model size. For the variant that imposes a strict bound on the model size, we show better performance against all other methods measured at the same model size value. Our empirical analysis is complemented by a theoretical performance guarantee which does not rely on any stochastic assumption on the source generating the stream.
△ Less
Submitted 20 August, 2015;
originally announced August 2015.
-
Parsing Natural Language Sentences by Semi-supervised Methods
Authors:
Rudolf Rosa
Abstract:
We present our work on semi-supervised parsing of natural language sentences, focusing on multi-source crosslingual transfer of delexicalized dependency parsers. We first evaluate the influence of treebank annotation styles on parsing performance, focusing on adposition attachment style. Then, we present KLcpos3, an empirical language similarity measure, designed and tuned for source parser weight…
▽ More
We present our work on semi-supervised parsing of natural language sentences, focusing on multi-source crosslingual transfer of delexicalized dependency parsers. We first evaluate the influence of treebank annotation styles on parsing performance, focusing on adposition attachment style. Then, we present KLcpos3, an empirical language similarity measure, designed and tuned for source parser weighting in multi-source delexicalized parser transfer. And finally, we introduce a novel resource combination method, based on interpolation of trained parser models.
△ Less
Submitted 16 June, 2015;
originally announced June 2015.
