-
A Methodology for Questionnaire Analysis: Insights through Cluster Analysis of an Investor Competition Data
Authors:
Carlos Henrique Q. Forster,
Paulo André Lima de Castro,
Andrei Ramalho
Abstract:
In this paper, we propose a methodology for the analysis of questionnaire data along with its application on discovering insights from investor data motivated by a day trading competition. The questionnaire includes categorical questions, which are reduced to binary questions, 'yes' or 'no'. The methodology reduces dimensionality by grou** questions and participants with similar responses using…
▽ More
In this paper, we propose a methodology for the analysis of questionnaire data along with its application on discovering insights from investor data motivated by a day trading competition. The questionnaire includes categorical questions, which are reduced to binary questions, 'yes' or 'no'. The methodology reduces dimensionality by grou** questions and participants with similar responses using clustering analysis. Rule discovery was performed by using a conversion rate metric. Innovative visual representations were proposed to validate the cluster analysis and the relation discovery between questions. When crossing with financial data, additional insights were revealed related to the recognized clusters.
△ Less
Submitted 9 February, 2024;
originally announced February 2024.
-
Foundational Competencies and Responsibilities of a Research Software Engineer
Authors:
Florian Goth,
Renato Alves,
Matthias Braun,
Leyla Jael Castro,
Gerasimos Chourdakis,
Simon Christ,
Jeremy Cohen,
Fredo Erxleben,
Jean-Noël Grad,
Magnus Hagdorn,
Toby Hodges,
Guido Juckeland,
Dominic Kempf,
Anna-Lena Lamprecht,
Jan Linxweiler,
Frank Löffler,
Michele Martone,
Moritz Schwarzmeier,
Heidi Seibold,
Jan Philipp Thiele,
Harald von Waldow,
Samantha Wittke
Abstract:
The term Research Software Engineer, or RSE, emerged a little over 10 years ago as a way to represent individuals working in the research community but focusing on software development. The term has been widely adopted and there are a number of high-level definitions of what an RSE is. However, the roles of RSEs vary depending on the institutional context they work in. At one end of the spectrum,…
▽ More
The term Research Software Engineer, or RSE, emerged a little over 10 years ago as a way to represent individuals working in the research community but focusing on software development. The term has been widely adopted and there are a number of high-level definitions of what an RSE is. However, the roles of RSEs vary depending on the institutional context they work in. At one end of the spectrum, RSE roles may look similar to a traditional research role. At the other extreme, they resemble that of a software engineer in industry. Most RSE roles inhabit the space between these two extremes. Therefore, providing a straightforward, comprehensive definition of what an RSE does and what experience, skills and competencies are required to become one is challenging. In this community paper we define the broad notion of what an RSE is, explore the different types of work they undertake, and define a list of fundamental competencies as well as values that define the general profile of an RSE. On this basis, we elaborate on the progression of these skills along different dimensions, looking at specific types of RSE roles, proposing recommendations for organisations, and giving examples of future specialisations. An appendix details how existing curricula fit into this framework.
△ Less
Submitted 12 April, 2024; v1 submitted 19 November, 2023;
originally announced November 2023.
-
Overview of LiLAS 2020 -- Living Labs for Academic Search
Authors:
Philipp Schaer,
Johann Schaible,
Leyla Jael Garcia Castro
Abstract:
Academic Search is a timeless challenge that the field of Information Retrieval has been dealing with for many years. Even today, the search for academic material is a broad field of research that recently started working on problems like the COVID-19 pandemic. However, test collections and specialized data sets like CORD-19 only allow for system-oriented experiments, while the evaluation of algor…
▽ More
Academic Search is a timeless challenge that the field of Information Retrieval has been dealing with for many years. Even today, the search for academic material is a broad field of research that recently started working on problems like the COVID-19 pandemic. However, test collections and specialized data sets like CORD-19 only allow for system-oriented experiments, while the evaluation of algorithms in real-world environments is only available to researchers from industry. In LiLAS, we open up two academic search platforms to allow participating research to evaluate their systems in a Docker-based research environment. This overview paper describes the motivation, infrastructure, and two systems LIVIVO and GESIS Search that are part of this CLEF lab.
△ Less
Submitted 31 October, 2023;
originally announced October 2023.
-
Living Lab Evaluation for Life and Social Sciences Search Platforms -- LiLAS at CLEF 2021
Authors:
Philipp Schaer,
Johann Schaible,
Leyla Jael Castro
Abstract:
Meta-evaluation studies of system performances in controlled offline evaluation campaigns, like TREC and CLEF, show a need for innovation in evaluating IR-systems. The field of academic search is no exception to this. This might be related to the fact that relevance in academic search is multilayered and therefore the aspect of user-centric evaluation is becoming more and more important. The Livin…
▽ More
Meta-evaluation studies of system performances in controlled offline evaluation campaigns, like TREC and CLEF, show a need for innovation in evaluating IR-systems. The field of academic search is no exception to this. This might be related to the fact that relevance in academic search is multilayered and therefore the aspect of user-centric evaluation is becoming more and more important. The Living Labs for Academic Search (LiLAS) lab aims to strengthen the concept of user-centric living labs for the domain of academic search by allowing participants to evaluate their retrieval approaches in two real-world academic search systems from the life sciences and the social sciences. To this end, we provide participants with metadata on the systems' content as well as candidate lists with the task to rank the most relevant candidate to the top. Using the STELLA-infrastructure, we allow participants to easily integrate their approaches into the real-world systems and provide the possibility to compare different approaches at the same time.
△ Less
Submitted 5 October, 2023;
originally announced October 2023.
-
VENOM: A Vectorized N:M Format for Unleashing the Power of Sparse Tensor Cores
Authors:
Roberto L. Castro,
Andrei Ivanov,
Diego Andrade,
Tal Ben-Nun,
Basilio B. Fraguela,
Torsten Hoefler
Abstract:
The increasing success and scaling of Deep Learning models demands higher computational efficiency and power. Sparsification can lead to both smaller models as well as higher compute efficiency, and accelerated hardware is becoming available. However, exploiting it efficiently requires kernel implementations, pruning algorithms, and storage formats, to utilize hardware support of specialized spars…
▽ More
The increasing success and scaling of Deep Learning models demands higher computational efficiency and power. Sparsification can lead to both smaller models as well as higher compute efficiency, and accelerated hardware is becoming available. However, exploiting it efficiently requires kernel implementations, pruning algorithms, and storage formats, to utilize hardware support of specialized sparse vector units. An example of those are the NVIDIA's Sparse Tensor Cores (SPTCs), which promise a 2x speedup. However, SPTCs only support the 2:4 format, limiting achievable sparsity ratios to 50%. We present the V:N:M format, which enables the execution of arbitrary N:M ratios on SPTCs. To efficiently exploit the resulting format, we propose Spatha, a high-performance sparse-library for DL routines. We show that Spatha achieves up to 37x speedup over cuBLAS. We also demonstrate a second-order pruning technique that enables sparsification to high sparsity ratios with V:N:M and little to no loss in accuracy in modern transformers.
△ Less
Submitted 3 October, 2023;
originally announced October 2023.
-
Role-playing software architecture styles
Authors:
Laura M. Castro
Abstract:
Software Architecture, from definition to maintenance and evolution, is a complex aspect of software development and, consequently, a challenging subject when it comes to teaching it, and learning it.
Many research efforts have been devoted to designing teaching approaches, strategies and tools. Most of them, however, focus on the knowledge itself and the ways to convey it to students, rather th…
▽ More
Software Architecture, from definition to maintenance and evolution, is a complex aspect of software development and, consequently, a challenging subject when it comes to teaching it, and learning it.
Many research efforts have been devoted to designing teaching approaches, strategies and tools. Most of them, however, focus on the knowledge itself and the ways to convey it to students, rather than on the different learning styles of students themselves.
Teaching methods which predominantly rely on verbal and written communication, are very well aligned with some learning styles. However, students with learning styles that benefit more from physical activity or first-hand experience, need to defer to cognitive processes that are less natural to them.
In this work, we propose an innovative use of role-playing as teaching strategy for architecture models of reference (i.e. layered, pipe and filter, client-server, etc.). This role-playing of different software architectures, in which students play the part of specific components in the system, intends to complement other classical teaching materials, such as in-person or recorded lectures, lab assignments, or development projects.
Addressing all learning styles within a classroom is key to ensure that we favour and foster the students' different learning processes, and give everyone an even playfield in which to best develop their capabilities as Software Architects.
△ Less
Submitted 28 February, 2023;
originally announced February 2023.
-
Relating Edge Computing and Microservices by means of Architecture Approaches and Features, Orchestration, Choreography, and Offloading: A Systematic Literature Review
Authors:
Lucas Fernando Souza de Castro,
Sandro Rigo
Abstract:
Context: Microservices running and being powered by Edge Computing have been gaining much attention in the industry and academia. Since 2014, when Martin Fowler popularized the Microservice term, many studies have been published relating these subjects to explore how the Edge's low-latency feature could be combined with the high throughput of the distributed paradigm from Microservices. Objective:…
▽ More
Context: Microservices running and being powered by Edge Computing have been gaining much attention in the industry and academia. Since 2014, when Martin Fowler popularized the Microservice term, many studies have been published relating these subjects to explore how the Edge's low-latency feature could be combined with the high throughput of the distributed paradigm from Microservices. Objective: Identifying how Microservices work together with Edge Computing whereas they take advantage when running on Edge. Method: In order to better understand this relationship, we first identified its key concepts, which are: architecture approaches and features, microservice composition (orchestration/choreography), and offloading. Afterward, we conducted a Systematic Literature Review (SLR) as the survey method. Results: We reviewed 111 selected studies and built a taxonomy of Microservices on Edge Computing demonstrating their current architecture approaches and features, composition, and offloading modes. Moreover, we identify the research gaps and trends. Conclusion: This paper is a step forward to help researchers and professionals get a general overview of how Microservices and Edge have been related in the last years. It also discusses gaps and research trends. This SLR will also be a good introduction for new researchers in Edge and Microservices.
△ Less
Submitted 23 January, 2023; v1 submitted 18 January, 2023;
originally announced January 2023.
-
Online Information Retrieval Evaluation using the STELLA Framework
Authors:
Timo Breuer,
Narges Tavakolpoursaleh,
Johann Schaible,
Daniel Hienert,
Philipp Schaer,
Leyla Jael Castro
Abstract:
Involving users in early phases of software development has become a common strategy as it enables developers to consider user needs from the beginning. Once a system is in production, new opportunities to observe, evaluate and learn from users emerge as more information becomes available. Gathering information from users to continuously evaluate their behavior is a common practice for commercial…
▽ More
Involving users in early phases of software development has become a common strategy as it enables developers to consider user needs from the beginning. Once a system is in production, new opportunities to observe, evaluate and learn from users emerge as more information becomes available. Gathering information from users to continuously evaluate their behavior is a common practice for commercial software, while the Cranfield paradigm remains the preferred option for Information Retrieval (IR) and recommendation systems in the academic world. Here we introduce the Infrastructures for Living Labs STELLA project which aims to create an evaluation infrastructure allowing experimental systems to run along production web-based academic search systems with real users. STELLA combines user interactions and log files analyses to enable large-scale A/B experiments for academic search.
△ Less
Submitted 24 October, 2022;
originally announced October 2022.
-
Faster and more diverse de novo molecular optimization with double-loop reinforcement learning using augmented SMILES
Authors:
Esben Jannik Bjerrum,
Christian Margreitter,
Thomas Blaschke,
Raquel López-Ríos de Castro
Abstract:
Using generative deep learning models and reinforcement learning together can effectively generate new molecules with desired properties. By employing a multi-objective scoring function, thousands of high-scoring molecules can be generated, making this approach useful for drug discovery and material science. However, the application of these methods can be hindered by computationally expensive or…
▽ More
Using generative deep learning models and reinforcement learning together can effectively generate new molecules with desired properties. By employing a multi-objective scoring function, thousands of high-scoring molecules can be generated, making this approach useful for drug discovery and material science. However, the application of these methods can be hindered by computationally expensive or time-consuming scoring procedures, particularly when a large number of function calls are required as feedback in the reinforcement learning optimization. Here, we propose the use of double-loop reinforcement learning with simplified molecular line entry system (SMILES) augmentation to improve the efficiency and speed of the optimization. By adding an inner loop that augments the generated SMILES strings to non-canonical SMILES for use in additional reinforcement learning rounds, we can both reuse the scoring calculations on the molecular level, thereby speeding up the learning process, as well as offer additional protection against mode collapse. We find that employing between 5 and 10 augmentation repetitions is optimal for the scoring functions tested and is further associated with an increased diversity in the generated compounds, improved reproducibility of the sampling runs and the generation of molecules of higher similarity to known ligands.
△ Less
Submitted 3 March, 2023; v1 submitted 22 October, 2022;
originally announced October 2022.
-
FAB: An FPGA-based Accelerator for Bootstrappable Fully Homomorphic Encryption
Authors:
Rashmi Agrawal,
Leo de Castro,
Guowei Yang,
Chiraag Juvekar,
Rabia Yazicigil,
Anantha Chandrakasan,
Vinod Vaikuntanathan,
Ajay Joshi
Abstract:
FHE offers protection to private data on third-party cloud servers by allowing computations on the data in encrypted form. However, to support general-purpose encrypted computations, all existing FHE schemes require an expensive operation known as bootstrap**. Unfortunately, the computation cost and the memory bandwidth required for bootstrap** add significant overhead to FHE-based computation…
▽ More
FHE offers protection to private data on third-party cloud servers by allowing computations on the data in encrypted form. However, to support general-purpose encrypted computations, all existing FHE schemes require an expensive operation known as bootstrap**. Unfortunately, the computation cost and the memory bandwidth required for bootstrap** add significant overhead to FHE-based computations, limiting the practical use of FHE. In this work, we propose FAB, an FPGA-based accelerator for bootstrappable FHE. Prior FPGA-based FHE accelerators have proposed hardware acceleration of basic FHE primitives for impractical parameter sets without support for bootstrap**. FAB, for the first time ever, accelerates bootstrap** (along with basic FHE primitives) on an FPGA for a secure and practical parameter set. The key contribution of our work is to architect a balanced FAB design, which is not memory bound. To this end, we leverage recent algorithms for bootstrap** while being cognizant of the compute and memory constraints of our FPGA. We use a minimal number of functional units for computing, operate at a low frequency, leverage high data rates to and from main memory, utilize the limited on-chip memory effectively, and perform operation scheduling carefully. For bootstrap** a fully-packed ciphertext, while operating at 300 MHz, FAB outperforms existing state-of-the-art CPU and GPU implementations by 213x and 1.5x respectively. Our target FHE application is training a logistic regression model over encrypted data. For logistic regression model training scaled to 8 FPGAs on the cloud, FAB outperforms a CPU and GPU by 456x and 6.5x and provides competitive performance when compared to the state-of-the-art ASIC design at a fraction of the cost.
△ Less
Submitted 24 July, 2022;
originally announced July 2022.
-
Overview of LiLAS 2021 -- Living Labs for Academic Search
Authors:
Philipp Schaer,
Timo Breuer,
Leyla Jael Castro,
Benjamin Wolff,
Johann Schaible,
Narges Tavakolpoursaleh
Abstract:
The Living Labs for Academic Search (LiLAS) lab aims to strengthen the concept of user-centric living labs for academic search. The methodological gap between real-world and lab-based evaluation should be bridged by allowing lab participants to evaluate their retrieval approaches in two real-world academic search systems from life sciences and social sciences. This overview paper outlines the two…
▽ More
The Living Labs for Academic Search (LiLAS) lab aims to strengthen the concept of user-centric living labs for academic search. The methodological gap between real-world and lab-based evaluation should be bridged by allowing lab participants to evaluate their retrieval approaches in two real-world academic search systems from life sciences and social sciences. This overview paper outlines the two academic search systems LIVIVO and GESIS Search, and their corresponding tasks within LiLAS, which are ad-hoc retrieval and dataset recommendation. The lab is based on a new evaluation infrastructure named STELLA that allows participants to submit results corresponding to their experimental systems in the form of pre-computed runs and Docker containers that can be integrated into production systems and generate experimental results in real-time. Both submission types are interleaved with the results provided by the productive systems allowing for a seamless presentation and evaluation. The evaluation of results and a meta-analysis of the different tasks and submission types complement this overview.
△ Less
Submitted 10 March, 2022;
originally announced March 2022.
-
Does Fully Homomorphic Encryption Need Compute Acceleration?
Authors:
Leo de Castro,
Rashmi Agrawal,
Rabia Yazicigil,
Anantha Chandrakasan,
Vinod Vaikuntanathan,
Chiraag Juvekar,
Ajay Joshi
Abstract:
Fully Homomorphic Encryption (FHE) allows arbitrarily complex computations on encrypted data without ever needing to decrypt it, thus enabling us to maintain data privacy on third-party systems. Unfortunately, sustaining deep computations with FHE requires a periodic noise reduction step known as bootstrap**. The cost of the bootstrap** operation is one of the primary barriers to the wide-spre…
▽ More
Fully Homomorphic Encryption (FHE) allows arbitrarily complex computations on encrypted data without ever needing to decrypt it, thus enabling us to maintain data privacy on third-party systems. Unfortunately, sustaining deep computations with FHE requires a periodic noise reduction step known as bootstrap**. The cost of the bootstrap** operation is one of the primary barriers to the wide-spread adoption of FHE. In this paper, we present an in-depth architectural analysis of the bootstrap** step in FHE. First, we observe that secure implementations of bootstrap** exhibit a low arithmetic intensity (<1 Op/byte), require large caches (>100 MB), and are heavily bound by the main memory bandwidth. Consequently, we demonstrate that existing workloads observe marginal performance gains from the design of bespoke high-throughput arithmetic units tailored to FHE. Second, we propose several cache-friendly algorithmic optimizations that improve the throughput in FHE bootstrap** by enabling up to 3.2x higher arithmetic intensity and 4.6x lower memory bandwidth. Our optimizations apply to a wide range of structurally similar computations such as private evaluation and training of machine learning models. Finally, we incorporate these optimizations into an architectural tool which, given a cache size, memory subsystem, the number of functional units and a desired security level, selects optimal cryptosystem parameters to maximize the bootstrap** throughput. Our optimized bootstrap** implementation represents a best-case scenario for compute acceleration of FHE. We show that despite these optimizations, bootstrap** continues to be bottlenecked by main memory bandwidth. We propose new research directions to address the underlying memory bottleneck. In summary, our answer to the titular question is: yes, but only after addressing the memory bottleneck!
△ Less
Submitted 14 December, 2021; v1 submitted 12 December, 2021;
originally announced December 2021.
-
Evaluating the Single-Shot MultiBox Detector and YOLO Deep Learning Models for the Detection of Tomatoes in a Greenhouse
Authors:
Sandro A. Magalhães,
Luís Castro,
Germano Moreira,
Filipe N. Santos,
mário Cunha,
Jorge Dias,
António P. Moreira
Abstract:
The development of robotic solutions for agriculture requires advanced perception capabilities that can work reliably in any crop stage. For example, to automatise the tomato harvesting process in greenhouses, the visual perception system needs to detect the tomato in any life cycle stage (flower to the ripe tomato). The state-of-the-art for visual tomato detection focuses mainly on ripe tomato, w…
▽ More
The development of robotic solutions for agriculture requires advanced perception capabilities that can work reliably in any crop stage. For example, to automatise the tomato harvesting process in greenhouses, the visual perception system needs to detect the tomato in any life cycle stage (flower to the ripe tomato). The state-of-the-art for visual tomato detection focuses mainly on ripe tomato, which has a distinctive colour from the background. This paper contributes with an annotated visual dataset of green and reddish tomatoes. This kind of dataset is uncommon and not available for research purposes. This will enable further developments in edge artificial intelligence for in situ and in real-time visual tomato detection required for the development of harvesting robots. Considering this dataset, five deep learning models were selected, trained and benchmarked to detect green and reddish tomatoes grown in greenhouses. Considering our robotic platform specifications, only the Single-Shot MultiBox Detector (SSD) and YOLO architectures were considered. The results proved that the system can detect green and reddish tomatoes, even those occluded by leaves. SSD MobileNet v2 had the best performance when compared against SSD Inception v2, SSD ResNet 50, SSD ResNet 101 and YOLOv4 Tiny, reaching an F1-score of 66.15%, an mAP of 51.46% and an inference time of 16.44 ms with the NVIDIA Turing Architecture platform, an NVIDIA Tesla T4, with 12 GB. YOLOv4 Tiny also had impressive results, mainly concerning inferring times of about 5 ms.
△ Less
Submitted 2 September, 2021;
originally announced September 2021.
-
Packaging research artefacts with RO-Crate
Authors:
Stian Soiland-Reyes,
Peter Sefton,
Mercè Crosas,
Leyla Jael Castro,
Frederik Coppens,
José M. Fernández,
Daniel Garijo,
Björn Grüning,
Marco La Rosa,
Simone Leo,
Eoghan Ó Carragáin,
Marc Portier,
Ana Trisovic,
RO-Crate Community,
Paul Groth,
Carole Goble
Abstract:
An increasing number of researchers support reproducibility by including pointers to and descriptions of datasets, software and methods in their publications. However, scientific articles may be ambiguous, incomplete and difficult to process by automated systems. In this paper we introduce RO-Crate, an open, community-driven, and lightweight approach to packaging research artefacts along with thei…
▽ More
An increasing number of researchers support reproducibility by including pointers to and descriptions of datasets, software and methods in their publications. However, scientific articles may be ambiguous, incomplete and difficult to process by automated systems. In this paper we introduce RO-Crate, an open, community-driven, and lightweight approach to packaging research artefacts along with their metadata in a machine readable manner. RO-Crate is based on Schema$.$org annotations in JSON-LD, aiming to establish best practices to formally describe metadata in an accessible and practical way for their use in a wide variety of situations.
An RO-Crate is a structured archive of all the items that contributed to a research outcome, including their identifiers, provenance, relations and annotations. As a general purpose packaging approach for data and their metadata, RO-Crate is used across multiple areas, including bioinformatics, digital humanities and regulatory sciences. By applying "just enough" Linked Data standards, RO-Crate simplifies the process of making research outputs FAIR while also enhancing research reproducibility.
An RO-Crate for this article is available at https://w3id.org/ro/doi/10.5281/zenodo.5146227
△ Less
Submitted 6 December, 2021; v1 submitted 14 August, 2021;
originally announced August 2021.
-
Detecting Oxbow Code in Erlang Codebases with the Highest Degree of Certainty
Authors:
Fernando Benavides Rodríguez,
Laura M. Castro
Abstract:
The presence of source code that is no longer needed is a handicap to project maintainability. The larger and longer-lived the project, the higher the chances of accumulating dead code in its different forms.
Manually detecting unused code is time-consuming, tedious, error-prone, and requires a great level of deep knowledge about the codebase. In this paper, we examine the kinds of dead code (sp…
▽ More
The presence of source code that is no longer needed is a handicap to project maintainability. The larger and longer-lived the project, the higher the chances of accumulating dead code in its different forms.
Manually detecting unused code is time-consuming, tedious, error-prone, and requires a great level of deep knowledge about the codebase. In this paper, we examine the kinds of dead code (specifically, oxbow code) that can appear in Erlang projects, and formulate rules to identify them with high accuracy.
We also present an open-source static analyzer that implements these rules, allowing for the automatic detection and confident removal of oxbow code in Erlang codebases, actively contributing to increasing their quality and maintainability.
△ Less
Submitted 19 July, 2021;
originally announced July 2021.
-
mt5se: An Open Source Framework for Building Autonomous Trading Robots
Authors:
Paulo André Lima de Castro
Abstract:
Autonomous trading robots have been studied in artificial intelligence area for quite some time. Many AI techniques have been tested for building autonomous agents able to trade financial assets. These initiatives include traditional neural networks, fuzzy logic, reinforcement learning but also more recent approaches like deep neural networks and deep reinforcement learning. Many developers claim…
▽ More
Autonomous trading robots have been studied in artificial intelligence area for quite some time. Many AI techniques have been tested for building autonomous agents able to trade financial assets. These initiatives include traditional neural networks, fuzzy logic, reinforcement learning but also more recent approaches like deep neural networks and deep reinforcement learning. Many developers claim to be successful in creating robots with great performance when simulating execution with historical price series, so called backtesting. However, when these robots are used in real markets frequently they present poor performance in terms of risks and return. In this paper, we propose an open source framework (mt5se) that helps the development, backtesting, live testing and real operation of autonomous traders. We built and tested several traders using mt5se. The results indicate that it may help the development of better traders. Furthermore, we discuss the simple architecture that is used in many studies and propose an alternative multiagent architecture. Such architecture separates two main concerns for portfolio manager (PM) : price prediction and capital allocation. More than achieve a high accuracy, a PM should increase profits when it is right and reduce loss when it is wrong. Furthermore, price prediction is highly dependent of asset's nature and history, while capital allocation is dependent only on analyst's prediction performance and assets' correlation. Finally, we discuss some promising technologies in the area.
△ Less
Submitted 28 June, 2022; v1 submitted 20 January, 2021;
originally announced January 2021.
-
Is it a great Autonomous FX Trading Strategy or you are just fooling yourself
Authors:
Murilo Sibrao Bernardini,
Paulo Andre Lima de Castro
Abstract:
In this paper, we propose a method for evaluating autonomous trading strategies that provides realistic expectations, regarding the strategy's long-term performance. This method addresses This method addresses many pitfalls that currently fool even experienced software developers and researchers, not to mention the customers that purchase these products. We present the results of applying our meth…
▽ More
In this paper, we propose a method for evaluating autonomous trading strategies that provides realistic expectations, regarding the strategy's long-term performance. This method addresses This method addresses many pitfalls that currently fool even experienced software developers and researchers, not to mention the customers that purchase these products. We present the results of applying our method to several famous autonomous trading strategies, which are used to manage a diverse selection of financial assets. The results show that many of these published strategies are far from being reliable vehicles for financial investment. Our method exposes the difficulties involved in building a reliable, long-term strategy and provides a means to compare potential strategies and select the most promising one by establishing minimal periods and requirements for the test executions. There are many developers that create software to buy and sell financial assets autonomously and some of them present great performance when simulating with historical price series (commonly called backtests). Nevertheless, when these strategies are used in real markets (or data not used in their training or evaluation), quite often they perform very poorly. The proposed method can be used to evaluate potential strategies. In this way, the method helps to tell if you really have a great trading strategy or you are just fooling yourself.
△ Less
Submitted 19 November, 2021; v1 submitted 15 January, 2021;
originally announced January 2021.
-
Discriminatory Expressions to Produce Interpretable Models in Short Documents
Authors:
Manuel Francisco,
Juan Luis Castro
Abstract:
Social Networking Sites (SNS) are one of the most important ways of communication. In particular, microblogging sites are being used as analysis avenues due to their peculiarities (promptness, short texts...). There are countless researches that use SNS in novel manners, but machine learning has focused mainly in classification performance rather than interpretability and/or other goodness metrics…
▽ More
Social Networking Sites (SNS) are one of the most important ways of communication. In particular, microblogging sites are being used as analysis avenues due to their peculiarities (promptness, short texts...). There are countless researches that use SNS in novel manners, but machine learning has focused mainly in classification performance rather than interpretability and/or other goodness metrics. Thus, state-of-the-art models are black boxes that should not be used to solve problems that may have a social impact. When the problem requires transparency, it is necessary to build interpretable pipelines. Although the classifier may be interpretable, resulting models are too complex to be considered comprehensible, making it impossible for humans to understand the actual decisions. This paper presents a feature selection mechanism that is able to improve comprehensibility by using less but more meaningful features while achieving good performance in microblogging contexts where interpretability is mandatory. Moreover, we present a ranking method to evaluate features in terms of statistical relevance and bias. We conducted exhaustive tests with five different datasets in order to evaluate classification performance, generalisation capacity and complexity of the model. Results show that our proposal is better and the most stable one in terms of accuracy, generalisation and comprehensibility.
△ Less
Submitted 15 February, 2021; v1 submitted 27 November, 2020;
originally announced December 2020.
-
It was never about the language: paradigm impact on software design decisions
Authors:
Laura M. Castro
Abstract:
Programming languages development has intensified in recent years. New ones are created; new features, often cross-paradigm, are featured in old ones. This new programming landscape makes language selection a more complex decision, both from the companies points of view (technical, recruiting) and from the developers point of view (career development). In this paper, however, we argue that program…
▽ More
Programming languages development has intensified in recent years. New ones are created; new features, often cross-paradigm, are featured in old ones. This new programming landscape makes language selection a more complex decision, both from the companies points of view (technical, recruiting) and from the developers point of view (career development). In this paper, however, we argue that programming languages have a secondary role in software development design decisions. We illustrate, based on a practical example, how the main influencer are higher-level traits: those traditionally assigned with programming paradigms. Following this renovated perspective, concerns about language choice are shifted for all parties. Beyond particular syntax, grammar, execution model or code organization, the main consequence of the predominance of one paradigm or another in the mind of the developer is the way solutions are designed.
△ Less
Submitted 16 October, 2020;
originally announced October 2020.
-
CryptoCredit: Securely Training Fair Models
Authors:
Leo de Castro,
Jiahao Chen,
Antigoni Polychroniadou
Abstract:
When develo** models for regulated decision making, sensitive features like age, race and gender cannot be used and must be obscured from model developers to prevent bias. However, the remaining features still need to be tested for correlation with sensitive features, which can only be done with the knowledge of those features. We resolve this dilemma using a fully homomorphic encryption scheme,…
▽ More
When develo** models for regulated decision making, sensitive features like age, race and gender cannot be used and must be obscured from model developers to prevent bias. However, the remaining features still need to be tested for correlation with sensitive features, which can only be done with the knowledge of those features. We resolve this dilemma using a fully homomorphic encryption scheme, allowing model developers to train linear regression and logistic regression models and test them for possible bias without ever revealing the sensitive features in the clear. We demonstrate how it can be applied to leave-one-out regression testing, and show using the adult income data set that our method is practical to run.
△ Less
Submitted 9 October, 2020;
originally announced October 2020.
-
A Multi-Agent System for Solving the Dynamic Capacitated Vehicle Routing Problem with Stochastic Customers using Trajectory Data Mining
Authors:
Juan Camilo Fonseca-Galindo,
Gabriela de Castro Surita,
José Maia Neto,
Cristiano Leite de Castro,
André Paim Lemos
Abstract:
The worldwide growth of e-commerce has created new challenges for logistics companies, one of which is being able to deliver products quickly and at low cost, which reflects directly in the way of sorting packages, needing to eliminate steps such as storage and batch creation. Our work presents a multi-agent system that uses trajectory data mining techniques to extract territorial patterns and use…
▽ More
The worldwide growth of e-commerce has created new challenges for logistics companies, one of which is being able to deliver products quickly and at low cost, which reflects directly in the way of sorting packages, needing to eliminate steps such as storage and batch creation. Our work presents a multi-agent system that uses trajectory data mining techniques to extract territorial patterns and use them in the dynamic creation of last-mile routes. The problem can be modeled as a Dynamic Capacitated Vehicle Routing Problem (VRP) with Stochastic Customer, being therefore NP-HARD, what makes its implementation unfeasible for many packages. The work's main contribution is to solve this problem only depending on the Warehouse system configurations and not on the number of packages processed, which is appropriate for Big Data scenarios commonly present in the delivery of e-commerce products. Computational experiments were conducted for single and multi depot instances. Due to its probabilistic nature, the proposed approach presented slightly lower performances when compared to the static VRP algorithm. However, the operational gains that our solution provides making it very attractive for situations in which the routes must be set dynamically.
△ Less
Submitted 26 September, 2020;
originally announced September 2020.
-
Analysis of Social Robotic Navigation approaches: CNN Encoder and Incremental Learning as an alternative to Deep Reinforcement Learning
Authors:
Janderson Ferreira,
Agostinho A. F. Júnior,
Letícia Castro,
Yves M. Galvão,
Pablo Barros,
Bruno J. T. Fernandes
Abstract:
Dealing with social tasks in robotic scenarios is difficult, as having humans in the learning loop is incompatible with most of the state-of-the-art machine learning algorithms. This is the case when exploring Incremental learning models, in particular the ones involving reinforcement learning. In this work, we discuss this problem and possible solutions by analysing a previous study on adaptive c…
▽ More
Dealing with social tasks in robotic scenarios is difficult, as having humans in the learning loop is incompatible with most of the state-of-the-art machine learning algorithms. This is the case when exploring Incremental learning models, in particular the ones involving reinforcement learning. In this work, we discuss this problem and possible solutions by analysing a previous study on adaptive convolutional encoders for a social navigation task.
△ Less
Submitted 5 September, 2020; v1 submitted 18 August, 2020;
originally announced August 2020.
-
Reusing Trained Layers of Convolutional Neural Networks to Shorten Hyperparameters Tuning Time
Authors:
Roberto L. Castro,
Diego Andrade,
Basilio Fraguela
Abstract:
Hyperparameters tuning is a time-consuming approach, particularly when the architecture of the neural network is decided as part of this process. For instance, in convolutional neural networks (CNNs), the selection of the number and the characteristics of the hidden (convolutional) layers may be decided. This implies that the search process involves the training of all these candidate network arch…
▽ More
Hyperparameters tuning is a time-consuming approach, particularly when the architecture of the neural network is decided as part of this process. For instance, in convolutional neural networks (CNNs), the selection of the number and the characteristics of the hidden (convolutional) layers may be decided. This implies that the search process involves the training of all these candidate network architectures.
This paper describes a proposal to reuse the weights of hidden (convolutional) layers among different trainings to shorten this process. The rationale is that if a set of convolutional layers have been trained to solve a given problem, the weights calculated in this training may be useful when a new convolutional layer is added to the network architecture.
This idea has been tested using the CIFAR-10 dataset, testing different CNNs architectures with up to 3 convolutional layers and up to 3 fully connected layers. The experiments compare the training time and the validation loss when reusing and not reusing convolutional layers. They confirm that this strategy reduces the training time while it even increases the accuracy of the resulting neural network. This finding opens up the future possibility of integrating this strategy in existing AutoML methods with the purpose of reducing the total search time.
△ Less
Submitted 30 July, 2020; v1 submitted 16 June, 2020;
originally announced June 2020.
-
A Hybrid Approach for Tracking Individual Players in Broadcast Match Videos
Authors:
Roberto L. Castro,
Diego Andrade,
Basilio Fraguela
Abstract:
Tracking people in a video sequence is a challenging task that has been approached from many perspectives. This task becomes even more complicated when the person to track is a player in a broadcasted sport event, the reasons being the existence of difficulties such as frequent camera movements or switches, total and partial occlusions between players, and blurry frames due to the codification alg…
▽ More
Tracking people in a video sequence is a challenging task that has been approached from many perspectives. This task becomes even more complicated when the person to track is a player in a broadcasted sport event, the reasons being the existence of difficulties such as frequent camera movements or switches, total and partial occlusions between players, and blurry frames due to the codification algorithm of the video. This paper introduces a player tracking solution which is both fast and accurate. This allows to track a player precisely in real-time. The approach combines several models that are executed concurrently in a relatively modest hardware, and whose accuracy has been validated against hand-labeled broadcast video sequences. Regarding the accuracy, the tests show that the area under curve (AUC) of our approach is around 0.6, which is similar to generic state of the art solutions. As for performance, our proposal can process high definition videos (1920x1080 px) at 80 fps.
△ Less
Submitted 10 March, 2020; v1 submitted 6 March, 2020;
originally announced March 2020.
-
A Probabilistic Adaptive Search System for Exploring the Face Space
Authors:
Andres G. Abad,
Luis I. Reyes Castro
Abstract:
Face recall is a basic human cognitive process performed routinely, e.g., when meeting someone and determining if we have met that person before. Assisting a subject during face recall by suggesting candidate faces can be challenging. One of the reasons is that the search space - the face space - is quite large and lacks structure. A commercial application of face recall is facial composite system…
▽ More
Face recall is a basic human cognitive process performed routinely, e.g., when meeting someone and determining if we have met that person before. Assisting a subject during face recall by suggesting candidate faces can be challenging. One of the reasons is that the search space - the face space - is quite large and lacks structure. A commercial application of face recall is facial composite systems - such as Identikit, PhotoFIT, and CD-FIT - where a witness searches for an image of a face that resembles his memory of a particular offender. The inherent uncertainty and cost in the evaluation of the objective function, the large size and lack of structure of the search space, and the unavailability of the gradient concept makes this problem inappropriate for traditional optimization methods. In this paper we propose a novel evolutionary approach for searching the face space that can be used as a facial composite system. The approach is inspired by methods of Bayesian optimization and differs from other applications in the use of the skew-normal distribution as its acquisition function. This choice of acquisition function provides greater granularity, with regularized, conservative, and realistic results.
△ Less
Submitted 28 April, 2016;
originally announced April 2016.
-
Incremental Sampling-based Algorithm for Minimum-violation Motion Planning
Authors:
Luis I. Reyes Castro,
Pratik Chaudhari,
Jana Tumova,
Sertac Karaman,
Emilio Frazzoli,
Daniela Rus
Abstract:
This paper studies the problem of control strategy synthesis for dynamical systems with differential constraints to fulfill a given reachability goal while satisfying a set of safety rules. Particular attention is devoted to goals that become feasible only if a subset of the safety rules are violated. The proposed algorithm computes a control law, that minimizes the level of unsafety while the des…
▽ More
This paper studies the problem of control strategy synthesis for dynamical systems with differential constraints to fulfill a given reachability goal while satisfying a set of safety rules. Particular attention is devoted to goals that become feasible only if a subset of the safety rules are violated. The proposed algorithm computes a control law, that minimizes the level of unsafety while the desired goal is guaranteed to be reached. This problem is motivated by an autonomous car navigating an urban environment while following rules of the road such as "always travel in right lane'' and "do not change lanes frequently''. Ideas behind sampling based motion-planning algorithms, such as Probabilistic Road Maps (PRMs) and Rapidly-exploring Random Trees (RRTs), are employed to incrementally construct a finite concretization of the dynamics as a durational Kripke structure. In conjunction with this, a weighted finite automaton that captures the safety rules is used in order to find an optimal trajectory that minimizes the violation of safety rules. We prove that the proposed algorithm guarantees asymptotic optimality, i.e., almost-sure convergence to optimal solutions. We present results of simulation experiments and an implementation on an autonomous urban mobility-on-demand system.
△ Less
Submitted 5 November, 2013; v1 submitted 6 May, 2013;
originally announced May 2013.
-
Minimum-violation LTL Planning with Conflicting Specifications
Authors:
Jana Tumova,
Luis I. Reyes Castro,
Sertac Karaman,
Emilio Frazzoli,
Daniela Rus
Abstract:
We consider the problem of automatic generation of control strategies for robotic vehicles given a set of high-level mission specifications, such as "Vehicle x must eventually visit a target region and then return to a base," "Regions A and B must be periodically surveyed," or "None of the vehicles can enter an unsafe region." We focus on instances when all of the given specifications cannot be re…
▽ More
We consider the problem of automatic generation of control strategies for robotic vehicles given a set of high-level mission specifications, such as "Vehicle x must eventually visit a target region and then return to a base," "Regions A and B must be periodically surveyed," or "None of the vehicles can enter an unsafe region." We focus on instances when all of the given specifications cannot be reached simultaneously due to their incompatibility and/or environmental constraints. We aim to find the least-violating control strategy while considering different priorities of satisfying different parts of the mission. Formally, we consider the missions given in the form of linear temporal logic formulas, each of which is assigned a reward that is earned when the formula is satisfied. Leveraging ideas from the automata-based model checking, we propose an algorithm for finding an optimal control strategy that maximizes the sum of rewards earned if this control strategy is applied. We demonstrate the proposed algorithm on an illustrative case study.
△ Less
Submitted 15 March, 2013;
originally announced March 2013.
-
Generating the Structure of a Fuzzy Rule under Uncertainty
Authors:
Juan Luis Castro,
Jose Manuel Zurita
Abstract:
The aim of this paper is to present a method for identifying the structure of a rule in a fuzzy model. For this purpose, an ATMS shall be used (Zurita 1994). An algorithm obtaining the identification of the structure will be suggested (Castro 1995). The minimal structure of the rule (with respect to the number of variables that must appear in the rule) will be found by this algorithm. Furthermo…
▽ More
The aim of this paper is to present a method for identifying the structure of a rule in a fuzzy model. For this purpose, an ATMS shall be used (Zurita 1994). An algorithm obtaining the identification of the structure will be suggested (Castro 1995). The minimal structure of the rule (with respect to the number of variables that must appear in the rule) will be found by this algorithm. Furthermore, the identification parameters shall be obtained simultaneously. The proposed method shall be applied for classification to an example. The {em Iris Plant Database} shall be learnt for all three kinds of plants.
△ Less
Submitted 20 February, 2013;
originally announced February 2013.
-
An Environment for the Exploration of Non Monotonic Logic Programs
Authors:
Luis F. Castro,
David S. Warren
Abstract:
Stable Model Semantics and Well Founded Semantics have been shown to be very useful in several applications of non-monotonic reasoning. However, Stable Models presents a high computational complexity, whereas Well Founded Semantics is easy to compute and provides an approximation of Stable Models. Efficient engines exist for both semantics of logic programs. This work presents a computational in…
▽ More
Stable Model Semantics and Well Founded Semantics have been shown to be very useful in several applications of non-monotonic reasoning. However, Stable Models presents a high computational complexity, whereas Well Founded Semantics is easy to compute and provides an approximation of Stable Models. Efficient engines exist for both semantics of logic programs. This work presents a computational integration of two of such systems, namely XSB and SMODELS. The resulting system is called XNMR, and provides an interactive system for the exploration of both semantics. Aspects such as modularity can be exploited in order to ease debugging of large knowledge bases with the usual Prolog debugging techniques and an interactive environment. Besides, the use of a full Prolog system as a front-end to a Stable Models engine augments the language usually accepted by such systems.
△ Less
Submitted 19 November, 2001;
originally announced November 2001.
-
XNMR: A tool for knowledge bases exploration
Authors:
L. Castro,
D. Warren
Abstract:
XNMR is a system designed to explore the results of combining the well-founded semantics system XSB with the stable-models evaluator SMODELS. Its main goal is to work as a tool for fast and interactive exploration of knowledge bases.
XNMR is a system designed to explore the results of combining the well-founded semantics system XSB with the stable-models evaluator SMODELS. Its main goal is to work as a tool for fast and interactive exploration of knowledge bases.
△ Less
Submitted 13 March, 2000;
originally announced March 2000.
