-
Naming the Pain in Machine Learning-Enabled Systems Engineering
Authors:
Marcos Kalinowski,
Daniel Mendez,
Görkem Giray,
Antonio Pedro Santos Alves,
Kelly Azevedo,
Tatiana Escovedo,
Hugo Villamizar,
Helio Lopes,
Teresa Baldassarre,
Stefan Wagner,
Stefan Biffl,
Jürgen Musil,
Michael Felderer,
Niklas Lavesson,
Tony Gorschek
Abstract:
Context: Machine learning (ML)-enabled systems are being increasingly adopted by companies aiming to enhance their products and operational processes. Objective: This paper aims to deliver a comprehensive overview of the current status quo of engineering ML-enabled systems and lay the foundation to steer practically relevant and problem-driven academic research. Method: We conducted an internation…
▽ More
Context: Machine learning (ML)-enabled systems are being increasingly adopted by companies aiming to enhance their products and operational processes. Objective: This paper aims to deliver a comprehensive overview of the current status quo of engineering ML-enabled systems and lay the foundation to steer practically relevant and problem-driven academic research. Method: We conducted an international survey to collect insights from practitioners on the current practices and problems in engineering ML-enabled systems. We received 188 complete responses from 25 countries. We conducted quantitative statistical analyses on contemporary practices using bootstrap** with confidence intervals and qualitative analyses on the reported problems using open and axial coding procedures. Results: Our survey results reinforce and extend existing empirical evidence on engineering ML-enabled systems, providing additional insights into typical ML-enabled systems project contexts, the perceived relevance and complexity of ML life cycle phases, and current practices related to problem understanding, model deployment, and model monitoring. Furthermore, the qualitative analysis provides a detailed map of the problems practitioners face within each ML life cycle phase and the problems causing overall project failure. Conclusions: The results contribute to a better understanding of the status quo and problems in practical environments. We advocate for the further adaptation and dissemination of software engineering practices to enhance the engineering of ML-enabled systems.
△ Less
Submitted 20 May, 2024;
originally announced June 2024.
-
High-Resolution Detection of Earth Structural Heterogeneities from Seismic Amplitudes using Convolutional Neural Networks with Attention layers
Authors:
Luiz Schirmer,
Guilherme Schardong,
Vinícius da Silva,
Rogério Santos,
Hélio Lopes
Abstract:
Earth structural heterogeneities have a remarkable role in the petroleum economy for both exploration and production projects. Automatic detection of detailed structural heterogeneities is challenging when considering modern machine learning techniques like deep neural networks. Typically, these techniques can be an excellent tool for assisted interpretation of such heterogeneities, but it heavily…
▽ More
Earth structural heterogeneities have a remarkable role in the petroleum economy for both exploration and production projects. Automatic detection of detailed structural heterogeneities is challenging when considering modern machine learning techniques like deep neural networks. Typically, these techniques can be an excellent tool for assisted interpretation of such heterogeneities, but it heavily depends on the amount of data to be trained.
We propose an efficient and cost-effective architecture for detecting seismic structural heterogeneities using Convolutional Neural Networks (CNNs) combined with Attention layers. The attention mechanism reduces costs and enhances accuracy, even in cases with relatively noisy data. Our model has half the parameters compared to the state-of-the-art, and it outperforms previous methods in terms of Intersection over Union (IoU) by 0.6% and precision by 0.4%. By leveraging synthetic data, we apply transfer learning to train and fine-tune the model, addressing the challenge of limited annotated data availability.
△ Less
Submitted 15 April, 2024;
originally announced April 2024.
-
Investigating the Impact of SOLID Design Principles on Machine Learning Code Understanding
Authors:
Raphael Cabral,
Marcos Kalinowski,
Maria Teresa Baldassarre,
Hugo Villamizar,
Tatiana Escovedo,
Hélio Lopes
Abstract:
[Context] Applying design principles has long been acknowledged as beneficial for understanding and maintainability in traditional software projects. These benefits may similarly hold for Machine Learning (ML) projects, which involve iterative experimentation with data, models, and algorithms. However, ML components are often developed by data scientists with diverse educational backgrounds, poten…
▽ More
[Context] Applying design principles has long been acknowledged as beneficial for understanding and maintainability in traditional software projects. These benefits may similarly hold for Machine Learning (ML) projects, which involve iterative experimentation with data, models, and algorithms. However, ML components are often developed by data scientists with diverse educational backgrounds, potentially resulting in code that doesn't adhere to software design best practices. [Goal] In order to better understand this phenomenon, we investigated the impact of the SOLID design principles on ML code understanding. [Method] We conducted a controlled experiment with three independent trials involving 100 data scientists. We restructured real industrial ML code that did not use SOLID principles. Within each trial, one group was presented with the original ML code, while the other was presented with ML code incorporating SOLID principles. Participants of both groups were asked to analyze the code and fill out a questionnaire that included both open-ended and closed-ended questions on their understanding. [Results] The study results provide statistically significant evidence that the adoption of the SOLID design principles can improve code understanding within the realm of ML projects. [Conclusion] We put forward that software engineering design principles should be spread within the data science community and considered for enhancing the maintainability of ML code.
△ Less
Submitted 7 February, 2024;
originally announced February 2024.
-
ML-Enabled Systems Model Deployment and Monitoring: Status Quo and Problems
Authors:
Eduardo Zimelewicz,
Marcos Kalinowski,
Daniel Mendez,
Görkem Giray,
Antonio Pedro Santos Alves,
Niklas Lavesson,
Kelly Azevedo,
Hugo Villamizar,
Tatiana Escovedo,
Helio Lopes,
Stefan Biffl,
Juergen Musil,
Michael Felderer,
Stefan Wagner,
Teresa Baldassarre,
Tony Gorschek
Abstract:
[Context] Systems incorporating Machine Learning (ML) models, often called ML-enabled systems, have become commonplace. However, empirical evidence on how ML-enabled systems are engineered in practice is still limited, especially for activities surrounding ML model dissemination. [Goal] We investigate contemporary industrial practices and problems related to ML model dissemination, focusing on the…
▽ More
[Context] Systems incorporating Machine Learning (ML) models, often called ML-enabled systems, have become commonplace. However, empirical evidence on how ML-enabled systems are engineered in practice is still limited, especially for activities surrounding ML model dissemination. [Goal] We investigate contemporary industrial practices and problems related to ML model dissemination, focusing on the model deployment and the monitoring of ML life cycle phases. [Method] We conducted an international survey to gather practitioner insights on how ML-enabled systems are engineered. We gathered a total of 188 complete responses from 25 countries. We analyze the status quo and problems reported for the model deployment and monitoring phases. We analyzed contemporary practices using bootstrap** with confidence intervals and conducted qualitative analyses on the reported problems applying open and axial coding procedures. [Results] Practitioners perceive the model deployment and monitoring phases as relevant and difficult. With respect to model deployment, models are typically deployed as separate services, with limited adoption of MLOps principles. Reported problems include difficulties in designing the architecture of the infrastructure for production deployment and legacy application integration. Concerning model monitoring, many models in production are not monitored. The main monitored aspects are inputs, outputs, and decisions. Reported problems involve the absence of monitoring practices, the need to create custom monitoring tools, and the selection of suitable metrics. [Conclusion] Our results help provide a better understanding of the adopted practices and problems in practice and support guiding ML deployment and monitoring research in a problem-driven manner.
△ Less
Submitted 7 February, 2024;
originally announced February 2024.
-
Status Quo and Problems of Requirements Engineering for Machine Learning: Results from an International Survey
Authors:
Antonio Pedro Santos Alves,
Marcos Kalinowski,
Görkem Giray,
Daniel Mendez,
Niklas Lavesson,
Kelly Azevedo,
Hugo Villamizar,
Tatiana Escovedo,
Helio Lopes,
Stefan Biffl,
Jürgen Musil,
Michael Felderer,
Stefan Wagner,
Teresa Baldassarre,
Tony Gorschek
Abstract:
Systems that use Machine Learning (ML) have become commonplace for companies that want to improve their products and processes. Literature suggests that Requirements Engineering (RE) can help address many problems when engineering ML-enabled systems. However, the state of empirical evidence on how RE is applied in practice in the context of ML-enabled systems is mainly dominated by isolated case s…
▽ More
Systems that use Machine Learning (ML) have become commonplace for companies that want to improve their products and processes. Literature suggests that Requirements Engineering (RE) can help address many problems when engineering ML-enabled systems. However, the state of empirical evidence on how RE is applied in practice in the context of ML-enabled systems is mainly dominated by isolated case studies with limited generalizability. We conducted an international survey to gather practitioner insights into the status quo and problems of RE in ML-enabled systems. We gathered 188 complete responses from 25 countries. We conducted quantitative statistical analyses on contemporary practices using bootstrap** with confidence intervals and qualitative analyses on the reported problems involving open and axial coding procedures. We found significant differences in RE practices within ML projects. For instance, (i) RE-related activities are mostly conducted by project leaders and data scientists, (ii) the prevalent requirements documentation format concerns interactive Notebooks, (iii) the main focus of non-functional requirements includes data quality, model reliability, and model explainability, and (iv) main challenges include managing customer expectations and aligning requirements with data. The qualitative analyses revealed that practitioners face problems related to lack of business domain understanding, unclear goals and requirements, low customer engagement, and communication issues. These results help to provide a better understanding of the adopted practices and of which problems exist in practical environments. We put forward the need to adapt further and disseminate RE-related practices for engineering ML-enabled systems.
△ Less
Submitted 10 October, 2023;
originally announced October 2023.
-
Identifying Concerns When Specifying Machine Learning-Enabled Systems: A Perspective-Based Approach
Authors:
Hugo Villamizar,
Marcos Kalinowski,
Helio Lopes,
Daniel Mendez
Abstract:
Engineering successful machine learning (ML)-enabled systems poses various challenges from both a theoretical and a practical side. Among those challenges are how to effectively address unrealistic expectations of ML capabilities from customers, managers and even other team members, and how to connect business value to engineering and data science activities composed by interdisciplinary teams. In…
▽ More
Engineering successful machine learning (ML)-enabled systems poses various challenges from both a theoretical and a practical side. Among those challenges are how to effectively address unrealistic expectations of ML capabilities from customers, managers and even other team members, and how to connect business value to engineering and data science activities composed by interdisciplinary teams. In this paper, we present PerSpecML, a perspective-based approach for specifying ML-enabled systems that helps practitioners identify which attributes, including ML and non-ML components, are important to contribute to the overall system's quality. The approach involves analyzing 59 concerns related to typical tasks that practitioners face in ML projects, grou** them into five perspectives: system objectives, user experience, infrastructure, model, and data. Together, these perspectives serve to mediate the communication between business owners, domain experts, designers, software and ML engineers, and data scientists. The creation of PerSpecML involved a series of validations conducted in different contexts: (i) in academia, (ii) with industry representatives, and (iii) in two real industrial case studies. As a result of the diverse validations and continuous improvements, PerSpecML stands as a promising approach, poised to positively impact the specification of ML-enabled systems, particularly hel** to reveal key components that would have been otherwise missed without using PerSpecML.
△ Less
Submitted 14 September, 2023;
originally announced September 2023.
-
A Systematic Map** Study and Practitioner Insights on the Use of Software Engineering Practices to Develop MVPs
Authors:
Silvio Alonso,
Marcos Kalinowski,
Bruna Ferreira,
Simone D. J. Barbosa,
Helio Lopes
Abstract:
[Background] The MVP concept has influenced the way in which development teams apply Software Engineering practices. However, the overall understanding of this influence of MVPs on SE practices is still poor. [Objective] Our goal is to characterize the publication landscape on practices that have been used in the context of software MVPs and to gather practitioner insights on the identified practi…
▽ More
[Background] The MVP concept has influenced the way in which development teams apply Software Engineering practices. However, the overall understanding of this influence of MVPs on SE practices is still poor. [Objective] Our goal is to characterize the publication landscape on practices that have been used in the context of software MVPs and to gather practitioner insights on the identified practices. [Method] We conducted a systematic map** study and discussed its results in two focus groups sessions involving twelve industry practitioners that extensively use MVPs in their projects to capture their perceptions on the findings of the map** study. [Results] We identified 33 papers published between 2013 and 2020 and observed some trends related to MVP ideation and evaluation practices. For instance, regarding ideation, we found six different approaches and mainly informal end-user involvement practices. Regarding evaluation, there is an emphasis on end-user validations based on practices such as usability tests, A/B testing, and usage data analysis. However, there is still limited research related to MVP technical feasibility assessment and effort estimation. Practitioners of the focus group sessions reinforced the confidence in our results regarding ideation and evaluation practices, being aware of most of the identified practices. They also reported how they deal with the technical feasibility assessments and effort estimation in practice. [Conclusion] Our analysis suggests that there are opportunities for solution proposals and evaluation studies to address literature gaps concerning technical feasibility assessment and effort estimation. Overall, more effort needs to be invested into empirically evaluating the existing MVP-related practices.
△ Less
Submitted 14 May, 2023;
originally announced May 2023.
-
A visão da BBChain sobre o contexto tecnológico subjacente à adoção do Real Digital
Authors:
Marcio G B de Avellar,
Alexandre A S Junior,
André H G Lopes,
André L S Carneiro,
João A Pereira,
Davi C B D da Cunha
Abstract:
We explore confidential computing in the context of CBDCs using Microsoft's CCF framework as an example. By develo** an experiment and comparing different approaches and performance and security metrics, we seek to evaluate the effectiveness of confidential computing to improve the privacy, security, and performance of CBDCs. Preliminary results suggest that confidential computing could be a pro…
▽ More
We explore confidential computing in the context of CBDCs using Microsoft's CCF framework as an example. By develo** an experiment and comparing different approaches and performance and security metrics, we seek to evaluate the effectiveness of confidential computing to improve the privacy, security, and performance of CBDCs. Preliminary results suggest that confidential computing could be a promising solution to the technological challenges faced by CBDCs. Furthermore, by implementing confidential computing in DLTs such as Hyperledger Besu and utilizing frameworks such as CCF, we increase transaction confidentiality and privacy while maintaining the scalability and interoperability required for a global digital financial system. In conclusion, confidential computing can significantly bolster CBDC development, fostering a secure, private, and efficient financial future.
--
Exploramos o uso da computação confidencial no contexto das CBDCs utilizando o framework CCF da Microsoft como exemplo. Via desenvolvimento de experimentos e comparação de diferentes abordagens e métricas de desempenho e segurança, buscamos avaliar a eficácia da computação confidencial para melhorar a privacidade, segurança e desempenho das CBDCs. Resultados preliminares sugerem que a computação confidencial pode ser uma solução promissora para os desafios tecnológicos enfrentados pelas CBDCs. Ao implementar a computação confidencial em DLTs, como o Hyperledger Besu, e utilizar frameworks como o CCF, aumentamos a confidencialidade e a privacidade das transações, mantendo a escalabilidade e a interoperabilidade necessárias para um sistema financeiro global e digital. Em conclusão, a computação confidencial pode reforçar significativamente o desenvolvimento do CBDC, promovendo um futuro financeiro seguro, privado e eficiente.
△ Less
Submitted 10 April, 2023;
originally announced April 2023.
-
Lessons Learned to Improve the UX Practices in Agile Projects Involving Data Science and Process Automation
Authors:
Bruna Ferreira,
Silvio Marques,
Marcos Kalinowski,
Helio Lopes,
Simone D. J. Barbosa
Abstract:
Context: User-Centered Design and Agile methodologies focus on human issues. Nevertheless, agile methodologies focus on contact with contracting customers and generating value for them. Usually, the communication between end users and the agile team is mediated by customers. However, they do not know the problems end users face in their routines. Hence, UX issues are typically identified only afte…
▽ More
Context: User-Centered Design and Agile methodologies focus on human issues. Nevertheless, agile methodologies focus on contact with contracting customers and generating value for them. Usually, the communication between end users and the agile team is mediated by customers. However, they do not know the problems end users face in their routines. Hence, UX issues are typically identified only after the implementation, during user testing and validation. Objective: Aiming to improve the understanding and definition of the problem in agile projects, this research investigates the practices and difficulties experienced by agile teams during the development of data science and process automation projects. Also, we analyze the benefits and the teams' perceptions regarding user participation in these projects. Method: We collected data from four agile teams in an academia-industry collaboration focusing on delivering data science and process automation solutions. Therefore, we applied a carefully designed questionnaire answered by developers, scrum masters, and UX designers. In total, 18 subjects answered the questionnaire. Results: From the results, we identify practices used by the teams to define and understand the problem and to represent the solution. The practices most often used are prototypes and meetings with stakeholders. Another practice that helped the team to understand the problem was using Lean Inceptions. Also, our results present some specific issues regarding data science projects. Conclusion: We observed that end-user participation can be critical to understanding and defining the problem. They help to define elements of the domain and barriers in the implementation. We identified a need for approaches that facilitate user-team communication in data science projects and the need for more detailed requirements representations to support data science solutions.
△ Less
Submitted 24 November, 2022;
originally announced November 2022.
-
Multiresolution Neural Networks for Imaging
Authors:
Hallison Paz,
Tiago Novello,
Vinicius Silva,
Luiz Schirmer,
Guilherme Schardong,
Fabio Chagas,
Helio Lopes,
Luiz Velho
Abstract:
We present MR-Net, a general architecture for multiresolution neural networks, and a framework for imaging applications based on this architecture. Our coordinate-based networks are continuous both in space and in scale as they are composed of multiple stages that progressively add finer details. Besides that, they are a compact and efficient representation. We show examples of multiresolution ima…
▽ More
We present MR-Net, a general architecture for multiresolution neural networks, and a framework for imaging applications based on this architecture. Our coordinate-based networks are continuous both in space and in scale as they are composed of multiple stages that progressively add finer details. Besides that, they are a compact and efficient representation. We show examples of multiresolution image representation and applications to texturemagnification, minification, and antialiasing. This document is the extended version of the paper [PNS+22]. It includes additional material that would not fit the page limitations of the conference track for publication.
△ Less
Submitted 10 September, 2022; v1 submitted 24 August, 2022;
originally announced August 2022.
-
Towards Perspective-Based Specification of Machine Learning-Enabled Systems
Authors:
Hugo Villamizar,
Marcos Kalinowski,
Helio Lopes
Abstract:
Machine learning (ML) teams often work on a project just to realize the performance of the model is not good enough. Indeed, the success of ML-enabled systems involves aligning data with business problems, translating them into ML tasks, experimenting with algorithms, evaluating models, capturing data from users, among others. Literature has shown that ML-enabled systems are rarely built based on…
▽ More
Machine learning (ML) teams often work on a project just to realize the performance of the model is not good enough. Indeed, the success of ML-enabled systems involves aligning data with business problems, translating them into ML tasks, experimenting with algorithms, evaluating models, capturing data from users, among others. Literature has shown that ML-enabled systems are rarely built based on precise specifications for such concerns, leading ML teams to become misaligned due to incorrect assumptions, which may affect the quality of such systems and overall project success. In order to help addressing this issue, this paper describes our work towards a perspective-based approach for specifying ML-enabled systems. The approach involves analyzing a set of 45 ML concerns grouped into five perspectives: objectives, user experience, infrastructure, model, and data. The main contribution of this paper is to provide two new artifacts that can be used to help specifying ML-enabled systems: (i) the perspective-based ML task and concern diagram and (ii) the perspective-based ML specification template.
△ Less
Submitted 20 June, 2022;
originally announced June 2022.
-
A Catalogue of Concerns for Specifying Machine Learning-Enabled Systems
Authors:
Hugo Villamizar,
Marcos Kalinowski,
Helio lopes
Abstract:
Requirements engineering (RE) activities for machine learning (ML) are not well-established and researched in the literature. Many issues and challenges exist when specifying, designing, and develo** ML-enabled systems. Adding more focus on RE for ML can help to develop more reliable ML-enabled systems. Based on insights collected from previous work and industrial experiences, we propose a catal…
▽ More
Requirements engineering (RE) activities for machine learning (ML) are not well-established and researched in the literature. Many issues and challenges exist when specifying, designing, and develo** ML-enabled systems. Adding more focus on RE for ML can help to develop more reliable ML-enabled systems. Based on insights collected from previous work and industrial experiences, we propose a catalogue of 45 concerns to be considered when specifying ML-enabled systems, covering five different perspectives we identified as relevant for such systems: objectives, user experience, infrastructure, model, and data. Examples of such concerns include the execution engine and telemetry for the infrastructure perspective, and explainability and reproducibility for the model perspective. We conducted a focus group session with eight software professionals with experience develo** ML-enabled systems to validate the importance, quality and feasibility of using our catalogue. The feedback allowed us to improve the catalogue and confirmed its practical relevance. The main research contribution of this work consists in providing a validated set of concerns grouped into perspectives that can be used by requirements engineers to support the specification of ML-enabled systems.
△ Less
Submitted 24 June, 2022; v1 submitted 15 April, 2022;
originally announced April 2022.
-
Neural Implicit Surface Evolution
Authors:
Tiago Novello,
Vinicius da Silva,
Guilherme Schardong,
Luiz Schirmer,
Helio Lopes,
Luiz Velho
Abstract:
This work investigates the use of smooth neural networks for modeling dynamic variations of implicit surfaces under the level set equation (LSE). For this, it extends the representation of neural implicit surfaces to the space-time $\mathbb{R}^3\times \mathbb{R}$, which opens up mechanisms for continuous geometric transformations. Examples include evolving an initial surface towards general vector…
▽ More
This work investigates the use of smooth neural networks for modeling dynamic variations of implicit surfaces under the level set equation (LSE). For this, it extends the representation of neural implicit surfaces to the space-time $\mathbb{R}^3\times \mathbb{R}$, which opens up mechanisms for continuous geometric transformations. Examples include evolving an initial surface towards general vector fields, smoothing and sharpening using the mean curvature equation, and interpolations of initial conditions.
The network training considers two constraints. A data term is responsible for fitting the initial condition to the corresponding time instant, usually $\mathbb{R}^3 \times \{0\}$. Then, a LSE term forces the network to approximate the underlying geometric evolution given by the LSE, without any supervision. The network can also be initialized based on previously trained initial conditions, resulting in faster convergence compared to the standard approach.
△ Less
Submitted 20 August, 2023; v1 submitted 24 January, 2022;
originally announced January 2022.
-
Exploring Differential Geometry in Neural Implicits
Authors:
Tiago Novello,
Guilherme Schardong,
Luiz Schirmer,
Vinicius da Silva,
Helio Lopes,
Luiz Velho
Abstract:
We introduce a neural implicit framework that exploits the differentiable properties of neural networks and the discrete geometry of point-sampled surfaces to approximate them as the level sets of neural implicit functions.
To train a neural implicit function, we propose a loss functional that approximates a signed distance function, and allows terms with high-order derivatives, such as the alig…
▽ More
We introduce a neural implicit framework that exploits the differentiable properties of neural networks and the discrete geometry of point-sampled surfaces to approximate them as the level sets of neural implicit functions.
To train a neural implicit function, we propose a loss functional that approximates a signed distance function, and allows terms with high-order derivatives, such as the alignment between the principal directions of curvature, to learn more geometric details. During training, we consider a non-uniform sampling strategy based on the curvatures of the point-sampled surface to prioritize points with more geometric details. This sampling implies faster learning while preserving geometric accuracy when compared with previous approaches.
We also use the analytical derivatives of a neural implicit function to estimate the differential measures of the underlying point-sampled surface.
△ Less
Submitted 20 August, 2022; v1 submitted 23 January, 2022;
originally announced January 2022.
-
Neural Implicit Map** via Nested Neighborhoods
Authors:
Vinícius da Silva,
Tiago Novello,
Guilherme Schardong,
Luiz Schirmer,
Hélio Lopes,
Luiz Velho
Abstract:
We introduce a novel approach for rendering static and dynamic 3D neural signed distance functions (SDF) in real-time. We rely on nested neighborhoods of zero-level sets of neural SDFs, and map**s between them. This framework supports animations and achieves real-time performance without the use of spatial data-structures. It consists of three uncoupled algorithms representing the rendering step…
▽ More
We introduce a novel approach for rendering static and dynamic 3D neural signed distance functions (SDF) in real-time. We rely on nested neighborhoods of zero-level sets of neural SDFs, and map**s between them. This framework supports animations and achieves real-time performance without the use of spatial data-structures. It consists of three uncoupled algorithms representing the rendering steps. The multiscale sphere tracing focuses on minimizing iteration time by using coarse approximations on earlier iterations. The neural normal map** transfers details from a fine neural SDF to a surface nested on a neighborhood of its zero-level set. It is smooth and it does not depend on surface parametrizations. As a result, it can be used to fetch smooth normals for discrete surfaces such as meshes and to skip later iterations when sphere tracing level sets. Finally, we propose an algorithm for analytic normal calculation for MLPs and describe ways to obtain sequences of neural SDFs to use with the algorithms.
△ Less
Submitted 6 December, 2022; v1 submitted 22 January, 2022;
originally announced January 2022.
-
Uncertainty quantification through Monte Carlo method in a cloud computing setting
Authors:
A. Cunha Jr,
R. Nasser,
R. Sampaio,
H. Lopes,
K. Breitman
Abstract:
The Monte Carlo (MC) method is the most common technique used for uncertainty quantification, due to its simplicity and good statistical results. However, its computational cost is extremely high, and, in many cases, prohibitive. Fortunately, the MC algorithm is easily parallelizable, which allows its use in simulations where the computation of a single realization is very costly. This work presen…
▽ More
The Monte Carlo (MC) method is the most common technique used for uncertainty quantification, due to its simplicity and good statistical results. However, its computational cost is extremely high, and, in many cases, prohibitive. Fortunately, the MC algorithm is easily parallelizable, which allows its use in simulations where the computation of a single realization is very costly. This work presents a methodology for the parallelization of the MC method, in the context of cloud computing. This strategy is based on the MapReduce paradigm, and allows an efficient distribution of tasks in the cloud. This methodology is illustrated on a problem of structural dynamics that is subject to uncertainties. The results show that the technique is capable of producing good results concerning statistical moments of low order. It is shown that even a simple problem may require many realizations for convergence of histograms, which makes the cloud computing strategy very attractive (due to its high scalability capacity and low-cost). Additionally, the results regarding the time of processing and storage space usage allow one to qualify this new methodology as a solution for simulations that require a number of MC realizations beyond the standard.
△ Less
Submitted 20 May, 2021;
originally announced May 2021.
-
Finite Variation Sensitivity Analysis for Discrete Topology Optimization of Continuum Structures
Authors:
Daniel Candeloro Cunha,
Breno Vincenzo de Almeida,
Heitor Nigro Lopes,
Renato Pavanello
Abstract:
This paper proposes two novel approaches to perform more suitable sensitivity analyses for discrete topology optimization methods. To properly support them, we introduce a more formal description of the Bi-directional Evolutionary Structural Optimization (BESO) method, in which the sensitivity analysis is based on finite variations of the objective function. The proposed approaches are compared to…
▽ More
This paper proposes two novel approaches to perform more suitable sensitivity analyses for discrete topology optimization methods. To properly support them, we introduce a more formal description of the Bi-directional Evolutionary Structural Optimization (BESO) method, in which the sensitivity analysis is based on finite variations of the objective function. The proposed approaches are compared to a naive strategy; to the conventional strategy, referred to as First-Order Continuous Interpolation (FOCI) approach; and to a strategy previously developed by other researchers, referred to as High-Order Continuous Interpolation (HOCI) approach. The novel Woodbury approach provides exact sensitivity values and is a better alternative to HOCI. Although HOCI and Woodbury approaches may be computationally prohibitive, they provide useful expressions for a better understanding of the problem. The novel Conjugate Gradient Method (CGM) approach provides sensitivity values with arbitrary precision and is computationally viable for a small number of steps. The CGM approach is a better alternative to FOCI since, for appropriate initial conditions, it is always more accurate than the conventional strategy. The standard compliance minimization problem with volume constraint is considered to illustrate the methodology. Numerical examples are presented together with a broad discussion about BESO-type methods.
△ Less
Submitted 17 May, 2021; v1 submitted 7 April, 2021;
originally announced April 2021.
-
Real-time rendering of complex fractals
Authors:
Vinícius da Silva,
Tiago Novello,
Hélio Lopes,
Luiz Velho
Abstract:
This chapter describes how to use intersection and closest-hit shaders to implement real-time visualizations of complex fractals using distance functions. The Mandelbulb and Julia Sets are used as examples.
This chapter describes how to use intersection and closest-hit shaders to implement real-time visualizations of complex fractals using distance functions. The Mandelbulb and Julia Sets are used as examples.
△ Less
Submitted 2 February, 2021;
originally announced February 2021.
-
Proceduray -- A light-weight engine for procedural primitive ray tracing
Authors:
Vinícius da Silva,
Tiago Novello,
Hélio Lopes,
Luiz Velho
Abstract:
We introduce Proceduray, an engine for real-time ray tracing of procedural geometry. Its motivation is the current lack of mid-level abstraction tools for scenes with primitives involving intersection shaders. Those scenes impose strict engine design choices since they need flexibility in the shader table setup. Proceduray aims at providing a fair tradeoff between that flexibility and productivity…
▽ More
We introduce Proceduray, an engine for real-time ray tracing of procedural geometry. Its motivation is the current lack of mid-level abstraction tools for scenes with primitives involving intersection shaders. Those scenes impose strict engine design choices since they need flexibility in the shader table setup. Proceduray aims at providing a fair tradeoff between that flexibility and productivity. It also aims to be didactic. Shader table behavior can be confusing because parameters for indexing come from different parts of a system, involving both host and device code. This is different in essence from ray tracing triangle meshes (which must use a built-in intersection shader for all objects) or rendering with the traditional graphics or compute pipelines. Additionals goals of the project include fomenting deeper discussions about DirectX RayTracing (DXR) host code and providing a good starting point for developers trying to deal with procedural geometry using DXR.
△ Less
Submitted 21 May, 2021; v1 submitted 18 December, 2020;
originally announced December 2020.
-
Second layer data governance for permissioned blockchains: the privacy management challenge
Authors:
Paulo Henrique Alves,
Isabella Z. Frajhof,
Fernando A. Correia,
Clarisse de Souza,
Helio Lopes
Abstract:
Data privacy is a trending topic in the internet era. Given such importance, many challenges emerged in order to collect, manage, process, and publish data. In this sense, personal data have got attention, and many regulations emerged, such as GDPR in the European Union and LGPD in Brazil. This regulation model aims to protect users' data from misusage and leakage and allow users to request an exp…
▽ More
Data privacy is a trending topic in the internet era. Given such importance, many challenges emerged in order to collect, manage, process, and publish data. In this sense, personal data have got attention, and many regulations emerged, such as GDPR in the European Union and LGPD in Brazil. This regulation model aims to protect users' data from misusage and leakage and allow users to request an explanation from companies when needed. In pandemic situations, such as the COVID-19 and Ebola outbreak, the action related to sharing health data between different organizations is/ was crucial to develop a significant movement to avoid the massive infection and decrease the number of deaths. However, the data subject, i.e., the users, should have the right to request the purpose of data use, anonymization, and data deletion. In this sense, permissioned blockchain technology emerges to empower users to get their rights providing data ownership, transparency, and security through an immutable, unified, and distributed database ruled by smart contracts. The governance model discussed in blockchain applications is usually regarding the first layer governance, i.e., public and permissioned models. However, this discussion is too superficial, and they do not cover compliance with the data regulations. Therefore, in order to organize the relationship between data owners and the stakeholders, i.e., companies and governmental entities, we developed a second layer data governance model for permissioned blockchains based on the Governance Analytical Framework principles applied in pandemic situations preserving the users' privacy and their duties. From the law perspective, we based our model on the UE GDPR in regard to data privacy concerns.
△ Less
Submitted 22 October, 2020;
originally announced October 2020.
-
A softwarized perspective of the 5G networks
Authors:
Kleber Vieira Cardoso,
Cristiano Bonato Both,
Lúcio Rene Prade,
Ciro J. A. Macedo,
Victor Hugo L. Lopes
Abstract:
The main goal of this article is to present the fundamental theoretical concepts for the tutorial presented in IEEE NetSoft 2020. The article explores the use of software in the 5G system composed of the Radio Access Network (RAN) and the core components, following the standards defined by 3GPP, particularly the Release 15. The article provides a brief overview of mobile cellular networks, includi…
▽ More
The main goal of this article is to present the fundamental theoretical concepts for the tutorial presented in IEEE NetSoft 2020. The article explores the use of software in the 5G system composed of the Radio Access Network (RAN) and the core components, following the standards defined by 3GPP, particularly the Release 15. The article provides a brief overview of mobile cellular networks, including basic concepts, operations, and evolution through the called `generations' of mobile networks. From a software perspective, RAN is presented in the context of 4G and 5G networks, which includes the virtualization and disaggregation concepts. A significant part of the article is dedicated to 5G networks and beyond, focusing on core, i.e., considering the Service-Based Architecture (SBA), due to its relevance and totally softwarized approach. Finally, the article briefly describes the demonstrations presented in IEEE NetSoft 2020, providing the link for the repository that has all material employed in the tutorial.
△ Less
Submitted 24 August, 2020; v1 submitted 18 June, 2020;
originally announced June 2020.
-
Brazilian Lyrics-Based Music Genre Classification Using a BLSTM Network
Authors:
Raul de Araújo Lima,
Rômulo César Costa de Sousa,
Simone Diniz Junqueira Barbosa,
Hélio Cortês Vieira Lopes
Abstract:
Organize songs, albums, and artists in groups with shared similarity could be done with the help of genre labels. In this paper, we present a novel approach for automatic classifying musical genre in Brazilian music using only the song lyrics. This kind of classification remains a challenge in the field of Natural Language Processing. We construct a dataset of 138,368 Brazilian song lyrics distrib…
▽ More
Organize songs, albums, and artists in groups with shared similarity could be done with the help of genre labels. In this paper, we present a novel approach for automatic classifying musical genre in Brazilian music using only the song lyrics. This kind of classification remains a challenge in the field of Natural Language Processing. We construct a dataset of 138,368 Brazilian song lyrics distributed in 14 genres. We apply SVM, Random Forest and a Bidirectional Long Short-Term Memory (BLSTM) network combined with different word embeddings techniques to address this classification task. Our experiments show that the BLSTM method outperforms the other models with an F1-score average of $0.48$. Some genres like "gospel", "funk-carioca" and "sertanejo", which obtained 0.89, 0.70 and 0.69 of F1-score, respectively, can be defined as the most distinct and easy to classify in the Brazilian musical genres context.
△ Less
Submitted 6 March, 2020;
originally announced March 2020.
-
Como Mensurar a Importância, Influência e a Relevância de Usuários do Twitter? Uma análise da interação dos candidatos à presidência do Brasil nas eleições de 2018
Authors:
Ademir Cristiano Gabardo,
Leandro Takeshi Hattori,
Brenda Cinthya Solari Berno,
Matheus Gutoski,
Wagner Rodrigues Ulian Agostinho,
Heitor Silverio Lopes
Abstract:
In the contemporary world, a significant number of people use social networking services for a variety of purposes, including, but not limited to, communicating, exchanging messages and searching for information. A popular social network in the political arena is Twitter, a microblogging service for posting messages of up to 280 characters, called "tweets," where influential politicians from vario…
▽ More
In the contemporary world, a significant number of people use social networking services for a variety of purposes, including, but not limited to, communicating, exchanging messages and searching for information. A popular social network in the political arena is Twitter, a microblogging service for posting messages of up to 280 characters, called "tweets," where influential politicians from various countries often use this medium to spread ideas and make public statements. In this work, an analysis was made of the connections of candidates for the presidency of the Republic of Brazil in the year 2018. Using the analysis of complex networks to measure influence and relevance, a metric was established able to quantify the importance of users in the network. As part of the analysis, a Memory Algorithm was used to detect communities, groups of strongly connected vertices (tweets) evidencing grou**s of users.
△ Less
Submitted 21 February, 2019;
originally announced February 2019.
-
A Study on the Perception of Researchers about the Application of Agile Software Development Methods in Research
Authors:
Nelson Marcelo Romero Aquino,
Adolfo Gustavo Serra Seca Neto,
Heitor Silvério Lopes
Abstract:
Papers on Agile Software Development methods are often focused on their applicability in commercial projects or organizations. There are no current studies that we know about addressing the application of these methods in research projects. The objective of this work is to describe the perception of researchers on the application of agile software development practices and principles for research…
▽ More
Papers on Agile Software Development methods are often focused on their applicability in commercial projects or organizations. There are no current studies that we know about addressing the application of these methods in research projects. The objective of this work is to describe the perception of researchers on the application of agile software development practices and principles for research projects. A study was conducted by constructing and applying a questionnaire to Brazilian researchers of different affiliations, formation and research areas in order to obtain information about their knowledge and openness to follow agile software development principles and practices.
△ Less
Submitted 5 December, 2017;
originally announced December 2017.