Showing 1–7 of 7 results for author: Naveed, H

Model-driven Engineering for Machine Learning Components: A Systematic Literature Review

Authors: Hira Naveed, Chetan Arora, Hourieh Khalajzadeh, John Grundy, Omar Haggag

Abstract: Context: Machine Learning (ML) has become widely adopted as a component in many modern software applications. Due to the large volumes of data available, organizations want to increasingly leverage their data to extract meaningful insights and enhance business profitability. ML components enable predictive capabilities, anomaly detection, recommendation, accurate image and text processing, and inf… ▽ More Context: Machine Learning (ML) has become widely adopted as a component in many modern software applications. Due to the large volumes of data available, organizations want to increasingly leverage their data to extract meaningful insights and enhance business profitability. ML components enable predictive capabilities, anomaly detection, recommendation, accurate image and text processing, and informed decision-making. However, develo** systems with ML components is not trivial; it requires time, effort, knowledge, and expertise in ML, data processing, and software engineering. There have been several studies on the use of model-driven engineering (MDE) techniques to address these challenges when develo** traditional software and cyber-physical systems. Recently, there has been a growing interest in applying MDE for systems with ML components. Objective: The goal of this study is to further explore the promising intersection of MDE with ML (MDE4ML) through a systematic literature review (SLR). Through this SLR, we wanted to analyze existing studies, including their motivations, MDE solutions, evaluation techniques, key benefits and limitations. Results: We analyzed selected studies with respect to several areas of interest and identified the following: 1) the key motivations behind using MDE4ML; 2) a variety of MDE solutions applied, such as modeling languages, model transformations, tool support, targeted ML aspects, contributions and more; 3) the evaluation techniques and metrics used; and 4) the limitations and directions for future work. We also discuss the gaps in existing literature and provide recommendations for future research. Conclusion: This SLR highlights current trends, gaps and future research directions in the field of MDE4ML, benefiting both researchers and practitioners △ Less

Submitted 1 November, 2023; originally announced November 2023.

Runtime Monitoring of Human-centric Requirements in Machine Learning Components: A Model-driven Engineering Approach

Authors: Hira Naveed

Abstract: As machine learning (ML) components become increasingly integrated into software systems, the emphasis on the ethical or responsible aspects of their use has grown significantly. This includes building ML-based systems that adhere to human-centric requirements, such as fairness, privacy, explainability, well-being, transparency and human values. Meeting these human-centric requirements is not only… ▽ More As machine learning (ML) components become increasingly integrated into software systems, the emphasis on the ethical or responsible aspects of their use has grown significantly. This includes building ML-based systems that adhere to human-centric requirements, such as fairness, privacy, explainability, well-being, transparency and human values. Meeting these human-centric requirements is not only essential for maintaining public trust but also a key factor determining the success of ML-based systems. However, as these requirements are dynamic in nature and continually evolve, pre-deployment monitoring of these models often proves insufficient to establish and sustain trust in ML components. Runtime monitoring approaches for ML are potentially valuable solutions to this problem. Existing state-of-the-art techniques often fall short as they seldom consider more than one human-centric requirement, typically focusing on fairness, safety, and trust. The technical expertise and effort required to set up a monitoring system are also challenging. In my PhD research, I propose a novel approach for the runtime monitoring of multiple human-centric requirements. This approach leverages model-driven engineering to more comprehensively monitor ML components. This doctoral symposium paper outlines the motivation for my PhD work, a potential solution, progress so far and future plans. △ Less

Submitted 9 October, 2023; originally announced October 2023.

A Comprehensive Overview of Large Language Models

Authors: Humza Naveed, Asad Ullah Khan, Shi Qiu, Muhammad Saqib, Saeed Anwar, Muhammad Usman, Naveed Akhtar, Nick Barnes, Ajmal Mian

Abstract: Large Language Models (LLMs) have recently demonstrated remarkable capabilities in natural language processing tasks and beyond. This success of LLMs has led to a large influx of research contributions in this direction. These works encompass diverse topics such as architectural innovations, better training strategies, context length improvements, fine-tuning, multi-modal LLMs, robotics, datasets,… ▽ More Large Language Models (LLMs) have recently demonstrated remarkable capabilities in natural language processing tasks and beyond. This success of LLMs has led to a large influx of research contributions in this direction. These works encompass diverse topics such as architectural innovations, better training strategies, context length improvements, fine-tuning, multi-modal LLMs, robotics, datasets, benchmarking, efficiency, and more. With the rapid development of techniques and regular breakthroughs in LLM research, it has become considerably challenging to perceive the bigger picture of the advances in this direction. Considering the rapidly emerging plethora of literature on LLMs, it is imperative that the research community is able to benefit from a concise yet comprehensive overview of the recent developments in this field. This article provides an overview of the existing literature on a broad range of LLM-related concepts. Our self-contained comprehensive overview of LLMs discusses relevant background concepts along with covering the advanced topics at the frontier of research in LLMs. This review article is intended to not only provide a systematic survey but also a quick comprehensive reference for the researchers and practitioners to draw insights from extensive informative summaries of the existing works to advance the LLM research. △ Less

Submitted 9 April, 2024; v1 submitted 12 July, 2023; originally announced July 2023.

Parameter-free version of Adaptive Gradient Methods for Strongly-Convex Functions

Authors: Deepak Gouda, Hassan Naveed, Salil Kamath

Abstract: The optimal learning rate for adaptive gradient methods applied to λ-strongly convex functions relies on the parameters λ and learning rate η. In this paper, we adapt a universal algorithm along the lines of Metagrad, to get rid of this dependence on λ and η. The main idea is to concurrently run multiple experts and combine their predictions to a master algorithm. This master enjoys O(d log T) reg… ▽ More The optimal learning rate for adaptive gradient methods applied to λ-strongly convex functions relies on the parameters λ and learning rate η. In this paper, we adapt a universal algorithm along the lines of Metagrad, to get rid of this dependence on λ and η. The main idea is to concurrently run multiple experts and combine their predictions to a master algorithm. This master enjoys O(d log T) regret bounds. △ Less

Submitted 14 July, 2023; v1 submitted 11 June, 2023; originally announced June 2023.

Biomedical image analysis competitions: The state of current participation practice

Authors: Matthias Eisenmann, Annika Reinke, Vivienn Weru, Minu Dietlinde Tizabi, Fabian Isensee, Tim J. Adler, Patrick Godau, Veronika Cheplygina, Michal Kozubek, Sharib Ali, Anubha Gupta, Jan Kybic, Alison Noble, Carlos Ortiz de Solórzano, Samiksha Pachade, Caroline Petitjean, Daniel Sage, Donglai Wei, Elizabeth Wilden, Deepak Alapatt, Vincent Andrearczyk, Ujjwal Baid, Spyridon Bakas, Niranjan Balu, Sophia Bano , et al. (331 additional authors not shown)

Abstract: The number of international benchmarking competitions is steadily increasing in various fields of machine learning (ML) research and practice. So far, however, little is known about the common practice as well as bottlenecks faced by the community in tackling the research questions posed. To shed light on the status quo of algorithm development in the specific field of biomedical imaging analysis,… ▽ More The number of international benchmarking competitions is steadily increasing in various fields of machine learning (ML) research and practice. So far, however, little is known about the common practice as well as bottlenecks faced by the community in tackling the research questions posed. To shed light on the status quo of algorithm development in the specific field of biomedical imaging analysis, we designed an international survey that was issued to all participants of challenges conducted in conjunction with the IEEE ISBI 2021 and MICCAI 2021 conferences (80 competitions in total). The survey covered participants' expertise and working environments, their chosen strategies, as well as algorithm characteristics. A median of 72% challenge participants took part in the survey. According to our results, knowledge exchange was the primary incentive (70%) for participation, while the reception of prize money played only a minor role (16%). While a median of 80 working hours was spent on method development, a large portion of participants stated that they did not have enough time for method development (32%). 25% perceived the infrastructure to be a bottleneck. Overall, 94% of all solutions were deep learning-based. Of these, 84% were based on standard architectures. 43% of the respondents reported that the data samples (e.g., images) were too large to be processed at once. This was most commonly addressed by patch-based training (69%), downsampling (37%), and solving 3D analysis tasks as a series of 2D tasks. K-fold cross-validation on the training set was performed by only 37% of the participants and only 50% of the participants performed ensembling based on multiple identical models (61%) or heterogeneous models (39%). 48% of the respondents applied postprocessing steps. △ Less

Submitted 12 September, 2023; v1 submitted 16 December, 2022; originally announced December 2022.

Survey: Image Mixing and Deleting for Data Augmentation

Authors: Humza Naveed, Saeed Anwar, Munawar Hayat, Kashif Javed, Ajmal Mian

Abstract: Neural networks are prone to overfitting and memorizing data patterns. To avoid over-fitting and enhance their generalization and performance, various methods have been suggested in the literature, including dropout, regularization, label smoothing, etc. One such method is augmentation which introduces different types of corruption in the data to prevent the model from overfitting and to memorize… ▽ More Neural networks are prone to overfitting and memorizing data patterns. To avoid over-fitting and enhance their generalization and performance, various methods have been suggested in the literature, including dropout, regularization, label smoothing, etc. One such method is augmentation which introduces different types of corruption in the data to prevent the model from overfitting and to memorize patterns present in the data. A sub-area of data augmentation is image mixing and deleting. This specific type of augmentation either deletes image regions or mixes two images to hide or make particular characteristics of images confusing for the network, forcing it to emphasize the overall structure of the object in an image. Models trained with this approach have proven to perform and generalize well compared to those trained without image mixing or deleting. An added benefit that comes with this method of training is robustness against image corruption. Due to its low computational cost and recent success, researchers have proposed many image mixing and deleting techniques. We furnish an in-depth survey of image mixing and deleting techniques and provide categorization via their most distinguishing features. We initiate our discussion with some fundamental relevant concepts. Next, we present essentials, such as each category's strengths and limitations, describing their working mechanism, basic formulations, and applications. We also discuss the general challenges and recommend possible future research directions for image mixing and deleting data augmentation techniques. Datasets and codes for evaluation are publicly available here. △ Less

Submitted 6 February, 2023; v1 submitted 13 June, 2021; originally announced June 2021.

OpTorch: Optimized deep learning architectures for resource limited environments

Authors: Salman Ahmed, Hammad Naveed

Abstract: Deep learning algorithms have made many breakthroughs and have various applications in real life. Computational resources become a bottleneck as the data and complexity of the deep learning pipeline increases. In this paper, we propose optimized deep learning pipelines in multiple aspects of training including time and memory. OpTorch is a machine learning library designed to overcome weaknesses i… ▽ More Deep learning algorithms have made many breakthroughs and have various applications in real life. Computational resources become a bottleneck as the data and complexity of the deep learning pipeline increases. In this paper, we propose optimized deep learning pipelines in multiple aspects of training including time and memory. OpTorch is a machine learning library designed to overcome weaknesses in existing implementations of neural network training. OpTorch provides features to train complex neural networks with limited computational resources. OpTorch achieved the same accuracy as existing libraries on Cifar-10 and Cifar-100 datasets while reducing memory usage to approximately 50%. We also explore the effect of weights on total memory usage in deep learning pipelines. In our experiments, parallel encoding-decoding along with sequential checkpoints results in much improved memory and time usage while kee** the accuracy similar to existing pipelines. OpTorch python package is available at available at https://github.com/cbrl-nuces/optorch △ Less

Submitted 4 May, 2021; v1 submitted 2 May, 2021; originally announced May 2021.