-
Characterising the Creative Process in Humans and Large Language Models
Authors:
Surabhi S. Nath,
Peter Dayan,
Claire Stevenson
Abstract:
Large language models appear quite creative, often performing on par with the average human on creative tasks. However, research on LLM creativity has focused solely on \textit{products}, with little attention on the creative \textit{process}. Process analyses of human creativity often require hand-coded categories or exploit response times, which do not apply to LLMs. We provide an automated meth…
▽ More
Large language models appear quite creative, often performing on par with the average human on creative tasks. However, research on LLM creativity has focused solely on \textit{products}, with little attention on the creative \textit{process}. Process analyses of human creativity often require hand-coded categories or exploit response times, which do not apply to LLMs. We provide an automated method to characterise how humans and LLMs explore semantic spaces on the Alternate Uses Task, and contrast with behaviour in a Verbal Fluency Task. We use sentence embeddings to identify response categories and compute semantic similarities, which we use to generate jump profiles. Our results corroborate earlier work in humans reporting both persistent (deep search in few semantic spaces) and flexible (broad search across multiple semantic spaces) pathways to creativity, where both pathways lead to similar creativity scores. LLMs were found to be biased towards either persistent or flexible paths, that varied across tasks. Though LLMs as a population match human profiles, their relationship with creativity is different, where the more flexible models score higher on creativity. Our dataset and scripts are available on \href{https://github.com/surabhisnath/Creative_Process}{GitHub}.
△ Less
Submitted 5 June, 2024; v1 submitted 1 May, 2024;
originally announced May 2024.
-
Simplicity in Complexity : Explaining Visual Complexity using Deep Segmentation Models
Authors:
Tingke Shen,
Surabhi S Nath,
Aenne Brielmann,
Peter Dayan
Abstract:
The complexity of visual stimuli plays an important role in many cognitive phenomena, including attention, engagement, memorability, time perception and aesthetic evaluation. Despite its importance, complexity is poorly understood and ironically, previous models of image complexity have been quite complex. There have been many attempts to find handcrafted features that explain complexity, but thes…
▽ More
The complexity of visual stimuli plays an important role in many cognitive phenomena, including attention, engagement, memorability, time perception and aesthetic evaluation. Despite its importance, complexity is poorly understood and ironically, previous models of image complexity have been quite complex. There have been many attempts to find handcrafted features that explain complexity, but these features are usually dataset specific, and hence fail to generalise. On the other hand, more recent work has employed deep neural networks to predict complexity, but these models remain difficult to interpret, and do not guide a theoretical understanding of the problem. Here we propose to model complexity using segment-based representations of images. We use state-of-the-art segmentation models, SAM and FC-CLIP, to quantify the number of segments at multiple granularities, and the number of classes in an image respectively. We find that complexity is well-explained by a simple linear model with these two features across six diverse image-sets of naturalistic scene and art images. This suggests that the complexity of images can be surprisingly simple.
△ Less
Submitted 6 May, 2024; v1 submitted 5 March, 2024;
originally announced March 2024.
-
Deep Learning Based Cyberbullying Detection in Bangla Language
Authors:
Sristy Shidul Nath,
Razuan Karim,
Mahdi H. Miraz
Abstract:
The Internet is currently the largest platform for global communication including expressions of opinions, reviews, contents, images, videos and so forth. Moreover, social media has now become a very broad and highly engaging platform due to its immense popularity and swift adoption trend. Increased social networking, however, also has detrimental impacts on the society leading to a range of unwan…
▽ More
The Internet is currently the largest platform for global communication including expressions of opinions, reviews, contents, images, videos and so forth. Moreover, social media has now become a very broad and highly engaging platform due to its immense popularity and swift adoption trend. Increased social networking, however, also has detrimental impacts on the society leading to a range of unwanted phenomena, such as online assault, intimidation, digital bullying, criminality and trolling. Hence, cyberbullying has become a pervasive and worrying problem that poses considerable psychological and emotional harm to the people, particularly amongst the teens and the young adults. In order to lessen its negative effects and provide victims with prompt support, a great deal of research to identify cyberbullying instances at various online platforms is emerging. In comparison to other languages, Bangla (also known as Bengali) has fewer research studies in this domain. This study demonstrates a deep learning strategy for identifying cyberbullying in Bengali, using a dataset of 12282 versatile comments from multiple social media sites. In this study, a two-layer bidirectional long short-term memory (Bi-LSTM) model has been built to identify cyberbullying, using a variety of optimisers as well as 5-fold cross validation. To evaluate the functionality and efficacy of the proposed system, rigorous assessment and validation procedures have been employed throughout the project. The results of this study reveals that the proposed model's accuracy, using momentum-based stochastic gradient descent (SGD) optimiser, is 94.46%. It also reflects a higher accuracy of 95.08% and a F1 score of 95.23% using Adam optimiser as well as a better accuracy of 94.31% in 5-fold cross validation.
△ Less
Submitted 6 January, 2024;
originally announced January 2024.
-
Exploring Relevant Artifacts of Release Notes: The Practitioners' Perspective
Authors:
Sristy Sumana Nath,
Banani Roy
Abstract:
A software release note is one of the essential documents in the software development life cycle. The software release contains a set of information, e.g., bug fixes and security fixes. Release notes are used in different phases, e.g., requirement engineering, software testing and release management. Different types of practitioners (e.g., project managers and clients) get benefited from the relea…
▽ More
A software release note is one of the essential documents in the software development life cycle. The software release contains a set of information, e.g., bug fixes and security fixes. Release notes are used in different phases, e.g., requirement engineering, software testing and release management. Different types of practitioners (e.g., project managers and clients) get benefited from the release notes to understand the overview of the latest release. As a result, several studies have been done about release notes production and usage in practice. However, two significant problems (e.g., duplication and inconsistency in release notes contents) exist in producing well-written & well-structured release notes and organizing appropriate information regarding different targeted users' needs. For that reason, practitioners face difficulties in writing and reading the release notes using existing tools. To mitigate these problems, we execute two different studies in our paper. First, we execute an exploratory study by analyzing 3,347 release notes of 21 GitHub repositories to understand the documented contents of the release notes. As a result, we find relevant key artifacts, e.g., issues (29%), pull-requests (32%), commits (19%), and common vulnerabilities and exposures (CVE) issues (6%) in the release note contents. Second, we conduct a survey study with 32 professionals to understand the key information that is included in release notes regarding users' roles. For example, project managers are more interested in learning about new features than less critical bug fixes. Our study can guide future research directions to help practitioners produce the release notes with relevant content and improve the documentation quality.
△ Less
Submitted 11 April, 2022;
originally announced April 2022.
-
Towards Automatically Generating Release Notes using Extractive Summarization Technique
Authors:
Sristy Sumana Nath,
Banani Roy
Abstract:
Release notes are admitted as an essential document by practitioners. They contain the summary of the source code changes for the software releases, such as issue fixes, added new features, and performance improvements. Manually producing release notes is a time-consuming and challenging task. For that reason, sometimes developers neglect to write release notes. For example, we collect data from G…
▽ More
Release notes are admitted as an essential document by practitioners. They contain the summary of the source code changes for the software releases, such as issue fixes, added new features, and performance improvements. Manually producing release notes is a time-consuming and challenging task. For that reason, sometimes developers neglect to write release notes. For example, we collect data from GitHub with over 1,900 releases, among them 37% of the release notes are empty. We propose an automatic generate release notes approach based on the commit messages and merge pull-request (PR) titles to mitigate this problem. We implement one of the popular extractive text summarization techniques, i.e., the TextRank algorithm. However, accurate keyword extraction is a vital issue in text processing. The keyword matching and topic extraction process of the TextRank algorithm ignores the semantic similarity among texts. To improve the keyword extraction method, we integrate the GloVe word embedding technique with TextRank. We develop a dataset with 1,213 release notes (after null filtering) and evaluate the generated release notes through the ROUGE metric and human evaluation. We also compare the performance of our technique with another popular extractive algorithm, latent semantic analysis (LSA). Our evaluation results show that the improved TextRank method outperforms LSA.
△ Less
Submitted 11 April, 2022;
originally announced April 2022.
-
Semantic Slicing of Architectural Change Commits: Towards Semantic Design Review
Authors:
Amit Kumar Mondal,
Chanchal K. Roy,
Kevin A. Schneider,
Banani Roy,
Sristy Sumana Nath
Abstract:
Software architectural changes involve more than one module or component and are complex to analyze compared to local code changes. Development teams aiming to review architectural aspects (design) of a change commit consider many essential scenarios such as access rules and restrictions on usage of program entities across modules. Moreover, design review is essential when proper architectural for…
▽ More
Software architectural changes involve more than one module or component and are complex to analyze compared to local code changes. Development teams aiming to review architectural aspects (design) of a change commit consider many essential scenarios such as access rules and restrictions on usage of program entities across modules. Moreover, design review is essential when proper architectural formulations are paramount for develo** and deploying a system. Untangling architectural changes, recovering semantic design, and producing design notes are the crucial tasks of the design review process. To support these tasks, we construct a lightweight tool [4] that can detect and decompose semantic slices of a commit containing architectural instances. A semantic slice consists of a description of relational information of involved modules, their classes, methods and connected modules in a change instance, which is easy to understand to a reviewer. We extract various directory and naming structures (DANS) properties from the source code for develo** our tool. Utilizing the DANS properties, our tool first detects architectural change instances based on our defined metric and then decomposes the slices (based on string processing). Our preliminary investigation with ten open-source projects (developed in Java and Kotlin) reveals that the DANS properties produce highly reliable precision and recall (93-100%) for detecting and generating architectural slices. Our proposed tool will serve as the preliminary approach for the semantic design recovery and design summary generation for the project releases.
△ Less
Submitted 1 September, 2021;
originally announced September 2021.
-
Designing for Recommending Intermediate States in A Scientific Workflow Management System
Authors:
Debasish Chakroborti,
Banani Roy,
Sristy Sumana Nath
Abstract:
To process a large amount of data sequentially and systematically, proper management of workflow components (i.e., modules, data, configurations, associations among ports and links) in a Scientific Workflow Management System (SWfMS) is inevitable. Managing data with provenance in a SWfMS to support reusability of workflows, modules, and data is not a simple task. Handling such components is even m…
▽ More
To process a large amount of data sequentially and systematically, proper management of workflow components (i.e., modules, data, configurations, associations among ports and links) in a Scientific Workflow Management System (SWfMS) is inevitable. Managing data with provenance in a SWfMS to support reusability of workflows, modules, and data is not a simple task. Handling such components is even more burdensome for frequently assembled and executed complex workflows for investigating large datasets with different technologies (i.e., various learning algorithms or models). However, a great many studies propose various techniques and technologies for managing and recommending services in a SWfMS, but only a very few studies consider the management of data in a SWfMS for efficient storing and facilitating workflow executions. Furthermore, there is no study to inquire about the effectiveness and efficiency of such data management in a SWfMS from a user perspective. In this paper, we present and evaluate a GUI version of such a novel approach of intermediate data management with two use cases (Plant Phenoty** and Bioinformatics). The technique we call GUI-RISPTS (Recommending Intermediate States from Pipelines Considering Tool-States) can facilitate executions of workflows with processed data (i.e., intermediate outcomes of modules in a workflow) and can thus reduce the computational time of some modules in a SWfMS. We integrated GUI-RISPTS with an existing workflow management system called SciWorCS. In SciWorCS, we present an interface that users use for selecting the recommendation of intermediate states (i.e., modules' outcomes). We investigated GUI-RISP's effectiveness from users' perspectives along with measuring its overhead in terms of storage and efficiency in workflow execution.
△ Less
Submitted 9 October, 2020;
originally announced October 2020.
-
Hear Her Fear: Data Sonification for Sensitizing Society on Crime Against Women in India
Authors:
Surabhi S Nath
Abstract:
Data sonification is a means of representing data through sound and has been utilized in a variety of applications. Crime against women has been a rising concern in India. We explore the potential of data sonification to provide an immersive engagement with sensitive data on crime against women in Indian states. The data for nine crime categories covering thirty-five Indian states over a period of…
▽ More
Data sonification is a means of representing data through sound and has been utilized in a variety of applications. Crime against women has been a rising concern in India. We explore the potential of data sonification to provide an immersive engagement with sensitive data on crime against women in Indian states. The data for nine crime categories covering thirty-five Indian states over a period of twelve years is acquired from National records. Sonification techniques of parameter map** and auditory icons are adopted: sound parameters such as frequencies, amplitudes and timbres are incorporated to represent the crime data, and audio sounds of women screams are employed as auditory icons to emphasize the traumatic experience. Higher crime rates are assigned higher frequencies, harsher scream textures and larger amplitudes. A user-friendly interface is developed with multiple options for sequential and comparative data sonification. Through the interface, a user can evaluate and compare the extent of crime against women in different states, years or crime categories. Sound spatialization is used to immerse the listener in the sound and further intensify the sonification experience. To assess and validate effectiveness, a user study on twenty participants is conducted with feedback obtained through questionnaires. The responses indicate that the participants could comprehend trends in the data easily and found the data sonification experience impactful. Sonification may therefore prove to be a valuable tool for data representation in fields related to social and human studies.
△ Less
Submitted 18 October, 2020; v1 submitted 29 September, 2020;
originally announced September 2020.
-
It's LeVAsa not LevioSA! Latent Encodings for Valence-Arousal Structure Alignment
Authors:
Surabhi S. Nath,
Vishaal Udandarao,
Jainendra Shukla
Abstract:
In recent years, great strides have been made in the field of affective computing. Several models have been developed to represent and quantify emotions. Two popular ones include (i) categorical models which represent emotions as discrete labels, and (ii) dimensional models which represent emotions in a Valence-Arousal (VA) circumplex domain. However, there is no standard for annotation map** be…
▽ More
In recent years, great strides have been made in the field of affective computing. Several models have been developed to represent and quantify emotions. Two popular ones include (i) categorical models which represent emotions as discrete labels, and (ii) dimensional models which represent emotions in a Valence-Arousal (VA) circumplex domain. However, there is no standard for annotation map** between the two labelling methods. We build a novel algorithm for map** categorical and dimensional model labels using annotation transfer across affective facial image datasets. Further, we utilize the transferred annotations to learn rich and interpretable data representations using a variational autoencoder (VAE). We present "LeVAsa", a VAE model that learns implicit structure by aligning the latent space with the VA space. We evaluate the efficacy of LeVAsa by comparing performance with the Vanilla VAE using quantitative and qualitative analysis on two benchmark affective image datasets. Our results reveal that LeVAsa achieves high latent-circumplex alignment which leads to improved downstream categorical emotion prediction. The work also demonstrates the trade-off between degree of alignment and quality of reconstructions.
△ Less
Submitted 30 November, 2020; v1 submitted 20 July, 2020;
originally announced July 2020.
-
An Exploratory Study to Find Motives Behind Cross-platform Forks from Software Heritage Dataset
Authors:
Avijit Bhattacharjee,
Sristy Sumana Nath,
Shurui Zhou,
Debasish Chakroborti,
Banani Roy,
Chanchal K. Roy,
Kevin Schneider
Abstract:
The fork-based development mechanism provides the flexibility and the unified processes for software teams to collaborate easily in a distributed setting without too much coordination overhead.Currently, multiple social coding platforms support fork-based development, such as GitHub, GitLab, and Bitbucket. Although these different platforms virtually share the same features, they have different em…
▽ More
The fork-based development mechanism provides the flexibility and the unified processes for software teams to collaborate easily in a distributed setting without too much coordination overhead.Currently, multiple social coding platforms support fork-based development, such as GitHub, GitLab, and Bitbucket. Although these different platforms virtually share the same features, they have different emphasis. As GitHub is the most popular platform and the corresponding data is publicly available, most of the current studies are focusing on GitHub hosted projects. However, we observed anecdote evidences that people are confused about choosing among these platforms, and some projects are migrating from one platform to another, and the reasons behind these activities remain unknown.With the advances of Software Heritage Graph Dataset (SWHGD),we have the opportunity to investigate the forking activities across platforms. In this paper, we conduct an exploratory study on 10popular open-source projects to identify cross-platform forks and investigate the motivation behind. Preliminary result shows that cross-platform forks do exist. For the 10 subject systems in this study, we found 81,357 forks in total among which 179 forks are on GitLab. Based on our qualitative analysis, we found that most of the cross-platform forks that we identified are mirrors of the repositories on another platform, but we still find cases that were created due to preference of using certain functionalities (e.g. Continuous Integration (CI)) supported by different platforms. This study lays the foundation of future research directions, such as understanding the differences between platforms and supporting cross-platform collaboration.
△ Less
Submitted 17 March, 2020;
originally announced March 2020.
-
Universal EEG Encoder for Learning Diverse Intelligent Tasks
Authors:
Baani Leen Kaur Jolly,
Palash Aggrawal,
Surabhi S Nath,
Viresh Gupta,
Manraj Singh Grover,
Rajiv Ratn Shah
Abstract:
Brain Computer Interfaces (BCI) have become very popular with Electroencephalography (EEG) being one of the most commonly used signal acquisition techniques. A major challenge in BCI studies is the individualistic analysis required for each task. Thus, task-specific feature extraction and classification are performed, which fails to generalize to other tasks with similar time-series EEG input data…
▽ More
Brain Computer Interfaces (BCI) have become very popular with Electroencephalography (EEG) being one of the most commonly used signal acquisition techniques. A major challenge in BCI studies is the individualistic analysis required for each task. Thus, task-specific feature extraction and classification are performed, which fails to generalize to other tasks with similar time-series EEG input data. To this end, we design a GRU-based universal deep encoding architecture to extract meaningful features from publicly available datasets for five diverse EEG-based classification tasks. Our network can generate task and format-independent data representation and outperform the state of the art EEGNet architecture on most experiments. We also compare our results with CNN-based, and Autoencoder networks, in turn performing local, spatial, temporal and unsupervised analysis on the data.
△ Less
Submitted 26 November, 2019;
originally announced November 2019.
