-
Detection of Seismic Infrasonic Elephant Rumbles Using Spectrogram-Based Machine Learning
Authors:
A. M. J. V. Costa,
C. S. Pallikkonda,
H. H. R. Hiroshan,
G. R. U. Y. Gamlath,
S. R. Munasinghe,
C. U. S. Edussooriya
Abstract:
This paper presents an effective method of identifying elephant rumbles in infrasonic seismic signals. The design and implementation of electronic circuitry to amplify, filter, and digitize the seismic signals captured through geophones are presented. A collection of seismic infrasonic elephant rumbles was collected at a free-ranging area of an elephant orphanage in Sri Lanka. The seismic rumbles…
▽ More
This paper presents an effective method of identifying elephant rumbles in infrasonic seismic signals. The design and implementation of electronic circuitry to amplify, filter, and digitize the seismic signals captured through geophones are presented. A collection of seismic infrasonic elephant rumbles was collected at a free-ranging area of an elephant orphanage in Sri Lanka. The seismic rumbles were converted to spectrograms, and several methods were used for spectral feature extraction. Using LasyPredict, the features extracted using different methods were fed into their corresponding machine-learning algorithms to train them for automatic seismic rumble identification. It was found that the Mel frequency cepstral coefficient (MFCC) together with the Ridge classifier machine learning algorithm produced the best performance in identifying seismic elephant rumbles. A novel method for denoising the spectrum that leads to enhanced accuracy in identifying seismic rumbles is also presented.
△ Less
Submitted 5 December, 2023;
originally announced December 2023.
-
Vision-Based Incoming Traffic Estimator Using Deep Neural Network on General Purpose Embedded Hardware
Authors:
K. G. Zoysa,
S. R. Munasinghe
Abstract:
Traffic management is a serious problem in many cities around the world. Even the suburban areas are now experiencing regular traffic congestion. Inappropriate traffic control wastes fuel, time, and the productivity of nations. Though traffic signals are used to improve traffic flow, they often cause problems due to inappropriate or obsolete timing that does not tally with the actual traffic inten…
▽ More
Traffic management is a serious problem in many cities around the world. Even the suburban areas are now experiencing regular traffic congestion. Inappropriate traffic control wastes fuel, time, and the productivity of nations. Though traffic signals are used to improve traffic flow, they often cause problems due to inappropriate or obsolete timing that does not tally with the actual traffic intensity at the intersection. Traffic intensity determination based on statistical methods only gives the average intensity expected at any given time. However, to control traffic accurately, it is required to know the real-time traffic intensity. In this research, image processing and machine learning have been used to estimate actual traffic intensity in real time. General-purpose electronic hardware has been used for in-situ image processing based on the edge-detection method. A deep neural network (DNN) was trained to infer traffic intensity in each image in real time. The trained DNN estimated traffic intensity accurately in 90% of the real-time images during road tests. The electronic system was implemented on a Raspberry Pi single-board computer; hence, it is cost-effective for large-scale deployment.
△ Less
Submitted 28 October, 2023;
originally announced November 2023.
-
PG-Video-LLaVA: Pixel Grounding Large Video-Language Models
Authors:
Shehan Munasinghe,
Rusiru Thushara,
Muhammad Maaz,
Hanoona Abdul Rasheed,
Salman Khan,
Mubarak Shah,
Fahad Khan
Abstract:
Extending image-based Large Multimodal Models (LMMs) to videos is challenging due to the inherent complexity of video data. The recent approaches extending image-based LMMs to videos either lack the grounding capabilities (e.g., VideoChat, Video-ChatGPT, Video-LLaMA) or do not utilize the audio-signals for better video understanding (e.g., Video-ChatGPT). Addressing these gaps, we propose PG-Video…
▽ More
Extending image-based Large Multimodal Models (LMMs) to videos is challenging due to the inherent complexity of video data. The recent approaches extending image-based LMMs to videos either lack the grounding capabilities (e.g., VideoChat, Video-ChatGPT, Video-LLaMA) or do not utilize the audio-signals for better video understanding (e.g., Video-ChatGPT). Addressing these gaps, we propose PG-Video-LLaVA, the first LMM with pixel-level grounding capability, integrating audio cues by transcribing them into text to enrich video-context understanding. Our framework uses an off-the-shelf tracker and a novel grounding module, enabling it to spatially localize objects in videos following user instructions. We evaluate PG-Video-LLaVA using video-based generative and question-answering benchmarks and introduce new benchmarks specifically designed to measure prompt-based object grounding performance in videos. Further, we propose the use of Vicuna over GPT-3.5, as utilized in Video-ChatGPT, for video-based conversation benchmarking, ensuring reproducibility of results which is a concern with the proprietary nature of GPT-3.5. Our framework builds on SoTA image-based LLaVA model and extends its advantages to the video domain, delivering promising gains on video-based conversation and grounding tasks. Project Page: https://github.com/mbzuai-oryx/Video-LLaVA
△ Less
Submitted 13 December, 2023; v1 submitted 22 November, 2023;
originally announced November 2023.
-
Accurate Crop Spraying with RTK and Machine Learning on an Autonomous Field Robot
Authors:
W. M. T. D. Wijesundara,
T. D. Wanigathunga,
M. N. C. Waas,
R. T. Hithanadura,
S. R. Munasinghe
Abstract:
The agriculture sector requires a lot of labor and resources. Hence, farmers are constantly being pressed for technology and automation to be cost-effective. In this context, autonomous robots can play a very important role in carrying out agricultural tasks such as spraying, sowing, inspection, and even harvesting. This paper presents one such autonomous robot that is able to identify plants and…
▽ More
The agriculture sector requires a lot of labor and resources. Hence, farmers are constantly being pressed for technology and automation to be cost-effective. In this context, autonomous robots can play a very important role in carrying out agricultural tasks such as spraying, sowing, inspection, and even harvesting. This paper presents one such autonomous robot that is able to identify plants and spray agro-chemicals precisely. The robot uses machine vision technologies to find plants and RTK-GPS technology to navigate the robot along a predetermined path. The experiments were conducted in a field of potted plants in which successful results have been obtained.
△ Less
Submitted 25 October, 2023;
originally announced October 2023.
-
A Cognitive Network Architecture for Vehicle-to-Network (V2N) Communications over Smart Meters for URLLC
Authors:
Shoaib Ahmed,
Sayonto Khan,
Kumudu S. Munasinghe,
Md. Farhad Hossain
Abstract:
With the rapid advancement of smart city infrastructure, vehicle-to-network (V2N) communication has emerged as a crucial technology to enable intelligent transportation systems (ITS). The investigation of new methods to improve V2N communications is sparked by the growing need for high-speed and dependable communications in vehicular networks. To achieve ultra-reliable low latency communication (U…
▽ More
With the rapid advancement of smart city infrastructure, vehicle-to-network (V2N) communication has emerged as a crucial technology to enable intelligent transportation systems (ITS). The investigation of new methods to improve V2N communications is sparked by the growing need for high-speed and dependable communications in vehicular networks. To achieve ultra-reliable low latency communication (URLLC) for V2N scenarios, we propose a smart meter (SM)-based cognitive network (CN) architecture for V2N communications. Our scheme makes use of SMs' available underutilized time resources to let them serve as distributed access points (APs) for V2N communications to increase reliability and decrease latency. We propose and investigate two algorithms for efficiently associating vehicles with the appropriate SMs. Extensive simulations are carried out for comprehensive performance evaluation of our proposed architecture and algorithms under diverse system scenarios. Performance is investigated with particular emphasis on communication latency and reliability, which are also compared with the conventional base station (BS)-based V2N architecture for further validation. The results highlight the value of incorporating SMs into the current infrastructure and open the door for future ITSs to utilize more effective and dependable V2N communications.
△ Less
Submitted 26 August, 2023;
originally announced August 2023.
-
Class-Aware Attention for Multimodal Trajectory Prediction
Authors:
Bimsara Pathiraja,
Shehan Munasinghe,
Malshan Ranawella,
Maleesha De Silva,
Ranga Rodrigo,
Peshala Jayasekara
Abstract:
Predicting the possible future trajectories of the surrounding dynamic agents is an essential requirement in autonomous driving. These trajectories mainly depend on the surrounding static environment, as well as the past movements of those dynamic agents. Furthermore, the multimodal nature of agent intentions makes the trajectory prediction problem more challenging. All of the existing models cons…
▽ More
Predicting the possible future trajectories of the surrounding dynamic agents is an essential requirement in autonomous driving. These trajectories mainly depend on the surrounding static environment, as well as the past movements of those dynamic agents. Furthermore, the multimodal nature of agent intentions makes the trajectory prediction problem more challenging. All of the existing models consider the target agent as well as the surrounding agents similarly, without considering the variation of physical properties. In this paper, we present a novel deep-learning based framework for multimodal trajectory prediction in autonomous driving, which considers the physical properties of the target and surrounding vehicles such as the object class and their physical dimensions through a weighted attention module, that improves the accuracy of the predictions. Our model has achieved the highest results in the nuScenes trajectory prediction benchmark, out of the models which use rasterized maps to input environment information. Furthermore, our model is able to run in real-time, achieving a high inference rate of over 300 FPS.
△ Less
Submitted 31 August, 2022;
originally announced September 2022.
-
Text-to-Face Generation with StyleGAN2
Authors:
D. M. A. Ayanthi,
Sarasi Munasinghe
Abstract:
Synthesizing images from text descriptions has become an active research area with the advent of Generative Adversarial Networks. The main goal here is to generate photo-realistic images that are aligned with the input descriptions. Text-to-Face generation (T2F) is a sub-domain of Text-to-Image generation (T2I) that is more challenging due to the complexity and variation of facial attributes. It h…
▽ More
Synthesizing images from text descriptions has become an active research area with the advent of Generative Adversarial Networks. The main goal here is to generate photo-realistic images that are aligned with the input descriptions. Text-to-Face generation (T2F) is a sub-domain of Text-to-Image generation (T2I) that is more challenging due to the complexity and variation of facial attributes. It has a number of applications mainly in the domain of public safety. Even though several models are available for T2F, there is still the need to improve the image quality and the semantic alignment. In this research, we propose a novel framework, to generate facial images that are well-aligned with the input descriptions. Our framework utilizes the high-resolution face generator, StyleGAN2, and explores the possibility of using it in T2F. Here, we embed text in the input latent space of StyleGAN2 using BERT embeddings and oversee the generation of facial images using text descriptions. We trained our framework on attribute-based descriptions to generate images of 1024x1024 in resolution. The images generated exhibit a 57% similarity to the ground truth images, with a face semantic distance of 0.92, outperforming state-of-the-artwork. The generated images have a FID score of 118.097 and the experimental results show that our model generates promising images.
△ Less
Submitted 25 May, 2022;
originally announced May 2022.
-
A Novel Transfer Learning-Based Approach for Screening Pre-existing Heart Diseases Using Synchronized ECG Signals and Heart Sounds
Authors:
Ramith Hettiarachchi,
Udith Haputhanthri,
Kithmini Herath,
Hasindu Kariyawasam,
Shehan Munasinghe,
Kithmin Wickramasinghe,
Duminda Samarasinghe,
Anjula De Silva,
Chamira U. S. Edussooriya
Abstract:
Diagnosing pre-existing heart diseases early in life is important as it helps prevent complications such as pulmonary hypertension, heart rhythm problems, blood clots, heart failure and sudden cardiac arrest. To identify such diseases, phonocardiogram (PCG) and electrocardiogram (ECG) waveforms convey important information. Therefore, effectively using these two modalities of data has the potentia…
▽ More
Diagnosing pre-existing heart diseases early in life is important as it helps prevent complications such as pulmonary hypertension, heart rhythm problems, blood clots, heart failure and sudden cardiac arrest. To identify such diseases, phonocardiogram (PCG) and electrocardiogram (ECG) waveforms convey important information. Therefore, effectively using these two modalities of data has the potential to improve the disease screening process. We evaluate this hypothesis on a subset of the PhysioNet Challenge 2016 Dataset which contains simultaneously acquired PCG and ECG recordings. Our novel Dual-Convolutional Neural Network based approach uses transfer learning to tackle the problem of having limited amounts of simultaneous PCG and ECG data that is publicly available, while having the potential to adapt to larger datasets. In addition, we introduce two main evaluation frameworks named record-wise and sample-wise evaluation which leads to a rich performance evaluation for the transfer learning approach. Comparisons with methods which used single or dual modality data show that our method can lead to better performance. Furthermore, our results show that individually collected ECG or PCG waveforms are able to provide transferable features which could effectively help to make use of a limited number of synchronized PCG and ECG waveforms and still achieve significant classification performance.
△ Less
Submitted 14 February, 2021; v1 submitted 2 February, 2021;
originally announced February 2021.
