Showing 1–50 of 65 results for author: Rodrigo, R

LiverUSRecon: Automatic 3D Reconstruction and Volumetry of the Liver with a Few Partial Ultrasound Scans

Authors: Kaushalya Sivayogaraj, Sahan T. Guruge, Udari Liyanage, Jeevani Udupihille, Saroj Jayasinghe, Gerard Fernando, Ranga Rodrigo, M. Rukshani Liyanaarachchi

Abstract: 3D reconstruction of the liver for volumetry is important for qualitative analysis and disease diagnosis. Liver volumetry using ultrasound (US) scans, although advantageous due to less acquisition time and safety, is challenging due to the inherent noisiness in US scans, blurry boundaries, and partial liver visibility. We address these challenges by using the segmentation masks of a few incomplete… ▽ More 3D reconstruction of the liver for volumetry is important for qualitative analysis and disease diagnosis. Liver volumetry using ultrasound (US) scans, although advantageous due to less acquisition time and safety, is challenging due to the inherent noisiness in US scans, blurry boundaries, and partial liver visibility. We address these challenges by using the segmentation masks of a few incomplete sagittal-plane US scans of the liver in conjunction with a statistical shape model (SSM) built using a set of CT scans of the liver. We compute the shape parameters needed to warp this canonical SSM to fit the US scans through a parametric regression network. The resulting 3D liver reconstruction is accurate and leads to automatic liver volume calculation. We evaluate the accuracy of the estimated liver volumes with respect to CT segmentation volumes using RMSE. Our volume computation is statistically much closer to the volume estimated using CT scans than the volume computed using Childs' method by radiologists: p-value of 0.094 (>0.05) says that there is no significant difference between CT segmentation volumes and ours in contrast to Childs' method. We validate our method using investigations (ablation studies) on the US image resolution, the number of CT scans used for SSM, the number of principal components, and the number of input US scans. To the best of our knowledge, this is the first automatic liver volumetry system using a few incomplete US scans given a set of CT scans of livers for SSM. △ Less

Submitted 28 June, 2024; v1 submitted 27 June, 2024; originally announced June 2024.

Comments: 10 pages, Accepted to MICCAI 2024

An efficient singlet-triplet spin qubit to fiber interface assisted by a photonic crystal cavity

Authors: Kui Wu, Sebastian Kindel, Thomas Descamps, Tobias Hangleiter, Jan Christoph Müller, Rebecca Rodrigo, Florian Merget, Hendrik Bluhm, Jeremy Witzens

Abstract: We introduce a novel optical interface between a singlet-triplet spin qubit and a photonic qubit which would offer new prospects for future quantum communication applications. The interface is based on a 220 nm thick GaAs/Al-GaAs heterostructure membrane and features a gate-defined singlet-triplet qubit, a gate-defined optically active quantum dot, a photonic crystal cavity and a bot-tom gold refl… ▽ More We introduce a novel optical interface between a singlet-triplet spin qubit and a photonic qubit which would offer new prospects for future quantum communication applications. The interface is based on a 220 nm thick GaAs/Al-GaAs heterostructure membrane and features a gate-defined singlet-triplet qubit, a gate-defined optically active quantum dot, a photonic crystal cavity and a bot-tom gold reflector. All essential components can be lithographically defined and deterministically fabricated, which greatly increases the scalability of on-chip in-tegration. According to our FDTD simulations, the interface provides an overall coupling efficiency of 28.7% into a free space Gaussian beam, assuming an SiO2 interlayer filling the space between the reflector and the membrane. The performance can be further increased to 48.5% by undercutting this SiO2 interlayer below the photonic crystal. △ Less

Submitted 20 June, 2024; originally announced June 2024.

Journal ref: The 25th European Conference on Integrated Optics, Springer Proceedings in Physics 402, pp. 365-372, 2024

Vessel Re-identification and Activity Detection in Thermal Domain for Maritime Surveillance

Authors: Yasod Ginige, Ransika Gunasekara, Darsha Hewavitharana, Manjula Ariyarathne, Ranga Rodrigo, Peshala Jayasekara

Abstract: Maritime surveillance is vital to mitigate illegal activities such as drug smuggling, illegal fishing, and human trafficking. Vision-based maritime surveillance is challenging mainly due to visibility issues at night, which results in failures in re-identifying vessels and detecting suspicious activities. In this paper, we introduce a thermal, vision-based approach for maritime surveillance with o… ▽ More Maritime surveillance is vital to mitigate illegal activities such as drug smuggling, illegal fishing, and human trafficking. Vision-based maritime surveillance is challenging mainly due to visibility issues at night, which results in failures in re-identifying vessels and detecting suspicious activities. In this paper, we introduce a thermal, vision-based approach for maritime surveillance with object tracking, vessel re-identification, and suspicious activity detection capabilities. For vessel re-identification, we propose a novel viewpoint-independent algorithm which compares features of the sides of the vessel separately (separate side-spaces) leveraging shape information in the absence of color features. We propose techniques to adapt tracking and activity detection algorithms for the thermal domain and train them using a thermal dataset we created. This dataset will be the first publicly available benchmark dataset for thermal maritime surveillance. Our system is capable of re-identifying vessels with an 81.8% Top1 score and identifying suspicious activities with a 72.4\% frame mAP score; a new benchmark for each task in the thermal domain. △ Less

Submitted 12 June, 2024; originally announced June 2024.

A Novel Algorithm for Digital Lithological Map**-Case Studies in Sri Lanka's Mineral Exploration

Authors: R. M. L. S. Ramanayake, D. C. Dammage, I. Z. M. Zumri, K. A. R. S. Rodrigo, A. A. P. Perera, D. Fernando, G. M. R. I. Godaliyadda, H. M. V. R. Herath, M. P. B. Ekanayake, A. Senaratne, Fadi Kizel

Abstract: Conventional manual lithological map** (MLM) through field surveys are resource-extensive and time-consuming. Digital lithological map** (DLM), harnessing remotely sensed spectral imaging techniques, provides an effective strategy to streamline target locations for MLM or an efficient alternative to MLM. DLM relies on laboratory-generated generic end-member signatures of minerals for spectral… ▽ More Conventional manual lithological map** (MLM) through field surveys are resource-extensive and time-consuming. Digital lithological map** (DLM), harnessing remotely sensed spectral imaging techniques, provides an effective strategy to streamline target locations for MLM or an efficient alternative to MLM. DLM relies on laboratory-generated generic end-member signatures of minerals for spectral analysis. Thus, the accuracy of DLM may be limited due to the presence of site-specific impurities. A strategy, based on a hybrid machine-learning and signal-processing algorithm, is proposed in this paper to tackle this problem of site-specific impurities. In addition, a soil pixel alignment strategy is proposed here to visualize the relative purity of the target minerals. The proposed methodologies are validated via case studies for map** of Limestone deposits in Jaffna, Ilmenite deposits in Pulmoddai and Mannar, and Montmorillonite deposits in Murunkan, Sri Lanka. The results of satellite-based spectral imaging analysis were corroborated with X-ray diffraction (XRD) and Magnetic Separation (MS) analysis of soil samples collected from those sites via field surveys. There exists a good correspondence between the relative availability of the minerals with the XRD and MS results. In particular, correlation coefficients of 0.8115 and 0.9853 were found for the sites in Pulmoddai and Jaffna respectively. △ Less

Submitted 5 May, 2024; v1 submitted 31 March, 2024; originally announced April 2024.

Forensic Video Analytic Software

Authors: Anton Jeran Ratnarajah, Sahani Goonetilleke, Dumindu Tissera, Kapilan Balagopalan, Ranga Rodrigo

Abstract: Law enforcement officials heavily depend on Forensic Video Analytic (FVA) Software in their evidence extraction process. However present-day FVA software are complex, time consuming, equipment dependent and expensive. Develo** countries struggle to gain access to this gateway to a secure haven. The term forensic pertains the application of scientific methods to the investigation of crime through… ▽ More Law enforcement officials heavily depend on Forensic Video Analytic (FVA) Software in their evidence extraction process. However present-day FVA software are complex, time consuming, equipment dependent and expensive. Develo** countries struggle to gain access to this gateway to a secure haven. The term forensic pertains the application of scientific methods to the investigation of crime through post-processing, whereas surveillance is the close monitoring of real-time feeds. The principle objective of this Final Year Project was to develop an efficient and effective FVA Software, addressing the shortcomings through a stringent and systematic review of scholarly research papers, online databases and legal documentation. The scope spans multiple object detection, multiple object tracking, anomaly detection, activity recognition, tampering detection, general and specific image enhancement and video synopsis. Methods employed include many machine learning techniques, GPU acceleration and efficient, integrated architecture development both for real-time and postprocessing. For this CNN, GMM, multithreading and OpenCV C++ coding were used. The implications of the proposed methodology would rapidly speed up the FVA process especially through the novel video synopsis research arena. This project has resulted in three research outcomes Moving Object Based Collision Free Video Synopsis, Forensic and Surveillance Analytic Tool Architecture and Tampering Detection Inter-Frame Forgery. The results include forensic and surveillance panel outcomes with emphasis on video synopsis and Sri Lankan context. Principal conclusions include the optimization and efficient algorithm integration to overcome limitations in processing power, memory and compromise between real-time performance and accuracy. △ Less

Submitted 17 September, 2023; originally announced January 2024.

Comments: The Forensic Video Analytic Software demo video is available https://www.youtube.com/watch?v=vsZlYKQxSkE

Moving Object Based Collision-Free Video Synopsis

Authors: Anton Jeran Ratnarajah, Sahani Goonetilleke, Dumindu Tissera, Kapilan Balagopalan, Ranga Rodrigo

Abstract: Video synopsis, summarizing a video to generate a shorter video by exploiting the spatial and temporal redundancies, is important for surveillance and archiving. Existing trajectory-based video synopsis algorithms will not able to work in real time, because of the complexity due to the number of object tubes that need to be included in the complex energy minimization algorithm. We propose a real-t… ▽ More Video synopsis, summarizing a video to generate a shorter video by exploiting the spatial and temporal redundancies, is important for surveillance and archiving. Existing trajectory-based video synopsis algorithms will not able to work in real time, because of the complexity due to the number of object tubes that need to be included in the complex energy minimization algorithm. We propose a real-time algorithm by using a method that incrementally stitches each frame of the synopsis by extracting object frames from the user specified number of tubes in the buffer in contrast to global energy-minimization based systems. This also gives flexibility to the user to set the threshold of maximum number of objects in the synopsis video according his or her tracking ability and creates collision-free summarized videos which are visually pleasing. Experiments with six common test videos, indoors and outdoors with many moving objects, show that the proposed video synopsis algorithm produces better frame reduction rates than existing approaches. △ Less

Submitted 17 September, 2023; originally announced January 2024.

Comments: The summarized output videos are available at https://anton-jeran.github.io/M2SYN/

SATHUR: Self Augmenting Task Hallucinal Unified Representation for Generalized Class Incremental Learning

Authors: Sathursan Kanagarajah, Thanuja Ambegoda, Ranga Rodrigo

Abstract: Class Incremental Learning (CIL) is inspired by the human ability to learn new classes without forgetting previous ones. CIL becomes more challenging in real-world scenarios when the samples in each incremental step are imbalanced. This creates another branch of problem, called Generalized Class Incremental Learning (GCIL) where each incremental step is structured more realistically. Grow When Req… ▽ More Class Incremental Learning (CIL) is inspired by the human ability to learn new classes without forgetting previous ones. CIL becomes more challenging in real-world scenarios when the samples in each incremental step are imbalanced. This creates another branch of problem, called Generalized Class Incremental Learning (GCIL) where each incremental step is structured more realistically. Grow When Required (GWR) network, a type of Self-Organizing Map (SOM), dynamically create and remove nodes and edges for adaptive learning. GWR performs incremental learning from feature vectors extracted by a Convolutional Neural Network (CNN), which acts as a feature extractor. The inherent ability of GWR to form distinct clusters, each corresponding to a class in the feature vector space, regardless of the order of samples or class imbalances, is well suited to achieving GCIL. To enhance GWR's classification performance, a high-quality feature extractor is required. However, when the convolutional layers are adapted at each incremental step, the GWR nodes corresponding to prior knowledge are subject to near-invalidation. This work introduces the Self Augmenting Task Hallucinal Unified Representation (SATHUR), which re-initializes the GWR network at each incremental step, aligning it with the current feature extractor. Comprehensive experimental results demonstrate that our proposed method significantly outperforms other state-of-the-art GCIL methods on CIFAR-100 and CORe50 datasets. △ Less

Submitted 12 August, 2023; originally announced November 2023.

Comments: 8 pages, 5 figures, ICCVW 2023

Modeling of an efficient singlet-triplet spin qubit to photon interface assisted by a photonic crystal cavity

Authors: Kui Wu, Sebastian Kindel, Thomas Descamps, Tobias Hangleiter, Jan Christoph Müller, Rebecca Rodrigo, Florian Merget, Hendrik Bluhm, Jeremy Witzens

Abstract: Efficient interconnection between distant semiconductor spin qubits with the help of photonic qubits would offer exciting new prospects for future quantum communication applications. In this paper, we optimize the extraction efficiency of a novel interface between a singlet-triplet spin qubit and a photonic qubit. The interface is based on a 220 nm thick GaAs/AlGaAs heterostructure membrane and co… ▽ More Efficient interconnection between distant semiconductor spin qubits with the help of photonic qubits would offer exciting new prospects for future quantum communication applications. In this paper, we optimize the extraction efficiency of a novel interface between a singlet-triplet spin qubit and a photonic qubit. The interface is based on a 220 nm thick GaAs/AlGaAs heterostructure membrane and consists of a gate-defined double quantum dot (GDQD) supporting a singlet-triplet qubit, an optically active quantum dot (OAQD) consisting of a gate-defined exciton trap, a photonic crystal cavity providing in-plane optical confinement and efficient out-coupling to an ideal free space Gaussian beam while accommodating the gate wiring of the GDQD and OAQD, and a bottom gold reflector to recycle photons and increase the optical extraction efficiency. All essential components can be lithographically defined and deterministically fabricated on the GaAs/AlGaAs heterostructure membrane, which greatly increases the scalability of on-chip integration. According to our simulations, the interface provides an overall coupling efficiency of 28.7% into a free space Gaussian beam, assuming an SiO2 interlayer filling the space between the reflector and the membrane. The performance can be further increased by undercutting this SiO2 interlayer below the photonic crystal. In this case, the overall efficiency is calculated to be 48.5%. △ Less

Submitted 28 October, 2023; originally announced October 2023.

Contrastive Deep Encoding Enables Uncertainty-aware Machine-learning-assisted Histopathology

Authors: Nirhoshan Sivaroopan, Chamuditha Jayanga, Chalani Ekanayake, Hasindri Watawana, Jathurshan Pradeepkumar, Mithunjha Anandakumar, Ranga Rodrigo, Chamira U. S. Edussooriya, Dushan N. Wadduwage

Abstract: Deep neural network models can learn clinically relevant features from millions of histopathology images. However generating high-quality annotations to train such models for each hospital, each cancer type, and each diagnostic task is prohibitively laborious. On the other hand, terabytes of training data -- while lacking reliable annotations -- are readily available in the public domain in some c… ▽ More Deep neural network models can learn clinically relevant features from millions of histopathology images. However generating high-quality annotations to train such models for each hospital, each cancer type, and each diagnostic task is prohibitively laborious. On the other hand, terabytes of training data -- while lacking reliable annotations -- are readily available in the public domain in some cases. In this work, we explore how these large datasets can be consciously utilized to pre-train deep networks to encode informative representations. We then fine-tune our pre-trained models on a fraction of annotated training data to perform specific downstream tasks. We show that our approach can reach the state-of-the-art (SOTA) for patch-level classification with only 1-10% randomly selected annotations compared to other SOTA approaches. Moreover, we propose an uncertainty-aware loss function, to quantify the model confidence during inference. Quantified uncertainty helps experts select the best instances to label for further training. Our uncertainty-aware labeling reaches the SOTA with significantly fewer annotations compared to random labeling. Last, we demonstrate how our pre-trained encoders can surpass current SOTA for whole-slide image classification with weak supervision. Our work lays the foundation for data and task-agnostic pre-trained deep networks with quantified uncertainty. △ Less

Submitted 13 September, 2023; originally announced September 2023.

Comments: 18 pages, 8 figures

3DLatNav: Navigating Generative Latent Spaces for Semantic-Aware 3D Object Manipulation

Authors: Amaya Dharmasiri, Dinithi Dissanayake, Mohamed Afham, Isuru Dissanayake, Ranga Rodrigo, Kanchana Thilakarathna

Abstract: 3D generative models have been recently successful in generating realistic 3D objects in the form of point clouds. However, most models do not offer controllability to manipulate the shape semantics of component object parts without extensive semantic attribute labels or other reference point clouds. Moreover, beyond the ability to perform simple latent vector arithmetic or interpolations, there i… ▽ More 3D generative models have been recently successful in generating realistic 3D objects in the form of point clouds. However, most models do not offer controllability to manipulate the shape semantics of component object parts without extensive semantic attribute labels or other reference point clouds. Moreover, beyond the ability to perform simple latent vector arithmetic or interpolations, there is a lack of understanding of how part-level semantics of 3D shapes are encoded in their corresponding generative latent spaces. In this paper, we propose 3DLatNav; a novel approach to navigating pretrained generative latent spaces to enable controlled part-level semantic manipulation of 3D objects. First, we propose a part-level weakly-supervised shape semantics identification mechanism using latent representations of 3D shapes. Then, we transfer that knowledge to a pretrained 3D object generative latent space to unravel disentangled embeddings to represent different shape semantics of component parts of an object in the form of linear subspaces, despite the unavailability of part-level labels during the training. Finally, we utilize those identified subspaces to show that controllable 3D object part manipulation can be achieved by applying the proposed framework to any pretrained 3D generative model. With two novel quantitative metrics to evaluate the consistency and localization accuracy of part-level manipulations, we show that 3DLatNav outperforms existing unsupervised latent disentanglement methods in identifying latent directions that encode part-level shape semantics of 3D objects. With multiple ablation studies and testing on state-of-the-art generative models, we show that 3DLatNav can implement controlled part-level semantic manipulations on an input point cloud while preserving other features and the realistic nature of the object. △ Less

Submitted 17 November, 2022; originally announced November 2022.

Visual-Semantic Contrastive Alignment for Few-Shot Image Classification

Authors: Mohamed Afham, Ranga Rodrigo

Abstract: Few-Shot learning aims to train and optimize a model that can adapt to unseen visual classes with only a few labeled examples. The existing few-shot learning (FSL) methods, heavily rely only on visual data, thus fail to capture the semantic attributes to learn a more generalized version of the visual concept from very few examples. However, it is a known fact that human visual learning benefits im… ▽ More Few-Shot learning aims to train and optimize a model that can adapt to unseen visual classes with only a few labeled examples. The existing few-shot learning (FSL) methods, heavily rely only on visual data, thus fail to capture the semantic attributes to learn a more generalized version of the visual concept from very few examples. However, it is a known fact that human visual learning benefits immensely from inputs from multiple modalities such as vision, language, and audio. Inspired by the human learning nature of encapsulating the existing knowledge of a visual category which is in the form of language, we introduce a contrastive alignment mechanism for visual and semantic feature vectors to learn much more generalized visual concepts for few-shot learning. Our method simply adds an auxiliary contrastive learning objective which captures the contextual knowledge of a visual category from a strong textual encoder in addition to the existing training mechanism. Hence, the approach is more generalized and can be plugged into any existing FSL method. The pre-trained semantic feature extractor (learned from a large-scale text corpora) we use in our approach provides a strong contextual prior knowledge to assist FSL. The experimental results done in popular FSL datasets show that our approach is generic in nature and provides a strong boost to the existing FSL baselines. △ Less

Submitted 19 October, 2022; originally announced October 2022.

Comments: ECCV 2022 Workshop on Computer Vision in the Wild

Realistic, Animatable Human Reconstructions for Virtual Fit-On

Authors: Gayal Kuruppu, Bumuthu Dilshan, Shehan Samarasinghe, Nipuna Madhushan, Ranga Rodrigo

Abstract: We present an end-to-end virtual try-on pipeline, that can fit different clothes on a personalized 3-D human model, reconstructed using a single RGB image. Our main idea is to construct an animatable 3-D human model and try-on different clothes in a 3-D virtual environment. The existing frame by frame volumetric reconstruction of 3-D human models are highly resource-demanding and do not allow clot… ▽ More We present an end-to-end virtual try-on pipeline, that can fit different clothes on a personalized 3-D human model, reconstructed using a single RGB image. Our main idea is to construct an animatable 3-D human model and try-on different clothes in a 3-D virtual environment. The existing frame by frame volumetric reconstruction of 3-D human models are highly resource-demanding and do not allow clothes switching. Moreover, existing virtual fit-on systems also lack realism due to predominantly being 2-D or not using user's features in the reconstruction. These shortcomings are due to either the human body or clothing model being 2-D or not having the user's facial features in the dressed model. We solve these problems by manipulating a parametric representation of the 3-D human body model and stitching a head model reconstructed from the actual image. Fitting the 3-D clothing models on the parameterized human model is also adjustable to the body shape of the input image. Our reconstruction results, in comparison with recent existing work, are more visually-pleasing. △ Less

Submitted 16 October, 2022; originally announced October 2022.

Vision Transformer with Convolutional Encoder-Decoder for Hand Gesture Recognition using 24 GHz Doppler Radar

Authors: Kavinda Kehelella, Gayangana Leelarathne, Dhanuka Marasinghe, Nisal Kariyawasam, Viduneth Ariyarathna, Arjuna Madanayake, Ranga Rodrigo, Chamira U. S. Edussooriya

Abstract: Transformers combined with convolutional encoders have been recently used for hand gesture recognition (HGR) using micro-Doppler signatures. We propose a vision-transformer-based architecture for HGR with multi-antenna continuous-wave Doppler radar receivers. The proposed architecture consists of three modules: a convolutional encoderdecoder, an attention module with three transformer layers, and… ▽ More Transformers combined with convolutional encoders have been recently used for hand gesture recognition (HGR) using micro-Doppler signatures. We propose a vision-transformer-based architecture for HGR with multi-antenna continuous-wave Doppler radar receivers. The proposed architecture consists of three modules: a convolutional encoderdecoder, an attention module with three transformer layers, and a multi-layer perceptron. The novel convolutional decoder helps to feed patches with larger sizes to the attention module for improved feature extraction. Experimental results obtained with a dataset corresponding to a two-antenna continuous-wave Doppler radar receiver operating at 24 GHz (published by Skaria et al.) confirm that the proposed architecture achieves an accuracy of 98.3% which substantially surpasses the state-of-the-art on the used dataset. △ Less

Submitted 12 September, 2022; originally announced September 2022.

Comments: Accepted to be published in IEEE Sensors Letters, 4 pages

DualCam: A Novel Benchmark Dataset for Fine-grained Real-time Traffic Light Detection

Authors: Harindu Jayarathne, Tharindu Samarakoon, Hasara Koralege, Asitha Divisekara, Ranga Rodrigo, Peshala Jayasekara

Abstract: Traffic light detection is essential for self-driving cars to navigate safely in urban areas. Publicly available traffic light datasets are inadequate for the development of algorithms for detecting distant traffic lights that provide important navigation information. We introduce a novel benchmark traffic light dataset captured using a synchronized pair of narrow-angle and wide-angle cameras cove… ▽ More Traffic light detection is essential for self-driving cars to navigate safely in urban areas. Publicly available traffic light datasets are inadequate for the development of algorithms for detecting distant traffic lights that provide important navigation information. We introduce a novel benchmark traffic light dataset captured using a synchronized pair of narrow-angle and wide-angle cameras covering urban and semi-urban roads. We provide 1032 images for training and 813 synchronized image pairs for testing. Additionally, we provide synchronized video pairs for qualitative analysis. The dataset includes images of resolution 1920$\times$1080 covering 10 different classes. Furthermore, we propose a post-processing algorithm for combining outputs from the two cameras. Results show that our technique can strike a balance between speed and accuracy, compared to the conventional approach of using a single camera frame. △ Less

Submitted 3 September, 2022; originally announced September 2022.

Comments: 6 pages with 7 figures. The dataset is available at https://github.com/harinduravin/DualCam

Class-Aware Attention for Multimodal Trajectory Prediction

Authors: Bimsara Pathiraja, Shehan Munasinghe, Malshan Ranawella, Maleesha De Silva, Ranga Rodrigo, Peshala Jayasekara

Abstract: Predicting the possible future trajectories of the surrounding dynamic agents is an essential requirement in autonomous driving. These trajectories mainly depend on the surrounding static environment, as well as the past movements of those dynamic agents. Furthermore, the multimodal nature of agent intentions makes the trajectory prediction problem more challenging. All of the existing models cons… ▽ More Predicting the possible future trajectories of the surrounding dynamic agents is an essential requirement in autonomous driving. These trajectories mainly depend on the surrounding static environment, as well as the past movements of those dynamic agents. Furthermore, the multimodal nature of agent intentions makes the trajectory prediction problem more challenging. All of the existing models consider the target agent as well as the surrounding agents similarly, without considering the variation of physical properties. In this paper, we present a novel deep-learning based framework for multimodal trajectory prediction in autonomous driving, which considers the physical properties of the target and surrounding vehicles such as the object class and their physical dimensions through a weighted attention module, that improves the accuracy of the predictions. Our model has achieved the highest results in the nuScenes trajectory prediction benchmark, out of the models which use rasterized maps to input environment information. Furthermore, our model is able to run in real-time, achieving a high inference rate of over 300 FPS. △ Less

Submitted 31 August, 2022; originally announced September 2022.

Dynamic Template Initialization for Part-Aware Person Re-ID

Authors: Kalana Abeywardena, Shechem Sumanthiran, Sanoojan Baliah, Nadarasar Bahavan, Nalith Udugampola, Ajith Pasqual, Chamira Edussooriya, Ranga Rodrigo

Abstract: Many of the existing Person Re-identification (Re-ID) approaches depend on feature maps which are either partitioned to localize parts of a person or reduced to create a global representation. While part localization has shown significant success, it uses either naıve position-based partitions or static feature templates. These, however, hypothesize the pre-existence of the parts in a given image… ▽ More Many of the existing Person Re-identification (Re-ID) approaches depend on feature maps which are either partitioned to localize parts of a person or reduced to create a global representation. While part localization has shown significant success, it uses either naıve position-based partitions or static feature templates. These, however, hypothesize the pre-existence of the parts in a given image or their positions, ignoring the input image-specific information which limits their usability in challenging scenarios such as Re-ID with partial occlusions and partial probe images. In this paper, we introduce a spatial attention-based Dynamic Part Template Initialization module that dynamically generates part-templates using mid-level semantic features at the earlier layers of the backbone. Following a self-attention layer, human part-level features of the backbone are used to extract the templates of diverse human body parts using a simplified cross-attention scheme which will then be used to identify and collate representations of various human parts from semantically rich features, increasing the discriminative ability of the entire model. We further explore adaptive weighting of part descriptors to quantify the absence or occlusion of local attributes and suppress the contribution of the corresponding part descriptors to the matching criteria. Extensive experiments on holistic, occluded, and partial Re-ID task benchmarks demonstrate that our proposed architecture is able to achieve competitive performance. Codes will be included in the supplementary material and will be made publicly available. △ Less

Submitted 24 August, 2022; originally announced August 2022.

Comments: 11 pages, 3 figures

HPGNN: Using Hierarchical Graph Neural Networks for Outdoor Point Cloud Processing

Authors: Arulmolivarman Thieshanthan, Amashi Niwarthana, Pamuditha Somarathne, Tharindu Wickremasinghe, Ranga Rodrigo

Abstract: Inspired by recent improvements in point cloud processing for autonomous navigation, we focus on using hierarchical graph neural networks for processing and feature learning over large-scale outdoor LiDAR point clouds. We observe that existing GNN based methods fail to overcome challenges of scale and irregularity of points in outdoor datasets. Addressing the need to preserve structural details wh… ▽ More Inspired by recent improvements in point cloud processing for autonomous navigation, we focus on using hierarchical graph neural networks for processing and feature learning over large-scale outdoor LiDAR point clouds. We observe that existing GNN based methods fail to overcome challenges of scale and irregularity of points in outdoor datasets. Addressing the need to preserve structural details while learning over a larger volume efficiently, we propose Hierarchical Point Graph Neural Network (HPGNN). It learns node features at various levels of graph coarseness to extract information. This enables to learn over a large point cloud while retaining fine details that existing point-level graph networks struggle to achieve. Connections between multiple levels enable a point to learn features in multiple scales, in a few iterations. We design HPGNN as a purely GNN-based approach, so that it offers modular expandability as seen with other point-based and Graph network baselines. To illustrate the improved processing capability, we compare previous point based and GNN models for semantic segmentation with our HPGNN, achieving a significant improvement for GNNs (+36.7 mIoU) on the SemanticKITTI dataset. △ Less

Submitted 5 June, 2022; originally announced June 2022.

Comments: Accepted for ICPR 2022

Towards Real-time Traffic Sign and Traffic Light Detection on Embedded Systems

Authors: Oshada Jayasinghe, Sahan Hemachandra, Damith Anhettigama, Shenali Kariyawasam, Tharindu Wickremasinghe, Chalani Ekanayake, Ranga Rodrigo, Peshala Jayasekara

Abstract: Recent work done on traffic sign and traffic light detection focus on improving detection accuracy in complex scenarios, yet many fail to deliver real-time performance, specifically with limited computational resources. In this work, we propose a simple deep learning based end-to-end detection framework, which effectively tackles challenges inherent to traffic sign and traffic light detection such… ▽ More Recent work done on traffic sign and traffic light detection focus on improving detection accuracy in complex scenarios, yet many fail to deliver real-time performance, specifically with limited computational resources. In this work, we propose a simple deep learning based end-to-end detection framework, which effectively tackles challenges inherent to traffic sign and traffic light detection such as small size, large number of classes and complex road scenarios. We optimize the detection models using TensorRT and integrate with Robot Operating System to deploy on an Nvidia Jetson AGX Xavier as our embedded device. The overall system achieves a high inference speed of 63 frames per second, demonstrating the capability of our system to perform in real-time. Furthermore, we introduce CeyRo, which is the first ever large-scale traffic sign and traffic light detection dataset for the Sri Lankan context. Our dataset consists of 7984 total images with 10176 traffic sign and traffic light instances covering 70 traffic sign and 5 traffic light classes. The images have a high resolution of 1920 x 1080 and capture a wide range of challenging road scenarios with different weather and lighting conditions. Our work is publicly available at https://github.com/oshadajay/CeyRo. △ Less

Submitted 4 May, 2022; originally announced May 2022.

Comments: Accepted to 33rd IEEE Intelligent Vehicles (IV) Symposium 2022

CrossPoint: Self-Supervised Cross-Modal Contrastive Learning for 3D Point Cloud Understanding

Authors: Mohamed Afham, Isuru Dissanayake, Dinithi Dissanayake, Amaya Dharmasiri, Kanchana Thilakarathna, Ranga Rodrigo

Abstract: Manual annotation of large-scale point cloud dataset for varying tasks such as 3D object classification, segmentation and detection is often laborious owing to the irregular structure of point clouds. Self-supervised learning, which operates without any human labeling, is a promising approach to address this issue. We observe in the real world that humans are capable of map** the visual concepts… ▽ More Manual annotation of large-scale point cloud dataset for varying tasks such as 3D object classification, segmentation and detection is often laborious owing to the irregular structure of point clouds. Self-supervised learning, which operates without any human labeling, is a promising approach to address this issue. We observe in the real world that humans are capable of map** the visual concepts learnt from 2D images to understand the 3D world. Encouraged by this insight, we propose CrossPoint, a simple cross-modal contrastive learning approach to learn transferable 3D point cloud representations. It enables a 3D-2D correspondence of objects by maximizing agreement between point clouds and the corresponding rendered 2D image in the invariant space, while encouraging invariance to transformations in the point cloud modality. Our joint training objective combines the feature correspondences within and across modalities, thus ensembles a rich learning signal from both 3D point cloud and 2D image modalities in a self-supervised fashion. Experimental results show that our approach outperforms the previous unsupervised learning methods on a diverse range of downstream tasks including 3D object classification and segmentation. Further, the ablation studies validate the potency of our approach for a better point cloud understanding. Code and pretrained models are available at http://github.com/MohamedAfham/CrossPoint. △ Less

Submitted 24 March, 2022; v1 submitted 1 March, 2022; originally announced March 2022.

Comments: CVPR 2022

PointCaps: Raw Point Cloud Processing using Capsule Networks with Euclidean Distance Routing

Authors: Dishanika Denipitiyage, Vinoj Jayasundara, Ranga Rodrigo, Chamira U. S. Edussooriya

Abstract: Raw point cloud processing using capsule networks is widely adopted in classification, reconstruction, and segmentation due to its ability to preserve spatial agreement of the input data. However, most of the existing capsule based network approaches are computationally heavy and fail at representing the entire point cloud as a single capsule. We address these limitations in existing capsule netwo… ▽ More Raw point cloud processing using capsule networks is widely adopted in classification, reconstruction, and segmentation due to its ability to preserve spatial agreement of the input data. However, most of the existing capsule based network approaches are computationally heavy and fail at representing the entire point cloud as a single capsule. We address these limitations in existing capsule network based approaches by proposing PointCaps, a novel convolutional capsule architecture with parameter sharing. Along with PointCaps, we propose a novel Euclidean distance routing algorithm and a class-independent latent representation. The latent representation captures physically interpretable geometric parameters of the point cloud, with dynamic Euclidean routing, PointCaps well-represents the spatial (point-to-part) relationships of points. PointCaps has a significantly lower number of parameters and requires a significantly lower number of FLOPs while achieving better reconstruction with comparable classification and segmentation accuracy for raw point clouds compared to state-of-the-art capsule networks. △ Less

Submitted 20 August, 2022; v1 submitted 21 December, 2021; originally announced December 2021.

Comments: Accepted to be published in Journal of Visual Communication and Image Representation (Elsevier), 16 Pages, 4 Figures, 5 Tables

KORSAL: Key-point Detection based Online Real-Time Spatio-Temporal Action Localization

Authors: Kalana Abeywardena, Shechem Sumanthiran, Sakuna Jayasundara, Sachira Karunasena, Ranga Rodrigo, Peshala Jayasekara

Abstract: Real-time and online action localization in a video is a critical yet highly challenging problem. Accurate action localization requires the utilization of both temporal and spatial information. Recent attempts achieve this by using computationally intensive 3D CNN architectures or highly redundant two-stream architectures with optical flow, making them both unsuitable for real-time, online applica… ▽ More Real-time and online action localization in a video is a critical yet highly challenging problem. Accurate action localization requires the utilization of both temporal and spatial information. Recent attempts achieve this by using computationally intensive 3D CNN architectures or highly redundant two-stream architectures with optical flow, making them both unsuitable for real-time, online applications. To accomplish activity localization under highly challenging real-time constraints, we propose utilizing fast and efficient key-point based bounding box prediction to spatially localize actions. We then introduce a tube-linking algorithm that maintains the continuity of action tubes temporally in the presence of occlusions. Further, we eliminate the need for a two-stream architecture by combining temporal and spatial information into a cascaded input to a single network, allowing the network to learn from both types of information. Temporal information is efficiently extracted using a structural similarity index map as opposed to computationally intensive optical flow. Despite the simplicity of our approach, our lightweight end-to-end architecture achieves state-of-the-art frame-mAP of 74.7% on the challenging UCF101-24 dataset, demonstrating a performance gain of 6.4% over the previous best online methods. We also achieve state-of-the-art video-mAP results compared to both online and offline methods. Moreover, our model achieves a frame rate of 41.8 FPS, which is a 10.7% improvement over contemporary real-time methods. △ Less

Submitted 5 November, 2021; originally announced November 2021.

Comments: 7 pages

ACM Class: I.4.8

A spatio-temporal analogue of the Omori-Utsu law of aftershock sequences

Authors: Marianito R. Rodrigo

Abstract: A spatio-temporal version of the well-known Omori-Utsu law of aftershock sequences is proposed. This 'diffusive Omori-Utsu law' satisfies a nonlinear partial differential equation (PDE). A similarity reduction is obtained that reduces the PDE to an ordinary differential equation (ODE). A nonzero constant solution of this ODE leads to the usual Omori-Utsu law. An exact and explicit similarity solut… ▽ More A spatio-temporal version of the well-known Omori-Utsu law of aftershock sequences is proposed. This 'diffusive Omori-Utsu law' satisfies a nonlinear partial differential equation (PDE). A similarity reduction is obtained that reduces the PDE to an ordinary differential equation (ODE). A nonzero constant solution of this ODE leads to the usual Omori-Utsu law. An exact and explicit similarity solution is found that corresponds to the original Omori law. An initial value problem for the 'diffusive Omori-Utsu law' is also considered, and whose spatio-temporal dynamics are described by bounding functions that satisfy nonlinear, but linearisable, PDEs. Numerical results are also provided. △ Less

Submitted 21 October, 2021; originally announced November 2021.

MSC Class: 86A15; 35K57; 35K55

A unified way to solve IVPs and IBVPs for the time-fractional diffusion-wave equation

Authors: Marianito R. Rodrigo

Abstract: The time-fractional diffusion-wave equation is revisited, where the time derivative is of order $2 ν$ and $0 < ν\le 1$. The behaviour of the equation is "diffusion-like" (respectively, "wave-like") when $0 < ν\le \frac{1}{2}$ (respectively, $\frac{1}{2} < ν\le 1$). Two types of time-fractional derivatives are considered, namely the Caputo and Riemann-Liouville derivatives. Initial value problems a… ▽ More The time-fractional diffusion-wave equation is revisited, where the time derivative is of order $2 ν$ and $0 < ν\le 1$. The behaviour of the equation is "diffusion-like" (respectively, "wave-like") when $0 < ν\le \frac{1}{2}$ (respectively, $\frac{1}{2} < ν\le 1$). Two types of time-fractional derivatives are considered, namely the Caputo and Riemann-Liouville derivatives. Initial value problems and initial-boundary value problems are investigated and handled in a unified way using an embedding method. A two-parameter auxiliary function is introduced and its properties are investigated. The time-fractional diffusion equation is used to generate a new family of probability distributions, and that includes the normal distribution as a particular case. △ Less

Submitted 21 October, 2021; originally announced October 2021.

MSC Class: 26A33; 35R11; 35K05; 35L05; 60E05

CeyMo: See More on Roads -- A Novel Benchmark Dataset for Road Marking Detection

Authors: Oshada Jayasinghe, Sahan Hemachandra, Damith Anhettigama, Shenali Kariyawasam, Ranga Rodrigo, Peshala Jayasekara

Abstract: In this paper, we introduce a novel road marking benchmark dataset for road marking detection, addressing the limitations in the existing publicly available datasets such as lack of challenging scenarios, prominence given to lane markings, unavailability of an evaluation script, lack of annotation formats and lower resolutions. Our dataset consists of 2887 total images with 4706 road marking insta… ▽ More In this paper, we introduce a novel road marking benchmark dataset for road marking detection, addressing the limitations in the existing publicly available datasets such as lack of challenging scenarios, prominence given to lane markings, unavailability of an evaluation script, lack of annotation formats and lower resolutions. Our dataset consists of 2887 total images with 4706 road marking instances belonging to 11 classes. The images have a high resolution of 1920 x 1080 and capture a wide range of traffic, lighting and weather conditions. We provide road marking annotations in polygons, bounding boxes and pixel-level segmentation masks to facilitate a diverse range of road marking detection algorithms. The evaluation metrics and the evaluation script we provide, will further promote direct comparison of novel approaches for road marking detection with existing methods. Furthermore, we evaluate the effectiveness of using both instance segmentation and object detection based approaches for the road marking detection task. Speed and accuracy scores for two instance segmentation models and two object detector models are provided as a performance baseline for our benchmark dataset. The dataset and the evaluation script is publicly available at https://github.com/oshadajay/CeyMo. △ Less

Submitted 3 May, 2022; v1 submitted 22 October, 2021; originally announced October 2021.

Comments: Accepted to 2022 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV 2022)

SwiftLane: Towards Fast and Efficient Lane Detection

Authors: Oshada Jayasinghe, Damith Anhettigama, Sahan Hemachandra, Shenali Kariyawasam, Ranga Rodrigo, Peshala Jayasekara

Abstract: Recent work done on lane detection has been able to detect lanes accurately in complex scenarios, yet many fail to deliver real-time performance specifically with limited computational resources. In this work, we propose SwiftLane: a simple and light-weight, end-to-end deep learning based framework, coupled with the row-wise classification formulation for fast and efficient lane detection. This fr… ▽ More Recent work done on lane detection has been able to detect lanes accurately in complex scenarios, yet many fail to deliver real-time performance specifically with limited computational resources. In this work, we propose SwiftLane: a simple and light-weight, end-to-end deep learning based framework, coupled with the row-wise classification formulation for fast and efficient lane detection. This framework is supplemented with a false positive suppression algorithm and a curve fitting technique to further increase the accuracy. Our method achieves an inference speed of 411 frames per second, surpassing state-of-the-art in terms of speed while achieving comparable results in terms of accuracy on the popular CULane benchmark dataset. In addition, our proposed framework together with TensorRT optimization facilitates real-time lane detection on a Nvidia Jetson AGX Xavier as an embedded system while achieving a high inference speed of 56 frames per second. △ Less

Submitted 22 October, 2021; originally announced October 2021.

Comments: Accepted to 20th IEEE International Conference on Machine Learning and Applications (ICMLA) 2021

Neural Mixture Models with Expectation-Maximization for End-to-end Deep Clustering

Authors: Dumindu Tissera, Kasun Vithanage, Rukshan Wijesinghe, Alex Xavier, Sanath Jayasena, Subha Fernando, Ranga Rodrigo

Abstract: Any clustering algorithm must synchronously learn to model the clusters and allocate data to those clusters in the absence of labels. Mixture model-based methods model clusters with pre-defined statistical distributions and allocate data to those clusters based on the cluster likelihoods. They iteratively refine those distribution parameters and member assignments following the Expectation-Maximiz… ▽ More Any clustering algorithm must synchronously learn to model the clusters and allocate data to those clusters in the absence of labels. Mixture model-based methods model clusters with pre-defined statistical distributions and allocate data to those clusters based on the cluster likelihoods. They iteratively refine those distribution parameters and member assignments following the Expectation-Maximization (EM) algorithm. However, the cluster representability of such hand-designed distributions that employ a limited amount of parameters is not adequate for most real-world clustering tasks. In this paper, we realize mixture model-based clustering with a neural network where the final layer neurons, with the aid of an additional transformation, approximate cluster distribution outputs. The network parameters pose as the parameters of those distributions. The result is an elegant, much-generalized representation of clusters than a restricted mixture of hand-designed distributions. We train the network end-to-end via batch-wise EM iterations where the forward pass acts as the E-step and the backward pass acts as the M-step. In image clustering, the mixture-based EM objective can be used as the clustering objective along with existing representation learning methods. In particular, we show that when mixture-EM optimization is fused with consistency optimization, it improves the sole consistency optimization performance in clustering. Our trained networks outperform single-stage deep clustering methods that still depend on k-means, with unsupervised classification accuracy of 63.8% in STL10, 58% in CIFAR10, 25.9% in CIFAR100, and 98.9% in MNIST. △ Less

Submitted 2 October, 2022; v1 submitted 6 July, 2021; originally announced July 2021.

Comments: Accepted and published at Neurocomputing 2022

MSC Class: 68T10; 62H30 ACM Class: I.2; I.4; I.5

End-To-End Data-Dependent Routing in Multi-Path Neural Networks

Authors: Dumindu Tissera, Rukshan Wijessinghe, Kasun Vithanage, Alex Xavier, Subha Fernando, Ranga Rodrigo

Abstract: Neural networks are known to give better performance with increased depth due to their ability to learn more abstract features. Although the deepening of networks has been well established, there is still room for efficient feature extraction within a layer which would reduce the need for mere parameter increment. The conventional widening of networks by having more filters in each layer introduce… ▽ More Neural networks are known to give better performance with increased depth due to their ability to learn more abstract features. Although the deepening of networks has been well established, there is still room for efficient feature extraction within a layer which would reduce the need for mere parameter increment. The conventional widening of networks by having more filters in each layer introduces a quadratic increment of parameters. Having multiple parallel convolutional/dense operations in each layer solves this problem, but without any context-dependent allocation of resources among these operations: the parallel computations tend to learn similar features making the widening process less effective. Therefore, we propose the use of multi-path neural networks with data-dependent resource allocation among parallel computations within layers, which also lets an input to be routed end-to-end through these parallel paths. To do this, we first introduce a cross-prediction based algorithm between parallel tensors of subsequent layers. Second, we further reduce the routing overhead by introducing feature-dependent cross-connections between parallel tensors of successive layers. Our multi-path networks show superior performance to existing widening and adaptive feature extraction, and even ensembles, and deeper networks at similar complexity in the image recognition task. △ Less

Submitted 28 February, 2023; v1 submitted 6 July, 2021; originally announced July 2021.

Comments: Neural Computing and Applications 2023

MSC Class: 68T10 ACM Class: I.2; I.4; I.5

Diverse Single Image Generation with Controllable Global Structure

Authors: Sutharsan Mahendren, Chamira Edussooriya, Ranga Rodrigo

Abstract: Image generation from a single image using generative adversarial networks is quite interesting due to the realism of generated images. However, recent approaches need improvement for such realistic and diverse image generation, when the global context of the image is important such as in face, animal, and architectural image generation. This is mainly due to the use of fewer convolutional layers… ▽ More Image generation from a single image using generative adversarial networks is quite interesting due to the realism of generated images. However, recent approaches need improvement for such realistic and diverse image generation, when the global context of the image is important such as in face, animal, and architectural image generation. This is mainly due to the use of fewer convolutional layers for mainly capturing the patch statistics and, thereby, not being able to capture global statistics very well. We solve this problem by using attention blocks at selected scales and feeding a random Gaussian blurred image to the discriminator for training. Our results are visually better than the state-of-the-art particularly in generating images that require global context. The diversity of our image generation, measured using the average standard deviation of pixels, is also better. △ Less

Submitted 25 January, 2023; v1 submitted 9 February, 2021; originally announced February 2021.

Comments: Published in the Neurocomputing Journal

Journal ref: Neurocomputing 528(2023)97-112

DEVI: Open-source Human-Robot Interface for Interactive Receptionist Systems

Authors: Ramesha Karunasena, Piumi Sandarenu, Madushi Pinto, Achala Athukorala, Ranga Rodrigo, Peshala Jayasekara

Abstract: Humanoid robots that act as human-robot interfaces equipped with social skills can assist people in many of their daily activities. Receptionist robots are one such application where social skills and appearance are of utmost importance. Many existing robot receptionist systems suffer from high cost and they do not disclose internal architectures for further development for robot researchers. More… ▽ More Humanoid robots that act as human-robot interfaces equipped with social skills can assist people in many of their daily activities. Receptionist robots are one such application where social skills and appearance are of utmost importance. Many existing robot receptionist systems suffer from high cost and they do not disclose internal architectures for further development for robot researchers. Moreover, there does not exist customizable open-source robot receptionist frameworks to be deployed for any given application. In this paper we present an open-source robot receptionist intelligence core -- "DEVI"(means 'lady' in Sinhala), that provides researchers with ease of creating customized robot receptionists according to the requirements (cost, external appearance, and required processing power). Moreover, this paper also presents details on a prototype implementation of a physical robot using the DEVI system. The robot can give directional guidance with physical gestures, answer basic queries using a speech recognition and synthesis system, recognize and greet known people using face recognition and register new people in its database, using a self-learning neural network. Experiments conducted with DEVI show the effectiveness of the proposed system. △ Less

Submitted 2 January, 2021; originally announced January 2021.

Comments: Published in: 2019 IEEE 4th International Conference on Advanced Robotics and Mechatronics (ICARM)

Fast and Accurate Light Field Saliency Detection through Deep Encoding

Authors: Sahan Hemachandra, Ranga Rodrigo, Chamira Edussooriya

Abstract: Light field saliency detection -- important due to utility in many vision tasks -- still lacks speed and can improve in accuracy. Due to the formulation of the saliency detection problem in light fields as a segmentation task or a memorizing task, existing approaches consume unnecessarily large amounts of computational resources for training, and have longer execution times for testing. We solve t… ▽ More Light field saliency detection -- important due to utility in many vision tasks -- still lacks speed and can improve in accuracy. Due to the formulation of the saliency detection problem in light fields as a segmentation task or a memorizing task, existing approaches consume unnecessarily large amounts of computational resources for training, and have longer execution times for testing. We solve this by aggressively reducing the large light field images to a much smaller three-channel feature map appropriate for saliency detection using an RGB image saliency detector with attention mechanisms. We achieve this by introducing a novel convolutional neural network based features extraction and encoding module. Our saliency detector takes $0.4$ s to process a light field of size $9\times9\times512\times375$ in a CPU and is significantly faster than state-of-the-art light field saliency detectors, with better or comparable accuracy. Furthermore, model size of our architecture is significantly lower compared to state-of-the-art light field saliency detectors. Our work shows that extracting features from light fields through aggressive size reduction and the attention mechanism results in a faster and accurate light field saliency detector leading to near real-time light field processing. △ Less

Submitted 13 December, 2021; v1 submitted 25 October, 2020; originally announced October 2020.

ACM Class: I.4; I.5

Feature-Dependent Cross-Connections in Multi-Path Neural Networks

Authors: Dumindu Tissera, Kasun Vithanage, Rukshan Wijesinghe, Kumara Kahatapitiya, Subha Fernando, Ranga Rodrigo

Abstract: Learning a particular task from a dataset, samples in which originate from diverse contexts, is challenging, and usually addressed by deepening or widening standard neural networks. As opposed to conventional network widening, multi-path architectures restrict the quadratic increment of complexity to a linear scale. However, existing multi-column/path networks or model ensembling methods do not co… ▽ More Learning a particular task from a dataset, samples in which originate from diverse contexts, is challenging, and usually addressed by deepening or widening standard neural networks. As opposed to conventional network widening, multi-path architectures restrict the quadratic increment of complexity to a linear scale. However, existing multi-column/path networks or model ensembling methods do not consider any feature-dependent allocation of parallel resources, and therefore, tend to learn redundant features. Given a layer in a multi-path network, if we restrict each path to learn a context-specific set of features and introduce a mechanism to intelligently allocate incoming feature maps to such paths, each path can specialize in a certain context, reducing the redundancy and improving the quality of extracted features. This eventually leads to better-optimized usage of parallel resources. To do this, we propose inserting feature-dependent cross-connections between parallel sets of feature maps in successive layers. The weighting coefficients of these cross-connections are computed from the input features of the particular layer. Our multi-path networks show improved image recognition accuracy at a similar complexity compared to conventional and state-of-the-art methods for deepening, widening and adaptive feature extracting, in both small and large scale datasets. △ Less

Submitted 1 January, 2021; v1 submitted 24 June, 2020; originally announced June 2020.

Comments: International Conference on Pattern Recognition (ICPR) 2020

TimeCaps: Capturing Time Series Data With Capsule Networks

Authors: Hirunima Jayasekara, Vinoj Jayasundara, Mohamed Athif, Jathushan Rajasegaran, Sandaru Jayasekara, Suranga Seneviratne, Ranga Rodrigo

Abstract: Capsule networks excel in understanding spatial relationships in 2D data for vision related tasks. Even though they are not designed to capture 1D temporal relationships, with TimeCaps we demonstrate that given the ability, capsule networks excel in understanding temporal relationships. To this end, we generate capsules along the temporal and channel dimensions creating two temporal feature detect… ▽ More Capsule networks excel in understanding spatial relationships in 2D data for vision related tasks. Even though they are not designed to capture 1D temporal relationships, with TimeCaps we demonstrate that given the ability, capsule networks excel in understanding temporal relationships. To this end, we generate capsules along the temporal and channel dimensions creating two temporal feature detectors which learn contrasting relationships. TimeCaps surpasses the state-of-the-art results by achieving 96.21% accuracy on identifying 13 Electrocardiogram (ECG) signal beat categories, while achieving on-par results on identifying 30 classes of short audio commands. Further, the instantiation parameters inherently learnt by the capsule networks allow us to completely parameterize 1D signals which opens various possibilities in signal processing. △ Less

Submitted 18 June, 2022; v1 submitted 26 November, 2019; originally announced November 2019.

Context-Aware Multipath Networks

Authors: Dumindu Tissera, Kumara Kahatapitiya, Rukshan Wijesinghe, Subha Fernando, Ranga Rodrigo

Abstract: Making a single network effectively address diverse contexts---learning the variations within a dataset or multiple datasets---is an intriguing step towards achieving generalized intelligence. Existing approaches of deepening, widening, and assembling networks are not cost effective in general. In view of this, networks which can allocate resources according to the context of the input and regulat… ▽ More Making a single network effectively address diverse contexts---learning the variations within a dataset or multiple datasets---is an intriguing step towards achieving generalized intelligence. Existing approaches of deepening, widening, and assembling networks are not cost effective in general. In view of this, networks which can allocate resources according to the context of the input and regulate flow of information across the network are effective. In this paper, we present Context-Aware Multipath Network (CAMNet), a multi-path neural network with data-dependant routing between parallel tensors. We show that our model performs as a generalized model capturing variations in individual datasets and multiple different datasets, both simultaneously and sequentially. CAMNet surpasses the performance of classification and pixel-labeling tasks in comparison with the equivalent single-path, multi-path, and deeper single-path networks, considering datasets individually, sequentially, and in combination. The data-dependent routing between tensors in CAMNet enables the model to control the flow of information end-to-end, deciding which resources to be common or domain-specific. △ Less

Submitted 26 July, 2019; originally announced July 2019.

Exploiting the Redundancy in Convolutional Filters for Parameter Reduction

Authors: Kumara Kahatapitiya, Ranga Rodrigo

Abstract: Convolutional Neural Networks (CNNs) have achieved state-of-the-art performance in many computer vision tasks over the years. However, this comes at the cost of heavy computation and memory intensive network designs, suggesting potential improvements in efficiency. Convolutional layers of CNNs partly account for such an inefficiency, as they are known to learn redundant features. In this work, we… ▽ More Convolutional Neural Networks (CNNs) have achieved state-of-the-art performance in many computer vision tasks over the years. However, this comes at the cost of heavy computation and memory intensive network designs, suggesting potential improvements in efficiency. Convolutional layers of CNNs partly account for such an inefficiency, as they are known to learn redundant features. In this work, we exploit this redundancy, observing it as the correlation between convolutional filters of a layer, and propose an alternative approach to reproduce it efficiently. The proposed 'LinearConv' layer learns a set of orthogonal filters, and a set of coefficients that linearly combines them to introduce a controlled redundancy. We introduce a correlation-based regularization loss to achieve such flexibility over redundancy, and control the number of parameters in turn. This is designed as a plug-and-play layer to conveniently replace a conventional convolutional layer, without any additional changes required in the network architecture or the hyperparameter settings. Our experiments verify that LinearConv models achieve a performance on-par with their counterparts, with almost a 50% reduction in parameters on average, and the same computational requirement and speed at inference. △ Less

Submitted 10 August, 2020; v1 submitted 26 July, 2019; originally announced July 2019.

Comments: Accepted to be published in Proceedings of IEEE Winter Conference on Applications of Computer Vision (WACV) 2021

Diurnal variation of dust and gas production in comet 67P/Churyumov-Gerasimenko at the inbound equinox as seen by OSIRIS and VIRTIS-M on board Rosetta

Authors: C. Tubiana, G. Rinaldi, C. Güttler, C. Snodgrass, X. Shi, X. Hu, R. Marschall, M. Fulle, D. Bockelée-Morvan, G. Naletto, F. Capaccioni, H. Sierks, G. Arnold, M. A. Barucci, J. -L. Bertaux, I. Bertini, D. Bodewits, M. T. Capria, M. Ciarniello, G. Cremonese, J. Crovisier, V. Da Deppo, S. Debei, M. De Cecco, J. Deller , et al. (31 additional authors not shown)

Abstract: On 27 Apr 2015, when 67P/C-G was at 1.76 au from the Sun and moving towards perihelion, the OSIRIS and VIRTIS-M instruments on Rosetta observed the evolving dust and gas coma during a complete rotation of the comet. We aim to characterize the dust, H2O and CO2 gas spatial distribution in the inner coma. To do this we performed a quantitative analysis of the release of dust and gas and compared the… ▽ More On 27 Apr 2015, when 67P/C-G was at 1.76 au from the Sun and moving towards perihelion, the OSIRIS and VIRTIS-M instruments on Rosetta observed the evolving dust and gas coma during a complete rotation of the comet. We aim to characterize the dust, H2O and CO2 gas spatial distribution in the inner coma. To do this we performed a quantitative analysis of the release of dust and gas and compared the observed H2O production rate with the one calculated using a thermo-physical model. For this study we selected OSIRIS WAC images at 612 nm (dust) and VIRTIS-M image cubes at 612 nm, 2700 nm (H2O) and 4200 nm (CO2). We measured the average signal in a circular annulus, to study spatial variation around the comet, and in a sector of the annulus, to study temporal variation in the sunward direction with comet rotation, both at a fixed distance of 3.1 km from the comet centre. The spatial correlation between dust and water, both coming from the sun-lit side of the comet, shows that water is the main driver of dust activity in this time period. The spatial distribution of CO2 is not correlated with water and dust. There is no strong temporal correlation between the dust brightness and water production rate as the comet rotates. The dust brightness shows a peak at 0deg sub-solar longitude, which is not pronounced in the water production. At the same epoch, there is also a maximum in CO2 production. An excess of measured water production, with respect to the value calculated using a simple thermo-physical model, is observed when the head lobe and regions of the Southern hemisphere with strong seasonal variations are illuminated. A drastic decrease in dust production, when the water production (both measured and from the model) displays a maximum, happens when typical Northern consolidated regions are illuminated and the Southern hemisphere regions with strong seasonal variations are instead in shadow. △ Less

Submitted 8 May, 2019; originally announced May 2019.

Comments: 15 pages, accepted for publication in A&A

Context-Aware Automatic Occlusion Removal

Authors: Kumara Kahatapitiya, Dumindu Tissera, Ranga Rodrigo

Abstract: Occlusion removal is an interesting application of image enhancement, for which, existing work suggests manually-annotated or domain-specific occlusion removal. No work tries to address automatic occlusion detection and removal as a context-aware generic problem. In this paper, we present a novel methodology to identify objects that do not relate to the image context as occlusions and remove them,… ▽ More Occlusion removal is an interesting application of image enhancement, for which, existing work suggests manually-annotated or domain-specific occlusion removal. No work tries to address automatic occlusion detection and removal as a context-aware generic problem. In this paper, we present a novel methodology to identify objects that do not relate to the image context as occlusions and remove them, reconstructing the space occupied coherently. The proposed system detects occlusions by considering the relation between foreground and background object classes represented as vector embeddings, and removes them through inpainting. We test our system on COCO-Stuff dataset and conduct a user study to establish a baseline in context-aware automatic occlusion removal. △ Less

Submitted 7 May, 2019; originally announced May 2019.

Comments: Accepted to be published in Proceedings of IEEE International Conference on Image Processing (ICIP), Taipei, Taiwan, September 2019

DeepCaps: Going Deeper with Capsule Networks

Authors: Jathushan Rajasegaran, Vinoj Jayasundara, Sandaru Jayasekara, Hirunima Jayasekara, Suranga Seneviratne, Ranga Rodrigo

Abstract: Capsule Network is a promising concept in deep learning, yet its true potential is not fully realized thus far, providing sub-par performance on several key benchmark datasets with complex data. Drawing intuition from the success achieved by Convolutional Neural Networks (CNNs) by going deeper, we introduce DeepCaps1, a deep capsule network architecture which uses a novel 3D convolution based dyna… ▽ More Capsule Network is a promising concept in deep learning, yet its true potential is not fully realized thus far, providing sub-par performance on several key benchmark datasets with complex data. Drawing intuition from the success achieved by Convolutional Neural Networks (CNNs) by going deeper, we introduce DeepCaps1, a deep capsule network architecture which uses a novel 3D convolution based dynamic routing algorithm. With DeepCaps, we surpass the state-of-the-art results in the capsule network domain on CIFAR10, SVHN and Fashion MNIST, while achieving a 68% reduction in the number of parameters. Further, we propose a class-independent decoder network, which strengthens the use of reconstruction loss as a regularization term. This leads to an interesting property of the decoder, which allows us to identify and control the physical attributes of the images represented by the instantiation parameters. △ Less

Submitted 21 April, 2019; originally announced April 2019.

TextCaps : Handwritten Character Recognition with Very Small Datasets

Authors: Vinoj Jayasundara, Sandaru Jayasekara, Hirunima Jayasekara, Jathushan Rajasegaran, Suranga Seneviratne, Ranga Rodrigo

Abstract: Many localized languages struggle to reap the benefits of recent advancements in character recognition systems due to the lack of substantial amount of labeled training data. This is due to the difficulty in generating large amounts of labeled data for such languages and inability of deep learning techniques to properly learn from small number of training samples. We solve this problem by introduc… ▽ More Many localized languages struggle to reap the benefits of recent advancements in character recognition systems due to the lack of substantial amount of labeled training data. This is due to the difficulty in generating large amounts of labeled data for such languages and inability of deep learning techniques to properly learn from small number of training samples. We solve this problem by introducing a technique of generating new training samples from the existing samples, with realistic augmentations which reflect actual variations that are present in human hand writing, by adding random controlled noise to their corresponding instantiation parameters. Our results with a mere 200 training samples per class surpass existing character recognition results in the EMNIST-letter dataset while achieving the existing results in the three datasets: EMNIST-balanced, EMNIST-digits, and MNIST. We also develop a strategy to effectively use a combination of loss functions to improve reconstructions. Our system is useful in character recognition for localized languages that lack much labeled training data and even in other related more general contexts such as object recognition. △ Less

Submitted 17 April, 2019; originally announced April 2019.

Journal ref: In 2019 IEEE Winter Conference on Applications of Computer Vision (WACV) (pp. 254-262). IEEE 2019

Surface evolution of the Anhur region on comet 67P from high-resolution OSIRIS images

Authors: S. Fornasier, C. Feller, P. H. Hasselmann, M. A. Barucci, J. Sunshine, J. -B. Vincent, X. Shi, H. Sierks, G. Naletto, P. L. Lamy, R. Rodrigo, D. Koschny, B. Davidsson, J. -L. Bertaux, I. Bertini, D. Bodewits, G. Cremonese, V. Da Deppo, S. Debei, M. De Cecco, J. Deller, S. Ferrari, M. Fulle, P. J. Gutierrez, C. Güttler , et al. (12 additional authors not shown)

Abstract: The southern hemisphere of comet 67P/Churyumov-Gerasimenko (67P) became observable by the Rosetta mission in March 2015, a few months before cometary southern vernal equinox. The Anhur region in the southern part of the comet's larger lobe was found to be highly eroded, enriched in volatiles, and highly active. We analyze high-resolution images of the Anhur region pre- and post-perihelion acquired… ▽ More The southern hemisphere of comet 67P/Churyumov-Gerasimenko (67P) became observable by the Rosetta mission in March 2015, a few months before cometary southern vernal equinox. The Anhur region in the southern part of the comet's larger lobe was found to be highly eroded, enriched in volatiles, and highly active. We analyze high-resolution images of the Anhur region pre- and post-perihelion acquired by the OSIRIS imaging system on board the Rosetta mission. The Narrow Angle Camera is particularly useful for studying the evolution in Anhur in terms of morphological changes and color variations.}{Radiance factor images processed by the OSIRIS pipeline were coregistered, reprojected onto the 3D shape model of the comet, and corrected for the illumination conditions. We find a number of morphological changes in the Anhur region that are related to formation of new scarps; removal of dust coatings; localized resurfacing in some areas, including boulders displacements; and vanishing structures, which implies localized mass loss that we estimate to be higher than 50 million kg. The strongest changes took place in and nearby the Anhur canyon-like structure, where significant dust cover was removed, an entire structure vanished, and many boulders were rearranged. All such changes are potentially associated with one of the most intense outbursts registered by Rosetta during its observations, which occurred one day before perihelion passage. Moreover, in the niche at the foot of a new observed scarp, we also see evidence of water ice exposure that persisted for at least six months. The abundance of water ice, evaluated from a linear mixing model, is relatively high (> 20%). Our results confirm that the Anhur region is volatile-rich and probably is the area on 67P with the most pristine exposures near perihelion. △ Less

Submitted 21 March, 2019; originally announced March 2019.

Comments: 19 pages, 16 figures; accepted for publication in Astronomy and Astrophysics for the Rosetta 2 special number

Journal ref: A&A 630, A13 (2019)

Constraining models of activity on comet 67P/Churyumov-Gerasimenko with Rosetta trajectory, rotation, and water production measurements

Authors: Nicholas Attree, Laurent Jorda, Olivier Groussin, Stefano Mottola, Nick Thomas, Yann Brouet, Ekkehard Kührt, Martin Knapmeyer, Frank Preusker, Frank Scholten, Jorg Knollenberg, Stubbe Hviid, Paul Hartogh, Rafael Rodrigo

Abstract: Aims. We use four observational data sets, mainly from the Rosetta mission, to constrain the activity pattern of the nucleus of comet 67P/Churyumov-Gerasimenko. Methods. We develop a numerical model that computes the production rate and non-gravitational acceleration of the nucleus of comet 67P as a function of time, taking into account its complex shape with a shape model reconstructed from OSIRI… ▽ More Aims. We use four observational data sets, mainly from the Rosetta mission, to constrain the activity pattern of the nucleus of comet 67P/Churyumov-Gerasimenko. Methods. We develop a numerical model that computes the production rate and non-gravitational acceleration of the nucleus of comet 67P as a function of time, taking into account its complex shape with a shape model reconstructed from OSIRIS imagery. We use this model to fit three observational data sets: the trajectory data from flight dynamics; the rotation state, as reconstructed from OSIRIS imagery; and the water production measurements from ROSINA, of 67P. The two key parameters of our model, adjusted to fit the three data sets all together, are the activity pattern and the momentum transfer efficiency (i.e., the so-called "$η$ parameter" of the non-gravitational forces). Results. We find an activity pattern able to successfully reproduce the three data sets simultaneously. The fitted activity pattern exhibits two main features: a higher effective active fraction in two southern super-regions ($\sim 10$~\%) outside perihelion compared to the northern ones ($< 4$~\%), and a drastic rise of the effective active fraction of the southern regions ($\sim 25-35$~\%) around perihelion. We interpret the time-varying southern effective active fraction by cyclic formation and removal of a dust mantle in these regions. Our analysis supports moderate values of the momentum transfer coefficient $η$ in the range $0.6-0.7$; values $η\leq0.5$ or $η\geq0.8$ degrade significantly the fit to the three data sets. Our conclusions reinforce the idea that seasonal effects linked to the orientation of the spin axis play a key role in the formation and evolution of dust mantles, and in turn largely control the temporal variations of the gas flux. △ Less

Submitted 9 January, 2019; originally announced January 2019.

Comments: 12 pages, 17 figures. Accepted for publication in forthcoming Rosetta issue of Astronomy and Astrophysics

Journal ref: A&A 630, A18 (2019)

ROSETTA/OSIRIS observations of the 67P nucleus during the April 2016 flyby: high-resolution spectrophotometry

Authors: C. Feller, S. Fornasier, S. Ferrari, P. H. Hasselmann, A. Barucci, M. Massironi, J. D. P Deshapriya, H. Sierks, G. Naletto, P. L. Lamy, R. Rodrigo, D. Koschny, B. J. R. Davidsson, J. -L. Bertaux, I. Bertini, D. Bodewits, G. Cremonese, V. Da Deppo, S. Debei, M. De Cecco, M. Fulle, P. J. Gutiérrez, C. Güttler, W. -H. Ip, H. U. Keller , et al. (13 additional authors not shown)

Abstract: In April 2016, the Rosetta spacecraft performed a low-altitude low-phase-angle flyby over the Imhotep-Khepry transition of 67P/Churyumov-Gerasimenko's nucleus. The OSIRIS/Narrow-Angle-Camera (NAC) acquired 112 images with mainly 3 broadband filters in the visible at a resolution of up to 0.53 m/px and for phase angles between 0.095° and 62°. Using those images, we have investigated the morphologic… ▽ More In April 2016, the Rosetta spacecraft performed a low-altitude low-phase-angle flyby over the Imhotep-Khepry transition of 67P/Churyumov-Gerasimenko's nucleus. The OSIRIS/Narrow-Angle-Camera (NAC) acquired 112 images with mainly 3 broadband filters in the visible at a resolution of up to 0.53 m/px and for phase angles between 0.095° and 62°. Using those images, we have investigated the morphological and spectrophotometrical properties of this area. We assembled the images into coregistered color cubes. Using a 3D shape model, we produced the illumination conditions and georeference for each image. We projected the observations on a map to investigate its geomorphology. Observations were photometrically corrected using the Lommel-Seeliger disk law. Spectrophotometric analyses were performed on the coregistered color cubes. These data were used to estimate the local phase reddening. This region of the nucleus hosts numerous and varied types of terrains and features. We observe an association between a feature's nature, its reflectance, and its spectral slope. Fine material deposits exhibit an average reflectance and spectral slope, while terrains with diamictons, consolidated material, degraded outcrops, or features such as somber boulders, present a lower-than-average reflectance and higher-than-average spectral slope. Bright surfaces present here a spectral behavior consistent with terrains enriched in water-ice. We find a phase-reddening slope of 0.064{\pm}0.001{\%}/100nm/° at 2.7 au outbound, similarly to the one obtained at 2.3 au inbound during the February 2015 flyby. Identified as the source region of multiple jets and a host of water-ice material, the Imhotep-Khepry transition appeared in April 2016, close to the frost line, to further harbor several potential locations with exposed water-ice material among its numerous different morphological terrain units. △ Less

Submitted 21 December, 2018; originally announced December 2018.

Comments: 23 pages, 14 figures, 5 tables

Combined Static and Motion Features for Deep-Networks Based Activity Recognition in Videos

Authors: Sameera Ramasinghe, Jathushan Rajasegaran, Vinoj Jayasundara, Kanchana Ranasinghe, Ranga Rodrigo, Ajith A. Pasqual

Abstract: Activity recognition in videos in a deep-learning setting---or otherwise---uses both static and pre-computed motion components. The method of combining the two components, whilst kee** the burden on the deep network less, still remains uninvestigated. Moreover, it is not clear what the level of contribution of individual components is, and how to control the contribution. In this work, we use a… ▽ More Activity recognition in videos in a deep-learning setting---or otherwise---uses both static and pre-computed motion components. The method of combining the two components, whilst kee** the burden on the deep network less, still remains uninvestigated. Moreover, it is not clear what the level of contribution of individual components is, and how to control the contribution. In this work, we use a combination of CNN-generated static features and motion features in the form of motion tubes. We propose three schemas for combining static and motion components: based on a variance ratio, principal components, and Cholesky decomposition. The Cholesky decomposition based method allows the control of contributions. The ratio given by variance analysis of static and motion features match well with the experimental optimal ratio used in the Cholesky decomposition based method. The resulting activity recognition system is better or on par with existing state-of-the-art when tested with three popular datasets. The findings also enable us to characterize a dataset with respect to its richness in motion information. △ Less

Submitted 16 October, 2018; originally announced October 2018.

Journal ref: IEEE Transactions on Circuits and Systems for Video Technology (2017)

Models of Rosetta/OSIRIS 67P dust coma phase function

Authors: Fernando Moreno, Daniel Guirado, Olga Muñoz, Ivano Bertini, Cecilia Tubiana, Carsten Guttler, Marco Fulle, Alessandra Rotundi, Vincenzo Della Corte, Stavro Ivanovski, Giovanna Rinaldi, Dominique Bockelee-Morvan, Vladimir Zakharov, Jessica Agarwal, Stefano Mottola, Imre Toth, Elisa Frattin, Luisa Lara, Pedro Gutierrez, Zhong Yi Lin, Ludmilla Kolokolova, Holger Sierks, Giampiero Naletto, Philippe Lamy, Rafael Rodrigo , et al. (17 additional authors not shown)

Abstract: The phase function of the dust coma of comet 67P has been determined from Rosetta/OSIRIS images \citep{Bertini17}. This function show a deep minimum at phase angles near 100$^\circ$, and a strong backscattering enhancement. These two properties cannot be reproduced by regular models of cometary dust, most of them based on wavelength-sized and randomly-oriented aggregate particles. We show, however… ▽ More The phase function of the dust coma of comet 67P has been determined from Rosetta/OSIRIS images \citep{Bertini17}. This function show a deep minimum at phase angles near 100$^\circ$, and a strong backscattering enhancement. These two properties cannot be reproduced by regular models of cometary dust, most of them based on wavelength-sized and randomly-oriented aggregate particles. We show, however, that an ensamble of oriented elongated particles of a wide variety of aspect ratios, with radii $r \gtrsim$10 $μ$m, and whose long axes are perpendicular to the direction of the solar radiation, are capable of reproducing the observed phase function. These particles must be absorbing, with an imaginary part of the refractive index of about 0.1 to match the expected geometric albedo, and with porosity in the 60-70\% range. △ Less

Submitted 27 September, 2018; originally announced September 2018.

Comments: Accepted by Astronomical Journal, September 26th, 2018. 21 pages, 5 figures

Linking surface morphology, composition, and activity on the nucleus of 67P/Churyumov-Gerasimenko

Authors: S. Fornasier, V. H. Hoang, P. H. Hasselmann, C. Feller, M. A. Barucci, J. D. P. Deshapriya, H. Sierks, G. Naletto, P. L. Lamy, R. Rodrigo, D. Koschny, B. Davidsson, J. Agarwal, C. Barbieri, J. -L. Bertaux, I. Bertini, D. Bodewits, G. Cremonese, V. Da Deppo, S. Debei, M. De Cecco, J. Deller, S. Ferrari, M. Fulle, P. J. Gutierrez , et al. (15 additional authors not shown)

Abstract: The Rosetta space probe accompanied comet 67P/Churyumov-Gerasimenko for more than two years, obtaining an unprecedented amount of unique data of the comet nucleus and inner coma. This work focuses identifying the source regions of faint jets and outbursts and on studying the spectrophotometric properties of some outbursts. We use observations acquired with the OSIRIS/NAC camera during July-October… ▽ More The Rosetta space probe accompanied comet 67P/Churyumov-Gerasimenko for more than two years, obtaining an unprecedented amount of unique data of the comet nucleus and inner coma. This work focuses identifying the source regions of faint jets and outbursts and on studying the spectrophotometric properties of some outbursts. We use observations acquired with the OSIRIS/NAC camera during July-October 2015, that is, close to perihelion. More than 200 jets of different intensities were identified directly on the nucleus. Some of the more intense outbursts appear spectrally bluer than the comet dark terrain in the vivible-to-near-infrared region. We attribute this spectral behavior to icy grains mixed with the ejected dust. Some of the jets have an extremely short lifetime. They appear on the cometary surface during the color sequence observations, and vanish in less than some few minutes after reaching their peak. We also report a resolved dust plume observed in May 2016 at a resolution of 55 cm/pixel, which allowed us to estimate an optical depth of $\sim$0.65 and an ejected mass of $\sim$ 2200 kg. We present the results on the location, duration, and colors of active sources on the nucleus of 67P from the medium-resolution (i.e., 6-10 m/pixel) images acquired close to perihelion passage. The observed jets are mainly located close to boundaries between different morphological regions. Jets depart not only from cliffs, but also from smooth and dust-covered areas, from fractures, pits, or cavities that cast shadows and favor the recondensation of volatiles. This study shows that faint jets or outbursts continuously contribute to the cometary activity close to perihelion passage, and that these events are triggered by illumination conditions. Faint jets or outbursts are not associated with a particular terrain type or morphology. △ Less

Submitted 11 September, 2018; originally announced September 2018.

Comments: Accepted for publication on Astronomy and Astrophysics on 27 August 2018. 27 pages, 18 figures, 2 tables

Journal ref: A&A 630, A7 (2019)

Traveling wave solutions in a model for tumor invasion with the acid-mediation hypothesis

Authors: P. N. Davis, P. van Heijster, R. Marangell, M. R. Rodrigo

Abstract: In this manuscript, we prove the existence of slow and fast traveling wave solutions in the original Gatenby--Gawlinski model. We prove the existence of a slow traveling wave solution with an interstitial gap. This interstitial gap has previously been observed experimentally, and here we derive its origin from a mathematical perspective. We give a geometric interpretation of the formal asymptotic… ▽ More In this manuscript, we prove the existence of slow and fast traveling wave solutions in the original Gatenby--Gawlinski model. We prove the existence of a slow traveling wave solution with an interstitial gap. This interstitial gap has previously been observed experimentally, and here we derive its origin from a mathematical perspective. We give a geometric interpretation of the formal asymptotic analysis of the interstitial gap and show that it is determined by the distance between a layer transition of the tumor and a dynamical transcritical bifurcation of two components of the critical manifold. This distance depends, in a nonlinear fashion, on the destructive influence of the acid and the rate at which the acid is being pumped. △ Less

Submitted 3 May, 2021; v1 submitted 27 July, 2018; originally announced July 2018.

Comments: 31 page, 5 figures

MSC Class: 35Q92; 35C07; 35B25; 92C17;

Tensile Strength of 67P/Churyumov-Gerasimenko Nucleus Material from Overhangs

Authors: N. Attree, O. Groussin, L. Jorda, D. Nébouy, N. Thomas, Y. Brouet, E. Kührt, F. Preusker, F. Scholten, J. Knollenberg, P. Hartogh, H. Sierks, C. Barbieri, P. Lamy, R. Rodrigo, D. Koschny, H. Rickman, H. U. Keller, M. F. A'Hearn, A. -T. Auger, M. A. Barucci, J. -L. Bertaux, I. Bertini, D. Bodewits, S. Boudreault , et al. (30 additional authors not shown)

Abstract: We directly measure twenty overhanging cliffs on the surface of comet 67P/Churyumov-Gerasimenko extracted from the latest shape model and estimate the minimum tensile strengths needed to support them against collapse under the comet's gravity. We find extremely low strengths of around one Pa or less (one to five Pa, when scaled to a metre length). The presence of eroded material at the base of mos… ▽ More We directly measure twenty overhanging cliffs on the surface of comet 67P/Churyumov-Gerasimenko extracted from the latest shape model and estimate the minimum tensile strengths needed to support them against collapse under the comet's gravity. We find extremely low strengths of around one Pa or less (one to five Pa, when scaled to a metre length). The presence of eroded material at the base of most overhangs, as well as the observed collapse of two features and implied previous collapse of another, suggests that they are prone to failure and that true material strengths are close to these lower limits (although we only consider static stresses and not dynamic stress from, for example, cometary activity). Thus, a tensile strength of a few pascals is a good approximation for the tensile strength of 67P's nucleus material, which is in agreement with previous work. We find no particular trends in overhang properties with size, over the $\sim10-100$ m range studied here, or location on the nucleus. There are no obvious differences, in terms of strength, height or evidence of collapse, between the populations of overhangs on the two cometary lobes, suggesting that 67P is relatively homogenous in terms of tensile strength. Low material strengths are supportive of cometary formation as a primordial rubble pile or by collisional fragmentation of a small (tens of km) body. △ Less

Submitted 20 December, 2017; originally announced December 2017.

Comments: 13 pages, 11 figures. Accepted for publication in Astronomy & Astrophysics

Journal ref: A&A 611, A33 (2018)

Evidence of sub-surface energy storage in comet 67P from the outburst of 2016 July 3

Authors: J. Agarwal, V. Della Corte, P. D. Feldman, B. Geiger, S. Merouane, I. Bertini, D. Bodewits, S. Fornasier, E. Gruen, P. Hasselmann, M. Hilchenbach, S. Hoefner, S. Ivanovski, L. Kolokolova, M. Pajola, A. Rotundi, H. Sierks, A. J. Steffl, N. Thomas, M. F. A'Hearn, C. Barbieri, M. A. Barucci, J. -L. Bertaux, S. Boudreault, G. Cremonese , et al. (45 additional authors not shown)

Abstract: On 3 July 2016, several instruments on board ESA's Rosetta spacecraft detected signs of an outburst event on comet 67P, at a heliocentric distance of 3.32 AU from the sun, outbound from perihelion. We here report on the inferred properties of the ejected dust and the surface change at the site of the outburst. The activity coincided with the local sunrise and continued over a time interval of 14 -… ▽ More On 3 July 2016, several instruments on board ESA's Rosetta spacecraft detected signs of an outburst event on comet 67P, at a heliocentric distance of 3.32 AU from the sun, outbound from perihelion. We here report on the inferred properties of the ejected dust and the surface change at the site of the outburst. The activity coincided with the local sunrise and continued over a time interval of 14 - 68 minutes. It left a 10m-sized icy patch on the surface. The ejected material comprised refractory grains of several hundred microns in size, and sub-micron-sized water ice grains. The high dust mass production rate is incompatible with the free sublimation of crystalline water ice under solar illumination as the only acceleration process. Additional energy stored near the surface must have increased the gas density. We suggest a pressurized sub-surface gas reservoir, or the crystallization of amorphous water ice as possible causes. △ Less

Submitted 27 October, 2017; originally announced October 2017.

Comments: 20 pages, 19 figures, 5 tables

Journal ref: MNRAS 469, S606-S625, 2017

Seasonal Mass Transfer on the Nucleus of Comet 67P/Chuyumov-Gerasimenko

Authors: H. U. Keller, S. Mottola, S. F. Hviid, J. Agarwal, E. Kührt, Y. Skorov, K. Otto, J. -B. Vincent, N. Oklay, S. E. Schröder, B. Davidsson, M. Pajola, X. Shi, D. Bodewits, I. Toth, F. Preusker, F. Scholten, H. Sierks, C. Barbieri, P. Lamy, R. Rodrigo, D. Koschny, H. Rickman, M. F. A'Hearn, M. A. Barucci , et al. (25 additional authors not shown)

Abstract: We collect observational evidence that supports the scheme of mass transfer on the nucleus of comet 67P/Churyumov-Gerasimenko. The obliquity of the rotation axis of 67P causes strong seasonal variations. During perihelion the southern hemisphere is four times more active than the north. Northern territories are widely covered by granular material that indicates back fall originating from the activ… ▽ More We collect observational evidence that supports the scheme of mass transfer on the nucleus of comet 67P/Churyumov-Gerasimenko. The obliquity of the rotation axis of 67P causes strong seasonal variations. During perihelion the southern hemisphere is four times more active than the north. Northern territories are widely covered by granular material that indicates back fall originating from the active south. Decimetre sized chunks contain water ice and their trajectories are influenced by an anti-solar force instigated by sublimation. OSIRIS observations suggest that up to 20 % of the particles directly return to the nucleus surface taking several hours of travel time. The back fall covered northern areas are active if illuminated but produce mainly water vapour. The decimetre chunks from the nucleus surface are too small to contain more volatile compounds such as CO 2 or CO. This causes a north-south dichotomy of the composition measurements in the coma. Active particles are trapped in the gravitational minimum of Hapi during northern winter. They are "shock frozen" and only reactivated when the comet approaches the sun after its aphelion passage. The insolation of the big cavity is enhanced by self-heating, i. e. reflection and IR radiation from the walls. This, together with the pristinity of the active back fall, explains the early observed activity of the Hapi region. Sobek may be a role model for the consolidated bottom of Hapi. Mass transfer in the case of 67P strongly influences the evolution of the nucleus and the interpretation of coma measurements. △ Less

Submitted 21 July, 2017; originally announced July 2017.

Comments: 17 pages, 20 figures

Journal ref: Monthly Notices of the Royal Astronomical Society stx1726, 13 July 2017

The highly active Anhur-Bes regions in the 67P/Churyumov - Gerasimenko comet: results from OSIRIS/ROSETTA observations

Authors: S. Fornasier, C. Feller, J. C. Lee, S. Ferrari, M. Massironi, P. H. Hasselmann, J. D. P Deshapriya, M. A. Barucci, M. R. El-Maarry, L. Giacomini, S. Mottola, H. U. Keller, W. H. Ip, Z. Y. Lin, H. Sierks, C. Barbieri, P. L. Lamy, R. Rodrigo, D. Koschny, H. Rickman, J. Agarwal, M. A'Hearn, J. -L. Bertaux, I. Bertini, G. Cremonese , et al. (29 additional authors not shown)

Abstract: The Southern hemisphere of the 67P/Churyumov-Gerasimenko comet has become visible from Rosetta only since March 2015. It was illuminated during the perihelion passage and therefore it contains the regions that experienced the strongest heating and erosion rate, thus exposing the subsurface most pristine material. In this work we investigate, thanks to the OSIRIS images, the geomorphology, the spec… ▽ More The Southern hemisphere of the 67P/Churyumov-Gerasimenko comet has become visible from Rosetta only since March 2015. It was illuminated during the perihelion passage and therefore it contains the regions that experienced the strongest heating and erosion rate, thus exposing the subsurface most pristine material. In this work we investigate, thanks to the OSIRIS images, the geomorphology, the spectrophotometry and some transient events of two Southern hemisphere regions: Anhur and part of Bes. Bes is dominated by outcrop** consolidated terrain covered with fine particle deposits, while Anhur appears strongly eroded with elongated canyon-like structures, scarp retreats, different kinds of deposits, and degraded sequences of strata indicating a pervasive layering. We discovered a new 140 m long and 10 m high scarp formed in the Anhur/Bes boundary during/after the perihelion passage, close to the area where exposed CO$_2$ and H$_2$O ices were previously detected. Several jets have been observed originating from these regions, including the strong perihelion outburst, an active pit, and a faint optically thick dust plume. We identify several areas with a relatively bluer slope (i.e. a lower spectral slope value) than their surroundings, indicating a surface composition enriched with some water ice. These spectrally bluer areas are observed especially in talus and gravitational accumulation deposits where freshly exposed material had fallen from nearby scarps and cliffs. The investigated regions become spectrally redder beyond 2 au outbound when the dust mantle became thicker, masking the underlying ice-rich layers. △ Less

Submitted 10 July, 2017; originally announced July 2017.

Comments: 16 pages, 15 figures, published online on 24 May 2017 on Mon. Not. R. Astron. Soc. stx1275, https://doi.org/10.1093/mnras/stx1275

Constraints on cometary surface evolution derived from a statistical analysis of 67P's topography

Authors: J. -B. Vincent, S. F. Hviid, S. Mottola, E. Kuehrt, F. Preusker, F. Scholten, H. U. Keller, N. Oklay, D. de Niem, B. Davidsson, M. Fulle, M. Pajola, M. Hofmann, X. Hu, H. Rickman, Z. -Y. Lin, C. Feller, A. Gicquel, S. Boudreault, H. Sierks, C. Barbieri, P. L. Lamy, R. Rodrigo, D. Koschny, M. F. A'Hearn , et al. (29 additional authors not shown)

Abstract: We present a statistical analysis of the distribution of large scale topographic features on comet 67P/Churyumov-Gerasimenko. We observe that the cumulative cliff height distribution across the surface follows a power law with a slope equal to -1.69 +- 0.02. When this distribution is studied independently for each region, we find a good correlation between the slope of the power law and the orbita… ▽ More We present a statistical analysis of the distribution of large scale topographic features on comet 67P/Churyumov-Gerasimenko. We observe that the cumulative cliff height distribution across the surface follows a power law with a slope equal to -1.69 +- 0.02. When this distribution is studied independently for each region, we find a good correlation between the slope of the power law and the orbital erosion rate of the surface. For instance, the northern hemisphere topography is dominated by structures on the 100~m scale while the southern hemisphere topography, illuminated at perihelion, is dominated by 10~m scale terrain features. Our study suggest that the current size of a cliff is controlled not only by material cohesion but by the dominant erosional process in each region. This observation can be generalized to other comets, where we argue that primitive nuclei are characterized by the presence of large cliffs with a cumulative height power index equal to or above -1.5, while older, eroded cometary surfaces have a power index equal to or below -2.3. In effect, our model shows that a measure of the topography provides a quantitative assessment of a comet's erosional history, i.e. its evolutionary age. △ Less

Submitted 24 July, 2017; v1 submitted 3 July, 2017; originally announced July 2017.