-
Improving Knowledge Distillation in Transfer Learning with Layer-wise Learning Rates
Authors:
Shirley Kokane,
Mostofa Rafid Uddin,
Min Xu
Abstract:
Transfer learning methods start performing poorly when the complexity of the learning task is increased. Most of these methods calculate the cumulative differences of all the matched features and then use them to back-propagate that loss through all the layers. Contrary to these methods, in this work, we propose a novel layer-wise learning scheme that adjusts learning parameters per layer as a fun…
▽ More
Transfer learning methods start performing poorly when the complexity of the learning task is increased. Most of these methods calculate the cumulative differences of all the matched features and then use them to back-propagate that loss through all the layers. Contrary to these methods, in this work, we propose a novel layer-wise learning scheme that adjusts learning parameters per layer as a function of the differences in the Jacobian/Attention/Hessian of the output activations w.r.t. the network parameters. We applied this novel scheme for attention map-based and derivative-based (first and second order) transfer learning methods. We received improved learning performance and stability against a wide range of datasets. From extensive experimental evaluation, we observed that the performance boost achieved by our method becomes more significant with the increasing difficulty of the learning task.
△ Less
Submitted 5 July, 2024;
originally announced July 2024.
-
DualContrast: Unsupervised Disentangling of Content and Transformations with Implicit Parameterization
Authors:
Mostofa Rafid Uddin,
Min Xu
Abstract:
Unsupervised disentanglement of content and transformation has recently drawn much research, given their efficacy in solving downstream unsupervised tasks like clustering, alignment, and shape analysis. This problem is particularly important for analyzing shape-focused real-world scientific image datasets, given their significant relevance to downstream tasks. The existing works address the proble…
▽ More
Unsupervised disentanglement of content and transformation has recently drawn much research, given their efficacy in solving downstream unsupervised tasks like clustering, alignment, and shape analysis. This problem is particularly important for analyzing shape-focused real-world scientific image datasets, given their significant relevance to downstream tasks. The existing works address the problem by explicitly parameterizing the transformation factors, significantly reducing their expressiveness. Moreover, they are not applicable in cases where transformations can not be readily parametrized. An alternative to such explicit approaches is self-supervised methods with data augmentation, which implicitly disentangles transformations and content. We demonstrate that the existing self-supervised methods with data augmentation result in the poor disentanglement of content and transformations in real-world scenarios. Therefore, we developed a novel self-supervised method, DualContrast, specifically for unsupervised disentanglement of content and transformations in shape-focused image datasets. Our extensive experiments showcase the superiority of DualContrast over existing self-supervised and explicit parameterization approaches. We leveraged DualContrast to disentangle protein identities and protein conformations in cellular 3D protein images. Moreover, we also disentangled transformations in MNIST, viewpoint in the Linemod Object dataset, and human movement deformation in the Starmen dataset as transformations using DualContrast.
△ Less
Submitted 26 May, 2024;
originally announced May 2024.
-
Image Data collection and implementation of deep learning-based model in detecting Monkeypox disease using modified VGG16
Authors:
Md Manjurul Ahsan,
Muhammad Ramiz Uddin,
Mithila Farjana,
Ahmed Nazmus Sakib,
Khondhaker Al Momin,
Shahana Akter Luna
Abstract:
While the world is still attempting to recover from the damage caused by the broad spread of COVID-19, the Monkeypox virus poses a new threat of becoming a global pandemic. Although the Monkeypox virus itself is not deadly and contagious as COVID-19, still every day, new patients case has been reported from many nations. Therefore, it will be no surprise if the world ever faces another global pand…
▽ More
While the world is still attempting to recover from the damage caused by the broad spread of COVID-19, the Monkeypox virus poses a new threat of becoming a global pandemic. Although the Monkeypox virus itself is not deadly and contagious as COVID-19, still every day, new patients case has been reported from many nations. Therefore, it will be no surprise if the world ever faces another global pandemic due to the lack of proper precautious steps. Recently, Machine learning (ML) has demonstrated huge potential in image-based diagnoses such as cancer detection, tumor cell identification, and COVID-19 patient detection. Therefore, a similar application can be adopted to diagnose the Monkeypox-related disease as it infected the human skin, which image can be acquired and further used in diagnosing the disease. Considering this opportunity, in this work, we introduce a newly developed "Monkeypox2022" dataset that is publicly available to use and can be obtained from our shared GitHub repository. The dataset is created by collecting images from multiple open-source and online portals that do not impose any restrictions on use, even for commercial purposes, hence giving a safer path to use and disseminate such data when constructing and deploying any type of ML model. Further, we propose and evaluate a modified VGG16 model, which includes two distinct studies: Study One and Two. Our exploratory computational results indicate that our suggested model can identify Monkeypox patients with an accuracy of $97\pm1.8\%$ (AUC=97.2) and $88\pm0.8\%$ (AUC=0.867) for Study One and Two, respectively. Additionally, we explain our model's prediction and feature extraction utilizing Local Interpretable Model-Agnostic Explanations (LIME) help to a deeper insight into specific features that characterize the onset of the Monkeypox virus.
△ Less
Submitted 3 June, 2022;
originally announced June 2022.
-
Monkeypox Image Data collection
Authors:
Md Manjurul Ahsan,
Muhammad Ramiz Uddin,
Shahana Akter Luna
Abstract:
This paper explains the initial Monkeypox Open image data collection procedure. It was created by assembling images collected from websites, newspapers, and online portals and currently contains around 1905 images after data augmentation.
This paper explains the initial Monkeypox Open image data collection procedure. It was created by assembling images collected from websites, newspapers, and online portals and currently contains around 1905 images after data augmentation.
△ Less
Submitted 3 June, 2022;
originally announced June 2022.
-
Cryo-shift: Reducing domain shift in cryo-electron subtomograms with unsupervised domain adaptation and randomization
Authors:
Hmrishav Bandyopadhyay,
Zihao Deng,
Leiting Ding,
Sinuo Liu,
Mostofa Rafid Uddin,
Xiangrui Zeng,
Sima Behpour,
Min Xu
Abstract:
Cryo-Electron Tomography (cryo-ET) is a 3D imaging technology that enables the visualization of subcellular structures in situ at near-atomic resolution. Cellular cryo-ET images help in resolving the structures of macromolecules and determining their spatial relationship in a single cell, which has broad significance in cell and structural biology. Subtomogram classification and recognition consti…
▽ More
Cryo-Electron Tomography (cryo-ET) is a 3D imaging technology that enables the visualization of subcellular structures in situ at near-atomic resolution. Cellular cryo-ET images help in resolving the structures of macromolecules and determining their spatial relationship in a single cell, which has broad significance in cell and structural biology. Subtomogram classification and recognition constitute a primary step in the systematic recovery of these macromolecular structures. Supervised deep learning methods have been proven to be highly accurate and efficient for subtomogram classification, but suffer from limited applicability due to scarcity of annotated data. While generating simulated data for training supervised models is a potential solution, a sizeable difference in the image intensity distribution in generated data as compared to real experimental data will cause the trained models to perform poorly in predicting classes on real subtomograms. In this work, we present Cryo-Shift, a fully unsupervised domain adaptation and randomization framework for deep learning-based cross-domain subtomogram classification. We use unsupervised multi-adversarial domain adaption to reduce the domain shift between features of simulated and experimental data. We develop a network-driven domain randomization procedure with `warp' modules to alter the simulated data and help the classifier generalize better on experimental data. We do not use any labeled experimental data to train our model, whereas some of the existing alternative approaches require labeled experimental samples for cross-domain classification. Nevertheless, Cryo-Shift outperforms the existing alternative approaches in cross-domain subtomogram classification in extensive evaluation studies demonstrated herein using both simulated and experimental data.
△ Less
Submitted 17 November, 2021;
originally announced November 2021.
-
A New Compression Based Index Structure for Efficient Information Retrieval
Authors:
Md. Abdullah al Mamun,
Md. Hanif,
Md. Rakib Uddin,
Tanvir Ahmed,
Md. Mofizul Islam
Abstract:
Finding desired information from large data set is a difficult problem. Information retrieval is concerned with the structure, analysis, organization, storage, searching, and retrieval of information. Index is the main constituent of an IR system. Now a day exponential growth of information makes the index structure large enough affecting the IR system's quality. So compressing the Index structure…
▽ More
Finding desired information from large data set is a difficult problem. Information retrieval is concerned with the structure, analysis, organization, storage, searching, and retrieval of information. Index is the main constituent of an IR system. Now a day exponential growth of information makes the index structure large enough affecting the IR system's quality. So compressing the Index structure is our main contribution in this paper. We compressed the document number in inverted file entries using a new coding technique based on run-length encoding. Our coding mechanism uses a specified code which acts over run-length coding. We experimented and found that our coding mechanism on an average compresses 67.34% percent more than the other techniques.
△ Less
Submitted 24 September, 2012;
originally announced September 2012.
-
Automatic Electric Meter Reading System: A Cost-Feasible Alternative Approach In Meter Reading For Bangladesh Perspective Using Low-Cost Digital Wattmeter And Wimax Technology
Authors:
Tanvir Ahmed,
Md. Suzan Miah,
Md. Manirul Islam,
Md. Rakib Uddin
Abstract:
Energy meter reading is a monotonous and an expensive task. Now the meter reader people goes to each meter and take the meter reading manually to issue the bill which will later be entered in the billing software for billing and payment automation. If the manual meter reading and bill data entry process can be automated then it would reduced the laborious task and financial wastage. "Automatic Ele…
▽ More
Energy meter reading is a monotonous and an expensive task. Now the meter reader people goes to each meter and take the meter reading manually to issue the bill which will later be entered in the billing software for billing and payment automation. If the manual meter reading and bill data entry process can be automated then it would reduced the laborious task and financial wastage. "Automatic Electric Meter Reading (AMR) System" is a metering system that is to be used for data collecting from the meter and processing the collected data for billing and other decision purposes. In this paper we have proposed an automatic meter reading system which is low cost, high performance, highest data rate, highest coverage area and most appropriate for Bangladesh perspective. In this AMR system there are four basic units. They are reading unit, communication unit, data receiving and processing unit and billing system. For reading unit we identified the disk rotation of the energy meter and stored the data in microcontroller. So it is not required to change the current analog energy meter. An external module will be added with the current energy meter. In the communication unit Wimax transceiver was used for wireless communication between meter end and the server end because of its wide coverage area. In the data receiving and processing unit meter reading will be collected from the transceiver which is controlled by another microcontroller. There will be a computer application that will take the data from the microcontroller. This will also help to avoid any tampering or break down of energy meter. There are various AMR system exists all over the world. Those systems were analyzed and we found they are not feasible for Bangladesh.
△ Less
Submitted 24 September, 2012;
originally announced September 2012.
