-
Transformer Architecture for NetsDB
Authors:
Subodh Kamble,
Kunal Sunil Kasodekar
Abstract:
Transformers models have become the backbone of the current state-of-the-art models in language, vision, and multimodal domains. These models, at their core, utilize multi-head self-attention to selectively aggregate context, generating dynamic contextual embeddings and modeling long-range dependencies for a clear contextual understanding. Lixi et al. \cite{zhou2022serving} proposed a method to us…
▽ More
Transformers models have become the backbone of the current state-of-the-art models in language, vision, and multimodal domains. These models, at their core, utilize multi-head self-attention to selectively aggregate context, generating dynamic contextual embeddings and modeling long-range dependencies for a clear contextual understanding. Lixi et al. \cite{zhou2022serving} proposed a method to use relational databases for deploying large-scale deep learning models and created an open-source implementation called NetsDB for the same. We build upon the previous work of these authors by creating an end-to-end implementation of the Encoder part of the transformer for model serving in NetsDB. Specifically, we construct a two-block encoder that includes Multi-Head Attention and its accompanying self-attention mechanism, Layer-Norm, Dropout, FeedForward Layers, and the necessary residual connections. We load out weights from our model for distributed processing, deployment, and efficient inferencing. To prove the efficacy of our implementation, we conduct a comprehensive performance analysis by comparing it with existing implementations in PyTorch, Tensorflow, Flax, and MxNet across key metrics such as inference time and model size.
△ Less
Submitted 9 May, 2024; v1 submitted 8 May, 2024;
originally announced May 2024.
-
Time Efficient Training of Progressive Generative Adversarial Network using Depthwise Separable Convolution and Super Resolution Generative Adversarial Network
Authors:
Atharva Karwande,
Pranesh Kulkarni,
Tejas Kolhe,
Akshay Joshi,
Soham Kamble
Abstract:
Generative Adversarial Networks have been employed successfully to generate high-resolution augmented images of size 1024^2. Although the augmented images generated are unprecedented, the training time of the model is exceptionally high. Conventional GAN requires training of both Discriminator as well as the Generator. In Progressive GAN, which is the current state-of-the-art GAN for image augment…
▽ More
Generative Adversarial Networks have been employed successfully to generate high-resolution augmented images of size 1024^2. Although the augmented images generated are unprecedented, the training time of the model is exceptionally high. Conventional GAN requires training of both Discriminator as well as the Generator. In Progressive GAN, which is the current state-of-the-art GAN for image augmentation, instead of training the GAN all at once, a new concept of progressing growing of Discriminator and Generator simultaneously, was proposed. Although the lower stages such as 4x4 and 8x8 train rather quickly, the later stages consume a tremendous amount of time which could take days to finish the model training. In our paper, we propose a novel pipeline that combines Progressive GAN with slight modifications and Super Resolution GAN. Super Resolution GAN up samples low-resolution images to high-resolution images which can prove to be a useful resource to reduce the training time exponentially.
△ Less
Submitted 24 February, 2022;
originally announced February 2022.
-
Plant Disease Detection Using Image Processing and Machine Learning
Authors:
Pranesh Kulkarni,
Atharva Karwande,
Tejas Kolhe,
Soham Kamble,
Akshay Joshi,
Medha Wyawahare
Abstract:
One of the important and tedious task in agricultural practices is the detection of the disease on crops. It requires huge time as well as skilled labor. This paper proposes a smart and efficient technique for detection of crop disease which uses computer vision and machine learning techniques. The proposed system is able to detect 20 different diseases of 5 common plants with 93% accuracy.
One of the important and tedious task in agricultural practices is the detection of the disease on crops. It requires huge time as well as skilled labor. This paper proposes a smart and efficient technique for detection of crop disease which uses computer vision and machine learning techniques. The proposed system is able to detect 20 different diseases of 5 common plants with 93% accuracy.
△ Less
Submitted 22 November, 2021; v1 submitted 20 June, 2021;
originally announced June 2021.
-
Hate Speech Detection from Code-mixed Hindi-English Tweets Using Deep Learning Models
Authors:
Satyajit Kamble,
Aditya Joshi
Abstract:
This paper reports an increment to the state-of-the-art in hate speech detection for English-Hindi code-mixed tweets. We compare three typical deep learning models using domain-specific embeddings. On experimenting with a benchmark dataset of English-Hindi code-mixed tweets, we observe that using domain-specific embeddings results in an improved representation of target groups, and an improved F-s…
▽ More
This paper reports an increment to the state-of-the-art in hate speech detection for English-Hindi code-mixed tweets. We compare three typical deep learning models using domain-specific embeddings. On experimenting with a benchmark dataset of English-Hindi code-mixed tweets, we observe that using domain-specific embeddings results in an improved representation of target groups, and an improved F-score.
△ Less
Submitted 13 November, 2018;
originally announced November 2018.
