-
DatUS^2: Data-driven Unsupervised Semantic Segmentation with Pre-trained Self-supervised Vision Transformer
Authors:
Sonal Kumar,
Arijit Sur,
Rashmi Dutta Baruah
Abstract:
Successive proposals of several self-supervised training schemes continue to emerge, taking one step closer to develo** a universal foundation model. In this process, the unsupervised downstream tasks are recognized as one of the evaluation methods to validate the quality of visual features learned with a self-supervised training scheme. However, unsupervised dense semantic segmentation has not…
▽ More
Successive proposals of several self-supervised training schemes continue to emerge, taking one step closer to develo** a universal foundation model. In this process, the unsupervised downstream tasks are recognized as one of the evaluation methods to validate the quality of visual features learned with a self-supervised training scheme. However, unsupervised dense semantic segmentation has not been explored as a downstream task, which can utilize and evaluate the quality of semantic information introduced in patch-level feature representations during self-supervised training of a vision transformer. Therefore, this paper proposes a novel data-driven approach for unsupervised semantic segmentation (DatUS^2) as a downstream task. DatUS^2 generates semantically consistent and dense pseudo annotate segmentation masks for the unlabeled image dataset without using any visual-prior or synchronized data. We compare these pseudo-annotated segmentation masks with ground truth masks for evaluating recent self-supervised training schemes to learn shared semantic properties at the patch level and discriminative semantic properties at the segment level. Finally, we evaluate existing state-of-the-art self-supervised training schemes with our proposed downstream task, i.e., DatUS^2. Also, the best version of DatUS^2 outperforms the existing state-of-the-art method for the unsupervised dense semantic segmentation task with 15.02% MiOU and 21.47% Pixel accuracy on the SUIM dataset. It also achieves a competitive level of accuracy for a large-scale and complex dataset, i.e., the COCO dataset.
△ Less
Submitted 23 January, 2024;
originally announced January 2024.
-
Exploiting Language Relatedness in Machine Translation Through Domain Adaptation Techniques
Authors:
Amit Kumar,
Rupjyoti Baruah,
Ajay Pratap,
Mayank Swarnkar,
Anil Kumar Singh
Abstract:
One of the significant challenges of Machine Translation (MT) is the scarcity of large amounts of data, mainly parallel sentence aligned corpora. If the evaluation is as rigorous as resource-rich languages, both Neural Machine Translation (NMT) and Statistical Machine Translation (SMT) can produce good results with such large amounts of data. However, it is challenging to improve the quality of MT…
▽ More
One of the significant challenges of Machine Translation (MT) is the scarcity of large amounts of data, mainly parallel sentence aligned corpora. If the evaluation is as rigorous as resource-rich languages, both Neural Machine Translation (NMT) and Statistical Machine Translation (SMT) can produce good results with such large amounts of data. However, it is challenging to improve the quality of MT output for low resource languages, especially in NMT and SMT. In order to tackle the challenges faced by MT, we present a novel approach of using a scaled similarity score of sentences, especially for related languages based on a 5-gram KenLM language model with Kneser-ney smoothing technique for filtering in-domain data from out-of-domain corpora that boost the translation quality of MT. Furthermore, we employ other domain adaptation techniques such as multi-domain, fine-tuning and iterative back-translation approach to compare our novel approach on the Hindi-Nepali language pair for NMT and SMT. Our approach succeeds in increasing ~2 BLEU point on multi-domain approach, ~3 BLEU point on fine-tuning for NMT and ~2 BLEU point on iterative back-translation approach.
△ Less
Submitted 3 March, 2023;
originally announced March 2023.
-
Explicit Context Integrated Recurrent Neural Network for Sensor Data Applications
Authors:
Rashmi Dutta Baruah,
Mario Muñoz Organero
Abstract:
The development and progress in sensor, communication and computing technologies have led to data rich environments. In such environments, data can easily be acquired not only from the monitored entities but also from the surroundings where the entity is operating. The additional data that are available from the problem domain, which cannot be used independently for learning models, constitute con…
▽ More
The development and progress in sensor, communication and computing technologies have led to data rich environments. In such environments, data can easily be acquired not only from the monitored entities but also from the surroundings where the entity is operating. The additional data that are available from the problem domain, which cannot be used independently for learning models, constitute context. Such context, if taken into account while learning, can potentially improve the performance of predictive models. Typically, the data from various sensors are present in the form of time series. Recurrent Neural Networks (RNNs) are preferred for such data as it can inherently handle temporal context. However, the conventional RNN models such as Elman RNN, Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU) in their present form do not provide any mechanism to integrate explicit contexts. In this paper, we propose a Context Integrated RNN (CiRNN) that enables integrating explicit contexts represented in the form of contextual features. In CiRNN, the network weights are influenced by contextual features in such a way that the primary input features which are more relevant to a given context are given more importance. To show the efficacy of CiRNN, we selected an application domain, engine health prognostics, which captures data from various sensors and where contextual information is available. We used the NASA Turbofan Engine Degradation Simulation dataset for estimating Remaining Useful Life (RUL) as it provides contextual information. We compared CiRNN with baseline models as well as the state-of-the-art methods. The experimental results show an improvement of 39% and 87% respectively, over state-of-the art models, when performance is measured with RMSE and score from an asymmetric scoring function. The latter measure is specific to the task of RUL estimation.
△ Less
Submitted 12 January, 2023;
originally announced January 2023.
-
Enhancing Dictionary Based Preprocessing For Better Text Compression
Authors:
R. R. Baruah,
V. Deka,
M. P. Bhuyan
Abstract:
With the rapid growing of data and number of applications, there is a crucial need of dictionary based reversible transformation techniques to increase the efficiency of the compression algorithms and hence contribute towards the enhancement in compression ratio. Performance analysis of compression methods in combination with the various transformation techniques is obtained for different text fil…
▽ More
With the rapid growing of data and number of applications, there is a crucial need of dictionary based reversible transformation techniques to increase the efficiency of the compression algorithms and hence contribute towards the enhancement in compression ratio. Performance analysis of compression methods in combination with the various transformation techniques is obtained for different text files of varying sizes. The popular block sorting lossless Burrows Wheeler Compression Algorithm (BWCA) is implemented along with one proposed method. For efficient compression a dictionary based transformation algorithm is also developed. It is observed that much increase in terms of compression ratio is attained when a source file is preprocessed with dictionary and then applied to BWCA and the proposed method.
△ Less
Submitted 19 March, 2014;
originally announced March 2014.
-
Design of A Low Power Low Voltage CMOS Opamp
Authors:
Ratul Kr. Baruah
Abstract:
In this paper a CMOS operational amplifier is presented which operates at 2V power supply and 1microA input bias current at 0.8 micron technology using non conventional mode of operation of MOS transistors and whose input is depended on bias current. The unique behaviour of the MOS transistors in subthreshold region not only allows a designer to work at low input bias current but also at low volta…
▽ More
In this paper a CMOS operational amplifier is presented which operates at 2V power supply and 1microA input bias current at 0.8 micron technology using non conventional mode of operation of MOS transistors and whose input is depended on bias current. The unique behaviour of the MOS transistors in subthreshold region not only allows a designer to work at low input bias current but also at low voltage. While operating the device at weak inversion results low power dissipation but dynamic range is degraded. Optimum balance between power dissipation and dynamic range results when the MOS transistors are operated at moderate inversion. Power is again minimised by the application of input dependant bias current using feedback loops in the input transistors of the differential pair with two current substractors. In comparison with the reported low power low voltage opamps at 0.8 micron technology, this opamp has very low standby power consumption with a high driving capability and operates at low voltage. The opamp is fairly small (0.0084 mm 2) and slew rate is more than other low power low voltage opamps reported at 0.8 um technology [1,2]. Vittoz at al [3] reported that slew rate can be improved by adaptive biasing technique and power dissipation can be reduced by operating the device in weak inversion. Though lower power dissipation is achieved the area required by the circuit is very large and speed is too small. So, operating the device in moderate inversion is a good solution. Also operating the device in subthreshold region not only allows lower power dissipation but also a lower voltage operation is achieved.
△ Less
Submitted 29 March, 2010;
originally announced March 2010.
