-
VASTA: A Vision and Language-assisted Smartphone Task Automation System
Authors:
Alborz Rezazadeh Sereshkeh,
Gary Leung,
Krish Perumal,
Caleb Phillips,
Minfan Zhang,
Afsaneh Fazly,
Iqbal Mohomed
Abstract:
We present VASTA, a novel vision and language-assisted Programming By Demonstration (PBD) system for smartphone task automation. Development of a robust PBD automation system requires overcoming three key challenges: first, how to make a particular demonstration robust to positional and visual changes in the user interface (UI) elements; secondly, how to recognize changes in the automation paramet…
▽ More
We present VASTA, a novel vision and language-assisted Programming By Demonstration (PBD) system for smartphone task automation. Development of a robust PBD automation system requires overcoming three key challenges: first, how to make a particular demonstration robust to positional and visual changes in the user interface (UI) elements; secondly, how to recognize changes in the automation parameters to make the demonstration as generalizable as possible; and thirdly, how to recognize from the user utterance what automation the user wishes to carry out. To address the first challenge, VASTA leverages state-of-the-art computer vision techniques, including object detection and optical character recognition, to accurately label interactions demonstrated by a user, without relying on the underlying UI structures. To address the second and third challenges, VASTA takes advantage of advanced natural language understanding algorithms for analyzing the user utterance to trigger the VASTA automation scripts, and to determine the automation parameters for generalization. We run an initial user study that demonstrates the effectiveness of VASTA at clustering user utterances, understanding changes in the automation parameters, detecting desired UI elements, and, most importantly, automating various tasks. A demo video of the system is available here: http://y2u.be/kr2xE-FixjI
△ Less
Submitted 4 November, 2019;
originally announced November 2019.
-
Semi-supervised and Unsupervised Methods for Categorizing Posts in Web Discussion Forums
Authors:
Krish Perumal
Abstract:
Web discussion forums are used by millions of people worldwide to share information belonging to a variety of domains such as automotive vehicles, pets, sports, etc. They typically contain posts that fall into different categories such as problem, solution, feedback, spam, etc. Automatic identification of these categories can aid information retrieval that is tailored for specific user requirement…
▽ More
Web discussion forums are used by millions of people worldwide to share information belonging to a variety of domains such as automotive vehicles, pets, sports, etc. They typically contain posts that fall into different categories such as problem, solution, feedback, spam, etc. Automatic identification of these categories can aid information retrieval that is tailored for specific user requirements. Previously, a number of supervised methods have attempted to solve this problem; however, these depend on the availability of abundant training data. A few existing unsupervised and semi-supervised approaches are either focused on identifying a single category or do not report category-specific performance. In contrast, this work proposes unsupervised and semi-supervised methods that require no or minimal training data to achieve this objective without compromising on performance. A fine-grained analysis is also carried out to discuss their limitations. The proposed methods are based on sequence models (specifically, Hidden Markov Models) that can model language for each category using word and part-of-speech probability distributions, and manually specified features. Empirical evaluations across domains demonstrate that the proposed methods are better suited for this task than existing ones.
△ Less
Submitted 24 April, 2016; v1 submitted 31 March, 2016;
originally announced April 2016.
-
A hybrid approach based segmentation technique for brain tumor in MRI Images
Authors:
D. Anithadevi,
K. Perumal
Abstract:
Automatic image segmentation becomes very crucial for tumor detection in medical image processing.In general, manual and semi automatic segmentation techniques require more time and knowledge. However these drawbacks had overcome by automatic segmentation still there needs to develop more appropriate techniques for medical image segmentation. Therefore, we proposed hybrid approach based image segm…
▽ More
Automatic image segmentation becomes very crucial for tumor detection in medical image processing.In general, manual and semi automatic segmentation techniques require more time and knowledge. However these drawbacks had overcome by automatic segmentation still there needs to develop more appropriate techniques for medical image segmentation. Therefore, we proposed hybrid approach based image segmentation using the combined features of region growing and threshold based segmentation techniques. It is followed by pre-processing stage to provide an accurate brain tumor extraction by the help of Magnetic Resonance Imaging (MRI). If the tumor has holes, the region growing segmentation algorithm cannot reveal but the proposed hybrid segmentation technique can be achieved and the result as well improved. Hence the result used to made assessment with the various performance measures as DICE, JACCARD similarity, accuracy, sensitivity and specificity. These similarity measures have been extensively used for evaluation with the ground truth of each processed image and its results are compared and analyzed.
△ Less
Submitted 8 March, 2016;
originally announced March 2016.
-
Supervised Classification Performance of Multispectral Images
Authors:
K. Perumal,
R. Bhaskaran
Abstract:
Nowadays government and private agencies use remote sensing imagery for a wide range of applications from military applications to farm development. The images may be a panchromatic, multispectral, hyperspectral or even ultraspectral of terra bytes. Remote sensing image classification is one amongst the most significant application worlds for remote sensing. A few number of image classification…
▽ More
Nowadays government and private agencies use remote sensing imagery for a wide range of applications from military applications to farm development. The images may be a panchromatic, multispectral, hyperspectral or even ultraspectral of terra bytes. Remote sensing image classification is one amongst the most significant application worlds for remote sensing. A few number of image classification algorithms have proved good precision in classifying remote sensing data. But, of late, due to the increasing spatiotemporal dimensions of the remote sensing data, traditional classification algorithms have exposed weaknesses necessitating further research in the field of remote sensing image classification. So an efficient classifier is needed to classify the remote sensing images to extract information. We are experimenting with both supervised and unsupervised classification. Here we compare the different classification methods and their performances. It is found that Mahalanobis classifier performed the best in our classification.
△ Less
Submitted 21 February, 2010;
originally announced February 2010.
