Veagle: Advancements in Multimodal Representation Learning
Authors:
Rajat Chawla,
Arkajit Datta,
Tushar Verma,
Adarsh Jha,
Anmol Gautam,
Ayush Vatsal,
Sukrit Chaterjee,
Mukunda NS,
Ishaan Bhola
Abstract:
Lately, researchers in artificial intelligence have been really interested in how language and vision come together, giving rise to the development of multimodal models that aim to seamlessly integrate textual and visual information. Multimodal models, an extension of Large Language Models (LLMs), have exhibited remarkable capabilities in addressing a diverse array of tasks, ranging from image cap…
▽ More
Lately, researchers in artificial intelligence have been really interested in how language and vision come together, giving rise to the development of multimodal models that aim to seamlessly integrate textual and visual information. Multimodal models, an extension of Large Language Models (LLMs), have exhibited remarkable capabilities in addressing a diverse array of tasks, ranging from image captioning and visual question answering (VQA) to visual grounding. While these models have showcased significant advancements, challenges persist in accurately interpreting images and answering the question, a common occurrence in real-world scenarios. This paper introduces a novel approach to enhance the multimodal capabilities of existing models. In response to the limitations observed in current Vision Language Models (VLMs) and Multimodal Large Language Models (MLLMs), our proposed model Veagle, incorporates a unique mechanism inspired by the successes and insights of previous works. Veagle leverages a dynamic mechanism to project encoded visual information directly into the language model. This dynamic approach allows for a more nuanced understanding of intricate details present in visual contexts. To validate the effectiveness of Veagle, we conduct comprehensive experiments on benchmark datasets, emphasizing tasks such as visual question answering and image understanding. Our results indicate a improvement of 5-6 \% in performance, with Veagle outperforming existing models by a notable margin. The outcomes underscore the model's versatility and applicability beyond traditional benchmarks.
△ Less
Submitted 18 January, 2024;
originally announced March 2024.
Measurements of Solar Differential Rotation Using the Century Long Kodaikanal Sunspot Data
Authors:
Bibhuti Kumar Jha,
Aditya Priyadarshi,
Sudip Mandal,
Subhamoy Chaterjee,
Dipankar Banerjee
Abstract:
The rotational profile of the Sun is considered to be one of the key inputs in a solar dynamo model. Hence, precise and long-term measurements of this quantity is important for our understanding of solar magnetism and its variability. In this study, we use the newly digitized, white light sunspot data (1923 -- 2011) from Kodaikanal Solar Observatory (KoSO) to derive the solar rotation profile. An…
▽ More
The rotational profile of the Sun is considered to be one of the key inputs in a solar dynamo model. Hence, precise and long-term measurements of this quantity is important for our understanding of solar magnetism and its variability. In this study, we use the newly digitized, white light sunspot data (1923 -- 2011) from Kodaikanal Solar Observatory (KoSO) to derive the solar rotation profile. An automated correlation based sunspot tracking algorithm is implemented to measure the rotation parameters, $A$, the equatorial rotation rate and $B$, the latitudinal gradient. Our measurements of $A=14.381\pm0.004$ and $B=-2.72\pm0.04$ compare well with previous studies. In our analysis, we find that the bigger sunspots (with area $>$400~$μ$Hem) rotate slower than the smaller ones. At the same time, we do not find any variation in the rotation rates between activity extremes, i.e solar maxima and minima. Lastly, we employ our tracking algorithm on the Michelson Doppler Imager (MDI) data and compare the MDI results with our KoSO values.
△ Less
Submitted 6 January, 2021;
originally announced January 2021.
