Showing 1–2 of 2 results for author: Czulo, O
-
The Case for Perspective in Multimodal Datasets
Authors:
Marcelo Viridiano,
Tiago Timponi Torrent,
Oliver Czulo,
Arthur Lorenzi Almeida,
Ely Edison da Silva Matos,
Frederico Belcavello
Abstract:
This paper argues in favor of the adoption of annotation practices for multimodal datasets that recognize and represent the inherently perspectivized nature of multimodal communication. To support our claim, we present a set of annotation experiments in which FrameNet annotation is applied to the Multi30k and the Flickr 30k Entities datasets. We assess the cosine similarity between the semantic re…
▽ More
This paper argues in favor of the adoption of annotation practices for multimodal datasets that recognize and represent the inherently perspectivized nature of multimodal communication. To support our claim, we present a set of annotation experiments in which FrameNet annotation is applied to the Multi30k and the Flickr 30k Entities datasets. We assess the cosine similarity between the semantic representations derived from the annotation of both pictures and captions for frames. Our findings indicate that: (i) frame semantic similarity between captions of the same picture produced in different languages is sensitive to whether the caption is a translation of another caption or not, and (ii) picture annotation for semantic frames is sensitive to whether the image is annotated in presence of a caption or not.
△ Less
Submitted 22 May, 2022;
originally announced May 2022.
-
Frame Shift Prediction
Authors:
Zheng-Xin Yong,
Patrick D. Watson,
Tiago Timponi Torrent,
Oliver Czulo,
Collin F. Baker
Abstract:
Frame shift is a cross-linguistic phenomenon in translation which results in corresponding pairs of linguistic material evoking different frames. The ability to predict frame shifts enables automatic creation of multilingual FrameNets through annotation projection. Here, we propose the Frame Shift Prediction task and demonstrate that graph attention networks, combined with auxiliary training, can…
▽ More
Frame shift is a cross-linguistic phenomenon in translation which results in corresponding pairs of linguistic material evoking different frames. The ability to predict frame shifts enables automatic creation of multilingual FrameNets through annotation projection. Here, we propose the Frame Shift Prediction task and demonstrate that graph attention networks, combined with auxiliary training, can learn cross-linguistic frame-to-frame correspondence and predict frame shifts.
△ Less
Submitted 5 January, 2022;
originally announced January 2022.