-
Submitting surveys via a conversational interface: an evaluation of user acceptance and approach effectiveness
Authors:
Irene Celino,
Gloria Re Calegari
Abstract:
Conversational interfaces are currently on the rise: more and more applications rely on a chat-like interaction pattern to increase their acceptability and to improve user experience. Also in the area of questionnaire design and administration, interaction design is increasingly looked at as an important ingredient of a digital solution. For those reasons, we designed and developed a conversationa…
▽ More
Conversational interfaces are currently on the rise: more and more applications rely on a chat-like interaction pattern to increase their acceptability and to improve user experience. Also in the area of questionnaire design and administration, interaction design is increasingly looked at as an important ingredient of a digital solution. For those reasons, we designed and developed a conversational survey tool to administer questionnaires with a colloquial form through a chat-like Web interface.
In this paper, we present the evaluation results of our approach, taking into account both the user point of view - by assessing user acceptance and preferences in terms of survey compilation experience - and the survey design perspective - by investigating the effectiveness of a conversational survey in comparison to a traditional questionnaire. We show that users clearly appreciate the conversational form and prefer it over a traditional approach and that, from a data collection point of view, the conversational method shows the same reliability and a higher response quality with respect to a traditional questionnaire.
△ Less
Submitted 5 March, 2020;
originally announced March 2020.
-
Interplay of Game Incentives, Player Profiles and Task Difficulty in Games with a Purpose
Authors:
Gloria Re Calegari,
Irene Celino
Abstract:
How to take multiple factors into account when evaluating a Game with a Purpose? How is player behaviour or participation influenced by different incentives? How does player engagement impact their accuracy in solving tasks? In this paper, we present a detailed investigation of multiple factors affecting the evaluation of a GWAP and we show how they impact on the achieved results. We inform our st…
▽ More
How to take multiple factors into account when evaluating a Game with a Purpose? How is player behaviour or participation influenced by different incentives? How does player engagement impact their accuracy in solving tasks? In this paper, we present a detailed investigation of multiple factors affecting the evaluation of a GWAP and we show how they impact on the achieved results. We inform our study with the experimental assessment of a GWAP designed to solve a multinomial classification task.
△ Less
Submitted 20 November, 2018;
originally announced November 2018.
-
A Framework to build Games with a Purpose for Linked Data Refinement
Authors:
Gloria Re Calegari,
Andrea Fiano,
Irene Celino
Abstract:
With the rise of linked data and knowledge graphs, the need becomes compelling to find suitable solutions to increase the coverage and correctness of datasets, to add missing knowledge and to identify and remove errors. Several approaches - mostly relying on machine learning and NLP techniques - have been proposed to address this refinement goal; they usually need a partial gold standard, i.e. som…
▽ More
With the rise of linked data and knowledge graphs, the need becomes compelling to find suitable solutions to increase the coverage and correctness of datasets, to add missing knowledge and to identify and remove errors. Several approaches - mostly relying on machine learning and NLP techniques - have been proposed to address this refinement goal; they usually need a partial gold standard, i.e. some "ground truth" to train automatic models. Gold standards are manually constructed, either by involving domain experts or by adopting crowdsourcing and human computation solutions.
In this paper, we present an open source software framework to build Games with a Purpose for linked data refinement, i.e. web applications to crowdsource partial ground truth, by motivating user participation through fun incentive. We detail the impact of this new resource by explaining the specific data linking "purposes" supported by the framework (creation, ranking and validation of links) and by defining the respective crowdsourcing tasks to achieve those goals.
To show this resource's versatility, we describe a set of diverse applications that we built on top of it; to demonstrate its reusability and extensibility potential, we provide references to detailed documentation, including an entire tutorial which in a few hours guides new adopters to customize and adapt the framework to a new use case.
△ Less
Submitted 7 November, 2018;
originally announced November 2018.
-
An Incremental Truth Inference Approach to Aggregate Crowdsourcing Contributions in Games with a Purpose
Authors:
Irene Celino,
Gloria Re Calegari
Abstract:
We introduce our approach for incremental truth inference over the contributions provided by players of Games with a Purpose: we motivate the need for such a method with the specificity of GWAP vs. traditional crowdsourcing; we explain and formalize the proposed process and we explain its positive consequences; finally, we illustrate the results of an experimental comparison with state-of-the-art…
▽ More
We introduce our approach for incremental truth inference over the contributions provided by players of Games with a Purpose: we motivate the need for such a method with the specificity of GWAP vs. traditional crowdsourcing; we explain and formalize the proposed process and we explain its positive consequences; finally, we illustrate the results of an experimental comparison with state-of-the-art approaches, performed on data collected through two different GWAPs, thus showing the properties of our proposed framework.
△ Less
Submitted 25 October, 2018;
originally announced October 2018.
-
Single Transverse-Spin Asymmetries in Drell-Yan Processes
Authors:
Gabriele Re Calegari,
Philip G. Ratcliffe
Abstract:
The asymmetry in the angular distribution of Drell--Yan dilepton pairs generated in collisions where just one nucleon is transversely polarised has been examined in the literature with a variety of results, mainly differing by factors of two. We re-evaluate the asymmetry via twist-3 contributions in collinear factorisation. In order to allow complete and in-depth comparison with existing calculati…
▽ More
The asymmetry in the angular distribution of Drell--Yan dilepton pairs generated in collisions where just one nucleon is transversely polarised has been examined in the literature with a variety of results, mainly differing by factors of two. We re-evaluate the asymmetry via twist-3 contributions in collinear factorisation. In order to allow complete and in-depth comparison with existing calculations, we supply all calculational details.
△ Less
Submitted 21 February, 2014; v1 submitted 19 July, 2013;
originally announced July 2013.
