-
Examining the role of context in statistical literacy outcomes using an isomorphic assessment instrument
Authors:
Sayali Phadke,
Matthew D Beckman,
Kari Lock Morgan
Abstract:
The central role of statistical literacy has been discussed extensively, emphasizing its importance as a learning outcome and in promoting a citizenry capable of interacting with the world in an informed and critical manner. Our work contributes to the growing literature on assessing and improving people's statistical literacy vis-a-vis contexts important in their professional and personal lives.…
▽ More
The central role of statistical literacy has been discussed extensively, emphasizing its importance as a learning outcome and in promoting a citizenry capable of interacting with the world in an informed and critical manner. Our work contributes to the growing literature on assessing and improving people's statistical literacy vis-a-vis contexts important in their professional and personal lives. We consider the measurement of contextualized statistics literacy - statistical literacy as applied to relevant contexts. We discuss the development of an isomorphic instrument modifying an existing assessment, design of a pilot study, and results which conclude that 1) the isomorphic assessment has comparable psychometric properties, and 2) test takers have lower statistical literacy scores on an assessment that incorporates relevant contexts.
△ Less
Submitted 11 May, 2022;
originally announced May 2022.
-
Foundations for NLP-assisted formative assessment feedback for short-answer tasks in large-enrollment classes
Authors:
Susan Lloyd,
Matthew Beckman,
Dennis Pearl,
Rebecca Passonneau,
Zhaohui Li,
Zekun Wang
Abstract:
Research suggests "write-to-learn" tasks improve learning outcomes, yet constructed-response methods of formative assessment become unwieldy with large class sizes. This study evaluates natural language processing algorithms to assist this aim. Six short-answer tasks completed by 1,935 students were scored by several human raters, using a detailed rubric, and an algorithm. Results indicate substan…
▽ More
Research suggests "write-to-learn" tasks improve learning outcomes, yet constructed-response methods of formative assessment become unwieldy with large class sizes. This study evaluates natural language processing algorithms to assist this aim. Six short-answer tasks completed by 1,935 students were scored by several human raters, using a detailed rubric, and an algorithm. Results indicate substantial inter-rater agreement using quadratic weighted kappa for rater pairs (each QWK > 0.74) and group consensus (Fleiss Kappa = 0.68). Additionally, intra-rater agreement was estimated for one rater who had scored 178 responses seven years prior (QWK = 0.89). With compelling rater agreement, the study then pilots cluster analysis of response text toward enabling instructors to ascribe meaning to clusters as a means for scalable formative assessment.
△ Less
Submitted 5 May, 2022;
originally announced May 2022.
-
Implementing version control with Git and GitHub as a learning objective in statistics and data science courses
Authors:
Matthew D. Beckman,
Mine Çetinkaya-Rundel,
Nicholas J. Horton,
Colin W. Rundel,
Adam J. Sullivan,
Maria Tackett
Abstract:
A version control system records changes to a file or set of files over time so that changes can be tracked and specific versions of a file can be recalled later. As such, it is an essential element of a reproducible workflow that deserves due consideration among the learning objectives of statistics and data science courses. This paper describes experiences and implementation decisions of four co…
▽ More
A version control system records changes to a file or set of files over time so that changes can be tracked and specific versions of a file can be recalled later. As such, it is an essential element of a reproducible workflow that deserves due consideration among the learning objectives of statistics and data science courses. This paper describes experiences and implementation decisions of four contributing faculty who are teaching different courses at a variety of institutions. Each of these faculty have set version control as a learning objective and successfully integrated one such system (Git) into one or more statistics courses. The various approaches described in the paper span different implementation strategies to suit student background, course type, software choices, and assessment practices. By presenting a wide range of approaches to teaching Git, the paper aims to serve as a resource for statistics and data science instructors teaching courses at any level within an undergraduate or graduate curriculum.
△ Less
Submitted 4 November, 2020; v1 submitted 7 January, 2020;
originally announced January 2020.
-
Statistics students' identification of inferential model elements within contexts of their own invention
Authors:
Matthew Beckman,
Robert delMas
Abstract:
Statistical thinking partially depends upon an iterative process by which essential features of a problem setting are identified and mapped onto an abstract model or archetype, and then translated back into the context of the original problem setting (Wild and Pfannkuch 1999). Assessment in introductory statistics often relies on tasks that present students with data in context and expects them to…
▽ More
Statistical thinking partially depends upon an iterative process by which essential features of a problem setting are identified and mapped onto an abstract model or archetype, and then translated back into the context of the original problem setting (Wild and Pfannkuch 1999). Assessment in introductory statistics often relies on tasks that present students with data in context and expects them to choose and describe an appropriate model. This study explores post-secondary student responses to an alternative task that prompts students to clearly identify a sample, population, statistic, and parameter using a context of their own invention. The data include free text narrative responses of a random sample of 500 students from a sample of more than 1600 introductory statistics students. Results suggest that students' responses often portrayed sample and population accurately. Portrayals of statistic and parameter were less reliable and were associated with descriptions of a wide variety of other concepts. Responses frequently attributed a variable of some kind to the statistic, or a study design detail to the parameter. Implications for instruction and research are discussed, including a call for emphasis on a modeling paradigm in introductory statistics.
△ Less
Submitted 20 February, 2019; v1 submitted 6 December, 2017;
originally announced December 2017.
-
Cognitive Transfer Outcomes for a Simulation-Based Introductory Statistics Curriculum
Authors:
Matthew D. Beckman,
Robert C. delMas,
Joan Garfield
Abstract:
Cognitive transfer is the ability to apply learned skills and knowledge to new applications and contexts. This investigation evaluates cognitive transfer outcomes for a tertiary-level introductory statistics course using the CATALST curriculum, which exclusively used simulation-based methods to develop foundations of statistical inference. A common assessment instrument administered at the end of…
▽ More
Cognitive transfer is the ability to apply learned skills and knowledge to new applications and contexts. This investigation evaluates cognitive transfer outcomes for a tertiary-level introductory statistics course using the CATALST curriculum, which exclusively used simulation-based methods to develop foundations of statistical inference. A common assessment instrument administered at the end of each course measured learning outcomes for students. CATALST students showed evidence of both near and far transfer outcomes while scoring as high, or higher on the assessed learning objectives, when compared with peers enrolled in similar courses that emphasized parametric inferential methods (e.g. the t-test).
△ Less
Submitted 20 July, 2017;
originally announced July 2017.
