Showing 1–8 of 8 results for author: Ericson, B

Insights from Social Sha** Theory: The Appropriation of Large Language Models in an Undergraduate Programming Course

Authors: Aadarsh Padiyath, Xinying Hou, Amy Pang, Diego Viramontes Vargas, Xingjian Gu, Tamara Nelson-Fromm, Zihan Wu, Mark Guzdial, Barbara Ericson

Abstract: The capability of large language models (LLMs) to generate, debug, and explain code has sparked the interest of researchers and educators in undergraduate programming, with many anticipating their transformative potential in programming education. However, decisions about why and how to use LLMs in programming education may involve more than just the assessment of an LLM's technical capabilities.… ▽ More The capability of large language models (LLMs) to generate, debug, and explain code has sparked the interest of researchers and educators in undergraduate programming, with many anticipating their transformative potential in programming education. However, decisions about why and how to use LLMs in programming education may involve more than just the assessment of an LLM's technical capabilities. Using the social sha** of technology theory as a guiding framework, our study explores how students' social perceptions influence their own LLM usage. We then examine the correlation of self-reported LLM usage with students' self-efficacy and midterm performances in an undergraduate programming course. Triangulating data from an anonymous end-of-course student survey (n = 158), a mid-course self-efficacy survey (n=158), student interviews (n = 10), self-reported LLM usage on homework, and midterm performances, we discovered that students' use of LLMs was associated with their expectations for their future careers and their perceptions of peer usage. Additionally, early self-reported LLM usage in our context correlated with lower self-efficacy and lower midterm scores, while students' perceived over-reliance on LLMs, rather than their usage itself, correlated with decreased self-efficacy later in the course. △ Less

Submitted 10 June, 2024; originally announced June 2024.

Comments: Accepted to the ACM Conference on International Computing Education Research V.1 (ICER '24 Vol. 1)

CodeTailor: LLM-Powered Personalized Parsons Puzzles for Engaging Support While Learning Programming

Authors: Xinying Hou, Zihan Wu, Xu Wang, Barbara J. Ericson

Abstract: Learning to program can be challenging, and providing high-quality and timely support at scale is hard. Generative AI and its products, like ChatGPT, can create a solution for most intro-level programming problems. However, students might use these tools to just generate code for them, resulting in reduced engagement and limited learning. In this paper, we present CodeTailor, a system that leverag… ▽ More Learning to program can be challenging, and providing high-quality and timely support at scale is hard. Generative AI and its products, like ChatGPT, can create a solution for most intro-level programming problems. However, students might use these tools to just generate code for them, resulting in reduced engagement and limited learning. In this paper, we present CodeTailor, a system that leverages a large language model (LLM) to provide personalized help to students while still encouraging cognitive engagement. CodeTailor provides a personalized Parsons puzzle to support struggling students. In a Parsons puzzle, students place mixed-up code blocks in the correct order to solve a problem. A technical evaluation with previous incorrect student code snippets demonstrated that CodeTailor could deliver high-quality (correct, personalized, and concise) Parsons puzzles based on their incorrect code. We conducted a within-subjects study with 18 novice programmers. Participants perceived CodeTailor as more engaging than just receiving an LLM-generated solution (the baseline condition). In addition, participants applied more supported elements from the scaffolded practice to the posttest when using CodeTailor than baseline. Overall, most participants preferred using CodeTailor versus just receiving the LLM-generated code for learning. Qualitative observations and interviews also provided evidence for the benefits of CodeTailor, including thinking more about solution construction, fostering continuity in learning, promoting reflection, and boosting confidence. We suggest future design ideas to facilitate active learning opportunities with generative AI techniques. △ Less

Submitted 30 May, 2024; v1 submitted 22 January, 2024; originally announced January 2024.

Comments: Accepted to the Eleventh ACM Conference on Learning @ Scale (L@S 24) as a full paper

Integrating Personalized Parsons Problems with Multi-Level Textual Explanations to Scaffold Code Writing

Authors: Xinying Hou, Barbara J. Ericson, Xu Wang

Abstract: Novice programmers need to write basic code as part of the learning process, but they often face difficulties. To assist struggling students, we recently implemented personalized Parsons problems, which are code puzzles where students arrange blocks of code to solve them, as pop-up scaffolding. Students found them to be more engaging and preferred them for learning, instead of simply receiving the… ▽ More Novice programmers need to write basic code as part of the learning process, but they often face difficulties. To assist struggling students, we recently implemented personalized Parsons problems, which are code puzzles where students arrange blocks of code to solve them, as pop-up scaffolding. Students found them to be more engaging and preferred them for learning, instead of simply receiving the correct answer, such as the response they might get from generative AI tools like ChatGPT. However, a drawback of using Parsons problems as scaffolding is that students may be able to put the code blocks in the correct order without fully understanding the rationale of the correct solution. As a result, the learning benefits of scaffolding are compromised. Can we improve the understanding of personalized Parsons scaffolding by providing textual code explanations? In this poster, we propose a design that incorporates multiple levels of textual explanations for the Parsons problems. This design will be used for future technical evaluations and classroom experiments. These experiments will explore the effectiveness of adding textual explanations to Parsons problems to improve instructional benefits. △ Less

Submitted 11 January, 2024; v1 submitted 6 January, 2024; originally announced January 2024.

Comments: Peer-Reviewed, Accepted for publication in Proceedings of the 55th ACM Technical Symposium on Computer Science Education V. 2 (SIGCSE 2024)

Critiquing Computing Artifacts through Programming Satirical Python Scripts

Authors: Aadarsh Padiyath, Tamara Nelson-Fromm, Barbara Ericson

Abstract: Computing artifacts tend to exclude marginalized students, so we must create new methods to critique and change them. We studied the potential for "satirical programming" to critique artifacts as part of culturally responsive computing (CRC) pedagogy. We conducted a one-hour session for three different BPC programs (N=51). We showed an example of a satirical Python script and taught elements of Py… ▽ More Computing artifacts tend to exclude marginalized students, so we must create new methods to critique and change them. We studied the potential for "satirical programming" to critique artifacts as part of culturally responsive computing (CRC) pedagogy. We conducted a one-hour session for three different BPC programs (N=51). We showed an example of a satirical Python script and taught elements of Python to create a script. Our findings suggest this method is a promising CRC pedagogical approach: 50% of marginalized students worked together to create a satirical script, and 80% enjoyed translating their "glitches" into satirical Python scripts. △ Less

Submitted 5 December, 2023; originally announced December 2023.

Comments: Peer-Reviewed, Accepted for publication in the proceedings of the 2023 IEEE Conference on Research in Equity and Sustained Participation in Engineering, Computing, and Technology (RESPECT)

Understanding the Effects of Using Parsons Problems to Scaffold Code Writing for Students with Varying CS Self-Efficacy Levels

Authors: Xinying Hou, Barbara J. Ericson, Xu Wang

Abstract: Introductory programming courses aim to teach students to write code independently. However, transitioning from studying worked examples to generating their own code is often difficult and frustrating for students, especially those with lower CS self-efficacy in general. Therefore, we investigated the impact of using Parsons problems as a code-writing scaffold for students with varying levels of C… ▽ More Introductory programming courses aim to teach students to write code independently. However, transitioning from studying worked examples to generating their own code is often difficult and frustrating for students, especially those with lower CS self-efficacy in general. Therefore, we investigated the impact of using Parsons problems as a code-writing scaffold for students with varying levels of CS self-efficacy. Parsons problems are programming tasks where students arrange mixed-up code blocks in the correct order. We conducted a between-subjects study with undergraduate students (N=89) on a topic where students have limited code-writing expertise. Students were randomly assigned to one of two conditions. Students in one condition practiced writing code without any scaffolding, while students in the other condition were provided with scaffolding in the form of an equivalent Parsons problem. We found that, for students with low CS self-efficacy levels, those who received scaffolding achieved significantly higher practice performance and in-practice problem-solving efficiency compared to those without any scaffolding. Furthermore, when given Parsons problems as scaffolding during practice, students with lower CS self-efficacy were more likely to solve them. In addition, students with higher pre-practice knowledge on the topic were more likely to effectively use the Parsons scaffolding. This study provides evidence for the benefits of using Parsons problems to scaffold students' write-code activities. It also has implications for optimizing the Parsons scaffolding experience for students, including providing personalized and adaptive Parsons problems based on the student's current problem-solving status. △ Less

Submitted 29 November, 2023; originally announced November 2023.

Comments: Peer-Reviewed, Accepted for publication in the proceedings of the 2023 ACM Koli Calling International Conference on Computing Education Research

How Novices Use LLM-Based Code Generators to Solve CS1 Coding Tasks in a Self-Paced Learning Environment

Authors: Majeed Kazemitabaar, Xinying Hou, Austin Henley, Barbara J. Ericson, David Weintrop, Tovi Grossman

Abstract: As Large Language Models (LLMs) gain in popularity, it is important to understand how novice programmers use them. We present a thematic analysis of 33 learners, aged 10-17, independently learning Python through 45 code-authoring tasks using Codex, an LLM-based code generator. We explore several questions related to how learners used these code generators and provide an analysis of the properties… ▽ More As Large Language Models (LLMs) gain in popularity, it is important to understand how novice programmers use them. We present a thematic analysis of 33 learners, aged 10-17, independently learning Python through 45 code-authoring tasks using Codex, an LLM-based code generator. We explore several questions related to how learners used these code generators and provide an analysis of the properties of the written prompts and the generated code. Specifically, we explore (A) the context in which learners use Codex, (B) what learners are asking from Codex, (C) properties of their prompts in terms of relation to task description, language, and clarity, and prompt crafting patterns, (D) the correctness, complexity, and accuracy of the AI-generated code, and (E) how learners utilize AI-generated code in terms of placement, verification, and manual modifications. Furthermore, our analysis reveals four distinct coding approaches when writing code with an AI code generator: AI Single Prompt, where learners prompted Codex once to generate the entire solution to a task; AI Step-by-Step, where learners divided the problem into parts and used Codex to generate each part; Hybrid, where learners wrote some of the code themselves and used Codex to generate others; and Manual coding, where learners wrote the code themselves. The AI Single Prompt approach resulted in the highest correctness scores on code-authoring tasks, but the lowest correctness scores on subsequent code-modification tasks during training. Our results provide initial insight into how novice learners use AI code generators and the challenges and opportunities associated with integrating them into self-paced learning environments. We conclude with various signs of over-reliance and self-regulation, as well as opportunities for curriculum and tool development. △ Less

Submitted 25 September, 2023; originally announced September 2023.

Comments: 12 pages, Peer-Reviewed, Accepted for publication in the proceedings of the 2023 ACM Koli Calling International Conference on Computing Education Research

Plated-Through-Hole Via Design Specifications for 112G Serial Links

Authors: Michael J. Degerstrom, Chad M. Smutzer, Richard B. Ericson, Clifton R Haider, Barry K. Gilbert

Abstract: An earlier study of a high layer-count test board using plated-through-hole (PTH) vias and a limited quantity of laser vias was shown to be capable of supporting 112 Gb/s PAM-4 links (or equivalent signaling having 28 GHz (Nyquist) bandwidth). This original board design was then rebuilt using a different fabricator, and the test results revealed a significant decrease in the bandwidth of the vias.… ▽ More An earlier study of a high layer-count test board using plated-through-hole (PTH) vias and a limited quantity of laser vias was shown to be capable of supporting 112 Gb/s PAM-4 links (or equivalent signaling having 28 GHz (Nyquist) bandwidth). This original board design was then rebuilt using a different fabricator, and the test results revealed a significant decrease in the bandwidth of the vias. These results led to the development of a set of design specifications that PCB vendors can easily validate, which will ensure that the use of high layer-count boards with PTH technology are viable for emerging 112 Gb/s PAM-4 links. △ Less

Submitted 17 January, 2023; originally announced April 2023.

Comments: 20 pages, 16 figures, DesignCon 2017

Studying the effect of AI Code Generators on Supporting Novice Learners in Introductory Programming

Authors: Majeed Kazemitabaar, Justin Chow, Carl Ka To Ma, Barbara J. Ericson, David Weintrop, Tovi Grossman

Abstract: AI code generators like OpenAI Codex have the potential to assist novice programmers by generating code from natural language descriptions, however, over-reliance might negatively impact learning and retention. To explore the implications that AI code generators have on introductory programming, we conducted a controlled experiment with 69 novices (ages 10-17). Learners worked on 45 Python code-au… ▽ More AI code generators like OpenAI Codex have the potential to assist novice programmers by generating code from natural language descriptions, however, over-reliance might negatively impact learning and retention. To explore the implications that AI code generators have on introductory programming, we conducted a controlled experiment with 69 novices (ages 10-17). Learners worked on 45 Python code-authoring tasks, for which half of the learners had access to Codex, each followed by a code-modification task. Our results show that using Codex significantly increased code-authoring performance (1.15x increased completion rate and 1.8x higher scores) while not decreasing performance on manual code-modification tasks. Additionally, learners with access to Codex during the training phase performed slightly better on the evaluation post-tests conducted one week later, although this difference did not reach statistical significance. Of interest, learners with higher Scratch pre-test scores performed significantly better on retention post-tests, if they had prior access to Codex. △ Less

Submitted 21 February, 2023; v1 submitted 14 February, 2023; originally announced February 2023.

Comments: To be published in Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (CHI '23), April 23--28, 2023, Hamburg, Germany 17 pages with 11 Figures, 2 Tables, 6 Page Appendix