-
Can ChatGPT Pass a Theory of Computing Course?
Authors:
Matei A. Golesteanu,
Garrett B. Vowinkel,
Ryan E. Dougherty
Abstract:
Large Language Models (LLMs) have had considerable difficulty when prompted with mathematical questions, especially those within theory of computing (ToC) courses. In this paper, we detail two experiments regarding our own ToC course and the ChatGPT LLM. For the first, we evaluated ChatGPT's ability to pass our own ToC course's exams. For the second, we created a database of sample ToC questions a…
▽ More
Large Language Models (LLMs) have had considerable difficulty when prompted with mathematical questions, especially those within theory of computing (ToC) courses. In this paper, we detail two experiments regarding our own ToC course and the ChatGPT LLM. For the first, we evaluated ChatGPT's ability to pass our own ToC course's exams. For the second, we created a database of sample ToC questions and responses to accommodate other ToC offerings' choices for topics and structure. We scored each of ChatGPT's outputs on these questions. Overall, we determined that ChatGPT can pass our ToC course, and is adequate at understanding common formal definitions and answering "simple"-style questions, e.g., true/false and multiple choice. However, ChatGPT often makes nonsensical claims in open-ended responses, such as proofs.
△ Less
Submitted 10 July, 2024;
originally announced July 2024.
-
Creation of a CS1 Course with Modern C++ Principles
Authors:
Ryan E. Dougherty
Abstract:
Best practices in programming need to be emphasized in a CS1 course as bad student habits persist if not reinforced well. The C++ programming language, although a relatively old language, has been regularly updated with new versions since 2011, on the pace of once every three years. Each new version contains important features that make the C++ language more complex for backwards compatibility, bu…
▽ More
Best practices in programming need to be emphasized in a CS1 course as bad student habits persist if not reinforced well. The C++ programming language, although a relatively old language, has been regularly updated with new versions since 2011, on the pace of once every three years. Each new version contains important features that make the C++ language more complex for backwards compatibility, but often introduce new features to make common use cases simpler to implement. This poster contains experiences in designing a CS1 course that uses the C++ programming language that incorporates ``modern'' versions of the language from the start, as well as recent conferences about the language. Our goals were to prevent many common bad habits among C++ programmers.
△ Less
Submitted 13 November, 2023;
originally announced November 2023.
-
Designing Theory of Computing Backwards
Authors:
Ryan E. Dougherty
Abstract:
The design of any technical Computer Science course must involve its context within the institution's CS program, but also incorporate any new material that is relevant and appropriately accessible to students. In many institutions, theory of computing (ToC) courses within undergraduate CS programs are often placed near the end of the program, and have a very common structure of building off previ…
▽ More
The design of any technical Computer Science course must involve its context within the institution's CS program, but also incorporate any new material that is relevant and appropriately accessible to students. In many institutions, theory of computing (ToC) courses within undergraduate CS programs are often placed near the end of the program, and have a very common structure of building off previous sections of the course. The central question behind any such course is ``What are the limits of computers?'' for various types of computational models. However, what is often intuitive for students about what a ``computer'' is--a Turing machine--is taught at the end of the course, which necessitates motivation for earlier models. This poster contains our experiences in designing a ToC course that teaches the material effectively ``backwards,'' with pedagogic motivation of instead asking the question ``What suitable restrictions can we place on computers to make their problems tractable?'' We also give recommendations for future course design.
△ Less
Submitted 13 November, 2023;
originally announced November 2023.
-
Faster Location in Combinatorial Interaction Testing
Authors:
Ryan E. Dougherty,
Dylan N. Green,
Grace M. Kim
Abstract:
Factors within a large-scale software system that simultaneously interact and strongly impact the system's response under a configuration are often difficult to identify. Although screening such a system for the existence of such interactions is important, determining their location is more useful for system engineers. Combinatorial interaction testing (CIT) concerns creation of test suites that n…
▽ More
Factors within a large-scale software system that simultaneously interact and strongly impact the system's response under a configuration are often difficult to identify. Although screening such a system for the existence of such interactions is important, determining their location is more useful for system engineers. Combinatorial interaction testing (CIT) concerns creation of test suites that nonadaptively either detect or locate the desired interactions, each of at most a specified size or show that no such set exists. Under the assumption that there are at most a given number of such interactions causing such a response, locating arrays (LAs) guarantee unique location for every such set of interactions and an algorithm to deal with outliers and nondeterministic behavior from real systems, we additionally require the LAs to have a "separation" between these collections. State-of-the-art approaches generate LAs that can locate at most one interaction of size at most three, due to the massive number of interaction combinations for larger parameters if no constraints are given. This paper presents LocAG, a two-stage algorithm that generates (unconstrained) LAs using a simple, but powerful partitioning strategy of these combinations. In particular, we are able to generate LAs with more factors, with any desired separation, and greater interaction size than existing approaches.
△ Less
Submitted 11 October, 2023;
originally announced October 2023.
-
Experiences with Research Processes in an Undergraduate Theory of Computing Course
Authors:
Ryan E. Dougherty
Abstract:
Theory of computing (ToC) courses are a staple in many undergraduate CS curricula as they lay the foundation of why CS is important to students. Although not a stated goal, an inevitable outcome of the course is enhancing the students' technical reading and writing abilities as it often contains formal reasoning and proof writing. Separately, many undergraduate students are interested in performin…
▽ More
Theory of computing (ToC) courses are a staple in many undergraduate CS curricula as they lay the foundation of why CS is important to students. Although not a stated goal, an inevitable outcome of the course is enhancing the students' technical reading and writing abilities as it often contains formal reasoning and proof writing. Separately, many undergraduate students are interested in performing research, but often lack these abilities. Based on this observation, we emulated a common research environment within our ToC course by creating a mock conference assignment, where students (in groups) both wrote a technical paper solving an assigned problem and (individually) anonymously refereed other groups' papers. In this paper we discuss the details of this assignment and our experiences, and conclude with reflections and future work about similar courses.
△ Less
Submitted 3 October, 2023;
originally announced October 2023.
-
Upper Bounds for Covering Arrays of Higher Index
Authors:
Mason R. Calbert,
Ryan E. Dougherty
Abstract:
A \emph{covering array} is an $N \times k$ array of elements from a $v$-ary alphabet such that every $N \times t$ subarray contains all $v^t$ tuples from the alphabet of size $t$ at least $λ$ times; this is denoted as $\CA_λ(N; t, k, v)$.
Covering arrays have applications in the testing of large-scale complex systems; in systems that are nondeterministic, increasing $λ$ gives greater confidence…
▽ More
A \emph{covering array} is an $N \times k$ array of elements from a $v$-ary alphabet such that every $N \times t$ subarray contains all $v^t$ tuples from the alphabet of size $t$ at least $λ$ times; this is denoted as $\CA_λ(N; t, k, v)$.
Covering arrays have applications in the testing of large-scale complex systems; in systems that are nondeterministic, increasing $λ$ gives greater confidence in the system's correctness.
The \emph{covering array number}, $\CAN_λ(t,k,v)$ is the smallest number of rows for which a covering array on the other parameters exists.
For general $λ$, only several nontrivial bounds are known, the smallest of which was asymptotically $\log k + λ\log \log k + o(λ)$ when $v, t$ are fixed.
Additionally it has been conjectured that the $\log \log k$ term can be removed.
First, we affirm the conjecture by deriving an asymptotically optimal bound for $\CAN_λ(t,k,v)$ for general $λ$ and when $v, t$ are constant using the Stein--Lovász--Johnson paradigm.
Second, we improve upon the constants of this method using the Lovász local lemma.
Third, when $λ=2$, we extend a two-stage paradigm of Sarkar and Colbourn that improves on the general bound and often produces better bounds than even when $λ=1$ of other results.
Fourth, we extend this two-stage paradigm further for general $λ$ to obtain an even stronger upper bound, including using graph coloring.
And finally, we determine a bound on how large $λ$ can be for when the number of rows is fixed.
△ Less
Submitted 4 June, 2023; v1 submitted 2 November, 2022;
originally announced November 2022.
-
Genetic Algorithms for Redundancy in Interaction Testing
Authors:
Ryan E. Dougherty
Abstract:
It is imperative for testing to determine if the components within large-scale software systems operate functionally. Interaction testing involves designing a suite of tests, which guarantees to detect a fault if one exists among a small number of components interacting together. The cost of this testing is typically modeled by the number of tests, and thus much effort has been taken in reducing t…
▽ More
It is imperative for testing to determine if the components within large-scale software systems operate functionally. Interaction testing involves designing a suite of tests, which guarantees to detect a fault if one exists among a small number of components interacting together. The cost of this testing is typically modeled by the number of tests, and thus much effort has been taken in reducing this number. Here, we incorporate redundancy into the model, which allows for testing in non-deterministic environments. Existing algorithms for constructing these test suites usually involve one "fast" algorithm for generating most of the tests, and another "slower" algorithm to "complete" the test suite. We employ a genetic algorithm that generalizes these approaches that also incorporates redundancy by increasing the number of algorithms chosen, which we call "stages." By increasing the number of stages, we show that not only can the number of tests be reduced compared to existing techniques, but the computational time in generating them is also greatly reduced.
△ Less
Submitted 13 February, 2020;
originally announced February 2020.
-
Counting Subwords and Regular Languages
Authors:
Charles J. Colbourn,
Ryan E. Dougherty,
Thomas F. Lidbetter,
Jeffrey Shallit
Abstract:
Let $x$ and $y$ be words. We consider the languages whose words $z$ are those for which the numbers of occurrences of $x$ and $y$, as subwords of $z$, are the same (resp., the number of $x$'s is less than the number of $y$'s, resp., is less than or equal). We give a necessary and sufficient condition on $x$ and $y$ for these languages to be regular, and we show how to check this condition efficien…
▽ More
Let $x$ and $y$ be words. We consider the languages whose words $z$ are those for which the numbers of occurrences of $x$ and $y$, as subwords of $z$, are the same (resp., the number of $x$'s is less than the number of $y$'s, resp., is less than or equal). We give a necessary and sufficient condition on $x$ and $y$ for these languages to be regular, and we show how to check this condition efficiently.
△ Less
Submitted 20 June, 2018; v1 submitted 30 April, 2018;
originally announced April 2018.
