Automated Repair of Programs from Large Language Models
Authors:
Zhiyu Fan,
Xiang Gao,
Martin Mirchev,
Abhik Roychoudhury,
Shin Hwei Tan
Abstract:
Large language models such as Codex, have shown the capability to produce code for many programming tasks. However, the success rate of existing models is low, especially for complex programming tasks. One of the reasons is that language models lack awareness of program semantics, resulting in incorrect programs, or even programs which do not compile. In this paper, we systematically study whether…
▽ More
Large language models such as Codex, have shown the capability to produce code for many programming tasks. However, the success rate of existing models is low, especially for complex programming tasks. One of the reasons is that language models lack awareness of program semantics, resulting in incorrect programs, or even programs which do not compile. In this paper, we systematically study whether automated program repair (APR) techniques can fix the incorrect solutions produced by language models in LeetCode contests. The goal is to study whether APR techniques can enhance reliability in the code produced by large language models. Our study revealed that: (1) automatically generated code shares common programming mistakes with human-crafted solutions, indicating APR techniques may have potential to fix auto-generated code; (2) given bug location information provided by a statistical fault localization approach, the newly released Codex edit mode, which supports editing code, is similar to or better than existing Java repair tools TBar and Recoder in fixing incorrect solutions. By analyzing the experimental results generated by these tools, we provide several suggestions: (1) enhancing APR tools to surpass limitations in patch space (e.g., introducing more flexible fault localization) is desirable; (2) as large language models can derive more fix patterns by training on more data, future APR tools could shift focus from adding more fix patterns to synthesis/semantics based approaches, (3) combination of language models with APR to curate patch ingredients, is worth studying.
△ Less
Submitted 1 January, 2023; v1 submitted 21 May, 2022;
originally announced May 2022.
Random Walk with Memory on Complex Networks
Authors:
Lasko Basnarkov,
Miroslav Mirchev,
Ljupco Kocarev
Abstract:
We study random walk on complex networks with transition probabilities which depend on the current and previously visited nodes. By using an absorbing Markov chain we derive an exact expression for the mean first passage time between pairs of nodes, for a random walk with a memory of one step. We have analyzed one particular model of random walk, where the transition probabilities depend on the nu…
▽ More
We study random walk on complex networks with transition probabilities which depend on the current and previously visited nodes. By using an absorbing Markov chain we derive an exact expression for the mean first passage time between pairs of nodes, for a random walk with a memory of one step. We have analyzed one particular model of random walk, where the transition probabilities depend on the number of paths to the second neighbors. The numerical experiments on paradigmatic complex networks verify the validity of the theoretical expressions, and also indicate that the flattening of the stationary occupation probability accompanies a nearly optimal random search.
△ Less
Submitted 28 August, 2020; v1 submitted 18 July, 2019;
originally announced July 2019.
