-
A manual categorization of new quality issues on automatically-generated tests
Authors:
Geraldine Galindo-Gutierrez,
Narea Maxilimiliano,
Blanco Alison Fernandez,
Nicolas Anquetil,
Alcocer Juan Pablo Sandoval
Abstract:
Diverse studies have analyzed the quality of automatically generated test cases by using test smells as the main quality attribute. But recent work reported that generated tests may suffer a number of quality issues not necessarily considered in previous studies. Little is known about these issues and their frequency within generated tests. In this paper, we report on a manual analysis of an exter…
▽ More
Diverse studies have analyzed the quality of automatically generated test cases by using test smells as the main quality attribute. But recent work reported that generated tests may suffer a number of quality issues not necessarily considered in previous studies. Little is known about these issues and their frequency within generated tests. In this paper, we report on a manual analysis of an external dataset consisting of 2,340 automatically generated tests. This analysis aimed at detecting new quality issues, not covered by past recognized test smells. We use thematic analysis to group and categorize the new quality issues found. As a result, we propose a taxonomy of 13 new quality issues grouped in four categories. We also report on the frequency of these new quality issues within the dataset and present eight recommendations that test generators may consider to improve the quality and usefulness of the automatically generated tests.
△ Less
Submitted 14 December, 2023;
originally announced December 2023.
-
Properties of maximum Lempel-Ziv complexity strings
Authors:
C. A. J. Nunes,
E. Estevez-Rams,
B. Aragón Fernández,
R. Lora Serrano
Abstract:
The properties of maximum Lempel-Ziv complexity strings are studied for the binary case. A comparison between MLZs and random strings is carried out. The length profile of both type of sequences show different distribution functions. The non-stationary character of the MLZs are discussed. The issue of sensitiveness to noise is also addressed. An empirical ansatz is found that fits well to the Lemp…
▽ More
The properties of maximum Lempel-Ziv complexity strings are studied for the binary case. A comparison between MLZs and random strings is carried out. The length profile of both type of sequences show different distribution functions. The non-stationary character of the MLZs are discussed. The issue of sensitiveness to noise is also addressed. An empirical ansatz is found that fits well to the Lempel-Ziv complexity of the MLZs for all lengths up to $10^6$ symbols.
△ Less
Submitted 4 November, 2013;
originally announced November 2013.
-
On the non-randomness of maximum Lempel Ziv complexity sequences of finite size
Authors:
E. Estevez-Rams,
R. Lora Serrano,
B. Aragón Fernández,
I. Brito Reyes
Abstract:
Random sequences attain the highest entropy rate. The estimation of entropy rate for an ergodic source can be done using the Lempel Ziv complexity measure yet, the exact entropy rate value is only reached in the infinite limit. We prove that typical random sequences of finite length fall short of the maximum Lempel-Ziv complexity, contrary to common belief. We discuss that, for a finite length, ma…
▽ More
Random sequences attain the highest entropy rate. The estimation of entropy rate for an ergodic source can be done using the Lempel Ziv complexity measure yet, the exact entropy rate value is only reached in the infinite limit. We prove that typical random sequences of finite length fall short of the maximum Lempel-Ziv complexity, contrary to common belief. We discuss that, for a finite length, maximum Lempel-Ziv sequences can be built from a well defined generating algorithm, which makes them of low Kolmogorov-Chaitin complexity, quite the opposite to randomness. It will be discussed that Lempel-Ziv measure is, in this sense, less general than Kolmogorov-Chaitin complexity, as it can be fooled by an intelligent enough agent. The latter will be shown to be the case for the binary expansion of certain irrational numbers. Maximum Lempel-Ziv sequences induce a normalization that gives good estimates of entropy rate for several sources, while kee** bounded values for all sequence length, making it an alternative to other normalization schemes in use.
△ Less
Submitted 3 November, 2013;
originally announced November 2013.
