-
The Empirical Reality of IT Project Cost Overruns: Discovering A Power-Law Distribution
Authors:
Bent Flyvbjerg,
Alexander Budzier,
Jong Seok Lee,
Mark Keil,
Daniel Lunn,
Dirk W. Bester
Abstract:
If managers assume a normal or near-normal distribution of Information Technology (IT) project cost overruns, as is common, and cost overruns can be shown to follow a power-law distribution, managers may be unwittingly exposing their organizations to extreme risk by severely underestimating the probability of large cost overruns. In this research, we collect and analyze a large sample comprised of…
▽ More
If managers assume a normal or near-normal distribution of Information Technology (IT) project cost overruns, as is common, and cost overruns can be shown to follow a power-law distribution, managers may be unwittingly exposing their organizations to extreme risk by severely underestimating the probability of large cost overruns. In this research, we collect and analyze a large sample comprised of 5,392 IT projects to empirically examine the probability distribution of IT project cost overruns. Further, we propose and examine a mechanism that can explain such a distribution. Our results reveal that IT projects are far riskier in terms of cost than normally assumed by decision makers and scholars. Specifically, we found that IT project cost overruns follow a power-law distribution in which there are a large number of projects with relatively small overruns and a fat tail that includes a smaller number of projects with extreme overruns. A possible generative mechanism for the identified power-law distribution is found in interdependencies among technological components in IT systems. We propose and demonstrate, through computer simulation, that a problem in a single technological component can lead to chain reactions in which other interdependent components are affected, causing substantial overruns. What the power law tells us is that extreme IT project cost overruns will occur and that the prevalence of these will be grossly underestimated if managers assume that overruns follow a normal or near-normal distribution. This underscores the importance of realistically assessing and mitigating the cost risk of new IT projects up front.
△ Less
Submitted 30 September, 2022;
originally announced October 2022.
-
The Cost-Benefit Fallacy: Why Cost-Benefit Analysis Is Broken and How to Fix It
Authors:
Bent Flyvbjerg,
Dirk W. Bester
Abstract:
Most cost-benefit analyses assume that the estimates of costs and benefits are more or less accurate and unbiased. But what if, in reality, estimates are highly inaccurate and biased? Then the assumption that cost-benefit analysis is a rational way to improve resource allocation would be a fallacy. Based on the largest dataset of its kind, we test the assumption that cost and benefit estimates of…
▽ More
Most cost-benefit analyses assume that the estimates of costs and benefits are more or less accurate and unbiased. But what if, in reality, estimates are highly inaccurate and biased? Then the assumption that cost-benefit analysis is a rational way to improve resource allocation would be a fallacy. Based on the largest dataset of its kind, we test the assumption that cost and benefit estimates of public investments are accurate and unbiased. We find this is not the case with overwhelming statistical significance. We document the extent of cost overruns, benefit shortfalls, and forecasting bias in public investments. We further assess whether such inaccuracies seriously distort effective resource allocation, which is found to be the case. We explain our findings in behavioral terms and explore their policy implications. Finally, we conclude that cost-benefit analysis of public investments stands in need of reform and we outline four steps to such reform.
△ Less
Submitted 12 October, 2021;
originally announced December 2021.
-
Marginal likelihood based model comparison in Fuzzy Bayesian Learning
Authors:
Indranil Pan,
Dirk Bester
Abstract:
In a recent paper [1] we introduced the Fuzzy Bayesian Learning (FBL) paradigm where expert opinions can be encoded in the form of fuzzy rule bases and the hyper-parameters of the fuzzy sets can be learned from data using a Bayesian approach. The present paper extends this work for selecting the most appropriate rule base among a set of competing alternatives, which best explains the data, by calc…
▽ More
In a recent paper [1] we introduced the Fuzzy Bayesian Learning (FBL) paradigm where expert opinions can be encoded in the form of fuzzy rule bases and the hyper-parameters of the fuzzy sets can be learned from data using a Bayesian approach. The present paper extends this work for selecting the most appropriate rule base among a set of competing alternatives, which best explains the data, by calculating the model evidence or marginal likelihood. We explain why this is an attractive alternative over simply minimizing a mean squared error metric of prediction and show the validity of the proposition using synthetic examples and a real world case study in the financial services sector.
△ Less
Submitted 29 March, 2017;
originally announced March 2017.
-
Fuzzy Bayesian Learning
Authors:
Indranil Pan,
Dirk Bester
Abstract:
In this paper we propose a novel approach for learning from data using rule based fuzzy inference systems where the model parameters are estimated using Bayesian inference and Markov Chain Monte Carlo (MCMC) techniques. We show the applicability of the method for regression and classification tasks using synthetic data-sets and also a real world example in the financial services industry. Then we…
▽ More
In this paper we propose a novel approach for learning from data using rule based fuzzy inference systems where the model parameters are estimated using Bayesian inference and Markov Chain Monte Carlo (MCMC) techniques. We show the applicability of the method for regression and classification tasks using synthetic data-sets and also a real world example in the financial services industry. Then we demonstrate how the method can be extended for knowledge extraction to select the individual rules in a Bayesian way which best explains the given data. Finally we discuss the advantages and pitfalls of using this method over state-of-the-art techniques and highlight the specific class of problems where this would be useful.
△ Less
Submitted 22 June, 2018; v1 submitted 28 October, 2016;
originally announced October 2016.
-
First science with SALT: peering at the accreting polar caps of the eclipsing polar SDSS J015543.40+002807.2
Authors:
D. O'Donoghue,
D. A. H. Buckley,
L. A. Balona,
D. Bester,
L. Botha,
J. Brink,
D. B. Carter,
P. A. Charles,
A. Christians,
F. Ebrahim,
R. Emmerich,
W. Esterhuyse,
G. P. Evans,
C. Fourie,
P. Fourie,
H. Gajjar,
M. Gordon,
C. Gumede,
M. de Kock,
A. Koeslag,
W. P. Koorts,
H. Kriel,
F. Marang,
J. G. Meiring,
J. W. Menzies
, et al. (26 additional authors not shown)
Abstract:
We describe briefly the properties of the recently completed Southern African Large Telescope (SALT), along with its first light imager SALTICAM. Using this instrument, we present 4.3 hr of high speed unfiltered photometric observations of the eclipsing polar SDSSJ015543.40+002807.2 with time resolution as short as 112 ms, the highest quality observations of this kind of any polar to date. The s…
▽ More
We describe briefly the properties of the recently completed Southern African Large Telescope (SALT), along with its first light imager SALTICAM. Using this instrument, we present 4.3 hr of high speed unfiltered photometric observations of the eclipsing polar SDSSJ015543.40+002807.2 with time resolution as short as 112 ms, the highest quality observations of this kind of any polar to date. The system was observed during its high luminosity state. Two accreting poles are clearly seen in the eclipse light curve. The binary system parameters have been constrained: the white dwarf mass is at the low end of the range expected for cataclysmic variables. Correlations between the positions of the accretion regions on or near the surface of the white dwarf and the binary system parameters were established. The sizes of the accretion regions and their relative movement from eclipse to eclipse were estimated: they are typically 4-7 deg depending on the mass of the white dwarf. The potential of these observations will only fully be realised when low state data of the same kind are obtained and the contact phases of the eclipse of the white dwarf are measured.
△ Less
Submitted 12 July, 2006;
originally announced July 2006.
