The accuracy of preliminary cost estimates in Public · PDF fileThe accuracy of preliminary...

12
The accuracy of preliminary cost estimates in Public Works Department (PWD) of Peninsular Malaysia Mohd Azrai Azman a, , Zulkiee Abdul-Samad b , Suraya Ismail b a Department of Quantity Surveying, Faculty of Architecture, Planning and Surveying, Universiti Teknologi MARA (UiTM), Kota Samarahan Campus, Jalan Meranek, 94300 Kota Samarahan, Sarawak, Malaysia b Department of Quantity Surveying, Faculty of Built Environment, University of Malaya, 50603 Kuala Lumpur, Malaysia Received 19 December 2011; received in revised form 6 November 2012; accepted 8 November 2012 Available online xxxx Abstract This paper presents the accuracy of preliminary cost estimates prepared by Public Works Department (PWD), which is the largest public construction management in Malaysia. It attempts to understand the Quantity Surveyors' (QS) estimation accuracy in relation to public projects. This study analyses 83 projects of estimates and tender bids. The analysis includes three (3) estimating targets i.e. lowest bid, accepted bid and mean of the bids. To broaden the study, 344 QS involved in the procurement answered the questionnaires. Linear multiple regression analysis on project characteristics shows that project size, number of bidders, location and type of schools affect the bias. Contract period affects the consistency. The use of mean of the bids is the best-t target to explain the bias in terms of adjusted R 2 . The accuracy may improve if sufcient design information is available, proper cost planning and improving the application of historical cost data. As an alternative, the phenomenon of overestimation is resulted from government directive instruction, which could challenge the rational of accurate estimate. © 2012 Elsevier Ltd. APM and IPMA. All rights reserved. Keywords: Public construction; Budget cost estimate; Quantity Surveyors; Tendering; Malaysia 1. Introduction Preliminary Cost Estimate determines the viability of public projects. It serves as a budgetary control during initial design stage. The government often considers the estimate as cost limit of projects. The estimate is based on size and functional unit of buildings using cost indicators of similar projects (Morton and Jaggar, 1995). It depends on information gathered at early design stage. The estimate is prepared according to desired quality, in accepted time and within budget (Karlsen and Lereim, 2005). The stakeholders of a project need to identify the possible cost before making monetary investment. The actual cost is known after the project is completed but the investment decisions need to be made before the construction begins. Public Works Department (PWD) of Malaysia is the largest public construction management organization, which supervises and provides in-house solutions for public agencies in Malaysia (Abdul-Aziz and Ali, 2004). The early cost advice of public funded projects is essential because budget constraint limit the government capacity to spend (Morton and Jaggar, 1995). If the estimate is too low, the proposed project design could be abandoned, and it may also lead to a lawsuit (Ashworth, 2010). Overestimation leads to a lesser fund available to other projects and underestimation results in difficulty to the contract (Odusami and Onukwube, 2008). To overcome these problems, Public Works Department could take Abbreviations: ANOVA, analysis of variance; CV, coefficient of variation; GFA, gross floor area; LMR, linear multiple regression analysis; m 2 , meter square, unit of area in the International System of Units (SI); MOF, Ministry of Finance; PDA, preliminary detailed abstract; PI, price intensity theory; PWD, Public Works Department; QS, quantity surveyor or quantity surveyors; RM, Malaysian Ringgit, currency of Malaysia; η 2 , partial eta-squared; N9, Negeri Sembilan (state in Malaysia). Corresponding author. Tel.: +60 82677200x7841. E-mail address: [email protected] (M.A. Azman). www.elsevier.com/locate/ijproman 0263-7863/$36.00 © 2012 Elsevier Ltd. APM and IPMA. All rights reserved. http://dx.doi.org/10.1016/j.ijproman.2012.11.008 Please cite this article as: Azman, M.A., et al., The accuracy of preliminary cost estimates in Public Works Department (PWD) of Peninsular Malaysia, International Journal of Project Management (2012), http://dx.doi.org/10.1016/j.ijproman.2012.11.008 Available online at www.sciencedirect.com International Journal of Project Management xx (2012) xxx xxx JPMA-01474; No of Pages 12

Transcript of The accuracy of preliminary cost estimates in Public · PDF fileThe accuracy of preliminary...

Page 1: The accuracy of preliminary cost estimates in Public · PDF fileThe accuracy of preliminary cost estimates in Public Works Department (PWD) of Peninsular Malaysia Mohd Azrai Azman

www.elsevier.com/locate/ijproman

Available online at www.sciencedirect.com

International Journal of Project Management xx (2012) xxx–xxx

JPMA-01474; No of Pages 12

The accuracy of preliminary cost estimates in Public Works Department(PWD) of Peninsular Malaysia

Mohd Azrai Azman a,⁎, Zulkiflee Abdul-Samad b, Suraya Ismail b

a Department of Quantity Surveying, Faculty of Architecture, Planning and Surveying, Universiti Teknologi MARA (UiTM), Kota Samarahan Campus,Jalan Meranek, 94300 Kota Samarahan, Sarawak, Malaysia

b Department of Quantity Surveying, Faculty of Built Environment, University of Malaya, 50603 Kuala Lumpur, Malaysia

Received 19 December 2011; received in revised form 6 November 2012; accepted 8 November 2012Available online xxxx

Abstract

This paper presents the accuracy of preliminary cost estimates prepared by Public Works Department (PWD), which is the largest publicconstruction management in Malaysia. It attempts to understand the Quantity Surveyors' (QS) estimation accuracy in relation to public projects.This study analyses 83 projects of estimates and tender bids. The analysis includes three (3) estimating targets i.e. lowest bid, accepted bid andmean of the bids. To broaden the study, 344 QS involved in the procurement answered the questionnaires. Linear multiple regression analysis onproject characteristics shows that project size, number of bidders, location and type of schools affect the bias. Contract period affects theconsistency. The use of mean of the bids is the best-fit target to explain the bias in terms of adjusted R2. The accuracy may improve if sufficientdesign information is available, proper cost planning and improving the application of historical cost data. As an alternative, the phenomenon ofoverestimation is resulted from government directive instruction, which could challenge the rational of accurate estimate.© 2012 Elsevier Ltd. APM and IPMA. All rights reserved.Keywords: Public construction; Budget cost estimate; Quantity Surveyors; Tendering; Malaysia

1. Introduction

Preliminary Cost Estimate determines the viability of publicprojects. It serves as a budgetary control during initial designstage. The government often considers the estimate as cost limitof projects. The estimate is based on size and functional unit ofbuildings using cost indicators of similar projects (Morton andJaggar, 1995). It depends on information gathered at early design

Abbreviations: ANOVA, analysis of variance; CV, coefficient of variationGFA, gross floor area; LMR, linear multiple regression analysis; m2, metesquare, unit of area in the International System of Units (SI); MOF, Ministry oFinance; PDA, preliminary detailed abstract; PI, price intensity theory; PWDPublic Works Department; QS, quantity surveyor or quantity surveyors; RMMalaysian Ringgit, currency of Malaysia; η2, partial eta-squared; N9, NegerSembilan (state in Malaysia).⁎ Corresponding author. Tel.: +60 82677200x7841.

E-mail address: [email protected] (M.A. Azman).

0263-7863/$36.00 © 2012 Elsevier Ltd. APM and IPMA. All rights reserved.http://dx.doi.org/10.1016/j.ijproman.2012.11.008

Please cite this article as: Azman, M.A., et al., The accuracy of preliminary coJournal of Project Management (2012), http://dx.doi.org/10.1016/j.ijproman.20

;rf,,i

st estim12.11

stage. The estimate is prepared according to desired quality, inaccepted time and within budget (Karlsen and Lereim, 2005).The stakeholders of a project need to identify the possible costbefore making monetary investment. The actual cost is knownafter the project is completed but the investment decisions need tobe made before the construction begins.

Public Works Department (PWD) of Malaysia is the largestpublic construction management organization, which supervisesand provides in-house solutions for public agencies in Malaysia(Abdul-Aziz andAli, 2004). The early cost advice of public fundedprojects is essential because budget constraint limit the governmentcapacity to spend (Morton and Jaggar, 1995). If the estimate is toolow, the proposed project design could be abandoned, and it mayalso lead to a lawsuit (Ashworth, 2010). Overestimation leads to alesser fund available to other projects and underestimation resultsin difficulty to the contract (Odusami and Onukwube, 2008). Toovercome these problems, Public Works Department could take

ates in Public Works Department (PWD) of Peninsular Malaysia, International.008

Page 2: The accuracy of preliminary cost estimates in Public · PDF fileThe accuracy of preliminary cost estimates in Public Works Department (PWD) of Peninsular Malaysia Mohd Azrai Azman

2 M.A. Azman et al. / International Journal of Project Management xx (2012) xxx–xxx

two (2) possible actions, that is design amendment, or requisitionof additional budget (Public Works Department, 2010). Even so,these recommendations may not resolve the inaccurate estima-tion. As usual, QS overestimated the estimate (Aibinu and Pasco,2008; Ashworth and Skitmore, 1982; Cheung et al., 2008;Skitmore and Drew, 2003; Skitmore and Picken, 2000). Most ofthe time, clients and QS are more likely to accept overestimationrather than underestimation (Cheung et al., 2008). They aregenerally contented with overestimation because it brings morebenefits with less risk than underestimation. A study made inSingapore by Ling and Boo (2001) shows the expected accuracyassured byQS are way off from the true values when compared toanalyzed samples.

According to official records, 57.39% of projects managed byPWD were public school projects and the department is still usingpre-design drawings for public facilities e.g. school, hospital,government office and public housing scheme (Public WorksDepartment, 2009a). The rapid development in Malaysianeconomy increases the demand for public facilities mentioned.To overcome this overwhelmed demand, the pre-design drawingshas been introduced by the PWD to speed-up the construction andit reduces the cost through the use of standardized constructionprocess and components (Norwina, 2006). However, according toofficial statement from the PWD itself, the reliability of QSdepartment in performing estimation was in doubt (Public WorksDepartment, 2004, 2005). This jeopardizes the merits of standarddesigns, as it should be able to help the government in terms ofprudent cost management advice by the QS. Construction IndustryResearch and Information Association (CIRIA) ascertain thatstandard design projects need to be managed effectively usingoptimized solutions to overcome some cost irregularities becausethe construction industry is still inherited with inefficiencies, evenstandardization comes into place (Gibb, 2001). Evidences showthat economies of scale in the construction industry are verydifficult to achieve because they involve high labor intensity andfew standardized products (Hillebrandt, 2000; Valence, 2011).Therefore, continuous improvement for pre-design projectsespecially in-cost estimation leads to more value for money forthe government. In addition, this paper offers interesting view ondisutility function, which might offer an alternative answer inrelation to the phenomena of overestimation of public projects. Itmight reject the rationale of accurate estimate (symmetrically nearto zero mean). This might help future research on the improvementof cost estimate.

2. Literature review

2.1. Review of Public Works Department estimating procedure

The PWD uses standard forms for cost plan. The estimate istabulated according to the elemental costs breakdown. The costestimate is calculated using historical and current prices (PublicWorks Department, 1992). Currently, there are only two (2) typesof estimating methods used by the PWD: single rate method andestimated quantities (Public Works Department, 2010). Theestimate is prepared using preliminary design plan. The QSofficers used average cost data (cost/m2GFA) of suitable building

Please cite this article as: Azman, M.A., et al., The accuracy of preliminary cost estimJournal of Project Management (2012), http://dx.doi.org/10.1016/j.ijproman.2012.11

and adjust it according to tender price index and locality factor.Then, the cost is multiplied by m2GFA of the area. The cost forPreliminaries, Internal Building Specialist Works, ExternalWorks and Professional Fees are calculated according to averagepercentage collected from previous projects. The cost data isobtained from the previous accepted tenders (Public WorksDepartment, 2009b). The current estimating policy in the PWDremains unchanged, as the new guideline introduced in 2010 doesnot provide any other methods or amendments in relation to theestimating procedures (Public Works Department, 2010). MostQS are not willing to change into new ways of preparing the costestimate (Fortune and Cox, 2005; Raftery, 1991). As for now, it isforeseeable that the improvement to current estimating processand procedure are the way forward to improve the estimatingaccuracy.

2.2. Estimating accuracy

Accurate means lack of error. It comprises two (2) aspects interms of “bias” and “consistency” (Ashworth and Skitmore,1982; Morrison, 1984; Skitmore, 1991). Bias is concern with theaverage differences between estimate and tender price accordingto arithmetical mean of percentage error (Skitmore, 1991). Thegreater the average differences, the more bias the estimate.Consistency is the degree of variation around the average whichmeans it measures how often the accuracy can be relied on(Ashworth and Skitmore, 1982). In theory, accurate estimation isthe one closer to the tender price. However, this concept mightraise some issues because overestimation could be rewarding dueto persistent trend of overestimation in almost all QS practices(Cheung et al., 2008; Skitmore and Cheung, 2007).

2.3. Target measures of forecast quality

The purpose of estimating is to make a prediction. There mustbe a reference point to measure the estimating performance. Someagree that the lowest tender price as the estimating target (Gunnerand Skitmore, 1999a; Morrison, 1984; Skitmore, 1991). Thequality of the estimating models depends on how accurate themodel predicts the bid value, which is usually an unknown value,but due to the nature of competition in bidding, the lowest bid isusually the target for most contracts. However, to some extent,using the lowest bid may not represent the value for money as itcould be a suicidal low bid price offered by a bidder (Skitmoreand Lo, 2002). This unrealistic low bid is a result of some bidders'desire to win the contract at any cost (Herdsman and Ellis, 2006;Murdoch and Hughes, 2007; Runeson, 2000). The loss from lowbid strategy could be recouped through project managementduring construction. The second lowest bid, in fact, might bebetter economically than the lowest bid (Lowe and Skitmore,2006). This shows that the lowest price cannot guarantee the bestvalue. To resolve this problem, McCaffer (1976) suggests the useof mean of the bids instead. His argument is that the mean has lessinterference variable (suicidal low bids), and more likely to bemore accurate. However, Raftery (1991) points out that the use ofmean of the bids as the target could be affected by uncompetitiveprices (high-priced bids) and this affects the average value as it

ates in Public Works Department (PWD) of Peninsular Malaysia, International.008

Page 3: The accuracy of preliminary cost estimates in Public · PDF fileThe accuracy of preliminary cost estimates in Public Works Department (PWD) of Peninsular Malaysia Mohd Azrai Azman

3M.A. Azman et al. / International Journal of Project Management xx (2012) xxx–xxx

will become one-sided to uncompetitive price. Some used acceptedtender bids to measure the estimating bias because it is the priceaccepted by clients (Aibinu and Pasco, 2008; Gunner andSkitmore, 1999a; Raftery, 1991). Skitmore claimed that thedrawback of using accepted bid is that it may have been influencedby the QS and therefore it is interdependent to some extent with theestimate (personal communication, December 17, 2010).

Some others used the final completed cost as the target becauseit provides the total commitment cost to the client (AbouRizk et al.,2002; Shane et al., 2009). However, Skitmore (2002) and Runeson(2000) explain that there are some problems of using finalcompleted cost. The problem is that the data is not readily availableand when it is available, it is not well recorded. They point out, dueto long delay between the estimate and final cost, significantchanges of design and price fluctuation occurs during construction.In addition, the contractor is reluctant to give the true cost thatshows the project's actual expenses (Skitmore, 1988). It becomes anorm for business practices not to expose information to others. Itshows that there are many views pertaining estimating targets. InPWDofMalaysia, the lowest tender price is not the main criteria inthe selection of a bidder to be awarded the contract. The PWD uses“average bid method” as against to “lowest bid method” (FaridahHalil, 2007; Public Works Department, 2004, 2005). This showsthat the use of mean of the bids is more appropriate.

Average bid method allows reasonable profit to contractors(Ioannou and Leu, 1993). The contract prices are high-priced butit could decrease risks of delay, substandard quality and disputes.The PWD states one of the reasons of using this method isbecause the QS estimates are more prone to errors and it maydiscourage insiders from giving the estimated figure to bidders(Public Works Department, 2004, 2005; Sharinah Hamid, 2008).This reduces the risk of inaccurate estimation being used as abenchmark. Countries like Italy and Taiwan used this tender priceassessment to reduce unrealistic low bids (Ioannou and Leu,1993; Kumaraswamy and Walker, 1999). The mean is calculatedfrom the total bidders' prices which include the estimate preparedby PWD QS officers to create “the lowest acceptable marketprice” or “cut-off price”. This is the main criteria of priceselection. The prices offered must not be lower than the pricedetermined by the method. Those bidders who priced the tenderlower than the amount calculated could also win the contractaward if they have good record of accomplishment and soundfinancial standing. The prices should not be lower than 20% ofthe mean of builder's work price (Public Works Department,2004, 2005).

However, not all tender evaluations used the average bidmethod e.g. selective tender, design and build tender or opentender that required expertise (Public Works Department, 2004,2005). In general, the procurement of standard design projects useopen tendering process because the design is uncomplicated andmore contractors are willing to bid. In open tender, the PWD tenderboard could reject “the lowest acceptable market price” if thebidder is in bad financial condition or bad record of accomplish-ment. Generally, projects that require expertise would go topre-qualification process before the bidders can bid the tender. Thepre-qualification process looks into financial and expertise record.This limits the number of tenders entering the bid.

Please cite this article as: Azman, M.A., et al., The accuracy of preliminary cost estimJournal of Project Management (2012), http://dx.doi.org/10.1016/j.ijproman.2012.11

2.4. Factors which affect the accuracy of the estimate

QS's pricing activities affect the quality of the estimate inrelation to many factors. This behavior causes systematic biases(Raftery, 1995). This includes heuristics in decision-making,personal biases and reporting biases. This happens because ofimprecise process and subjective decision during estimation(Ashworth and Skitmore, 1982). Factors contribute to theestimation bias could be influenced by more causes and inter-correlation between variables may be presented. It could be a resultfrom the effect of confounding variables that camouflage theimportant variable (Gunner and Skitmore, 1999b). Skitmore(1991) established that the project characteristics determine theQS estimating performance. These factors are building function,contract type, contract condition, contract sum, contract period,number of bidders, economic condition, procurement basis, projectsector and location. Subsequently, price intensity (cost/m2) wasadded to the existing variables. In common practice, QS takes theprice per meter square floor of comparable building as a startingpoint then adjusts it to the anticipated value (Gunner and Skitmore,1999b; Skitmore and Drew, 2003). Therefore, in this theory,judgmental bias by QS will be the significant deciding factor whenadjustment or pooling cost data (cost/m2) is made. The followingTable 1 is the summary of empirical evidences on project charac-teristics which affects quality of the estimate.

QS's biases lead to overestimation in many countries whencomparing the differences between estimate and bid (Skitmore etal., 1990). This brings to an opinion that QS simply add on the‘raw’ estimate with the amount they felt suitable (Cheung et al.,2008). QS aims to be on the safe side when preparing the estimate.QS could predict his estimate will be too high or too low becauseof his years of experience in preparing cost estimates (Skitmoreand Cheung, 2007). Some said no QS wants to prepare under-estimated estimate because they want to avoid unpleasant surpriseto the clients if his estimate is the lowest (Magnussen and Olsson,2006). QS may be at risk of being sued by his client or his projectcould be abandoned (Chappell et al., 2001). A bidder who knows afair amount of information about a project decreases their pricemargin compare to bidders who were given partial information(Soo and Oo, 2007). QS is suspected to increase his pricing if littleinformation is obtained. A PhD thesis by Yeung (2009) found thatprojects with less variance in the project specification using thesame target group e.g. residential, car-park, social community,hotel, residential and school buildings provide an acceptable levelof accuracy. However, the problem regarding projects with largevariances in the project specification e.g. hospital, university andcommercial center buildings is that there are only few costreferences available. The variance means the variations in projectspecification and construction price range. Inaccurate estimatecould also happen because bidders may deliberately be biasduring estimation rather than due to random effect (Flyvbjerg,2009; Flyvbjerg et al., 2002; Magnussen and Olsson, 2006). Theyshow that underestimation or cost overrun occurred when theestimate is lower than the value of final cost. They explain that abid price by a project promoter is more likely to be underestimatedduring bidding because public clients prefer inexpensive optionand it happens because of political pressure to secure funding. In

ates in Public Works Department (PWD) of Peninsular Malaysia, International.008

Page 4: The accuracy of preliminary cost estimates in Public · PDF fileThe accuracy of preliminary cost estimates in Public Works Department (PWD) of Peninsular Malaysia Mohd Azrai Azman

Table 1Empirical evidences of project characteristics affecting quality of the estimate.

Researcher Evidence

Project size and valueMorrison andStevens (1980)

Bias reduces with size. Consistencyimproves with size

Skitmore andTan (1988)

Bias reduces and consistency improves with size

Ogunlana andThorpe (1991)

Consistency reduce with larger contract size

Gunner andSkitmore (1999a)

Bias reduces with size. Consistency improveswith size

Skitmore andDrew (2003)

Bias increases with size

Aibinu andPasco (2008)

Bias reduces with size

Kiew (2009) Bias increases with size

Type of projectHarvey(1979) Different bias for buildings, non buildings,

special trades and othersMorrison andStevens (1980)

Different bias and consistency for schools,new housing, housing modifications, and others

Number of biddersHarvey (1979) Estimates lower with more biddersFlanagan andNorman (1983)

Estimates lower with more bidders

Skitmore(2002)

Consistency decrease with additional numberof bidders in seven data sets

Price intensityGunner andSkitmore (1999b)

High value contract were underestimatedand low value contracts over estimated

Skitmore andDrew (2003)

High value contract were underestimatedand low value contracts over estimated

Contract periodSkitmore (1988) No difference between groups of contract periodGunner andSkitmore (1999a)

No conclusion due to different result obtained

LocationHarvey (1979) Bias differences between Canadian regionsOgunlana andThorpe (1991)

No conclusion although bias and consistencydifference between regions of the United Kingdom

Note: it depends on how the measures of target are selected which is bid/estimateor estimate/bid ratio.

Table 2Frequency and percentage of respondents' years of experience.

Years of experience Respondents' organizations

PWD Privateconsultant

Total

Count N % Count N % Count N %

Less than 5 years 0 0.0% 13 24.5% 13 8.3%5–10 years 30 28.8% 13 24.5% 43 27.4%11–15 years 33 31.7% 5 9.4% 38 24.2%N15 years 41 39.4% 22 41.5% 63 40.1%

4 M.A. Azman et al. / International Journal of Project Management xx (2012) xxx–xxx

addition, promoters and public officials overestimate the benefitsof the project. However, according to Shane et al. (2009),contractors' bias estimate is a tendency to be overoptimisticabout the significance of cost factors which results in the bid to beunderestimated. There are many breakthroughs in alternativeestimating method and quality management system. Most of thetime, QS do not change their estimating policy in reaction toprevious estimates because they are not aware of the error trendthey developed during estimation (Morrison, 1984). This leads tothe estimate not improving over time (Aibinu and Pasco, 2008).The following are the research questions based on the researchproblems:

a) How accurate are the preliminary cost estimates using standarddesign plans prepared by PWD?

Please cite this article as: Azman, M.A., et al., The accuracy of preliminary cost estimJournal of Project Management (2012), http://dx.doi.org/10.1016/j.ijproman.2012.11

b) What is the best estimating target to explain the bias in theestimate?

c) To what extent could the project characteristics significantlyaffect the accuracy of preliminary cost estimates prepared bythe PWD?

d) Why biases happen in PWD procurement and how to overcomethis problem?

3. Research method

This research explores the significant relationship betweenbiases of the estimate (observed variables) and project character-istics (predictor variables). In addition, it compares which esti-mating target i.e. lowest bid, accepted tender and mean of the bidsto find the best-fit linear model to explain the estimating bias.Therefore, this research makes comparison using the same set ofdata. QS in PWD priced the estimate using elemental cost order.These costs determine the total cost of a project. The only way toexamine cost distribution in the estimate is to compare it withtenders' elemental prices. A large variance in elemental cost isexpected to influence the total accuracy. It was assumed theunbalanced bids by the bidders might not affect the analysisbecause the bid prices transferred to contract bills of quantities arerationalized to ensure the prices are acceptable. It means that afterthe signing of the contract between the accepted tender and thegovernment, it is the duty of the QS officers tomake sure the tenderdo not price high rate on some items and low rate on the others. Theeffect of unbalanced bid is that the comparison between elementalprices of the samples collected may not reasonably explain the truedistribution of elemental prices.

3.1. Data collection

The data from 83 projects were collected randomly fromCKUB, PWD files. The projects were designed and estimated byPWD. The total approved estimates were RM280,740,476.45. Thetotal approved contract prices were RM 250,539,830.40. All theseprojects are located in the west coast of PeninsularMalaysia. Theseare the state of Kedah, Perak, Negeri Sembilan (N9), Malacca andJohor. In addition, 344 questionnaires were also sent to senior QSwho work in PWD and private QS consultants which provide costconsultancy service to the government (refer to Table 2). Abouthalf of the questionnaires were returned (46% — 157 responses).PWD officers and QS consultants returned 45% and 46% of thetotal respectively. All respondents from PWD are senior officers

ates in Public Works Department (PWD) of Peninsular Malaysia, International.008

Page 5: The accuracy of preliminary cost estimates in Public · PDF fileThe accuracy of preliminary cost estimates in Public Works Department (PWD) of Peninsular Malaysia Mohd Azrai Azman

5M.A. Azman et al. / International Journal of Project Management xx (2012) xxx–xxx

while 75.5% of respondents from private consultants are seniorexecutives. Almost 92% of total respondents are considered havingexperience in estimating because most of them have been workingfor more than 5 years as QS. 40.1% of respondents have beenworking for more than 15 years in their respective organization.Therefore, this survey has achieved its target in having experiencedrespondents.

3.2. Sample size for linear multiple regression

Dattalo (2008) suggests the use of G*Power software todetermine the sample size needed. In empirical research, statisticalinference about a population could only be made throughadequate sample size. However, the access to data depends onthe permission of the owner; the author decides to limit the sampleto a minimum. Field (2009) suggests a sample size of 80 is alwaysenough with 20 predictors to find a large effect size for linearmultiple regression analysis. The following calculation usedG*Power software in order to determine the sample size for multiregression model (Faul et al., 2007):

λ ¼ f 2N

26:95 ¼ 0:35ð Þ � NN ¼ 77

Effect size ¼ f 2 at 0:35

Non� centrality parameter

¼ λ at 26:95 α ¼ 0:05; power of 1−β error probability ¼ 0:80and number of predictors ¼ 20

� �

Estimated number of sample ¼ N:

3.3. Conventional analysis on the accuracy

Percentage error or a multiplicative approach measures the bias.It measures the percentage differences between estimate and tenderbid. Here, the percentage difference between estimate and bid ischosen because in theory, the value of the bid is the one that QSwould like to estimate. Arithmetic mean of forecast/tender bid ratiomeasures the distribution of biases. Coefficient of variation of ratioforecast/tender bid (cv) measures consistency of the estimate:

cv ¼ X=Yð Þ � 100%X ¼ standard deviation of ratio forecasts=tender bidsY ¼ mean of ratio forecasts=tender bids:

Adjustment has to be made using mean of the bids to measurethe forecast quality. The mean calculation uses only eight (8)bidders, as they are limited and if more than eight bidders, thechances of uncompetitive tenders remained in the analysis is high(Carr, 2005; Dell'Isola, 2002; Skitmore, 2002).

Please cite this article as: Azman, M.A., et al., The accuracy of preliminary cost estimJournal of Project Management (2012), http://dx.doi.org/10.1016/j.ijproman.2012.11

3.4. ANOVA test on mean estimate bias

Factorial ANOVA between groups examines the bias (error) ofthe estimate against the project characteristics, which includes thegroups of factors. It looks into the main effect of one predictorvariable on an outcome variable while ignoring the effects of allother predictor variables. The data must meet certain assumptionsof the parameter. Field (2009) points out the distributionswithin groups must be normally distributed and it must meetcondition in relation to homogeneity of variance. The Shapiro–Wilk test examines the normality of data for small sample size(Hun Myoung Park, 2008). It compares a set of measures againstthe normal distribution. It tests whether the data is normallydistributed. Levene's test measures the homogeneity of varianceby accessing the equality of variances of the groups. Tukey's testis used for pair-wise comparisons (Fellows and Liu, 2008). It isbecause project characteristics are divided into different groups.

3.5. Levene's test on consistency

Levene's test investigates whether the consistency varies ineach project characteristics. It measures the equality of thevariance. The analysis used Levene's test because it is lesssensitive to skew distribution. Analysis uses median value as itprovides robustness against non-normal data (Schultz, 1985).

3.6. Linear multiple regression analysis on the estimate bias

It investigates the relationship between the estimate bias andproject characteristics. The analysis assumes linear relationshipbetween variables. According to Field (2009), any research looksfor real-life relationships should adopt linear multiple regression(LMR). The analysis uses “stepwise technique” for selecting andremoving predictors in the model. It must meet the parametricassumptions of the model that includes several assumptions onmulticolinearity, homoscedasticity, normally distributed errorsand number of extreme cases that affects the model. Errorsincrease or decrease with some variable such as contract valuewhich depends on the way the error is measured. For example, ifthe dollar difference between bids and estimates, the LMR mayfind a positive correlation with project value. Then again, usingpercentage error, it may find a negative correlation with contractvalue. It is very important because the correlations betweenvariables are much depending on the way bias is measured.

3.7. Analysis on questionnaire survey

The chi-square test is used to examine differences withcategorical variables. It tests whether the groups of respondents'responses are different significantly from the expected frequenciesand the observed frequencies (Field, 2009). Mann–Whitney test isused on ordinal questions that are based on five (5) points Likert'sScale. It looks for differences in the rank between two inde-pendent samples. It tests whether the populations from the twosamples are drawn from the same distribution (Field, 2009). Itis assumed the answered questionnaire survey follows non-parametric distribution.

ates in Public Works Department (PWD) of Peninsular Malaysia, International.008

Page 6: The accuracy of preliminary cost estimates in Public · PDF fileThe accuracy of preliminary cost estimates in Public Works Department (PWD) of Peninsular Malaysia Mohd Azrai Azman

Table 4Bias parameter according to different estimating targets.

Target 95% confidenceinterval

Percentile 25th,50th and 75th

Lowest bid 22.01 (L), 28.09 (U) 16.43, 25.48, 33.57Accepted bid 8.84 (L), 13.52 (U) 4.88, 11.28, 18.54Mean of the bids 8.57 (L), 13.19 (U) 3.57, 11.86, 19.10

Note: L=lower value, U=upper value

Table 5Initial analysis on project characteristics.

Variable N Lowest bid Accepted bid Mean of thebids

Bias cv Bias cv Bias cv

Total 83 25.05 11.13 11.18 9.63 10.88 9.54

Project value (RM)1–1,000,000 16 17.39 14.96 3.62 12.08 2.61 12.431,000,001–3,000,000 33 26.20 10.56 12.43 8.93 13.18 8.553,000,001–5,000,000 22 26.81 9.92 11.09 8.62 10.38 8.28N5,000,000 12 28.88 7.63 18.01 5.24 16.52 4.47

Project size (m2)1–1500 31 19.84 12.49 5.85 10.95 5.23 10.791501–2500 23 27.26 11.39 13.78 7.95 14.02 8.272501–3500 21 28.30 8.20 15.45 8.14 14.07 7.62N3500 8 30.38 9.03 13.19 6.31 15.39 4.86

PI (cost/m2GFA)Lowest bid*1–1000 13 40.25 10.501001–1500 39 23.65 7.601501–2000 18 23.21 9.52N2000 13 16.63 14.81Accepted bid1–1200 16 16.22 7.541201–1700 40 11.85 8.331701–2200 14 10.79 9.15N2200 13 3.37 13.49Mean of the bids1–1200 16 18.96 6.451201–1700 38 11.07 7.931701–2200 15 8.62 8.05N2200 14 3.55 13.21

Story height1 story 20 20.96 13.78 6.29 11.79 6.46 12.222 story 9 23.20 9.63 10.48 9.09 6.52 8.143 story 20 22.41 7.95 11.10 9.68 10.87 8.504 story 34 29.51 10.85 14.30 7.80 14.64 7.76

Contract Period (weeks)

6 M.A. Azman et al. / International Journal of Project Management xx (2012) xxx–xxx

4. Results

4.1. Initial data analysis

Tables 3 and 4 show the parameter of estimating bias. Thescores of accepted bid and mean of the bids give a similarparameter value. According to paired-samples t-test, there is nosignificant mean difference between the mean of the bids'errors and accepted bid's errors (PN0.05) but there is asignificant mean difference between lowest bid's errors ifcompared to other estimating targets (mean of the bids andaccepted bid) (Pb0.05). At 95% confidence interval, allestimates were overestimated. Table 5 shows initial analysison project characteristics.

4.2. Analysis on elemental cost estimate

Table 6 shows the accuracy according to elemental costs.Most of the time, area method is used to estimate the buildingcosts. Other elements used various methods such as approximatequantities and percentage allowances. Piling and foundationrecorded the highest mean error at 44.52%. Two (2) projects wereestimated without piling works but four (4) projects did notrequire piling works at all during tender stage. It is followed byexternal works at 42.86%, building works at 24.26% and internalservices at 3.63%. Preliminaries are found with the least meanerror at 0.58%. Internal services are the most consistent estimatesat 20.61cv. It is followed by building works at 49.26cv, prelim-inaries at 59.61cv and external works at 69.20cv. Piling andfoundation is the most inconsistent estimates at 198.25cv. Buildingworks contributed 48.33% of the total construction cost, which isthe highest. It is followed by internal services at 24.67%, piling andfoundation at 11.66% and external works at 9.99%. The lowest ispreliminaries, which is at 5.35%.

4.3. ANOVA test on mean bias

ANOVA test examines mean bias of the estimate varies indifferent project characteristic. Shapiro–Wilk test shows onlyfew groups are not equally distributed (Pb0.05). These includethe group of more than RM 5,000,000, 2501–3500 m2 andsecondary school in lowest bid and 2 story heights in acceptedbid. It was found that all the groups in estimate/mean of the bidsare equally distributed. Table 7 shows the error variances ofoutcome variables are equal across the groups in all models.

ANOVA test results show there are significant differencesbetween the mean biases in the groups of factors (Pb0.05)which is price intensity (F=8.122, η2 =0.296), state of project(F=3.674, η2 =0.202) and type of school (F=5.917, η2 =

Table 3Mean bias using different estimating targets.

Comparison N Bias (%) Consistency (cv)

Estimate and lowest bid 83 25.05 11.13Estimate and accepted tender bid 83 11.18 9.63Estimate and mean of the bids 83 10.88 9.54

1–35 18 19.15 14.54 5.91 12.93 4.96 13.2636–52 43 25.66 10.23 13.65 9.12 11.99 8.74N52 22 28.70 9.25 10.69 6.14 13.57 5.99

Number of tenders1–7 13 21.04 10.97 14.15 8.25 8.45 10.058–15 33 23.63 11.21 9.75 10.90 7.93 9.4516–23 26 27.60 11.17 11.44 9.91 13.83 9.57N23 11 28.02 11.28 11.37 6.69 15.62 6.59

Please cite this article as: Azman, M.A., et al., The accuracy of preliminary cost estimates in Public Works Department (PWD) of Peninsular Malaysia, InternationalJournal of Project Management (2012), http://dx.doi.org/10.1016/j.ijproman.2012.11.008

Page 7: The accuracy of preliminary cost estimates in Public · PDF fileThe accuracy of preliminary cost estimates in Public Works Department (PWD) of Peninsular Malaysia Mohd Azrai Azman

Table 5 (continued)

Variable N Lowest bid Accepted bid Mean of thebids

Bias cv Bias cv Bias cv

StateA — Kedah 36 26.64 10.30 12.28 8.08 13.55 8.10B — Perak 7 21.93 10.90 8.66 7.24 7.48 5.26C — Negeri Sembilan (N9) 5 35.49 1.99 14.31 8.67 19.29 5.76C1 — Malacca 11 13.69 17.02 −4.34 9.25 −2.84 9.68D — Johor 24 26.62 8.95 16.74 7.03 12.41 8.55

Type of schoolPrimary 28 19.10 12.57 5.87 10.66 6.33 10.73Secondary 55 28.08 9.73 13.89 8.28 13.20 8.29

Class of projectMain building 33 26.64 11.43 13.57 8.49 13.67 8.76Secondary building 23 20.96 13.48 5.99 11.03 5.85 11.66Mix building 27 26.60 8.28 12.69 8.84 11.75 7.41

Note: PI=price intensity. The groups of PI have a different interval values inorder to ensure the group sizes are almost balance.

Table 7Levene's test on equality of error variances.

Test subject F df1 df2 P

Estimate/lowest bid 0.909 72 10 0.626Estimate/accepted bid 0.937 74 8 0.605Estimate/mean of the bids 1.996 72 10 0.116

Note: statistical significance is set at the 5% level.

7M.A. Azman et al. / International Journal of Project Management xx (2012) xxx–xxx

0.093) which are statistically different in estimate/lowest bid. Inestimate/accepted bid, contract value (F=2.985, η2 =0.134)and state (F=7.223, η2 =0.332) are statistically different. Inestimate/mean of the bids, project size (F=3.713, η2 =0.161),price intensity (F=4.271, η2 =0.181) state (F=9.344, η2 =0.392) and school (F=5.414, η2 =0.085) are statisticallydifferent. The following are the results from Tukey's test,which examines the significant mean differences between thegroups:

4.3.1. Estimate/lowest bid

• Price intensity of RM 1–1000/m2GFA has a significantmean difference if compared to other price intensity values.

• Malacca has a significant mean difference if compared toother states except Perak.

4.3.2. Estimate/accepted bid

• Project value of RM 1–1,000,000 has a significant meandifference if compared to other project values.

• The state of Malacca has a significant mean difference ifcompared to other states.

Table 6Accuracy of elemental costs.

Element Amount (RM) % ofelement

Error (%) sd cv

Total 232,508,387.06 100.00 11.18 10.71 9.63Preliminaries 12,446,277.61 5.35 0.58 59.95 59.61Piling and

foundation27,101,527.60 11.66 44.52 286.52 198.25

Building works 112,370,446.99 48.33 24.26 61.22 49.26Internal services 57,356,052.02 24.67 3.63 21.36 20.61External works 23,234,082.84 9.99 42.86 98.87 69.20

Please cite this article as: Azman, M.A., et al., The accuracy of preliminary cost estimJournal of Project Management (2012), http://dx.doi.org/10.1016/j.ijproman.2012.11

4.3.3. Estimate/mean of the bids• Project size of 1–1500 m2GFA has a significant meandifference if compared to other project sizes.

• Price intensity value of RM 1–1000/m2GFA has a significantmean difference if compared to other price intensity values.Price intensity value of more than RM 2201/m2GFA has asignificant mean difference if compared to other price intensityvalues but it has no significant mean difference if compared toRM 1701–2200/m2GFA.

• The state of Malacca has a significant mean difference ifcompared to other states. N9 has a significant mean differenceif compared to the state of Perak.

4.4. Levene's test on consistency

Levene's test of homogeneity of variance examines the equalityof variances in each of the project characteristics. Contract periodhas unequal variances (heterogeneous) if compared to other projectcharacteristics. Thus, it affects the estimating consistency. Inestimate/accepted bid, the result shows df1=2, df2=80 andstatistic=3.476(Pb0.05).In estimate/mean of the bids, the resultshows df1=2, df2=80 and statistic=4.193(Pb0.05).

4.5. Linear multiple regression analysis on estimating bias

LMR demonstrates the relationship between project charac-teristics and estimating bias. It uses a number of estimatingtargets from lowest bid, accepted bid and mean of the bids asoutcome variables. There are the nine (9) project characteristicsregressed for linear relationships. Predictors of state, type ofschools and class of projects are converted into dummyvariables because these are categorical variables (nominalscale). β0 is for beta (intercept), β1 for beta (predictors) and ε israndom error of Y. The adjusted formula contains 14 predictorsas follows:

Y ¼ β0 þ β1 Contract Valueð Þ þ β2 m2GFA� �

þβ3 cost=m2GFA� �þ β4 Storeyð Þ

þβ5 Contract Periodð Þ þ β6 Number of Biddersð Þþβ7 Kedahð Þ þ β8 Perakð Þ þ β9 N9ð Þ þ β10 Malaccað Þþβ11 Johorð Þ þ β12 Primary schoolð Þþβ13 Secondary schoolð Þ þ β14 Main buildingð Þþβ15 Ancillary buildingð Þ þ β16 Mix buildingð Þ þ ε:

Table 8 shows the results from LMR. All models fromdifferent estimating targets are useful (Pb0.05) with F-value at10.720, 25.984 and 13.762 respectively. The predictors of

ates in Public Works Department (PWD) of Peninsular Malaysia, International.008

Page 8: The accuracy of preliminary cost estimates in Public · PDF fileThe accuracy of preliminary cost estimates in Public Works Department (PWD) of Peninsular Malaysia Mohd Azrai Azman

Table 8Linear multiple regression test results on percentage error.

Model Predictor Standard error t-value P-value

Estimate/lowest bid Constant 2.619 13.359 0.000cost/m2GFA 0.001 −3.447 0.001Primary 2.914 −2.810 0.006

Estimate/accepted bid Constant 1.166 13.198 0.000Malacca 2.774 −5.936 0.000Primary 1.989 −3.009 0.004

Estimate/mean ofthe bids

Constant 2.993 0.196 0.845Malacca 2.838 −4.132 0.000Size 0.001 2.309 0.024Number of bidders 0.138 2.639 0.010Secondary school 2.024 2.278 0.025

Note: statistical significance is set at the 5% level.

8 M.A. Azman et al. / International Journal of Project Management xx (2012) xxx–xxx

the models have taken into account 19.2%, 37.9% and 38.4% (R2

Adjusted) of variance in Y. The model from estimate/mean of thebids has the lowest standard error of the estimate than the othersat 8.306. The model from lowest bid and accepted bid have ahigher standard error of estimate at 12.514 and 8.441 respective-ly. A model with a smaller standard error of the estimate is better.

Residuals are randomly dispersed around zero, which suggestsno violation of homoscedasticity. Co-linearity diagnostics showno models have a condition index that more than 15, whichsuggests no real serious problem in relation to multicolinearity(Tabachnick and Fidell, 2007). Table 9 shows that residuals in allmodels are normally distributed (Pb0.05). Case-wise diagnosticsshow that six cases of standardized residuals in lowest bid modelhave values outside +/−2.00. Meanwhile, accepted bid and meanof the bids models only have three cases outside this limit and nocases in all models are more than +/−3.00. The regressionequations are as follows:

Y(estimate/lowest bid) model 1 =34.986–0.004 (cost/m2GFA)–8.190(primary school)

Y(estimate/accepted bid) model 2 =15.386–16.465 (Malacca)–5.986(primary school)

Y(estimate/mean of the bids) model 3 =0.588–11.727 (Malacca)+0.002(m2GFA)+0.363 (number of bidders)+4.610 (second-ary school).

4.6. Questionnaire survey on estimating bias

The respondents were asked which estimation value is themostacceptable for public client. More than half (59.2%) of therespondents from PWD and private QS consultants agreed that theoverestimate was the most acceptable value for public client.

Table 9Kolgomorov–Smirnov test on standardized residual.

Subject Kolmogorov–Smirnov a

Statistic df Sig.

Standardized Residual (lowest bid) .080 83 .200 ⁎

Standardized Residual (accepted bid) .067 83 .200 ⁎

Standardized Residual (mean of the bids) .047 83 .200 ⁎

a Lilliefors Significance Correction.⁎ This is a lower bound of the true significance.

Please cite this article as: Azman, M.A., et al., The accuracy of preliminary cost estimJournal of Project Management (2012), http://dx.doi.org/10.1016/j.ijproman.2012.11

Underestimate was the least agreed by respondents (10.2%).Morethan a quarter of total respondents, (30.6%) did not know aboutwhich values were more acceptable. Chi-square test shows there isa significant difference between type of respondent's organizationand level of tolerances, χ2 (2)=7.060, Pb0.05. Even thoughmorethan half from both sectors agreed with overestimated value, therewere considerable respondents (37.5%) from PWD who did notknow which value is more acceptable to government.

The respondents were asked whether they intentionallymark-up their estimates to reduce the risk of underestimation.Close to half the number of respondents (48.4%) did it. Less thanhalf of respondents (43.9%) disapprove of it. Few respondents(7.6%) said they are clueless. Chi-square test shows χ2 (2)=9.446, Pb0.05. There are differences between types of re-spondents' responses because more than half of private QS(56.6%) consultants did not do it.

The respondents were asked about a number of methodswhich could improve the accuracy according to Ling and Boo(2001). It shows that QS in PWD ranked the use of costplanning and cost control during design stage lower (meanrank=71.98) while QS in private consultants ranked it higher(mean rank=88.03). Mann–Whitney test shows a significantdifference in rank between QS in PWD and QS in privatesector, U=2118.500, Pb0.05, r=−0.181.

5. Discussion

5.1. Estimating accuracy in Public Works Department

Most of the time, the estimates are overestimated. The occur-rences of underestimated estimates are unlikely. Research byGunner and Skitmore (1999b) and Aibinu and Pasco (2008) alsoshows the same result. Overestimate is more acceptable to QS andClient (Cheung et al., 2008; Magnussen and Olsson, 2006;Skitmore and Cheung, 2007). QS are deliberately bias rather thanby mistake because they want to achieve certain result (Flyvbjerget al., 2002). Nevertheless Soo and Oo (2007) point out QSoverestimate the estimate because of the lack of information. Thisshows that QS try to reduce the risks by increasing the level ofprice. However, in terms of estimation bias, the estimate isacceptably accurate because the value is +/−20% which is withinthe value of +/−26% by Morrison (1984). He finds that theestimate never getting any better when it is prepared based ontraditional estimating method.

The estimate is consistent because the value of coefficientvariation (cv) is at 10% to 11% which is nearer to 9% cv. It isbecause substantial design is already completed when usingstandard pre-design plans (Ashworth and Skitmore, 1982).According to Skitmore and Ng (2000), consistency of preliminaryestimate is suggested at around 15% cv if the estimate usespreliminary design. Accuracy of elemental items shows a highmargin of bias and highly inconsistent estimate. The mean biasusing area method is 25% for building works. The mean bias forfoundation and external works using average percentage allocationis 45%. The use of area method and approximate quantities gives amean bias of around 20–30% and 15–25% respectively(Boussabaine, 2007; Skitmore and Patchell, 1990). Perhaps, this

ates in Public Works Department (PWD) of Peninsular Malaysia, International.008

Page 9: The accuracy of preliminary cost estimates in Public · PDF fileThe accuracy of preliminary cost estimates in Public Works Department (PWD) of Peninsular Malaysia Mohd Azrai Azman

.00 10.00 20.00 30.00 40.00

Number of bidders

-20.00

.00

20.00

40.00

60.00

80.00

Bia

s (%

)

Fig. 1. Scatter plot shows bias (%) against number of bidders.

9M.A. Azman et al. / International Journal of Project Management xx (2012) xxx–xxx

gives us the indication that a less rigorous estimating method wasused or lack of information supplied to prepare the estimate. Theuse of more rigorous estimating method may improve the accuracybut if the underestimate is unfavorable; the persistent bias couldstill be presented. The estimate appears to be accurate because ofthe effect of error canceling e.g. underestimated estimates werecovered by other overestimated elements (Morrison, 1984).

5.2. Best model to explain bias estimate

The estimate is extremely biased if lowest bid is usedbecause of the lowest price submission. The PWD selects thegroup of tender prices from the mean of the bids and the systemis based on average bid method (Public Works Department,2004, 2005). The use of accepted bid as target is bias becauseQS and clients could determine the level of price. The use ofmean of the bids is better because it has less suicidal low bidsbut it could be intervening by the uncompetitive bids(McCaffer, 1976; Raftery, 1991). The regression model frommean of the bids shows project size, number of tenders, type ofschools and state are the best-fit predictors to explain the biasestimate. It has the largest value of adjusted R2. It shows thatmean of the bids is the most appropriate target to measure theaccuracy of the estimate in the PWD. Lowest bid is not thesuitable target to measure the estimating performance in therespect of competitive tendering in the PWD.

5.3. Factors which affects the accuracy

5.3.1. Project value (RM) and project size (m2GFA)Bias increases with project size is consistent with the

findings by Kiew (2009) in Sarawak and Skitmore and Drew(2003) but the former is more negatively biased. It is different ifcompared to Morrison and Stevens (1980), Skitmore and Tan(1988) and Gunner and Skitmore (1999a) and Aibinu and Pasco(2008). There is some reason on this phenomenon. QS in thePWD are more biased towards more expensive and largeproject size as those projects are more complicated with lessinformation especially on foundation and external works. Theestimate of expensive projects is more consistent. It contradictsthe finding by Ogunlana and Thorpe (1991) in which largeproject is less consistent. It may happen due to the importanceof large projects to QS than small and inexpensive projects. QSare consistent in increasing their pricing margin for expensiveprojects regularly but not for cheap projects.

5.3.2. Price intensity theory (RM/m2GFA)The finding of price intensity theory in terms of cost/m2GFA

which was suggested by Gunner and Skitmore (1999b) andSkitmore and Drew (2003) is similar to this finding. Buildingswith low unit rate were overestimated, while high unit rate wereunderestimated. This price intensity variable is significant inLMR but if lowest bid is the target. Nevertheless, the outcomeshows that price intensity is not the significant predictor in meanof the bids. This happens because of some confounding variablesin the model (Gunner and Skitmore, 1999b).

Please cite this article as: Azman, M.A., et al., The accuracy of preliminary cost estimJournal of Project Management (2012), http://dx.doi.org/10.1016/j.ijproman.2012.11

5.3.3. Number of biddersThe additional number of bidders led the estimate to be more

biased. LMR shows the predictor is significant in estimate/meanof the bids. It contradicts the findings by Harvey (1979), andFlanagan and Norman (1983). Here, bidders decreased their bidamount when they are expecting more bidders enter the bid butQS does not expect this trend and they are unlikely to follow suit.Consistency improves in the mean of the bids with additionalbidders entering the bid. Findings by Skitmore (2002) show thatconsistency decreases with additional number of bidders in sevendata sets and the trend was found reversed in other three data sets.Therefore, the relationship between consistency and number ofbidders is unclear. It contradicts the bidding model efficacy thatsuggests that consistency decrease with the additional number ofbidders. One possible explanation is most of cost data are pooledfrom a large number of bidders, which has almost same level ofcompetition. The estimation consistency reduce when QS needsto predict future projects that have a smaller number of bidders.The use of accepted bid as the target has no significant linearcorrelation with the bias (refer to Fig. 1). It affects the observationof the trend for future research especially for cost modelingpurposes due to the intervention of clients and designers inselecting the appropriate prices. The use of accepted bid maycontradict the effect of competition. The number of bidders wasregressed using linear and logarithmic curve estimation. Thenumber of bidders is found significant (Pb0.05) if estimate/meanof the bids is used but not for other targets. Fig. 2 shows mean ofthe bids above estimate against number of bidders.

5.3.4. Location (state)There are bias differences between state to state (location)

which is similar to the findings by Ogunlana and Thorpe (1991)and Harvey (1979). According to Pegg (1984), the different pricesfor labor and materials in each location have caused the bias.ANOVA test shows high impact (effect size) of state variable.Malacca and Negeri Sembilan are grouped in the same location(refer to Table 10). However, the finding shows that the differencesbetween these two states are statistically significant. AlthoughPeninsular Malaysia is small, there is a local economic impact on

ates in Public Works Department (PWD) of Peninsular Malaysia, International.008

Page 10: The accuracy of preliminary cost estimates in Public · PDF fileThe accuracy of preliminary cost estimates in Public Works Department (PWD) of Peninsular Malaysia Mohd Azrai Azman

42403836343230282624222018160 2 4 6 8 10 12 14

number of bidders

1.20

1.10

1.00

.90

.80

.70

Fig. 2. Mean of the bids above estimate against number of bidders.

10 M.A. Azman et al. / International Journal of Project Management xx (2012) xxx–xxx

the construction prices due to economic disparity between thestates as the demand and supply in the states are different.

5.3.5. Types of schoolsANOVA test shows that the type of school has significant

mean differences in lowest bid and mean of the bids. LMR showsthat all comparisons are statistical significant. These are similar tofindings of Harvey (1979) and Morrison and Stevens (1980), buttheir results are based on different types of schemes. Theinformation regarding the complexity of the design involved isnot available but it was made known to the authors that thedesigns of these schools are different. It could happen because ofthe design complexity for secondary schools. It shows that thesecondary school is biased than primary school.

5.3.6. Contract periodIt was found that only contract period is statistically significant

for consistency homogeneity. The consistency improves whencontract period increases. Findings from Skitmore (1988) andGunner and Skitmore (1999a) show that no conclusion can bemade. The estimates for small projects, which have shorterconstruction period, are most likely unpredictable than biggerprojects. The use of the same group of historical cost data may notbe reliable if there is a difference between the contract period of

Table 10States grouping for locality index adjustments in the year 2009.Abstracted from Public Works Department, 2009.

Group A B C D E F

States Perlis,Kedah andPulauPinang

Perak Selangor,WilayahPersekutuan,NegeriSembilanand Malacca

Johor Pahang TerengganuandKelantan

1.0816 1.0466 1.0000 1.0567 1.0438 1.0417

Please cite this article as: Azman, M.A., et al., The accuracy of preliminary cost estimJournal of Project Management (2012), http://dx.doi.org/10.1016/j.ijproman.2012.11

new project and previous project. It is suggested that the poolingof cost data for the estimate of new projects should also reflect thecontract period of historical project data.

5.4. Overestimating bias in PWD cost estimation

The earlier discussion discusses the findings based on leastsquares but not the procedure biases embedded in the PWDpractice. As usual, the Government of Malaysia throughMinistry of Finance (MOF) sets aside allocation for publicprojects according to its annual plan, which depends on expectedeconomic conditions. What happened during that year was thatthe government tried to improve the decline economy throughfiscal instrument under the Ninth Malaysia Plan (2006–2011).The development plan took place ahead of general election in2008. Under the treasury requirement, public money allocatedyearly must be fully spend by all project implementers e.g. PWD(National Audit Department, 2007). One of the allocationsprovided for PWD was to construct additional pre-design schoolprojects around Malaysia, in which the data samples werecollected for this research. To confirm the allocation provided,PWD prepares preliminary cost estimate.

The trend shows overestimation of projects when projectvalue (RM) increased. One possible explanation is the QSofficers try to maximize the budget allocated. In order to do so,elemental costs for piling and external works were pricedhigher as these are more risky than the other elements. Over-estimation could provide a safety net if the tender prices are onthe high side because QS officers also need to make sure that thebudget allocated is sufficient during the tender stage and thisleads to the project being started within the one-year time frameprovided by MOF. This reduces another round of process torequest for more budgets.

It shows that QS officers are more likely to fulfill the budgetrequirement rather than to leave the estimate as it is. According toRaftery (1995), public clients prefer not to under-spend budgetsbecause of opportunity costs public money lying idle. Thequestionnaire result shows that the private QS consultantswhich are hired by PWD are unlikely to “adjust” their estimatebecause they are less likely to work under the pressure of thegovernment procedure. According to Wachs (1990), biasestimation happened because of organization loyalty towardsits superior directive order rather than scientific objectivity.Questionnaire survey shows that QS in PWD expected theestimate to be priced higher than the theoretical value whichhas a near to zero mean.

The bigger the project, the more extensive are the piling andexternal works required. It is because only design works forbuildings are standardized and others are not. Lack of informationduring this stage may lead QS officers to find justification tomaximize the use of the budget. This might be true becauseCheung et al. (2008) describe that most clients did not consideraccurate estimate as the important criteria for estimating perfor-mance. Their primary concern is more on cost sensitive elements.The reward of making such estimation error (overestimation) is notto underutilize the budget. The questionnaire result also shows thatthe importance of cost planning is not a critical procedure for PWD

ates in Public Works Department (PWD) of Peninsular Malaysia, International.008

Page 11: The accuracy of preliminary cost estimates in Public · PDF fileThe accuracy of preliminary cost estimates in Public Works Department (PWD) of Peninsular Malaysia Mohd Azrai Azman

11M.A. Azman et al. / International Journal of Project Management xx (2012) xxx–xxx

officers. The result rejects the idea that the client could have bettercost planning provided by Cheung et al. (2008). QS in PWD usedintuition to identify cost sensitive elements. It means PWD is lesslikely to prepare a detailed estimate through more definite costplanning because they are more concerned about the time taken toroll out the projects and amount of money to be allocated.

6. Conclusion

Overestimation bias was found in this study. However, theestimates are found to be acceptably accurate in terms of bias andconsistency. The best linear model to explain the accuracy ofpreliminary cost estimate in the PWD is the data from estimate/mean of the bids when compared to other estimating targets.Project size, number of bidders, type of school and location aresignificant predictors affect the estimate. Nevertheless, locationof project (state) has a larger effect size if compared to others.Contract period affects the consistency of the estimate. Thedifference when using accepted bid as the target is that it couldnot show the impending competition in bidding when morebidders enter the contract bidding. This affects the observation ofsystematic bias in the analysis when using different targets. Thecomparison between project characteristics and elemental costsshows that lack of accuracy in early design stage is caused by lackof information from designers. Data analysis shows that theelement of substructure (Piling and Foundations) and externalwork elements are the most bias and inconsistent. Foundationwork and external work are the ones with less information. This isbecause piling works and external works are not identical fromone project to another due to different soil condition. Perhaps, inorder to improve the accuracy of these two (2) elements, engineersconcerned should provide more detailed information for these twoelements. In addition, building work still constitutes the highestamount of construction works. A larger error in this elementaffects greatly the total accuracy of construction works. The use ofpercentage allocation and average cost contributes to more biasestimate. The use of more reliable estimating methods and theimprovement of cost data pooling improves the accuracy. Inaddition, PWD needs to embrace proper cost planning procedure.The present practice that divides state according to group shouldbe revised, as it was found that some state in the same group arestatistically different from each other. However, as an alternative,the overestimation bias happened because of directive order fromthe government that is affected by procedure biases, whichrequired public projects to initiate according to the developmentplan. Public projects are more often political in nature. The risk ofunderestimation is that the project could not be initiated on timethus, it may put public officers in difficult position and only withthe overestimation bias could they deal with it and the resultpleases the top officials and politicians.

Acknowledgments

The authors thankfully acknowledge the support of the PublicWorks Department of Malaysia, which contributed the informa-tion needed and Universiti Teknologi MARA for providing thescholarship to the corresponding author.

Please cite this article as: Azman, M.A., et al., The accuracy of preliminary cost estimJournal of Project Management (2012), http://dx.doi.org/10.1016/j.ijproman.2012.11

ReferencesAbdul-Aziz, Abdul-Rashid, Ali, Normah, 2004. Outsourcing and quality perfor-

mance: Malaysia's public works department. Structural Survey 22, 53–60.AbouRizk, S.M., Babey, G.M., Karumanasseri, G., 2002. Estimating the cost of

capital projects: an empirical study of accuracy levels for municipalprojects. Canadian Journal of Civil Engineering 29, 653–661.

Aibinu, A.A., Pasco, T., 2008. The accuracy of pre-tender building cost estimatesin Australia. Construction Management and Economics 26, 1257–1269.

Ashworth, A., 2010. Cost Studies of Buildings, 5 ed. Pearson EducationLimited, Essex.

Ashworth, A., Skitmore, M., 1982. Accuracy in estimating. Occasional PaperNo, 27, pp. 412–437.

Boussabaine, A., 2007. Cost Planning of PFI and PPP Building Projects. Taylor& Francis, Oxford.

Carr, P.G., 2005. Investigation of bid price competition measured throughprebid project estimates, actual bid prices, and numbers of bidders. Journalof Construction Engineering and Management 131, 1165–1172.

Chappell, D., Marshall, D., Powell-Smith, V., Cavender, S., 2001. BuildingContract Dictionary. Blackwell Science Ltd, Oxford.

Cheung, F.K.T., Wong, M.W.L., Skitmore, M., 2008. A study of clients' andestimators' tolerance towards estimating errors. Construction Managementand Economics 26, 349–362.

Dattalo, P., 2008. Pocket Guide to Social Work Research Methods:Determining Sample Size: Balancing Power, Precision and Practicality.Oxford University Press, New York.

Dell'Isola, M.D., 2002. Architect's Essentials of Cost Management. John Wiley& Sons, Inc., New York.

Public Works Department, 2010. Guideline for Public Works' ContractManagement. In: P.W. Department (Ed.), 3rd ed. Public Works Department,Kuala Lumpur.

Faul, F., Erdfelder, E., Lang, A.-G., Buchner, A., 2007. G*Power 3: a flexiblestatistical power analysis program for the social, behavioral, and biomedicalsciences. Behaviour Research Methods 39, 175–191.

Fellows, R., Liu, A., 2008. Research Methods for Construction, 3rd ed.Blackwell Publishing, Oxford.

Field, A., 2009. Discovering Statistics using SPSS, 2nd ed. Sage PublicationsLtd, London.

Flanagan, R., Norman, G., 1983. The accuracy and monitoring of quantitysurveyors' price forecasting for building work. Construction Management& Economics 1, 157–180.

Flyvbjerg, B., 2009. Survival of the unfittest: why the worst infrastructure getsbuilt — and what we can do about it. Oxford Review of Economic Policy25, 344–367.

Flyvbjerg, B., Holm, M.S., Buhl, S., 2002. Underestimating the costs in publicworks projects: error or lie. Journal of the American Planning Association68, 279–295.

Fortune, C., Cox, O., 2005. Current practices in building project contract priceforecasting in the UK. Engineering Construction and ArchitecturalManagement 12, 446–457.

Gibb, A.G.F., 2001. Standardization and pre-assembly distinguishing mythfrom reality using case study research. Construction Management &Economics 19, 307–315.

Gunner, J., Skitmore, M., 1999a. Comparative analysis of pre-bid forecasting ofbuilding prices based on Singapore data. Construction Management andEconomics 17, 635–646.

Gunner, J., Skitmore, M., 1999b. Pre-bid building price forecasting accuracy:price intensity theory. Engineering Construction and Architectural Man-agement 6, 267–275.

Halil, Faridah, 2007. Contractor's perception of the use of statistical approach inthe tender evaluation at the public works department, Malaysia. AmericanJournal of Applied Sciences 4, 1084–1089.

Hamid, Sharinah, 2008. Briefing on Tender Evaluation Criteria for ElectricalContractors. Public Works Department, Kuala Lumpur, Electrical Work Branch.

Harvey, J.R., 1979. Competitive Bidding on Canadian Public ConstructionContracts: Stochastic Analysis for Optimization. University of WesternOntario, Ontario.

Herdsman, Z., Ellis, R.D., 2006. The cost/time/quality integrated bidding system— an innovation in contract administration, In: Bezelga, A., Brandon, P.

ates in Public Works Department (PWD) of Peninsular Malaysia, International.008

Page 12: The accuracy of preliminary cost estimates in Public · PDF fileThe accuracy of preliminary cost estimates in Public Works Department (PWD) of Peninsular Malaysia Mohd Azrai Azman

12 M.A. Azman et al. / International Journal of Project Management xx (2012) xxx–xxx

(Eds.), Management, Quality and Economics in Building, 2nd ed. FN Spon,London, pp. 109–121.

Hillebrandt, P.M., 2000. Economic Theory and the Construction Industry, 3rded. Macmillan Press Ltd, London.

Hun Myoung Park, 2008. Univariate analysis and normality test using SAS,Stata, and SPSS. The University Information Technology Services (UITS)Center for Statistical and Mathematical Computing. Indiana University,Indiana.

Ioannou, P.G., Leu, S.-S., 1993. Average-bid method: competitive biddingstrategy. Journal of Construction Engineering and Management 119.

Karlsen, J.T., Lereim, J., 2005. Management of project contingency andallowance. Cost Engineering 47, 24–29.

Kiew, Y.H., 2009. Accuracy of Pre-tender Cost Estimates of Quantity SurveyorConsultants Faculty of Built Environment. Universiti Teknologi Malaysia,Skudai.

Kumaraswamy, M.M., Walker, D.H.T., 1999. Multiple performance criteria forevaluating construction contractors. In: Rowlinson, S., McDermott, P.(Eds.), Procurement Systems: A Guide to Best Practice in Construction. E &FN Spon London.

Ling, Y.Y., Boo, J.H.S., 2001. Improving the accuracy of approximate estimatesof building projects. Building Research & Information 29, 312–318.

Lowe, D., Skitmore, M., 2006. Bidding. In: Lowe, D., Leiringer, R. (Eds.),Commercial Management of Projects: Defining the Disipline. BlackwellPublishing, Oxford.

Magnussen, O.M., Olsson, N.O.E., 2006. Comparative analysis of costestimates of major public investment projects. International Journal ofProject Management 24, 281–288.

McCaffer, R., 1976. Contractors' Bidding Behaviour and Tender PricePrediction. Loughborough University of Technology, Loughborough.

Morrison, N., 1984. The accuracy of quantity surveyor's cost estimating.Construction Management & Economics 2, 57–75.

Morrison, N., Stevens, S., 1980. Construction Cost Data Base, 2nd AnnualReport of Research Project. Department of Construction Management,University of Reading for Property Services Agency, Directorate ofQuantity Surveying Services, DoE, Reading, London.

Morton, R., Jaggar, D., 1995. Design and the Economics of Building. Spon Press.Murdoch, J., Hughes, W., 2007. Construction Contracts: Law and Management,

4th ed. Spon Press, London.National Audit Department, 2007. Auditor-general Reports. National Audit

Department, Putrajaya.Norwina, M.N., 2006. Standardisation and customisation of healthcare facility

design in nation building, Health facilities symposium on designing forsustainable and safe healthcare facilities in Africa. The Public Health Groupof the International Union of Architects (UIA-PHG), Pretoria.

Odusami, K.T., Onukwube, H.N., 2008. Factors affecting the accuracy of pre-tender cost estimate in Nigeria. The Construction and Building ResearchConference of the Royal Institution of Chartered Surveyors. RoyalInstitution of Chartered Surveyors, Dublin Institute of Technology.

Ogunlana, S., Thorpe, A., 1991. The nature of estimating accuracy: developingcorrect associations. Building and Environment 26, 77–86.

Pegg, I., 1984. The effect of location and other measureable parameters on tenderlevels, cost study F33, BCIS 1983-4-219. BCIS, RICS, London . pp. 5–17.

Public Works Department, 1992. Guidelines for preparation and submission ofpreliminary detailed abstract, as tendered detailed abstract and as completeddetailed abstract for building works. In: P.W. Department (Ed.), PublicWorks Department, Kuala Lumpur.

Public Works Department, 2004. Utilization and cut-off pricing based onstatistical methods in the evaluation of tenders and recommendations. In:P.W. Department (Ed.), Public Works Department, Kuala Lumpur.

Public Works Department, 2005. Utilization and Cut-off Pricing Based onStatistical Methods in the Evaluation of Tenders and Recommendations:Addendum No.1. In: P.W. Department (Ed.), Public Works Department,Kuala Lumpur.

Please cite this article as: Azman, M.A., et al., The accuracy of preliminary cost estimJournal of Project Management (2012), http://dx.doi.org/10.1016/j.ijproman.2012.11

PublicWorks Department, 2009a. Annual Report 2009. PublicWorks Department,Kuala Lumpur.

Public Works Department, 2009b. Average cost per metre square of buildingconstruction cost. In: Department P.W (Ed.), Public Works Department,Kuala Lumpur.

Raftery, J., 1991. Principles of Building Economics. BSP Professional Books,Oxford.

Raftery, J., 1995. Risk Analysis in Project Management. E & FN Spon, London.Runeson, G., 2000. Building Economics. Deakin University Press, Victoria.Schultz, B.B., 1985. Levene's test for relative variation. Systematic Zoology 34,

449–456.Shane, J.S., Molenaar, K.R., Anderson, S., Schexnayder, C., 2009. Construc-

tion project cost escalation factors. Journal of Construction Engineering andManagement 25, 221–229.

Skitmore, M., 1988. Fundamental research in bidding and estimating, British/Israeli Seminar on Building Economics: The International Council forBuilding Research Studies and Documentation CIB W-55. Haifa, Israel.

Skitmore, M., 1991. Early Stage Construction Price Forecasting: A Review ofPerformance. RICS, London.

Skitmore, M., 2002. Raftery curve construction for tender price forecasts.Construction Management & Economics 20, 83–89.

Skitmore, M., Cheung, F.K.T., 2007. Explorations in specifying constructionprice forecast loss functions. Construction Management & Economics 25,449–465.

Skitmore, M., Drew, D., 2003. The analysis of pre-tender building priceforecasting performance: a case study. Engineering Construction andArchitectural Management 10, 36–42.

Skitmore, M., Lo, H.P., 2002. A method for identifying high outliers inconstruction contract auctions. Engineering Construction and ArchitecturalManagement 9, 90–130.

Skitmore, M., Ng, T., 2000. The variance of elemental estimates. In: Serpell, A.(Ed.), Proceedings Information and Communication in ConstructionProcurement, The International Council for Building Research Studies andDocumentation (CIB) W-92 Procurement System Symposium 1. TheInternational Council for Building Research Studies and Documentation(CIB), Department of Construction Engineering and Management, PontificiaUniversidad Catolica de Chile, Santiago Chile, pp. 557–567.

Skitmore, M., Patchell, B.R.T., 1990. Developments in contract price forecastingand bidding technique. In: Brandon, P.S. (Ed.), Quantity Surveying TechniquesNew Directions. Blackwell Scientific, London, pp. 75–120.

Skitmore, M., Picken, D.H., 2000. The accuracy of pre-tender building priceforecasts: an analysis of USA data. Australian Institute of Quantity SurveyorsRefered Journal 4, 33–39.

Skitmore, M., Stradling, S.G., Tuohy, A.P., Mkwezalamba, H., 1990. TheAccuracy of Construction Price Forecasts, A Study of Quantity Surveyors'Performance in Early Stage Estimating. Department of Surveying, Universityof Salford, Salford.

Skitmore, M., Tan, S.H., 1988. Factors Affecting Accuracy of Engineer'sEstimates. Cost Engineering 30 (10), 16–23.

Soo, A., Oo, B.L., 2007. The effect of information feedback in constructionbidding. Australasian Journal of Construction Economics and Building 10,65–75.

Tabachnick, B.G., Fidell, L.S., 2007. Using Multivariate Statistics 5th ed.Pearson, Boston.

Valence, G.d, 2011. Market types and construction market. In: Valence, G.d(Ed.), Modern Construction Economics. Spon press, Oxford.

Wachs, M., 1990. Ethics and advocacy in forecasting for public policy.Business & Professional Ethics Journal 9, 141–157.

Yeung, K.L.T., 2009. Statistical Model Development to Identify the Best DataPooling for Early Stage Construction Price Forecasts, School of UrbanDevelopment, Faculty of Built Environment and Engineering. QueenslandUniversity of Technology, Queensland.

ates in Public Works Department (PWD) of Peninsular Malaysia, International.008