Parliam Aff 2012 Benton Pa Gss009

Assessing the Impact of ParliamentaryOversight Committees: The Select Committeesin the British House of Commons

Meghan Benton and Meg Russell*

Department of Political Science, University College London, London, UK

*Correspondence: [email protected]

The departmental select committees of the British House of Commons deal with

executive oversight, but not with legislation. This has two important knock-on

effects. First, although these committees are permanent, expert, and largely well

regarded in Britain, they are often overlooked by comparative scholars. Second,

and connectedly, their impact on policy is very difficult to assess. Even in the UK

there is considerable scepticism about whether the committees really matter to

policy outcomes. This paper reports the first detailed cross-departmental investiga-

tion of the select committees’ impact on government policy for many years,

employing both quantitative and qualitative methods. It concludes that numerous

committee recommendations are implemented by government, including many

for major policy change. But counting successful recommendations is a poor

proxy for committee influence overall, and this also takes many other, perhaps

more important, forms.

The British House of Commons is unusual in splitting the functions of legislativeand executive oversight between two separate sets of committees. The committeestage of legislation is taken in non-specialist and temporary ‘public bill commit-tees’ (formerly ‘standing committees’). Executive oversight and other investiga-tive work meanwhile is carried out by a set of specialist ‘select committees’which shadow government departments. The inexpert and relatively party-dominated legislative committees have been subject to ‘an extraordinary levelof opprobrium’, from both practitioners and academics (Modernisation Com-mittee, 2006, Ev. 108). Comparative scholars are similarly dismissive, even treat-ing the House of Commons as if it had no specialist committees at all (Lees andShaw, 1979; Mattson and Strøm, 1995). But in fact the House of Commons pre-sents complex challenges for assessing committee impact on policy. Since its only

# The Author [2012]. Published by Oxford University Press on behalf of the Hansard Society; all rights reserved.

For permissions, please e-mail: [email protected]

Parliamentary Affairs (2012) 1–26 doi:10.1093/pa/gss009

Parliamentary Affairs Advance Access published May 16, 2012 at Q

ueen's University B

elfast on Decem

ber 16, 2014http://pa.oxfordjournals.org/

Dow

nloaded from

http://pa.oxfordjournals.org/

specialist committees play no formal part in the legislative process, the mostobvious measures—such as numbers of amendments made to bills—do notapply to them. This contributes to an impression that committees at Westminsterare unusually weak.

This article asks how influential the House of Commons select committees arein the policy process, and particularly on the policies of central government. Thecommittees are generally well regarded, and their establishment in 1979 isroutinely cited as a key event in British parliamentary history (Ryle, 2005;Kelso, 2009). But the challenges of assessing their policy contribution meanthat there has been no major cross-committee study of their outputs for manyyears. Likewise outside the UK the study of committees’ executive oversight func-tion has been very limited. We therefore seek to provide two things: much-neededevidence about the select committees at Westminster and more general insightsabout the evaluation of executive oversight committees.

There are presently 20 departmental select committees in the House ofCommons, including the quasi-departmental Public Administration Committee(PASC).1 Since the creation of the new system 30 years ago, the committees haveconsiderably developed their role in scrutinising the work of government, largelythrough holding ‘inquiries’ into different aspects of executive policy-making.Over this period they have become better established, gained resources (Kelso,2009) and attracted increasing media attention (Barnett and Gaber, 1992;Kubala, 2011). Recent high profile inquiries into the banking crisis and theNews of the World phone hacking scandal, for example, have thrust them intothe limelight. In 2008 the committees were given new powers to hold hearingswith senior government appointees (Hazell et al., to appear). In 2010 their mem-bership was also democratised, so that chairs and members are now elected,having previously been chosen by party whips (Russell, 2011). BBC parliamentarycorrespondent Mark D’Arcy has thus described the committees in the 2010parliament as ‘more powerful and independent-minded than ever before’.2

But there remains scepticism about whether select committees make a realdifference to the policy process. They obviously have no power over legislation,and although government must respond formally to their reports, the committeeshave no means to require government to act on their recommendations. AsGuardian journalist David Hencke put it recently on BBC Radio 4:

1PASC is not listed in House of Commons standing orders as a departmental committee, but the

House of Commons’ own Information Office factsheet (P2, August 2010, page 7) notes that it ‘has

become in effect a departmental select committee for the Cabinet Office’. PASC was one of our

sample committees.

2‘Time to salute the post-2010 election Parliament’, 28 July 2011. http://www.bbc.co.uk/news/

uk-politics-14330865.

of 26 Parliamentary Affairs

at Queen's U

niversity Belfast on D

ecember 16, 2014

http://pa.oxfordjournals.org/D

ownloaded from

http://www.bbc.co.uk/news/uk-politics-14330865










The biggest weakness for select committees is, really, that they don’thave enough impact. Their reports are very good, very detailed, andthey are getting much better at extracting information from civilservants, witnesses and ministers. But when it comes to their findings,government seems to be quite happy to brush aside, in a sort of tersememorandum, not taking any real notice of anything the committeehas recommended, unless it happens to be in the interest of thegovernment at the time.3

Comments such as these have been made in a virtual vacuum in terms of evi-dence. Our research therefore set out to test such claims in a robust and objectiveway. This article is one output from a large-scale study on select committee influ-ence, and a longer discussion of the findings can be found in the project report.4

We start with a review of the limited literature on the policy influence ofexecutive oversight committees, in the UK and more widely. We then brieflyoutline our methods. The main body of the paper is divided into quantitativeand qualitative results, both exploring the extent to which the House ofCommons select committees are influential, and what form(s) this influencetakes. In the quantitative section we first present findings on the acceptanceand implementation by government of the recommendations in committeeinquiry reports, and then examine which factors seem to affect this. In the quali-tative section we explore further the factors affecting the influence of committeereports, and then consider other forms of influence beyond these reports, settingout a comprehensive typology of eight forms of select committee influence. Weend with a brief conclusion.

In short we find that government by no means ignores select committeerecommendations, and that it is not just the trivial or more supportive recom-mendations which succeed. But take-up by government of recommendations isonly one form of committee influence, and arguably not even the most import-ant. The select committees have many less tangible and measurable effects on thepolicy process. This presents significant challenges to those wishing to objectivelyassess the work of oversight committees.

1. Existing studies

The select committees are a breed apart from the specialist committees found inthe US Congress, or in most parliaments in Europe, which combine legislativeand executive oversight. Analysis of the policy impact of committees tends to

3Westminster Hour, 27 March 2011.

4Russell and Benton (2011). This work was funded by the Nuffield Foundation.

Assessing the Impact of Parliamentary Oversight Committees of 26

at Queen's U


ecember 16, 2014


ownloaded from


focus on their legislative impact, and thus to adopt measures which are not ap-plicable to the select committees (Mattson and Strøm, 1995).

Methods for studying the impact of non-legislative committees have thereforemainly been developed in the Westminster context. Here one of the more com-prehensive studies was also the earliest. An edited collection on the first parlia-ment following the establishment of the new committee system containedchapters on each of 13 committees, though the methods of analysis appliedwere not uniform (Drewry, 1985b, 1989). Most of the contributors, and thevolume as a whole, concluded that select committee influence was relativelyslight, but that it was too early to pass judgement on the new system.

In another early study Flegmann (1986) considered committees’ oversight ofpublic expenditure through documentary analysis and interviews. She concludedthat the committees were taken seriously, but did not seek to quantify theirimpact. A more comprehensive study by Hawes (1993) of the influence ofselect committees on environmental policy drew on case studies, quantitativeanalysis of government responses and interviews. Although he found that60% of recommendations were accepted by government to some degree, hisconclusions—like those of several authors in the Drewry volume—were thatmuch select committee influence is indirect, invisible and long term. Morelimited recent snapshots have indicated that the select committees are influentialon policy outcomes at least some of the time (Horne, 2006; Rogers and Walters,2006; Klug and Wildbore, 2007).

Various authors note the problems with using a simple count of recommenda-tions accepted by government as an indicator of committee influence (e.g. Baines,1985, Drewry, 1985a, Giddings, 1985). One problem is that if all recommenda-tions are treated equally, the success of trivial recommendations may skew theresults (Aldons, 2000). Another is that government acceptance cannot necessarilybe taken at face value. For example, government might sometimes accept recom-mendations but never put them into effect. Likewise, some recommendationsmight initially be rejected but then be implemented later, in what has been re-ferred to as the ‘delayed drop effect’ (Hawes, 1992).

These problems were to some extent dealt with by the only recent systematicstudy of select committee policy influence: a mixed methods study of the Educa-tion Committee conducted by Hindmoor et al. (2009). The quantitative part oftheir analysis again focused on success of committee recommendations, but theauthors logged both the initial government response and whether the recommen-dation went on to be implemented in legislation. They found that relatively fewrecommendations were implemented in government bills, but they made noattempt to trace recommendations calling for non-legislative action. Their inter-views did suggest that the committee had other types of influence, much of whichwent beyond the recommendations in committee reports.


at Queen's U


ecember 16, 2014


ownloaded from


Outside Westminster the oversight function of committees, and particularlyany policy impact that may result, has been surprisingly little studied. Even scho-lars of the US Congress note that committees’ investigatory role is underexa-mined (Kriner and Schwartz, 2008). In a rare example of such work outsidethe UK, Monk (2009b) studied several committees in the Australian Houseand Senate. He considered the initial acceptance by government of committeerecommendations, but not their eventual implementation. In a subsequentarticle Monk (2009a) suggested that measuring the proportion of recommenda-tions accepted is arbitrary, and that a better measure of minimal influence iswhether the government accepts at least one recommendation in a committeereport.5 Comparative studies of oversight committees, meanwhile, are evenrarer than single country studies. One example—a study of Public AccountsCommittees—surveyed committee chairs to assess how frequently recommenda-tions were both accepted and implemented (Pelizzo et al., 2006). However, find-ings were based on only 33 respondents, and relied on chairs’ judgement ofwhether their committees were influential, which is potentially open to bias.

Ours is therefore the first cross-committee study of Westminster for manyyears. It is also the first that we are aware of in any parliament to apply consistentimpact measures across a set of non-legislative committees. We have sought tolearn from both the methods and perceived shortcomings of earlier studies. Tothis end, we trace government responses to recommendations as well as theirultimate degree of implementation, and go beyond implementation in legislationalone. We also complement our quantitative evidence with extensive interviews.

2. Data and methods

Our project was a collaborative study between the Constitution Unit at UniversityCollege London and the Committee Office of the House of Commons. Thereduced demands on select committee staff during the period when parliamentwas dissolved for the 2010 general election allowed us to recruit seven volunteers.Each volunteer collected and coded data on the activities of one committee overthe entire period of Labour government May 1997–May 2010.6 The sevencommittees were chosen to be representative across a broad range of domesticand international policy. Five researchers were able to work on their own commit-tee, while two were deployed to other committees, with support provided by the

5This is also of course a very crude measure, and does not seem useful in the British context. Of our

216 sample reports, only 19 received a government response in which not a single recommendation

was accepted.

6For our analysis of government responses, the study period was extended to 31 October 2010 to allow

time for receipt of these responses.


at Queen's U


ecember 16, 2014


ownloaded from


staff of these committees where necessary.7 The project was directed, and the ana-lysis (and interviews) conducted by the authors.

Our main focus was the impact of committees on government policy, and ourquantitative work concentrated on the impact of recommendations in committeeinquiry reports. We thus first collected basic data on every report published by theseven select committees, including the form of report, i.e. associated with aninquiry or other work. For all inquiry reports we then recorded the point inthe policy process at which it intervened, the number of recommendations andconclusions that it included, and the extent to which it was controversialwithin the committee, measured by the number of committee divisions (votes)on the report. This allowed us to select a stratified random sample of inquiryreports, based on committee, parliament (1997–2001, 2001–2005, or 2005–2010), and the three aforementioned variables. This sample included 216reports, which provides results at a 95% level of confidence.

For these sample reports we collected data on every conclusion and recom-mendation. For recommendations we coded who they were aimed at (includingprivate industry, local government, etc.), and then collected further informationon all those aimed at central government. This included the type of action thatthe recommendation called for, the extent to which it would be possible tomeasure progress against it, the level of policy change that the recommendationdemanded and whether it referred to a high-profile policy mentioned in the gov-erning party’s election manifesto or annual Queen’s speech (see Appendix formain codes). When it came to the more painstaking process of tracing govern-ment acceptance and implementation of recommendations, however, thisdataset exceeded our capacity. Thus, we selected a slightly smaller random sub-sample of 1906 recommendations, and traced all of these which called forsome clear form of action (1831 in total). At the recommendation level, thisagain provides results at a 95% confidence level.

All coding was conducted according to a strict set of written guidelines. Thesewere discussed and developed collectively, with training exercises held at regularteam meetings, and an online forum for discussion of any coding problems.The quantitative data collected in this way was then complemented by 56 semi-structured interviews with those who had been directly involved in the workof the seven case study committees. These included committee insiders

7We are extremely grateful to these volunteers for their hard work. The committees were BIS, Defence,

Foreign Affairs, Health, Home Affairs, Public Administration (PASC) and Treasury. As our focus in

this paper is on the broad question of whether select committees are influential, rather than on the

differences between committees, we present data for committees as a whole. Where we found

significant differences across the committees we highlight this. Full data broken down by

committee is provided in Russell and Benton (2011).


at Queen's U


ecember 16, 2014


ownloaded from


(members, chairs and staff), as well as representatives of government (civil ser-vants and ministers) and outside groups.8

3. Initial findings

Before presenting our findings on government acceptance and implementation ofcommittee recommendations, we summarise some basic data about the volumeand content of committee work in order to situate these findings in context. Theinformation collected by the project represents the largest dataset ever gatheredabout the House of Commons select committees. Key facts uncovered by the re-search included the following.

† The seven select committees produced 695 reports over the 13-year period,most of which—505—were reports of inquiries. Inquiries made up a majorityfor all committees, but there was variation—ranging from 58% of reports fromthe Foreign Affairs Committee to 95% from the Health Committee.9

† Most inquiries were reactive rather than proactive in policy terms. In codingfor point in the policy process, only 8% of reports were judged to be‘agenda setting’, with most instead reviewing government progress (42%),examining new government proposals (22%) or responding to perceived gov-ernment failures (15%).

† The 505 reports contained a total of 13,216 conclusions and recommenda-tions. Our 216 sample reports included 5682, of which 3471 (61%) wererecommendations, and 2865 (50%) were recommendations aimed at centralgovernment. This suggests that the committees produced around 6500central government recommendations in total over this period, and thatthere were some 19,000 such recommendations from the 20 departmentalselect committees overall.

† The average number of recommendations per inquiry report was 16, but thisvaried substantially. Some reports made no recommendations at all, while thereport with the largest number of recommendations (from the Foreign AffairsCommittee) made 98.

† Recommendations called for a wide variety of actions by government. Thelargest category (23%) proposed that government should review a policy, orcarry out new research. Many recommendations (14%) called for disclosure

8We would also like to thank the interviewees for giving up their time. A full list of acknowledgements

is provided in Russell and Benton (2011).

9Other kinds of reports produced included reviews of departmental annual reports, or occasional

legislation published in early draft form, and reports of hearings held with government appointees.

We chose not to explore these for capacity reasons, and instead to concentrate on the inquiries

which make up the bulk of select committee work.


at Queen's U


ecember 16, 2014


ownloaded from


of information by government. In contrast, only 4% explicitly called for legis-lative action, while 4% called for new guidance from government departments(e.g. to regulators or arm’s-length agencies) and 3% for new funding.

† Relatively few recommendations (around 20%) related directly to flagship pol-icies as identified in government manifestos and Queen’s speeches.

† Around 40% of recommendations called for only a small policy change or acontinuation of existing policy, while the remainder called for larger changes.

4. How influential are select committee recommendations?

These figures illustrate that the select committees are prolific, and that a good dealof what they do attempts to influence central government policy through recom-mendations. As already indicated, the literature suggests two ways of measuringwhether these attempts are successful. The first is the most straightforward, inthat it is based on information on the public record: whether the governmentaccepts the recommendation in its written response to an inquiry report. Butthis may be a weak proxy for influence, if the government makes vague commit-ments aimed at appeasing the committee or does not follow through on what itpromises. For this reason we also traced recommendations to see whether theywere ultimately put into effect.

4.1 Government acceptance of recommendations

Previous scholars have noted that committees may couch recommendations interms that are easier for the government to accept, thereby exaggerating their ap-parent influence (e.g. Aldons, 2000). For this reason we coded recommendationsaccording to whether they called for a small, medium or large change to govern-ment policy. Government acceptance was coded on a five-point scale, rangingfrom fully accepted to fully rejected (see Appendix). Our coding scheme builton categories used by previous studies (Hawes, 1993; Hindmoor et al., 2009;Monk, 2009b). However, we disaggregated responses where the governmentsaid that something was already being done (which other authors have treatedas one group). Sometimes this form of response clearly indicated a denial by gov-ernment that the action was necessary; but in others it was obvious that the com-mittee had brought a new issue to the government’s attention and caused it to act,before the committee inquiry was complete. The first of these indicates a lack ofcommittee influence, while the second suggests the reverse.

Table 1 shows that 40% of recommendations in our sample were initiallyaccepted (fully or partially) in the government response. This initially appearsquite impressive. However, it would show little sign of meaningful influence if,as predicted by David Hencke at the start of this article, only unambitious or


at Queen's U


ecember 16, 2014


ownloaded from


Parliam Aff 2012 Benton Pa Gss009

Documents

Transcript of Parliam Aff 2012 Benton Pa Gss009