WP12 NSW Scaling Committee Tablemembers.webone.com.au/~markld/PubPol/Edu/UAIs/WPs/WP12.pdf · 2....

1

Working Paper 12

The "NSW Scaling Committee Table" which the Authorities Responsible for Determining ACT UAIs will Not Let us (i.e. the

Public) See

by Mark Drummond February 2008

The "NSW Scaling Committee Table" assumes a central role in the determination of ACT system Universities Admission Indices (UAIs) but students

whose UAIs are determined using this table and others hoping to examine it to check the table's validity are unable to view the table, as documented in

Table 1 below.

Please note especially the developments set out on pages 13 to 22, spanning from February 2006 until late 2007, including several questions in the

ACT Legislative Assembly which have sought to draw attention to this scandalous state of affairs. The documents shown on pages 1-12 spanning from

1995 to early 2006 provide significant background information.

Table 1: Evidence of the existence of the NSW Scaling Committee Table, its role in ACT UAI calculations, and efforts to bring it

into the open so we can actually see it.

Date of

Statement

Person(s)

Making

Statement

Form of

Statement Actual Statements Comment by Mark D

2 December 1995

Stephanie Raethel

'Teachers Call for Scrapping

of 'failed' TER', Sydney Morning

Herald, p. 5.

... The chairman of the technical committee on scaling, Professor George Cooney, of Macquarie University, said the TER had never been designed to report on students' overall performance.

Note that George Cooney is a key player here.

20 February 1997

Stephanie Raethel

'HSC English: Poor Rewards

for Hard Work', Sydney Morning

Herald, p. 3.

... The head of the Technical Scaling Committee, Professor George Cooney, said the committee of chairs of university academic boards and senates did not want to make such a significant change before the release of the HSC review being prepared by Professor Barry McGaw.

Note the mention of the "committee of chairs of [NSW and ACT] university academic boards and senates", which the "Technical Scaling Committee" is a creature of.

2

Table 1 (continued)

Date of

Statement

Person(s)

Making

Statement

Form of


27 June 1997 not stated Paper titled Technical

Committee on Scaling, available online at

http://www.secretariat.unsw.edu.au/acboard/committee_chairs/tcs_tor.p

df

Page 1: Shortly after the Technical Committee became a committee of the New South Wales Vice-Chancellors’ Conference, reporting to it through the Committee of Chairs of Academic Boards/Senates of Universities in New South Wales and the ACT. The committee, at that stage, comprised representatives from all New South Wales and ACT universities. There was, and is, no fixed term of office, and members tended to be drawn from Mathematics and Statistics disciplines. Since that time the Technical Committee has provided advice on scaling matters to the Committee of Chairs which accepted responsibility for the calculation of the aggregate. The scaling operation itself is currently carried out by Professor Mack, Professor Cooney and Ms McAllister (the programmer).

Pages 1 and 2: 2. Proposed Structure for the Technical Committee on Scaling. The Technical Committee on Scaling The Technical Committee on Scaling is a committee of the New South Wales Vice-Chancellors’ Conference which reports through the Committee of Chairs of Academic Boards/Senates of Universities in New South Wales and the ACT. A. Composition 1. The committee comprises a Chair, and one representative from each of the universities in New South Wales and the ACT, the Board of Studies and the Universities Admissions Centre. 2. The Chair is appointed by the Committee of Chairs. 3. A Deputy Chair will be appointed from the committee members. 4. University representatives will be appointed by their Vice-Chancellors. 5. The representative of the Board of Studies will be appointed by the President of the Board. 6. The representative of UAC will be appointed by the Director of UAC. 7. The Chair and members of the committee will be appointed initially for up to three (3) year terms with the possibility of renewal. In the first instance terms will be arranged so that continuity of membership is maintained and a regular rotation pattern be established. 8. The Chair will : • be an expert in educational measurement and the derivation of selection indices for universities, • have demonstrated research skills, • have an interest in, and experience of, assessment and reporting at secondary and tertiary levels, • have an extensive knowledge and understanding of procedures used by the Board of Studies to determine its patterns of reported marks, and • be able to work with the organisations involved in the production of the TER. 9. The members of the committee will have extensive knowledge and understanding of the New South Wales Higher School Certificate and selection methods for universities, be aware of methods for developing selection indices and have demonstrated research skills.

This document provides a useful historical background of the "NSW Technical Committee on Scaling". It also makes it clear that this committee should be referred to as the "NSW and ACT Technical Committee on Scaling".

3

Table 1 (continued)

Date of

Statement

Person(s)

Making

Statement

Form of


1997 to 2004 ANU ANU Annual Reports

referring to Dr Daryl Daley as

the ANU representative of the NSW Technical

Committee on Scaling

At http://wwwmaths.anu.edu.au/annual-reports/1997/AR1-3.html (1997 annual report for the ANU Centre For Mathematics and its Applications):

Service To The University

... Daley, member, Board of Studies, Graduate Program in Mathematics; Graduate Program in Statistics: adviser/consultant, ACT Board of Senior Secondary Studies (BSSS); ANU representative, Technical Committee on Scaling, NSW (BSSS).

At http://info.anu.edu.au/Discover_ANU/University-wide_Publications/_ANU_Annual_Reports/_1997/Cooperation_with_Government.htm: Cooperation with Government and other public institutions

The Australian National University encourages members of the academic staff to give specialist advice and assistance to Federal and State Government departments and to other public institutions, both within Australia and internationally. This help takes many forms — consultancies, membership of committees, involvement in particular projects, secondments, etc. The following list records such cooperation for 1997. This is not necessarily comprehensive. ... D J Daley, adviser, consultant, ACT Board of Senior Secondary Studies; representative, Technical Committee on Scaling; trustee, Applied Probability Trust; technical editor, Australian Journal of Statistics.

At http://wwwmaths.anu.edu.au/annual-reports/1998/html/node19.html (1998 annual report for the ANU Centre For Mathematics and its Applications):

SERVICE TO OUTSIDE ORGANISATIONS

... D J Daley, advisor, consultant, ACT Board of Senior Secondary Studies; ANU representative, Technical Committee on Scaling, NSW (BSSS); trustee, Applied Probability Trust; technical editor Australian Journal of Statistics.

At http://wwwmaths.anu.edu.au/annual-reports/2000/entire.html (2000 annual report for the ANU School of Mathematical Sciences): Examination marks

Interest revived in questions regarding scaling examination marks, both with regard to NSW system and the ACT system. Of interest to both is the simulation of examination mark datasets. For the latter, Daley revisited his 1990 Report to the ACT BSSS and the system now implements another of its recommendations with respect to the use of a reference scale (i.e. SAT) scores. Work has begun on studying the question of significant variability in the error variance between colleges as a possible source of excessive numbers of students from larger colleges gaining 'top' scores. ...

University and School Service

...

These entries prove that Daryl Daley of the ANU has for many years done work for or with the ACT BSSS and the NSW Technical Committee on Scaling. Note especially the following 1997 entry which implies that Daryl did work jointly for the NSW Technical Committee on Scaling and also for the ACT BSSS:

ANU representative, Technical Committee on Scaling, NSW (BSSS).

The BSSS in brackets here implies that Daryl's work for the NSW Technical Committee on Scaling was done as part of his BSSS work, and highlights the manner in which the ANU, ACT BSSS and this NSW Technical Committee on Scaling have been jointly responsible for ACT UAI calculations by virtue of Daryl Daley's involvement on behalf of all of these entities: the ANU, the BSSS and the NSW Technical Committee on Scaling.

This raises questions as to intellectual property and the manner in which Daryl has been allowed to monopolise advice on the ACT UAI calculation process. Should the work Daryl has done in relation to ACT UAI calculations be considered Daryl's own intellectual property, or that of the ANU, or the ACT BSSS, or the NSW Technical Committee on Scaling? But UAI receivers are the ones to whom the ANU, BSSS, Technical Committee on Scaling et al. owe duties of care.

Various BSSS documents imply that the BSSS and this NSW Technical Committee on Scaling have separate roles in relation to the ACT UAI calculation process, but all the main steps have been designed by Daryl Daley (buidling on the work of others from the past).

Daryl has made several collossal blunders in relation to the ACT UAI calculation process going back more than two decades but the ACT BSSS and NSW Technical Committee on Scaling (especially George Cooney) have sided with Daryl and against the thousands of ACT school leavers who have wrongly missed out on funded university places as a result of the collossal mistakes and oversights by Daryl, George Cooney and others over many years – oversights that would have been better avoided if Daryl and George hadn't monopolised the design of and advice on ACT UAI calculation matters to the extent they've been allowed to by the ACT government and other stakeholders for far too many years.

4

D J Daley

Member: Board of Studies Mathematical Sciences Program Member: Graduate Studies Committee Statistics Member: School 4th Year Honours Committee ANU representative: NSW Vice-Chancellors Committee of Chairs Technical Committee on Scaling At http://wwwmaths.anu.edu.au/annual-reports/2001/uni_service.html (2001 annual report for the ANU School of Mathematical Sciences): ... University and School Service

... D J Daley

Program Convenor: Stochastic Analysis Program Board of Studies Mathematical Sciences Program Graduate Studies Committee Statistics School 4th Year Honours Committee ANU representative: NSW Vice-Chancellors Committee of Chairs Technical Committee on Scaling Liaison Committee on the Appointment of the new Dean, SMS

At http://wwwmaths.anu.edu.au/annual-reports/2003/CMA_collab.html (2003 annual report for the ANU Mathematical Sciences Institute): Dr D J Daley, Applied Probability Trust, Trustee and Associate Editor; NSW Vice-Chancellors Committee, ANU representative, Technical Committee on Scaling

At http://wwwmaths.anu.edu.au/annual-reports/2004/CMA_collab.html (2004 annual report for the ANU Mathematical Sciences Institute): ... Joint Research Projects undertaken with Universities, CSIRO and other

Institutions

... • Daley, Dr D.J., Applied Probability Trust, Trustee and Associate Editor; NSW Vice-Chancellors Committee, ANU representative, Technical Committee on Scaling; ACT Board of Senior Secondary Studies on Scaling Matters, Consultant.

5

Table 1 (continued)

Date of

Statement

Person(s)

Making

Statement

Form of Statement Actual Statements Comment by Mark D

1 January 1998

ACT Government

Board of Senior Secondary Studies Act

1997, original version,

Effective 1 January 2008, available online

at http://www.legislation.

act.gov.au/a/1997-87/19980101-

6938/pdf/1997-87.pdf.

Page 12: ... Protection of members

20. (1) No action or other proceeding, civil or criminal, lies against a person who is or has been a member of the Board in relation to an act done or omitted to be done in good faith in the performance or exercise, or purported performance or exercise, of a function or power under this Act. (2) Nothing in subsection (1) shall be taken to affect any liability that the Territory would, but for that subsection, have in respect of an act or omission mentioned in that subsection.

This legislation emphasises the protection of the board itself and its members but does not provide any expectation that the Board would seek to serve ACT students in a competent and equitable manner.

Surely the legislation should seek to protect ACT students from wrongs which unjustly reduce their chances of gaining funded university places?

9 March 1998

Stephanie Raethel

'The New Numbers Game', Sydney

Morning Herald, p. 12.

(emphasis added here in italics) The TER has been replaced by a new "fairer" index. Experts hope it will also make it easier for students to study interstate, writes STEPHANIE RAETHEL. ... Well, now the TER is gone, and the HSC class of 1998 will be the first to receive the new Universities Admissions Index (UAI). But while the name has changed, university officials concede the calculation of the new index will be almost exactly the same as the old TER, ... On the question of fairness, though, the experts believe the new UAI is an improvement. ... The key change that NSW students will notice with the index is that they will receive a higher UAI than they would have a TER. For example, a TER of 99 in 1997 would be equivalent to UAI of 99.45. Students who gained a TER of 80 would have received a UAI of 87.35. ... Despite the Government's claims of major changes, the process for calculating the new index will be almost the same as the system used for the TER. Except for one vital last step. This will determine what ranking the students would have received if all Year 10 students had completed Year 12. To do this the anticipated performance of the students who left school at Year 10 will be included in the ranking along with the students who completed the HSC. Because most of the students who left school at Year 10 are likely to be less academically able they will fill many of the lower ranking positions, resulting in students being pushed up

the scale and receiving a higher UAI. But don't panic if you did poorly in Year 10 because the ranking of HSC students depends only on their performance in Year 12, says the Universities Admissions Centre.

The head of the Committee of Chairs of Academic Boards, Professor Jeremy Davis, admits that the change will take some getting used to. But he compares the change with the move from Fahrenheit to Centigrade -the scale is changing but the information is the same. He says the new system will be fairer and will also encourage students to be more mobile because the index will make it easier for students to apply for university places outside NSW. "The only thing we are trying to be is more equitable," he says.

Cooney denies the change will artificially inflate the results for HSC students. He compares the process with an Olympic swimming final. The swimmer who comes eighth in the final is considered to have come last, but when compared with the entire

field of swimmers who competed, he/she is the eighth best swimmer in the world. "I think it is fairer because it is saying to the

students `All right, this is your ranking within your age group' and that gives them a better indication of how they have

performed, rather than saying this is your performance in an elite group," he says. Cooney says one of the advantages is that universities will be able to compare students from year to year. Under the old system the TER is only applicable to that year and a TER of 80 in 1989 is not comparable with a TER of 80 in 1997, which makes it difficult when students defer for a year or two before applying for university. As with the old TER, the UAI will not be a mark but a ranking. It will not be a score out of 100 but will tell students how they performed against other Year 12 students.

6

Table 1 (continued)

Date of

Statement

Person(s)

Making

Statement

Form of


20 December

1999

Gerard Noonan and Julia Baird

'HSC Lists Confirm The Rise

Of Girl Power', Sydney Morning

Herald, p. 1.

... However, Professor George Cooney, from Macquarie University and the chair of the University Admission Centre's technical committee on scaling, said the top students on the UAI list this year those with a UAI of 100 were evenly split, with 11 boys and 11 girls. ...

Here we see that the Technical Committee on Scaling is described as part of UAC – i.e. UAC NSW & ACT Pty Ltd.

11 November

2000

Gerard Noonan

'Top School Considers Court

Over Uni Scores', Sydney Morning

Herald, p. 2.

Noonan (2000: 2, emphasis added here in italics): One of Sydney's top private schools is considering bringing a Supreme Court action against the body controlling university entrance over the way it discriminates against students taking art, biology and geography in the new HSC. Sydney Grammar has commissioned legal opinion from a Senior Counsel on the controversial ``capping'' process used for calculating marks in the three subjects which this year had more than 30,000 candidates. The opinion says the calculation of the Universities Admission Index (UAI) for 2001 "is susceptible to judicial review'' and there is a prospect that a court order could be obtained to restrain the decision. ... Calculation of each student's UAI is an extremely complex business involving mathematical formulas which convert marks for each subject into

an overall score measured against all other students doing the same subject. In turn, the marks are also adjusted against the marks for the other

HSC subjects a student undertakes. The whole process is administered by the so-called Technical Scaling Committee of the combined

universities, which reports to the heads of each university. The universities use these UAI scores to set quotas for each of the subjects they offer.

The lowermost extract well describes the link between the "Technical Scaling Committee" and the universities this committee serves.

3 January 2001

Sydney Morning Herald

Editorial Leader titled 'Open

Admissions', p. 8.

Universities should be open about the basis on which they admit students to courses of study. However much governments may encourage them – and they themselves be inclined to entrepreneurship, they remain public institutions. They are sustained by public money and serve the public purpose of providing higher education to those best able to benefit from it.

On principle, then, universities and their creature, the Universities Admissions Centre, should be open about how they adjust students' Higher School Certificate marks to calculate the Universities Admission Index. As every student and parent knows, it is the UAI which matters when deciding what course of university study to take. Naturally, schools see the UAI as vital for formulating teaching strategies for the HSC and offering advice to students.

Yet schools are prevented from knowing how their students have performed according to the UAI ranking system. The HSC results of students are published, but the UAI ranking of a student is known only to the student and to the universities. If schools want to know their students' UAI marks, they must ask each student. Not all students will disclose their UAI mark, or give it accurately.

The pressure for change to end the secrecy and to change the way the UAI is calculated is mounting. A private school, Sydney Grammar, is considering Supreme Court action against the UAC and its Technical Scaling Committee to challenge the basis on which the UAI is calculated. There is concern that students who perform at the highest level in the HSC, but who take subjects such as visual arts, biology and geography, have little hope of maximising their UAI score. Sydney Grammar's head of mathematics, Dr Bill Pender, told a meeting of visual arts teachers in October that his school regarded subjects such as art, geography and biology highly, as part of the intellectual foundation for serious university study. ... ... Principals of several schools have spoken out against the withholding from them of UAI results for their students. They say this deprives them of information which would help them prepare future students better for the HSC. And, disturbingly, they say the secrecy makes it harder for them to help particular students challenge the accuracy of the calculation of a UAI mark when they may have good reason to.

The integrity of the HSC is vital to public confidence in the system of secondary education. Confidence in the UAI ranking system is equally vital because of its crucial importance in determining a student's future through tertiary education. The State Government attempts to wash its hands

of the matter, saying the decision is for the UAC. That is disingenuous. In reality the Government opposes disclosure for political reasons,

following controversy over the publication of the 1996 HSC results at Mt Druitt High School. That controversy was hurtful and unnecessary. But

it is not enough reason to maintain a blanket secrecy that deprives schools of useful information to the possible disadvantage of many student.

Sound familiar?

The secrecy with the ACT system UAI calculation process is much worse than for the NSW system UAIs as referred to here because authorities continue to prevent the public from seeing the "NSW Scaling Committee Table" that establishes the entire ACT UAI distribution!!!

7

Table 1 (continued)

Date of

Statement

Person(s)

Making

Statement


10 January 2001

Julia Baird ' Aquilina: Unis Scale Down Humanities Students' Hopes', Sydney Morning

Herald, p. 1.

The State Government has urged university chiefs to reconsider changes to the controversial scaling system for HSC results amid concerns from teachers and students that the system is unfair and discriminatory.

The Minister for Education, Mr Aquilina, has written to the heads of universities and the Universities Admissions Centre, expressing concerns about changes to the scaling system to be introduced this year. ... But the chairman of the technical committee on scaling, Professor George Cooney, who is is responsible for the UAI calculations, rejected the claims.

This is just included to show that George Cooney is responsible for UAI calculations – for ACT system students as well as NSW system students – though others are obviously responsible as well.

11 January 2001

Professor George

Cooney is the chairman of the technical committee on scaling which calculates the

UAI.

'In Defence Of University Scaling Systems', Sydney

Morning Herald, p. 12.

Cooney (2001a: 12, emphasis added here in italics): A few urban myths about the admission index need to be put to rest, writes George Cooney. ... As the UAI is the index universities use to rank school-leaver applicants for most courses, it is not surprising that it is the focus of attention from students, teachers and parents. It is also not surprising that students will adopt strategies they think will maximise their UAIs.

NSW universities are no different from their interstate counterparts. While they do use interviews, portfolios, principals' recommendations and special tests where appropriate, the UAI serves as a selection measure for most courses. Local and overseas research has shown that an achievement measure based on the last year of schooling is the best single predictor of subsequent success at tertiary study.

Universities calculate the UAIs using HSC marks provided by the NSW Board of Studies. These marks are provided under strict conditions set by the board: the UAIs can only be given to the Universities Admission Centre (UAC) for selection purposes. They cannot be sent to schools or even to the board itself. The UAC, on behalf of the universities, distributes the UAIs to individual students who request them. ... Despite allegations to the contrary, the procedures are fully documented and in the public domain through

publications issued by the UAC on behalf of the university sector. An increasing number of these can be found

on the UAC Web site.

... Professor George Cooney is the chairman of the technical committee on scaling which calculates the UAI.

What George says here for NSW system UAIs is all great – I actually agree with just about everything George says about the NSW system, including his defence of the NSW system, but UAC NSW & ACT Pty Ltd is meant to serve the ACT system as well as the NSW system!

George's statement as follows is very true for NSW system UAIs but very false for ACT UAIs in view of the failure to publicly release the "NSW Scaling committee Table" – the focus of this documentary evidence compilation – which establishes ACT system UAIs:

the procedures are fully documented

and in the public domain through

publications issued by the UAC on

behalf of the university sector. An

increasing number of these can be

found on the UAC Web site.

12 September

2001

ACT Government

Board of Senior Secondary Studies Act

1997, Republication No. 2,

Effective 1 12 September 2001,

available online at http://www.legislation.

act.gov.au/a/1997-87/20010912-

6937/pdf/1997-87.pdf.

Page 12: ... 20 Protection of members

(1) No action or other proceeding, civil or criminal, lies against a person who is or has been a member of the board in relation to an act done or omitted to be done in good faith in the exercise, or purported exercise, of a function under this Act. (2) Nothing in subsection (1) shall be taken to affect any liability that the Territory would, apart from that subsection, have in respect of an act or omission mentioned in that subsection.

Thwe wording here just slightly differs from that used in the original version effective 1 January 1998, as above.

Section 20 remains as stated here unchanged until 1 June 2005.

8

Table 1 (continued)

Date of

Statement

Person(s)

Making

Statement


31 December

2001

Professor George Cooney,

Chairman of the Technical Committee on

Scaling

Letter to the editor in the Sydney Morning

Herald, p. 11.

Cooney (2001b: 11, emphasis added here in italics): Admission control

The Universities Admission Index (UAI) and controversy seem to be inexorably linked, and each year brings its own story. This year the focus has been on the apparent disjunction between HSC marks and the UAI, and the release of UAIs to schools.

The HSC presents a profile of student achievement across a very broad range of subjects, including both academic and vocational. Achievement is described against standards. On the other hand, the UAI is not a mark but an index used for ranking school-leaver applicants for tertiary places. It indicates a student's position in relation to his or her age group.

Universities calculate the UAIs using HSC marks provided by the board. These marks are provided under

very strict conditions: UAIs can only be given to the Universities Admission Centre for selection purposes.

They cannot be sent to schools or to the board. The UAC, on behalf of the universities, distributes UAIs to

individual students who have requested them.

Professor George Cooney, Chairman, Technical Committee on Scaling, Sydney, December 28.

George again ignores the ACT system here. NSW and ACT universities, through the Vice Chancellors' Committee's Technical Committee on Scaling which George chairs, calculate UAIs for ACT system students who don't do the NSW HSC as well as for NSW system students who do complete the HSC.

1 February 2002

Professor George Cooney,

Chair of The Universities admissions

centre's technical

committee on scaling.

'The UAI Is A Top Scorer As A Predictor Of Success', article by

George Cooney, Sydney Morning

Herald, p. 8.

Professor George Cooney, From The School of Education At Macquarie University, Is The Chair of The Universities admissions centre's technical committee on scaling. ...

Here we again see that the Technical Committee on Scaling is described as part of UAC – i.e. UAC NSW & ACT Pty Ltd.

4 February 2002

Gerard Noonan

'Scales, Standards, and a lot of Confused

Teenagers, Sydney Morning Herald, p. 10.

... To understand how and why this occurred for many students required taking a deep breath and following the logic of two quite different approaches to reporting academic performance one by the board of studies and the other by the oddly named technical scaling committee, a creature of the universities.

Note especially the confirmation here that technical scaling committee is "a creature of the universities".

9

Table 1 (continued)

Date of

Statement

Person(s)

Making

Statement


18 November

2003

Jenny Allum 'Give Schools Some Marks For Tertiary

Prep', Sydney Morning Herald, p. 13.

[emphasis added here in italics] For the sake of fairness, UAI results should be made available to all, writes Jenny Allum.

EDUCATION is, in all of our schools, a rich mix of academic work, co-curricular activities and pastoral networks that help students grow and develop as full, active participants in society. Schools use many measures to see how they are performing, and strive to improve wherever they can. This is as it should be. We want schools to be reflective, encouraging continual appraisal of their strengths and weaknesses. We want schools to be accountable to the community for the outcomes they achieve. And so, we should give schools all the information they need to examine their performance, to reflect and improve.

Perhaps the most important measure, for many of our students, is their Universities Admission Index (UAI) the university entrance score. Of course, this doesn't represent the summation of 12 or 13 years of schooling, and no one claims that it does. But, for students wishing to go on to university, it represents a passport to further education.

Schools are not given their students' UAIs when they leave year 12. Schools are given the students' marks in each subject from the Board of Studies, and these are important measures of how they have performed. But the UAIs are given only to students, not to the schools who have helped them at every stage in attaining it. The NSW Government has a policy of withholding from schools information about how each of their students performed in the UAI. It is curious, isn't it, that it is seen to be appropriate to give the schools marks in individual subjects but not an aggregate? This policy prevents schools from being able to provide students, parents and the community with the accurate advice they need and are entitled to have about how to maximise their UAIs. It also denies educators the ability to make fair comparisons of performance within their own schools. The marks the Board of Studies provides in each course are not able to be compared from subject to subject. The board is quite explicit about that. But UAI scaling does provide a reliable and robust mechanism for making comparisons between achievements of students and cohorts across courses and across years.

Given that some information is released (in the form of complicated mathematical scaling points), the information can be obtained (with effort and expense) by any school in a position to communicate with its former students and/or buy commercially available software. It therefore gives advantage to well-resourced schools with high socio-economic students over the most disadvantaged schools, which will find it extremely difficult.

Assessment authorities should not demand that schools accept the "correctness" of an aggregating procedure

when they deny schools the information they need to evaluate that procedure. The Government should

immediately move to ensure all schools get this information.

It is also quite curious that there are no public or non-government school representatives on the committee responsible for UAI calculation. The Technical Committee on Scaling oversees UAI calculations and decides the principles on which those calculations are made, including relative differences between subjects.

It is important that school sectors and the community see the calculations used in the UAI as credible, and

representation of schools on the committee would strengthen a positive perception. In Victoria such school

representation routinely occurs. We have a number of teachers and principals within our schools who are

competent to understand the calculations and contribute to the discussion.

Our education system will seem somewhat shady, rather than open and transparent, until we have an

accountable system for the calculation and dissemination of the UAI in NSW.

All of the valid points raised in this article apply to ACT system UAIs to an even grater extent than NSW UAIs, noting the failure of the "NSW [& ACT] Scaling Committee" to make publicly available the "NSW [& ACT] Scaling Committee Table" which establishes the entire ACT system UAI distribution.

10

Table 1 (continued)

Date of

Statement

Person(s)

Making

Statement

Form of


26 November

2004

National International

Baccalaureate Working Party

Record of meeting dated 26 November 2004 which

included Professors

George Cooney and

Rob Hyndman.

A document at https://www.ecu.edu.au/GPPS/acad_secret/assets/asc/050713_Final%20Agenda.pdf (cached version accessed via Google in March 2007; this page still shows up on Google searches as at April 2007 but the pdf file itself and the cached copy can no longer be accessed) shows that Rob Hyndman was a member of the ITI Technical Group in 2004 at a time when George Cooney was the Chairman of this ITI Technical Group, as follows:

This document also states as follows, as shown in the results of a Google search carried out on 11 April 2007:

[PDF] AGENDA File Format: PDF/Adobe Acrobat ACTAC asked Prof George Cooney from Macquarie University, the Chair of the ITI Technical. Group to chair a National IB Working Party to review each of the ...

Cooney and Hyndman have both been brought in by the ACT BSSS to do "hatchet jobs" on analyses by me (Mark D) which have criticised the ACT UAI calculation process. It seems clear to me (Mark D) that George, Rob and others are upset with me because I have been able to identify serious technical flaws which they themselves overlooked despite their heavy involvement in the processes used to calculate not only UAI scores (and their equivalents) but also the Interstate Transfer Indices that were supposed to improve interstate cross-border transferability of tertiary entrance scores. I didn't even know who these guys were until I received copies of the dishonest and incompetent hatchet jobs on my criticisms the BSSS had them do! It was never my original intention to set out to criticise these individuals for their technical mistakes, but I'm not prepared to see students continue to wrongly miss out on funded uni places because these high ranking academics lack the integrity and backbone to own up to the technical errors they have been responsible for!

11

Table 1 (continued)

Date of

Statement

Person(s)

Making

Statement

Form of


2 June 2005 ACT Government

Board of Senior Secondary Studies

Act 1997, Republication No.

5, Effective 2 June 2005,

available online at http://www.legislation.act.gov.au/a/1

997-87/20050602-

19299/pdf/1997-87.pdf.

Page 14: ...

20 Protection of board members from liability

(1) A board member does not incur civil liability for an act or omission done honestly and without recklessness for this Act. (2) Any civil liability that would, apart from this section, attach to a person, attaches instead to the Territory.

Section 20 here remains current as of 2008.

9 November 2005

ACT BSSS Executive

Officer Bob Edwards

Email to Mark Drummond

From: Edwards, Bob To: Mark Drummond Sent: Wednesday, November 09, 2005 4:35 PM Subject: RE: Recommend the BSSS set the ACT system wide median UAI to 84 instead of 76 or so Mark There is an assumption that all states are equal in the UAI processes. ... Dr Daley has examined the equivalence with NSW and has agreed with the current outcome. ...

Dr Daley is largely responsible for the whole ACT UAI calculation system. If he acknowledged the full extent of the problem now he'd in effect have to admit that he has failed to address such problems for more than two decades! It is ridiculous that the ACT BSSS and ACT Government continue to seek advice from Dr Daley on Dr Daley's very own UAI calculation model.

10 December 2005

ACT BSSS Executive

Officer Bob Edwards


From: "Edwards, Bob" <[email protected]> To: "Mark Drummond" <[email protected]> Sent: Saturday, December 10, 2005 10:26 AM Subject: RE: NSW meshing Mark Professor Cooney has responded to your question about meshing UAIs with NSW. He states that the solution of fixing the ACT distribution by referencing it to the NSW North Shore distribution is attractive at a superficial level but it is not based on any research data. It is possible to find different regions that are similar to the ACT in terms of SES profile but with different participation rates and UAI profiles. He thinks that we need to think further about the issues before making any major change. The best way forward for this year is to follow the current procedures. He will be working with Professor Daley of ANU early next year to see if there is a better way for fix the ACT distribution of UAIs. ...

It is just ridiculous that Daryl Daley and George Cooney continue to be given the first and last word on ACT UAI calculation procedures that Daryl and George themselves are so dominantly responsible for!

Daryl Daley produced a ridiculosuly confused paper not early in 2006, but in November 2006 (see below), and in early 2006 Rob Hyndman (George Cooney's colleague on the ITI Technical Group as per the 26 November 2004 entry in this compilation as above, and a long time colleague of BSSS chairman Tim Brown) was brought in and paid $5600 by ACT taxpayers to "look after" Daryl, George, Bob Edwards, Selwtn Cornish, the BSSS, the NSW Technical Committee on Scaling, the ANU (through Daryl as ANU's representative on the NSW Techniocal Committee on Scaling) and others responsible for the gross mistakes and oversights affecting ACT UAIs which have robbed thousands of ACT UAI receivers of funded university places over many years.

Daryl, George, Bob Edwards, Selwyn Cornish, Rob Hyndman, Tim Brown, ANU leaders and others have all been real bleeding hearts towards one another (i.e. those most directly responsible for the problems with ACT UAI calculations over many years) whilst continuing to just completely dump on the ACT Year 12s who continue to miss out on funded uni places whilst this grubbly little old boys' network comes up with new and "cleverer" ways of concealing their huge past mistakes and oversights.

12

Table 1 (continued)

Date of

Statement

Person(s)

Making

Statement

Form of


14 December 2005

ACT BSSS Executive

Officer Bob Edwards


From: Edwards, Bob To: Mark Drummond Sent: Wednesday, December 14, 2005 12:24 PM Subject: RE: My email of 30 Dec 2004 in which I first raised the NSW v ACT issue Mark In ACT, the Board of Senior Secondary Studies calculates the UAI using a table provided by the NSW Technical Committee on Scaling that enables a Rank by Year 10 Cohort to be calculated for each student that places ACT UAIs on a equivalent basis as NSW. The method of meshing with NSW was agreed to Board of Senior Secondary Studies upon a recommendation by the Chair of this committee following discussion with both ACT universities. The NSW Technical Committee has a representative from each ACT university and is responsible to the NSW (and ACT) Vice-Chancellors Conference Committee ...

This provides useful background information, but glosses over the manner in which Daryl Daley has monopolised the processes "agreed" to by the ACT BSSS and this NSW (and ACT, indeed!) Tecvhniocal Committee on Scaling.

February 2006

ACT Board of Senior

Secondary Studies, signed by chair Selwyn Cornish of the

ANU

ACT BSSS, Policy and

Procedures

Manual 2006, ACT

Government, Canberra.

BSSS (2006, emphasis added here in italics):

5.4.1 Universities Admission Index (UAI)

The Tertiary Entrance Statement is produced for students who qualify for the Universities Admission Index (UAI). It only reports information useful for tertiary admission. The method of calculation of the UAI will be: ▪ calculate aggregate score for each student ▪ calculate notional aggregate score for those students completing Year 12 and completing one T subject ▪ calculate candidate ranking for each student including notionals ▪ translate candidate ranking to UAI ranking using NSW Scaling Committee Table. BSSS (2006: 52, emphasis added here in italics):

5.4.3.2 Aggregate Scores and Universities Admission Index (UAI)

The Aggregate Score is reported on the Tertiary Entrance Statement. The Aggregate Score will be calculated for all students who have studied an appropriate package and sat the AST examination. The Aggregate Score will be calculated and reported on the Tertiary Entrance Statement as an integer. A percentile rank, named the Universities Admission Index, will also be calculated, based on the Aggregate Score. The UAI will be reported on the Tertiary Entrance Statement as a decimal value to the nearest .05.

5.4.3.3 Ranking of Candidates

The Year 12 Candidature is the group of Year 12 students who completed at least one T minor course and who were enrolled in Semester 2 of Year 12. The Rank by Year 12 Candidature indicates the percentage of the Year 12 Candidature placed higher than the student on the Aggregate Score. This is translated to a Universities Admission Index (UAI) ranking using NSW Scaling Committee Table. The UAI is a ranking of a student relative to the full age cohort ie. relative to the set of students who would be in the group if all students stayed on and completed Year 12. It is reported with a range from 100.00 for the highest ranked students down to 30.00.

These extracts show that the "NSW Scaling Committee Table" plays a central role in the ACT UAI calculation process. Indeed all other steps "merely" establish a rank ordering of ACT system UAI receivers. But this "NSW Scaling Committee Table" is not shown in this Policy and Procedures Manual nor in any other publicly available BSSS or ACT government publication. The phrase "NSW Scaling Committee" doesn't appear anywhere in this manual besides these two entries on pages 51 and 52.

The question of whether the ACT system receives just 50 or so 99+ UAIs or 100 or 300 (in recent years it has been around 50 to 50; I believe we should be receiving 100 or so 99+ UAIs each year – James Ruse Agricultural College usually achieves 100 or more 99+ UAIs each year and it is just one school with typically less than 200 Year 12 students!) is entirely decided by this NSW Scaling Committee Table.

We obviously should be allowed to see what this table looks like in order to check whether it passes the "common sense test" etc. etc., and if it clearly looks to be incorrect we should have it changed. Academics Daryl Daley of the ANU and George Cooney of Macquarie Uni have been responsible for the table in recent years and these guys seem to think that it's their table and purely for their own benefit rather than something that surely should be transparently available and competently produced to provide equitable tertiary admissions opportunities for ACT system UAI receivers.

The relevant authorities have displayed a total lack of transparency and accountability and have basically been just downright contemptuous and rude by failing to openly present this NSW Scaling Committee Table.

13

Table 1 (continued)

Date of

Statement

Person(s)

Making

Statement

Form of


15 February 2006

Dr Deb Foskey

MLA and Minister for Education Ms Katie Gallagher

ACT Legislative Assembly Question, Hansard

pages 128 and 129, available online at

http://www.hansard.act.gov.au/HANSARD/2006/week01/128.

htm and http://www.hansard.act.gov.au/HANSARD/2006/week01/129.

htm.

(emphasis added here in italics) Universities admission index

DR FOSKEY: My question is to the minister for education and is in regard to the universities admission index-UAI-scores awarded to ACT students. The minister would know well that, on average, kids from families with high incomes and good education usually do very well at getting into university and that Canberra people, compared to New South Wales as a whole, have significantly higher incomes and education. So you would expect that year 12 kids from right across Canberra would be getting university entrance scores much higher than the New South Wales-wide average, but it seems that Canberra kids who do not go to grammar, Narrabundah or Radford are actually getting fewer scores in the high 90s, which is the cut-off point for many scholarships and select university courses, than the New South Wales average and considerably less than students from Sydney's North Shore, which is the area of New South Wales with the demographic closest to ours. Could the minister please advise the Assembly whether students going to the other colleges or senior high schools in Canberra are not taught as well as they would be in New South Wales or whether they are disadvantaged by the existing UAI scheme?

MS GALLAGHER: I think that the question has a bit of an historical basis in that a lot of correspondence has been going around the place from a person who is concerned about the way the current university index is working in the territory and has asked for further information. I understand that the Chief Minister, as acting minister whilst I was on leave, has written to the Board of Senior Secondary Studies seeking further information on some of the concerns which have been raised with us and which I think formed the basis of Dr Foskey's question.

From the information that I have seen and from the assurances that the board has given me on the rather complex way that scores are determined, I am confident at this stage that there does not seem to be any issue with the way scores are provided across the colleges, although it is a complex area. As I said, the Chief Minister has asked for further information about it. We do see within the school system some colleges doing well and performing very strongly and other colleges not doing so well. Independent advice has been sought about that over the history of the system in the ACT and all the advice to us at this stage has been that the system is fair and delivers results that are equally measured across the system for all students who participate in it. As I said, we are seeking further advice on some of the concerns that have been raised recently, advice which I believe would answer your question more fully once we have it, but at this stage I have no reason to doubt the system in place, in that it is delivering the scores that students are achieving through their year 12 and year 11 performance at college.

DR FOSKEY: I have a supplementary question. Will you make the scaling committee table which links the ACT to New South Wales schools available to the Assembly?

MS GALLAGHER: I believe that the table that Dr Foskey is talking about is not publicly available. It is a New South Wales table and it is not publicly available in New South Wales. I just would not be

in a position to table that document in the Assembly. It is a New South Wales table which is not

publicly available. As I said, extensive concerns have been raised by a person about the system, and

that person has sought to obtain the table. We are currently working on getting some external advice

through the board so as to look at that person's concerns and see whether there is anything legitimate

about them, in which case we would need to respond. At this stage, all the advice to me is that the

concerns are not valid, but we will look at it further. As to the table, it is not publicly available.

The lowermost question and answer is the one specifically intended for this compilation, but the uppermost answer by Ms Gallagher gives a good example of what the goodies in this UAI debate have been up against. Given other statements, acts and omissions prior to and after this statement was made, I believe Ms Gallagher's statements here prove that she and others in the ACT Government have never made any sincere effort to have this matter looked at properly.

Ms Gallagher's lowermost response here is quite farcical. Here we have, according to Ms Gallagher, a "New South Wales table" that is "not publicly available in New South Wales", such that Ms Gallagher "just would not be in a position to table that document in the Assembly". So Ms Gallagher effectively admits here that the ACT senior secondary education system's highest stakes "pointiest end" just totally lacks transparency, accountability and autonomy. So much for school-based autonomy when ACT UAIs are determined by a secretive table which is owned by NSW and not publicly available anywhere! You'd really think the ACT Education Minister would seek a better deal for the ACT here rather than just meekly go along with the crazy situation where a table not used to determine NSW UAIs and only used to determine ACT UAIs is not even publicly available in the ACT! The sought after table here doesn't need to be publicly available in NSW because it only applies to ACT UAIs!

The NSW Scaling Committee (aka NSW Technical Committee on Scaling), like UAC, is a creature of the NSW Vice-Chancellors' Committee. But UAC's full title is the Universities Admissions Centre (NSW & ACT) Pty Ltd A.C.N. 070 055 935 ABN 19 070 055 935 (as at http://www.uac.edu.au/), so the NSW Vice Chancellors' Committee should really be called the NSW and ACT Vice Chancellors' Committee, and the NSW Scaling Committee should really be called the NSW and ACT Scaling Committee. It follows, therefore, that the table used to determine ACT UAIs, which authorities variously refer to as the "NSW Scaling Committee Table", or the "lookup table" or "conversion table" provided by the NSW Scaling Committee, etc., etc., should at least be called the NSW and ACT Scaling Committee Table. And given that the ACT system relies wholly upon this table to determine the entire ACT UAI distribution, it is obviously quite absurd and outrageous that the ACT public are not allowed to see the table.

Ms Gallagher may have supernatural powers, but mere mortals like me (Mark D) can only properly assess something like a "Scaling Committee Table" after we've actually seen the thing!

14

Table 1 (continued)

Date of

Statement

Person(s)

Making

Statement

Form of


16 February 2006

Minister for Education Ms

Katie Gallagher

ACT Legislative Assembly Question,

Hansard page 237, available

online at http://www.hansard.act.gov.au/HANSARD/2006/week01/237.htm

.

Universities admission index

MS GALLAGHER: Yesterday in question time Dr Foskey asked me a question about the universities admission index, and specifically about a table. I indicated in my answer that the table was not publicly available.

Just to clarify that answer. I am advised that the Universities Admissions Centre intends to make available a table that converts the tertiary entrance rank to the universities admissions index. The table is not readily available but can be obtained on request from the New South Wales Technical Scaling Committee. There is a partial table available through the New South Wales University Admissions Centre website.

It is good that Ms Gallagher provided this update, but the table was never made available. I phoned UAC and asked for the table and left a message with my contact details asking for this table to be sent to me. Nobody from UAC ever sent me a copy of the table nor explained to me why they were not prepared to send it to me.

The partial table Ms Gallagher refers to was the 2005 version of the Table A8 as at http://www.uac.edu.au/pubs/pdf/2007_table_A8.pdf. On 14 February Ms Gallagher's senior adviser Brendan Ryan stated to me as follows in an email of 14 February 2006: Mark, The Chair of the Scaling Committee suggests that you be directed to the UAC website. There is a preliminary report on the 2005 examination which contains a table (Table 8) giving the relationship between UAIs and year 12 percentiles for the past four years.

As everyone can see at http://www.uac.edu.au/pubs/pdf/2007_table_A8.pdf, this Table A8 has NSW HSC aggregates in it rather than ACT system aggregates. So why on earth is it that the website of UAC (NSW & ACT) Pty Ltd has this table for the NSW system but no corresponding table for the ACT system, especially given the way ACT system UAIs piggy back on to NSW system UAIs and hence very heavily rely upon the ACT-NSW calibration process here which is wholly determined by this NSW Scaling Committee Table.

15

Table 1 (continued)

Date of

Statement

Person(s)

Making

Statement

Form of


April 2006 Professor Rob Hyndman of

Monash University

Report commissioned

by the ACT BSSS under

direction from Acting

Education Minister Jon Stanhope in late 2005:

Hyndman R. (2006a),

Review of ACT

University

Admission

Index

Calculation:

Report for

ACT Board of

Senior

Secondary

Studies, Monash

University. Melbourne, April 2006.

Hyndman (2006: 2): In this report, I review the calculation of University Admission Indexes (UAIs) in the ACT and the procedures used to relate ACT UAIs with NSW UAIs. In particular, I review the claims of Mr Mark Drummond about the fairness of these procedures.

Hyndman (2006: 2-3 [emphasis added here]): For the 2504 UAI-eligible students [in 2005], the following procedure was followed: 1. Each college calculates a score for each student completing a “T course”. 2. The BSSS scales the college course scores using the ACT Scaling Test results to give statistically moderated scores known as “scaled scores”. 3. For each student, the aggregate score is equal to the sum of their best three scaled scores plus 0.6 of the next best scaled score. (All scaled scores must be from T courses.) 4. Students who have completed the ACT year 12 certificate and at least one tertiary subject, but who have not received an aggregate score, are given a notional aggregate score. 5. Students with aggregate scores or notional aggregate scores are given a percentile rank based on their scores.

6. For eligible students, these percentile ranks are converted to UAIs using the same conversion table as for

NSW TERs to UAIs. The criticisms surrounding the calculation of the ACT UAIs primarily concern Step 6 above, and so this step is the focus of the rest of my report. I do not consider Steps 1–5 further.

Hyndman (2006: 4): Because there are no comparable Year 10 results available for the ACT, the TER-UAI conversion in the ACT is the same as that for NSW, where the percentile rank (TER in the above equation) is based on students with aggregate scores and those with notional aggregate scores.

By using the NSW conversion table based on equation (1), the following implicit assumptions have been made: 1. the age cohort in the ACT has the same distribution of ability as the age cohort in NSW; and 2. the Year 12 students receiving an aggregate or notional aggregate score have the same distribution of ability as the students receiving a UAI in NSW. It is these assumptions that Mr Drummond is questioning. In his various emails, he alleges either that the age cohort in the ACT is different from the age cohort in NSW, or that the population of the ACT receiving either an aggregate or nominal aggregate score is different from the population of NSW who receive a UAI.

Hyndman (2006: 5-6 [emphasis added here]): The ACT experiences greater immigration of students in years 11 and 12 than do other states. So the age cohort of students is trickier to define than it is in states with more stable residential populations. Nevertheless, there is some statistical evidence that the age cohort in the ACT is relatively similar to that in NSW. The “participation rate” (the proportion of the age cohort receiving a UAI) in the ACT for 2005 is 55.7%. The participation rate in NSW was 57.0%. In comparison, the only other state with a participation rate between 50 and 60% was South Australia with 53.3%. Victoria had a participation rate of 65.5% in 2005. The differing participation rates between states has been shown (Hyndman, 2005, 2006) as the major contributing factor to the different state conversion tables. This suggests that the NSW age cohort is

quite similar to the ACT age cohort in terms of their propensity to undertake preparation for tertiary

education.

Whilst Hyndman's paper contains significant errors and arrives at conclusions that were both ridiculous and incredibly hostile towards ACT students, he does at least explain here that:

For eligible students, these percentile ranks are converted to UAIs using the same conversion table as for NSW TERs to UAIs.

and By using the NSW conversion table based on equation (1), the following implicit assumptions have been made: 1. the age cohort in the ACT has the same distribution of ability as the age cohort in NSW ...

So this appears to be some confirmation that this "lookup table" or "NSW Scaling Committee Table" or "NSW conversion table" is indeed based on the assumption of State-Territory and hence ACT-NSW equality that follows from the mid 1990s MCEETYA agreement.

The passages from pages 5 and 6 here that are emphasised are probably the most treacherous ones in this appallingly incompetent and dishonest paper by Rob Hyndman. The extract from pages 5 and 6 here are riddled with numerical errors: the 55.7% for the ACT should be about 45% and the 57.0% for NSW should be 60%; the true would ring loud alarm bells that even a grossly dishonest report like Hyndman's here would be compelled to address – i.e. how could it be that a part of Australia with a socio-economic status that is vastly higher than that of NSW and the other States and Northern Territory have UAI (or equivalent) participation rates that are much lower than that of NSW, VIC etc.? Hyndman's incompetence "helps" him to arrive at absurd conclusions that really are a huge kick in the guts to all ACT students and indeed the entire ACT senior secondary system – but the BSSS and ACT Government have been too stupid and dishonest to realise and admit to this plain truth!

16

Table 1 (continued)

Date of

Statement

Person(s)

Making

Statement


11 May 2006 Mr Zed Seselja MLA and Minister

for Education

Mr Andrew Barr

ACT Legislative Assembly Question on Notice No.1045 parts (3) and (7)-

(11), Hansard pages 1708-1709,

available online at http://www.hansard.act.gov.au/hansard/2006/week05/1708.htm and following

pages.

Education—university admissions index

(Question No 1045)

Mr Seselja asked the Minister for Education and Training, upon notice, on 30 March 2006: ... (3) For each calendar year from 1998 to 2005 inclusive, how has the (a) ACT UAI distribution been established relative to that of NSW and (b) ACT's median UAI been established; ... (7) What role is exercised by the (a) University Admissions Centre (NSW and ACT Pty Ltd) and (b) NSW Technical Committee on Scaling, in the determination of the UAIs of ACT students; (8) In what publicly available document are full details of the NSW Scaling Committee Table, referred to in paragraphs 5.4.1 and 5.4.3.3 on pages 50 and 51 of the 2005 BSSS Policy and Procedures Manual; (9) Noting that this NSW Scaling Committee Table has until now been kept from public view, when will it be released for public view and scrutiny; (10) Who within the ACT, as of this current date, has been allowed to view and examine the NSW Scaling Committee Table as referred to in the 2005 BSSS Policy and Procedures Manual; (11) What checks, as of this current date, has the (a) ACT Government (b) ACT Board of Senior Secondary Studies and (c) NSW Technical Committee on Scaling conducted to ensure the general accuracy and validity of this NSW Scaling Committee Table in respect of its role in determining ACT UAIs.

Mr Barr: The answer to the member's question is as follows: ... (3) (a) UAIs in the ACT are determined by calculating the percentile rank of aggregate scores including notional aggregate scores. Eligible students' UAIs are then calculated with reference to a 'Lookup table' provided by the NSW Technical Scaling Committee. (b) The median score is calculated each year by identifying the middle score from each cohort. ... 7) (a) All UAIs are submitted to the University Admission Centre, which allocates places to NSW and ACT universities. (b) The NSW Technical Scaling Committee provides the lookup table referred to above. (8) The Lookup table is received in digital form by the BSSS and is used in digital form. It is available from the University Admission centre. (9) The Lookup table is available from the University Admission Centre. (10) The table is routinely used by officers of the BSSS. (11) The BSSS considered a detailed evaluation in April 2006.

Mr Barr's answers here are certainly very different from those provided by Ms Gallagher on 15 and 16 February 2006 as above. We now here that the sought after table here "is routinely used by officers of the BSSS". So the BSSS are allowed to see it, but students, parents and researchers like me (and I'm a past student and parent of prospective students as well) can't, though a former student I spoke to once told me that he received a copy of the table sought here to assist with a maths assignment. So the BSSS or UAC (or whoever feels they have the power to control who sees this table) are obviously happy for some people to see this table, but just not the main stakeholders and people like me who raise concerns about the accuracy of the table.

The answer to 3(b) is just plain rude – further proof of the culture we goodies have been up against in the fight to have students' tertiary education aspirations taken seriously. The median is by definition the "middle score", but part (3)(b) here obviously sought the substantive rationale which determined whether the median should be say 75 or 86 or 48 or whatever ...

The "detailed evaluation in April 2006" referred to here is the ridiculous report by Rob Hyndman of Monash University that was riddled with mistakes and concluded that the ACT-NSW equality assumption was all fine and dandy. The Hyndman report does not include a copy of the "NSW Scaling Committee Table" and does not carry out any examination of the table's validity.

The response to part 3(a) shows that ACT UAIs can only be accurate if (1) the "Lookup table" referred to here (aka "NSW Scaling committee Table") is accurate and (2) the correct number of notional aggregates are employed in the ACT UAI determination process.

We see here that ACT system kids are simply at the mercy of a NSW-based "NSW Technical Scaling Committee". So what's the point in having a separate ACT education system and so-called "school based autonomy" if ACT citizens via their government and the ACT BSSS have no control over the highest stakes "pointy end" of the system that determines ACT system UAIs. This "NSW Scaling Committee Table" is not used to determine NSW UAIs at all. It is only used to determine ACT UAIs! What a farce!

The responses to parts (7) and (8) are evasive and false statement by Mr Barr, noting that the "NSW Scaling Committee Table" or "Lookup Table" has never appeared in any document accessible by the general public in the ACT. It has never been made available.

The response to part (10) shows that the ACT citizens who happen to be officers of the BSSS can see this table but the students whose UAIs are calculated using this table, and parents, teachers, researchers and the public generally cannot. Isn't the BSSS supposed to be there for ACT students' benefit, not the other way around?

The so-called "detailed evaluation" referred to in the response to part (11) was a corrupt sham working to pre-determined conclusions and documented in an error-ridden report by Monash University's Professor Rob Hyndman – a senior academic who should have done much better. An extract from this Hyndman report is shown above. I acknowledge that Hyndman has no secondary education qualifications (i.e. a DipEd or BEd) and hence and otherwise should never have been called upon to report on a matter which he is poorly qualified to report upon.

17

Table 1 (continued)

Date of

Statement

Person(s)

Making

Statement


17 May 2006 ACT Department of Education and Training

Colleges ACT

Handbook 2007, available online at http://www.det.act.gov.au/schools/pdf/CollegesACT_Han

dbook_2007.pdf

DET (2006: 31, emphasis added here in italics): UAI

A student’s UAI is calculated from the student’s best 3 T major scaled course scores plus 0.6 of the next best scaled course score. These scaled course scores are added to form the aggregate score. Students are then ranked based on their aggregate score. The UAI is calculated based on this ranking and information supplied by the NSW Scaling Committee.

We're again told ACT UAIs are based on (1) rankings determined within the ACT system (fair enough) and (2) "information supplied by the "NSW Scaling Committee", but the phrase "NSW Scaling Committee" doesn't appear anywhere in this document besides this single entry on page 31. This document makes no mention of any "Scaling Committee Table" and contains the word "table" only twice, both times in the phrase "Table of Contents".

November 2006

Dr Daryl Daley of the

ANU

Paper prepared for the ACT BSSS,

available online at http://www.bsss.act.edu.au/__data/assets/pdf_file/0010/58969/Daryls_meshing

_with_NSW.pdf

Paper titled Relating ACT and NSW UAI populations via PISA and other scores This 11 page paper doesn't once mention any scaling committee table or conversion table or lookup table.

The web address here, ending in "Daryls_meshing_with_NSW.pdf" seems to indicate the cosy relationship between Daryl and the BSSS. Daryl has been on the NSW Technical Committee on Scaling for many years and would perhaps be prepared to have the "Scaling Committee Table" which he, George Cooney and others have created placed up on the BSSS website just as Daryl's paper here is.

January 2007 ACT Board of Senior

Secondary Studies,

signed by chair Tim

Brown of the ANU

ACT BSSS, Policy and Procedures

Manual 2007, ACT Government,

Canberra.

BSSS (2007, emphasis added here in italics):

5.4.1 Universities Admission Index (UAI)

The Tertiary Entrance Statement is produced for students who qualify for the Universities Admission Index (UAI). It only reports information useful for tertiary admission. The method of calculation of the UAI will be: ▪ calculate aggregate score for each student ▪ calculate notional aggregate score for those students completing Year 12 and completing one T subject ▪ calculate candidate ranking for each student including notionals ▪ translate candidate ranking to UAI ranking using NSW Scaling Committee Table.

BSSS (2006: 52, emphasis added here in italics): 5.4.3.2 Aggregate Scores and Universities Admission Index (UAI)

The Aggregate Score is reported on the Tertiary Entrance Statement. The Aggregate Score will be calculated for all students who have studied an appropriate package and sat the AST examination. The Aggregate Score will be calculated and reported on the Tertiary Entrance Statement as an integer. A percentile rank, named the Universities Admission Index, will also be calculated, based on the Aggregate Score. The UAI will be reported on the Tertiary Entrance Statement as a decimal value to the nearest 0.05.

5.4.3.3 Ranking of Candidates

The Year 12 Candidature is the group of Year 12 students who completed at least one T minor course and who were enrolled in Semester 2 of Year 12. The Rank by Year 12 Candidature indicates the percentage of the Year 12 Candidature placed higher than the student on the Aggregate Score. This is translated to a Universities Admission Index (UAI) ranking using NSW Scaling Committee Table. The UAI is a ranking of a student relative to the full age cohort ie. relative to the set of students who would be in the group if all students stayed on and completed Year 12. It is reported with a range from 100.00 for the highest ranked students down to 30.00.

These extracts are identical to the corresponding extracts in the 2006 edition of this manual, except that the 2007 version writes "0.05" rather than ".05" in para 5.4.3.2.

18

Table 1 (continued)

Date of

Statement

Person(s)

Making

Statement

Form of


2007 ACT BSSS Brochure titled What's the UAI?, available online at

http://www.bsss.act.edu.au/__data/assets/pdf_file/0003/45435/Whats_the_UAI.pdf

BSSS (2007a: 3, emphasis in original): The main steps in the process are: 1 Colleges calculate a course score for each student completing a T course. 2 The Board of Senior Secondary Studies scales the college course scores against the AST results, using the OCS scaling method (see Appendix A). This ensures that all course scores can be meaningfully compared. This process can also be referred to as statistical moderation. 3 The scaled course scores are used to produce an Aggregate Score for each student. This is done by adding together the scaled scores in the best three T majors plus 0.6 of the next best T score, whether a major or minor. 4 The Aggregate Scores for all students who have met the appropriate requirements are listed in order from highest to lowest. Starting from the top of the list students are given a candidate rank. 5 This rank is then converted to an age rank by using a Lookup table supplied by the NSW Technical Committee on Scaling. BSSS (2007a: 6-7): step 4: Calculating the UAI

The Aggregate Scores for all students who have met the requirements are listed in order from highest to lowest. The students are then given a rank called the rank by candidature. This rank is then converted to an age rank by using a lookup table supplied by the NSW Technical Committee on Scaling. (The NSW mapping of candidate ranking to age ranking is based on the distribution of Year 10 student results for those students who meet the requirements for the UAI in NSW). This gives a rank for ACT students as if they were part of the NSW age cohort. The ACT, because of its size and demographic nature, can be more readily considered to be within the NSW age cohort for comparison to the states. By using this method the relativities that have been in place for ACT students for a number of years are maintained. The UAI calculated in the ACT is directly comparable to the UAI calculated in NSW and the same cutoffs at NSW and ACT universities apply to both ACT and NSW students. If you are applying to interstate universities, that state’s admissions centre will use the UAI for entry to its universities. All Australian universities have agreed to use the ranking based on the age cohort for each state. This means that most interstate applicants will be able to directly compare their ranks with university cut-offs irrespective of their state of origin.

The claim that "this ensures that all course scores can be meaningfully compared" is complete rubbish in view of the low and sometimes negative correlations between the AST and subject/course scores, but that is a separate issue from that which this compilation is designed to highlight.

Here we are told that the ranks determined within the ACT system are "converted to an age rank by using a Lookup table supplied by the NSW Technical Committee on Scaling".

This "Lookup table supplied by the NSW Technical Committee on Scaling" is presumably the table referred to as the "NSW Scaling Committee" table in the BSSS policy and Procedures Manual.

The explanation on pages 6 and 7 here is the most detailed explanation available in any BSSS or ACT Government document available through Google searches, but it makes incoherent and highly questionable assertions as if they were simple gospel truths, and generally raises more questions than answers. Specifically, it is claimed that the ACT, "because of its size and demographic nature, can be more readily considered to be within the NSW age cohort for comparison to the states". What on earth does this mean? And how exactly does the "lookup table" reflect whatever it is that the author of this statement wants us to believe?

We all know that the ACT lies geographically within the bounds of NSW, and has a population and land area much smaller than those of NSW, but how does it follow from these facts that ACT students should be simplistically considered "as if they were part of the NSW age cohort" for UAI determination purposes?

The explanation on pages 6 and 7 seems to be the BSSS's way of breaking to us here in the ACT that for UAI purposes the ACT age cohort is considered to be of equal ability overall and on average to that of NSW and the other States and NT – as follows from the MCEETYA agreement which simplistically imposes this equality assumption.

This "lookup table" or "Scaling Committee Table" must obviously reflect the MCEETYA State-Territory tertiary admissions score equality assumption, and we should be able to read much more explicit explanations on how the "lookup table" or "Scaling Committee Table" reflects this equality assumption and any other research and analysis that has been conducted to quality control the ACT UAI calculation process.

19

Table 1 (continued)

Date of

Statement

Person(s)

Making

Statement

Form of


2007 ACT BSSS Brochure titled What Certificates Could You Obtain?, available online at

http://www.bsss.act.edu.au/__data/assets/pdf_file/0020/45434/What_certificates_.pdf

BSSS (2007b: 13): Calculating the UAI (7) The Aggregate Scores for all students who have met the requirements are listed in order from highest to lowest. The students are then given a rank called the rank by candidature. This rank is then converted to an age rank by using a lookup table supplied by the NSW Technical Committee on Scaling. (The NSW mapping of candidate ranking to age ranking is based on the distribution of Year 10 student results for those students who meet the requirements for the UAI in NSW). This gives a rank for ACT students as if they were part of the NSW age cohort. The ACT, because of its size and demographic nature, can be best considered to be within the NSW age cohort for comparisons to other states. By using this method, the relativities that have been in place for ACT students for a number of years are maintained.

The UAI calculated in the ACT is directly comparable to the UAI calculated in NSW and the same entrance cutoffs at NSW and ACT universities apply to both NSW and ACT students. If you are applying to interstate universities, state admission centres will use the UAI for entry to their universities. All Australian universities have agreed to use the ranking based on the Year 10 age cohort for each state. This means that interstate applicants will be able to directly compare their ranks with university entrance cut-offs irrespective of their state of origin.

This almost exactly repeats the similar statements which appear in the brochure What's the UAI? as above.

So the incoherent explanations which really explain very little are being spread quite widely ... all incredibly inconsiderate towards the students, parents and others who are obviously entitled to much clearer explanations on how ACT UAIs are determined.

2007 (but still current

in 2008)

UAC Website describing the UAI at

http://www.uac.edu.au/admin/uai.html

... Calculation of the UAI is the responsibility of the Technical Committee on Scaling on behalf of the NSW Vice-Chancellors' Committee. ... Information about the calculation of the ACT UAI is available from the ACT Board of Senior Secondary Studies. Call (02) 6205 7181. ... © 2007 Universities Admissions Centre (NSW & ACT) Pty Ltd A.C.N. 070 055 935 ABN 19 070 055 935

The uppermost sentence here implies that the calculation of the UAI for ACT and NSW systems alike "is the responsibility of the Technical Committee on Scaling on behalf of the NSW Vice-Chancellors' Committee", noting of course that this "NSW Vice-Chancellors' Committee" should really be called the "NSW and ACT Vice-Chancellors' Committee".

The second sentence here – the only mention of ACT UAIs on this entire page (except the contents listing pointing to this little endnote/afterthought) – says that "information about the calculation of the ACT UAI is available from the ACT Board of Senior Secondary Studies". But if a person called the ACT BSSS on (02) 6205 7181 as advised they'd probably be shown one of the brochures (especially the What’s the UAI? brochure) or the BSSS Policy and Procedures Manual as described above in this compilation, all of which – along with this UAC webpage here - clearly state or imply that the ACT UAI calculation involves significant input from this "NSW Technical Committee on Scaling" or "NSW Scaling Committee". The ACT BSSS documents here all refer to the "NSW Scaling Committee Table" or the "lookup table supplied by the NSW Technical Committee on Scaling".

A person wanting to see how ACT UAIs are calculated is sent on a circular wild goose chase with no end point. If the person starts with the BSSS they'll be informed that the NSW Technical Committee on Scaling provides a lookup table that sets the entire ACT UAI distribution, but when one searches for details on how the Technical Committee calculates ACT UAIs, one is informed on the relevant UAC webpage at http://www.uac.edu.au/admin/uai.html that "information about the calculation of the ACT UAI is available from the ACT Board of Senior Secondary Studies". I wonder if an accountability black hole as had as this arises anywhere else in the world? This situation certainly appears to be world's worst practice!

20

Table 1 (continued)

Date of

Statement

Person(s)

Making

Statement

Form of


June 2007 Governor General's

Office

List of people awarded AMs

in 2007 Queen's Birthday

Honours, as shown online

at http://www.gg.gov.au/res/File/PDFs/honours/qb07/Media%20notes%20AM%20(A-L).pdf, on page 56.

[emphasis added here]

MEMBER (AM) IN THE GENERAL DIVISION OF THE ORDER OF AUSTRALIA Professor George Henry COONEY, 90 Mitchell Street, Croydon Park NSW 2133 For service to secondary education, particularly through contributions to scholarship, research and policy development in the areas of curriculum, educational testing, and standards based assessment. Current Professor of Education, Australian Centre for Educational Studies, Macquarie University. Patron and benefactor, Professor George Cooney Scholarship, since 2004. Director Teacher Education Program, 1993-2000.

Current Member, Accreditation Panels, New South Wales Vice Chancellors Committee. Adviser, School Partners Program, since 1999. Expert Adviser to School Partners Program since 1999; facilitates links between the University and over 60 member schools; voluntarily visits program member schools to assist staff, students and parents to understand university entrance scores. Extensive involvement with the Gifted and Talented Program (recently launched). Recent Presenter, fairer assessment procedures to Beijing Education Authorities, China. Chair of Education, 1993. New South Wales Department of Education and Training

Chairman, New South Wales Technical Committee on Scaling (responsible for the calculation of the Universities Admission Index), since 1977; appointed to conduct an independent review of New South Wales Student Assessments, 2005 (Interim report released 2006).

Current Chairman, Interstate Transfer Index Technical Group; responsible for ensuring each state's methodology adheres to the

agreed principles for calculation of the index.

Board of Studies New South Wales Long-standing Member, Technical Advisory Committee; advises the Board on statistical issues associated with scaling and assessment moderation procedures. Member, Higher School Certificate Consultative Committee (responsible for determining results that are reported to students), for approximately 25 years; Auditor, Higher School Certificate results (prior to release to students), since 1996. Adviser (curriculum development and assessment matters in secondary education and the process to determine University Admission Index), since 1990s; Senior University Member, Assessment Committee, since 1990. Key role in Introduction of the New Higher School Certificate Scaling System, 1977. Member Evaluation Team, 2006 National Literacy and Numeracy Trial, Department of Education, Training and the Arts, Queensland.

Advises and assists, University Admissions Index Enquiry Centre, University Admissions Centre (following the release of

University Admissions Index each year).

Statistical Expert, Consultant and mentor to staff of the Educational Measurement and School Accountability Directorate, voluntary capacity. Research Fellow, Edinburgh University, 1988; Princeton University, 1980. Author and co-author to many research papers in diverse fields. Current Parish Warden, Service Leader and Lay Preacher, Anglicans of Enfield; former Synod representative, Council Member, Church Warden, for approximately 40 years. President, Rotary Club of Burwood; various Board roles, for approximately 15 years. Awards include: Paul Harris Fellow, Rotary Club of Burwood.

Shows that Professor George Cooney is the current chairman of the NSW Technical Committee on Scaling, and has been the chairman since 1977 [or maybe this should be 1997? a typo perhaps ...], and is also the chairman of the Interstate Transfer Index Technical Group. This also shows that George is a member of the NSW Vice Chancellors Committee and someone who advises and assists for the University Admissions Index Enquiry Centre, which is part of the University Admissions Centre.

21

Table 1 (continued)

Date of

Statement

Person(s)

Making

Statement


1 July 2007 ACT Government

Board of Senior Secondary Studies Act 1997,

Reproduction No. 6, Effective 1 July 2007,

available online at http://www.legislation.act.go

v.au/a/1997-87/current/pdf/1997-87.pdf.

Page 14: ...

20 Protection of board members from liability

(1) A board member does not incur civil liability for an act or omission done honestly and without recklessness for this Act. (2) Any civil liability that would, apart from this section, attach to a person, attaches instead to the Territory.

This legislation is the same as that which became effective on 2 June 2005, as above.

30 August 2007

Question by Mrs Vicki

Dunne MLA dated 23

August and response by

ACT Education Minister

Andrew Barr on 30

August 2007

ACT Legislative Assembly Question on Notice Number 1662 parts (1)-(6), Hansard,

pages 2569 to 2573, available online at

http://www.hansard.act.gov.au/hansard/2007/week08/2569.htm and following pages.

Universities admissions index

(Question No 1662)

Mrs Dunne asked the Minister for Education and Training, upon notice, on 23 August 2007: (1) In relation to the process used by the ACT Board of Senior Secondary Studies (BSSS) to calculate ACT Universities Admission Indexes (UAIs) as set out in the current 2007 edition of the BSSS Policy and Procedures Manual (the Manual), why does the Manual not contain any details of the changes to the UAI calculation process that were made in late 2006, as reported in The Canberra Times on 20 December 2006 and 28 April 2007; (1) Are there any plans to amend the Manual to explain the changes referred to in part (1); if not, why not; (2) Has a NSW Scaling Committee Table (Table) played a role in the UAI determination process, as described in sections 5.4.1 (page 51) and 5.4.3.3 (page 52) of the Manual; (3) Why does the Manual not include a copy of the Table; (4) Is the Table available for inspection anywhere on the BSSS or Department of Education and Training (DET) websites; (5) Did the Table used in 2006 differ from that used in 2005; if so, (a) what were the differences and (b) how did they affect UAIs; (6) Can the Tables used to determine UAIs in 2006, 2005 and other past years be placed on the BSSS and/or DET websites so that affected students, parents and other members of the public may inspect them; ... Mr Barr: The answer to the member's question is as follows: (1) The changes that were made in late 2006 should have appeared in the ACT Board of Senior Secondary Studies (BSSS) Policy and Procedures Manual 2007 (the manual). This was an omission. The changes will be added as an addendum to the version on the BSSS website. The change to policy was that the year 12 candidature will be defined as 'the group of students who complete at least one T course'. (1) The 2008 edition of the manual will contain the new policy. (2) Yes. (3) The purpose of the manual is to outline BSSS policies and general procedures. Apart from the fact that the table is not the intellectual property of the BSSS, operational procedures are not part of the manual. (4) No. (5) Yes; (a) a new table relevant to the cohort of students is produced by the NSW Scaling Committee at the end of each year; (b) the 2006 UAIs were slightly higher than those in 2005, for each TER. (6) As the table is not the property of the BSSS or the Department of Education and Training, I am unable to comment. ...

In response to part (3), Mr Barr cites two reasons why the 2007 ACT BSSS Policy and Procedures <anual does not include the "NSW Scaling Committee Table" which – as the manual itself describes on pages 51 and 52 – sets the entire ACT UAI distribution. He first claims that the "NSW Scaling committee Table" is "not the intellectual property of the BSSS". He then claims further that "operational procedures are not part of the manual" – a strange claim for a Policy and Procedures Manual, and quite inaccurate given that quite a lot of the various ACT UAI calculation procedures certainly are described in the manual here, albeit with huge gaps in the descriptions such as the "NSW Scaling Committee Table" itself. The ACT BSSS Policy and Procedures Manual certainly raises and leaves unanswered numerous critically important questions about the ACT UAI deterination process. Documents describing how UAIs are calculated for students completing the NSW HSC system, in contrast, are vastly superior to those provided in this ACT BSSS manual. Whereas the ACT authorites won't publish the "Scaling Committee Table" used to determine ACT UAIs, documentation on the NSW HSC system available via the UAC website provides tables and descriptions generally which very clearly explain how NSW UAIs are calculated in terms of the NSW system Year 10 School Certificate cohort.

Mr Barr correctly states in response to part (4) that the "NSW Scaling Committee Table" which sets the entire ACT UAI distribution each year is not available for inspection anywhere on the BSSS or Department of Education and Training (DET) websites. What a farcical state of affairs this continues to be!

22

Table 1 (continued)

Date of

Statement

Person(s)

Making

Statement

Form of


14 December 2007

Question by Mrs Vicki

Dunne MLA dated 14

November 2007 (see

www.parliament.act.gov.au/downloads/noti

ce-paper/06NP11

8.pdf) and response by

ACT Education Minister

Andrew Barr on 14

December 2007.

ACT Legislative Assembly

Question on Notice

Number 1763 part

(2), Hansard pages 2077-

2079, available online at

http://www.parliament.act.gov.au/downloads/notic

e-paper/06NP1

18.pdf.

MRS DUNNE: To ask the Minister for Education and Training— ... (2) Did the NSW Scaling Committee Table used to determine ACT UAIs need to be modified as a result of the use of PISA results in the 2006 UAI calculation process; if so, how and on what publicly available document is the rationale for such changes documented; ... Mr Barr's response dated 14 December 2007 as shown in Mr Barr's letter to Mrs Dunne:

Mr Barr claims here in response to part (2) that he is unable to comment on the "NSW Scaling Committee Table" which sets the entire ACT UAI distribution each year because of "intellectual property" reasons. He was, however, prepared to comment on the table in response to part (5) of Question 1662 as above.

The BSSS and "NSW Scaling Committee" clearly consider themselves beyond accountability in relation to this table and Mr Barr seems to have no problem with this and seems happy to continue to come up with different excuses for why the public is unable to see the very instrument that sets the entire ACT UAI distribution.

The lengths the ACT Government, BSSS and this "NSW Scaling Committee" continue to go to in order to hide this Scaling Committee Table from public view, along with the rationale behind it, and how PISA results affected it from late 2006 etc., is something I never thought would be considered acceptable or even possible in an advanced democracy, let alone in the supposedly highly advanced ACT.

I believe the lack of transparency and accountability on display here, coupled with the obviously incompetent and dishonest processes that led to the Hyndman report in 2006, amount to a serious form of corruption in which the ACT Government, BSSS and the "NSW Scaling Committee" have clearly all been involved.

Mr Barr's response to part (1)((b) further explains how the BSSS and this "NSW Scaling Committee" are able to carry out secretive research and analysis used to "inform BSSS policy" without in any way being made accountable for its impact upon the UAIs which our Year 12s receive each year. Mr Barr's responses here indicate that Mr Barr himself has no problem with this outrageous lack of accountability and transparency.

Why is the ACT government continuing to side with those responsible for robbing thousands of ACT kids of funded uni places over many years? Is this a legal liability issue in that the ACT Government are worried they'll be successfully sued? If so, wouldn't it be better to do the right thing to reduce such exposure to liability?

WP12 NSW Scaling Committee Tablemembers.webone.com.au/~markld/PubPol/Edu/UAIs/WPs/WP12.pdf · 2....

Documents

Transcript of WP12 NSW Scaling Committee Tablemembers.webone.com.au/~markld/PubPol/Edu/UAIs/WPs/WP12.pdf · 2....