Dv Clauses Monitoring Evalulation Framework 2011

8/11/2019 Dv Clauses Monitoring Evalulation Framework 2011

1/13

Framework to Monitor and Evaluate

the Outcomes of the Introduction of

Domestic Violence Clauses

For the Australian Domestic and Family Violence

Clearinghouse

Social Policy Research Centre

University of New South Wales

May 2011


2/13

DRAFT Framework to Monitor and Evaluate the Outcomes of the Introduction of Domestic Violence Clauses

Social Policy Research Centre i

Contents

1 Introduction ......................................................................................................... 1

2 Evaluation framework and program logic ........................................................ 1

3 Data sources ......................................................................................................... 33.1 Existing data sources ............................................................................................... 4

4 Project-specific evaluation data ......................................................................... 54.1 Establishment of baseline data and instrument development .................................. 5

4.2 Workplace survey .................................................................................................... 6

4.3 Union survey ........................................................................................................... 7

5 Scope and Sampling ............................................................................................ 7

References ...................................................................................................................... 8


3/13


Social Policy Research Centre 1

1 Introduction

The Domestic Violence Workplace Rights and Entitlements project is being

conducted by the Australian Domestic and Family Violence Clearinghouse and

funded by the Commonwealth Department of Education, Employment and Workplace

Relations (DEEWR).

The goal of the project, which runs from July 2010 to December 2011, is to reduce the

impact of family violence on working women by achieving better workplace rights

that support them to stay safely in their jobs and in their homes. This is to be achieved

by:

Briefing a broad range of unions and employers nationally on the issue of familyviolence as a workplace issue, the adoption of family violence provisions in

enterprise agreements and other workplace instruments

Developing with unions and employers a set of model workplace information andtraining resources for general staff, human resources personnel, union delegates

and supervisors.

Producing model policies and safety plans to assist in the informed introduction offamily violence clauses.

Surveying union members to provide essential information on the impact offamily violence, and enabling future monitoring of the outcomes of introducing

family violence clauses and other instruments.

The first Australian family violence clauses have been incorporated into the enterprise

agreement between the Surf Coast Shire Council (Victoria) and the Australian

Services Union, Victorian Authorities and Services Branch (September 2010)providing worlds best practice with up to twenty day extra paid leave for reasons of

family violence.

This document proposes an evaluation and monitoring framework for the introduction

of domestic violence clauses into industrial awards and agreements and the Domestic

Violence Workplace Rights and Entitlements project.

2 Evaluation framework and program logic

A program logic is a systematic, visual representation of the underlying assumptions

of a planned program. It illustrates why and how a program is presumed to work(AIFS and SPRC, 2010).

Figure 2.1 is a program logic for the implementation and impact of the introduction of

Domestic Violence clauses into industrial agreements, based on the anticipated

activities, and short-, medium- and long-term outcomes.

The model should be read from bottom to top. Inputs are at the bottom. These are

anticipated to drive changes in the content of industrial agreements, and in the

development of workplace initiatives including training. These changes are

anticipated in turn to lead to improvements in key measures related to domestic and

family violence.


4/13



A program logic sets out the hypothesised results of an intervention, and evaluation

and monitoring activities should ascertain whether or not they occurred. This requires

measuring if and to what extent changes have occurred as anticipated. Possible

measures and data sources for monitoring whether the project has been implemented

as intended (inputs) are identified inTable 2.1.No one data set is available to monitor

the outcomes of the project. The ABS Conceptual Framework for Measuring Familyand Domestic Violence (2009) identifies a range of data and definitional needs that

are require to improve data on victims, perpetrators and responses. Possible measures

and data sources for monitoring whether the anticipated outcomes have occurred are

identified inTable 2.2.

It should be noted that evaluation frameworks such as this one alone are insufficient

for the attribution of any changes to the introduction of the clauses and associated

initiatives. The relationship between family and domestic violence prevalence and

reporting is generally agreed to be characterised by under-reporting and under-

recording. The prevalence of family and domestic violence is therefore likely to be

much higher than reported rates, and changes in reported prevalence could meanimprovements to reporting, or actual changes in prevalence, or both. Moreover, even

if it were possible to be confident that the anticipated outcomes had really happened,

this framework does not allow for any calculation of the impact of the project on these

changes. It is possible that other industrial, social, economic or cultural changes could

bring about changes to workplace cultures and the prevalence of domestic violence;

and this framework cannot disentangle those changes from the impact of the project.

Notwithstanding these limitations, the framework should provide a robust, relatively

straightforward, mechanism for monitoring the implementation of the clauses.


5/13



Figure 2.1: A program logic for evaluating the implementation and impact of Domestic Violence clauses into industrial agreements.

Incorporation of domestic violence clauses

into industrial instruments.

Introduction of training and other significant

workplace initiatives related to implementing the

clauses.

Uptake of entitlements and related

safety policies and procedures in

workplaces where these are

available, by employees

experiencing domestic violence

1. A reported reduction in the rates and types of domestic violence

experienced by workers, impacting on their work performance, and

manifesting in the workplace (e.g. abusive phone calls/e-mails; performance

issues).

2. Reduction in adverse action experienced by women at work (loss of job,

performance management, disciplinary action) as a result of domestic

violence.

3. The effectiveness of safety plans and measures such as telephone and e-

mail screening.

4.The uptake of protection orders and their effectiveness.

5. Increased activity by the workplace on community related domestic violence

prevention.6. Development of workplace cultures which are supportive of employees

experiencing domestic violence.

Reform area

Medium term

outcomes

Long term

outcomes

1. Development of model domestic violence entitlements

clauses appropriate for inclusion in enterprise agreements

2. Meeting with and presenting issues relating to the impact of

domestic violence on the workplace and the use of the clauses

to minimise this to interested unions and employers across

the country.

3. Development of model policies and procedures to assist

workplaces operationalise domestic violence clauseseffectively

1. Development of fact sheets with information about domestic

violence and its impact on employees, employers and union

representatives

2. Development of a generic training module which can be

customised to particular employers

3. Customising the training module for the partner unions and

employers

4. Development of a website providing information andresources for employers (including model domestic violence

clauses), and an online training tool

Inputs

Incorporation of domestic

violence clauses into industrial

instruments.

Introduction of training and

other significant workplace

initiatives related to

implementing the clauses.

Short term

outcomes


6/13



Table 2.1: Possible measures and data sources: inputs

Measure Source

Clauses

Development of model domestic

violence entitlements clausesappropriate for inclusion in

enterprise agreements

Number of clauses

developed

ADFCV

Meeting with and presenting issues

relating to the impact of domestic

violence on the workplace and the

use of the clauses to minimise this to

interested unions and employers

across the country.

Number of meetings,

workshops, forums

attended

ADFCV

Development of model policies and

procedures to assist workplaces

operationalise domestic violenceclauses effectively

Number of policies and

procedures developed,

distributed

ADFCV

Training

Development of fact sheets with

information about domestic

violence and its impact on

employees, employers and union

representatives

Fact sheets developed ADFCV

Development of a generic training

module which can be customised to

particular employers

Training models

developed

ADFCV

Customising the training module for

the partner unions and employers

Number of customised

training models

ADFCV

Development of a website

providing information and

resources for employers (including

model domestic violence clauses),

and an online training tool

Website developed ADFCV


7/13



Table 2.2: Possible measures and data sources: short-, medium and long-term outcomes

Outcome Measure Source

Short term

Incorporation of domestic violence

clauses into industrial instruments.

Number of negotiations

commenced to insert clausesinto industrial instruments

Number of industrial

instruments which

incorporate clauses

Union survey

(possible long-termsource: ABS

SEARS/TUS)

Introduction of training and other

significant workplace initiatives related

to implementing the clauses.

Number of workplaces

which introduce training and

other initiatives

Take-up and effectiveness of

training

Workplace survey

Union survey

Medium term

Uptake of entitlements and relatedsafety policies and procedures in

workplaces where these are available by

employees experiencing domestic

violence.

Rates of leave and otherentitlements used

Workplace survey(possible long-

term source: ABS

SEARS/TUS)

Long term

A reported reduction in the rates and

types of domestic violence experienced

by workers, impacting on their work

performance, and manifesting in the

workplace (e.g. abusive phone calls ande-mails; performance issues).

Number of workers

reporting domestic violence

at work

Reported impact of domestic

violence on workperformance and workplace

Workplace survey

Union survey

Reduction in adverse action

experienced by women at work (loss of

job, performance management,

disciplinary action) as a result of

domestic violence.

Reported impact of domestic

violence on work

performance and workplace

Focus groups

Workplace survey

Union survey

The effectiveness of safety plans and

measures such as telephone and e-mail

screening.

Experience and perceived

impact of safety plans

Satisfaction ratings

Focus groups

Workplace survey

Union survey

The uptake of protection orders and

their effectiveness.

Number of protection

orders

Outcome of protection

orders

Specific research

required. Victoria is

the only

jurisdiction with a

family violence

database.

Individual

jurisdictions have

some data available

on protection

orders, (e.g. NSWBOCSAR) but this


8/13



would require

further analysis.

Most research on

protection orders is

conducted via case

analysis orrecruitment from

court hearings.

Increased activity by the workplace on

community related domestic violence

prevention.

Focus groups

Workplace survey

Union survey

Development of workplace cultures

which are supportive of employees

experiencing domestic violence

Focus groups

Workplace survey

Union survey

3 Data sources

Based on the program logic, which indicates that the anticipated outcomes of the clauses are

short-term, medium-term and long-term, a mixed method approach to data collection and

analysis is suggested. This should include:

Exploitation of data already routinely collected

Modifying and expanding existing instruments used for routine data collection

Collecting and analysing project-specific data on implementation and impact

For most evaluations, these options present a trade-off between precision and efficiency. In

general, analysis of data that collected by a third-party agency for different purposes, for

example ABS survey or census data, has fewer costs than the project or evaluation

researchers collecting the data themselves. This is also true of analysis of administrative data

that projects collect in the course of doing business: for example, number of staff employed,

number and type of services delivered. The disadvantage of this first approach, however, is

that these data sources can only provide, in most cases, proxies for what the evaluation is

actually trying to measure, and proxies vary in how well they fit the intended outcome.

The second method involves adding items to existing surveys and reports. This processinvolves identifying existing instruments that could be adapted, and negotiating with the

agency responsible to (normally) add new questions. This can be done with large-scale

surveys such as ABS surveys, and with routinely collected project data. The advantage of this

method is that it combines better efficiency and precision than simply working with existing

data sources, with fewer costs than a totally new survey. However, it is dependent both on the

identification of surveys that could be augmented to fit the evaluation, and on the capacity

and willingness of the relevant agencies to make these changes.

A third method is for projectsor independent evaluatorsto design and implement

evaluation instruments themselves, and ensure that these instruments collect data that does fit

very closely with what the evaluation is trying to measure. However, this demands resources,from design to implantation to analysis of the evaluation instruments, and cannot be used for


9/13



ongoing evaluations with confidence, unless it is likely that the resources will also be

ongoing.

Using a combination of methods may therefore address the disadvantages, and utilise the

benefits, of each. The following section describes possible existing data sources that may be

used and/or augmented, and options for developing project-specific instruments.

3.1 Existing data sources

ABS SEARS and TUS

The ABS Survey of Employment Arrangements, Retirement and Superannuation (SEARS) is

conducted every six years. The next survey will be in 2013 (see box). It collects detailed

information about:

the diversity of employment arrangements in Australia, including the types ofemployment people have, aspects of job stability and flexibility, and working patterns;

the working arrangements that people use, or would like to use to balance their work andcaring responsibilities;

plans that people aged 45 years and over have for retirement, including transitions toretirement, expected sources of income at retirement and factors that might influence the

decision to retire;

characteristics of retirement including age at retirement, reasons for retiring andretirement income; and

the superannuation coverage of individuals, including superannuation contributions and

account balances.The Australian Bureau of Statistics (ABS) conducted the Survey of Employment

Arrangements, Retirement and Superannuation (SEARS) in 2007 and the Time Use Survey

(TUS) was last conducted in 2006. SEARS 2007 and TUS 2006 will be combined and

components updated to form SEARS/TUS 2013. SEARS/TUS 2013 will be comprised of a

SEARS questionnaire to be completed by the entire sample of households, and a Time Use

module and Time Use Diary to be completed by a proportion of households within that

sample.

SEARS/TUS 2013 has a target sample size of 13,750 households from all States and

Territories. Information will be collected via Computer Assisted Personal Interview (CAPI).

The Time Use component has a target sample size of 4,000 responding households withinthe overall sample. Households in the TUS subsample will also be required to complete a

time use module and fill in a Time Use Diary in which they will record their activity over a 48

hour period.

(From the ABS Discussion Paper: Development of the 2013 Survey of Employment

Arrangements, Retirement and Superannuation and Time Use Survey)

Items in the survey relevant to monitoring the impact of the Domestic Violence clauses

include employment arrangements and work, caring and family balance.


10/13



Whether entitled to paid sick leave in job

Whether entitled to paid sick leave in job

Whether entitled to paid holiday leave in job

Leave entitlements in job

Whether entitled to paid carers leave in job

Whether had time off work to provide care last week

Time absent from work due to providing care

Whether used working arrangements to provide care last week

Type of working arrangements used to provide care last week

Whether wanted to make more use of working arrangements to care for someone

Type of working arrangements wanted to make more use of

Main reason did not make more use of working arrangements to care for someone

Whether would like to have made use of working arrangement to care but didn't

Type of working arrangement would like to have used but did not

Main reason did not make use of working arrangement would like to have used

Ideally, a question specifically on access to domestic violence workplace entitlements could

be added to this survey.

4 Project-specific evaluation data

4.1 Establishment of baseline data and instrument development

The development of effective instruments to monitor the effects of the introduction of the

clauses, their effects, and their interactions with other workplace initiatives, should be

established by the use of broad-ranging, qualitative research. This type of research cannot

provide baseline data on either the prevalence and workplace effects of domestic violence, or

the take-up of domestic violence clauses in industrial agreements (no one data source has

been identified that could provide this quantitative data). However, given that the anticipated

outcomes do not relate only to the introduction of the clauses, but also their role in changing

workplace cultures, broader outcome measures are needed.

Research indicates that the relationship between family violence and supportive workplacesis complex. Disclosure by victims of family violence to supervisors can be beneficial, but

different kinds of support are desired and warranted (Yragui et al., 2011). For example, has

been hypothesised that women who are closer to ending an abusive relationship will want

more comprehensive assistance from workplaces, whereas those in a pre-contemplation

phase will be interested in specific, limited and confidential support (Perrin et al., 2010). The

introduction of policies and initiatives to encourage disclosure, and support victims, can haveunanticipated consequences (O'Leary-Kelly et al., 2008). The impact of the introduction of

clauses into industrial agreements will vary between individual workplaces, and be affected

by the existing workplace culture and systems already in place to protect safety and

confidentiality (for example: screening of phone calls, security systems and measures, and

employee assistance programs) (Swanberg et al., 2006; Swanberg and Ojha, 2010).


11/13



Therefore, in addition to monitoring anticipated changes, the evaluation should also attempt

to identify unanticipated consequences, both positive and negative. Qualitative approaches

are most appropriate, and could include open-ended questions on surveys (see below) and

periodic focus groups and interviews with employers, employees and unions.

4.2 Focus groups and interviewsFocus groups with key stakeholders should be conducted prior to the development of survey

instruments. As an alternative to focus groups, or as an additional method, individual

interviews (face-to-face, in person, email or phone) could also be used. These baseline

activities will be important to:

Identify workplace- and industry-specific issues likely to affect implementation oroutcomes, so that these can be included in the surveys

Identify common concerns and anticipated benefits, so that these can be included in thesurveys

Gather baseline information on existing safety plans, workplace cultures and communityactivity around domestic violence.

Focus groups should then be conducted throughout the implementation of the project to

identify unanticipated outcomes, positive and negative, and the interaction between different

elements. Ideally, the second round of focus groups should be held around 12 months from

commencement. The focus groups should also gather qualitative information on:

Experiences of negotiating the insertion of clauses into industrial agreement

Factors which facilitate and hinder introduction of clauses into industrial agreements

Perceived impact of safety plans, and barriers and facilitators to implementation

Perceptions of workplace culture and changes over time as a result of the project

Facilitators and barriers training, awareness programs and other initiatives (employer- andunion-initiated

Perceived effectiveness of workplace activity on community related domestic violenceprevention

4.3 Workplace survey

Online surveys for employees and employers should be administered to monitor changes in

key outcome areas, and include items on the following topics:

Knowledge of training, awareness programs and other initiatives

Take-up and effectiveness of training (satisfaction ratings, qualitative assessment)

Access to and take-up of leave (DV, carers, unpaid etc)

Incidence of experiencing family and domestic violence at work

Reported impact of domestic violence on work performance and workplace

Knowledge of safety plans

Perceived impact of safety plans (satisfaction ratings, qualitative assessment)


12/13



Knowledge and perceived effectiveness of workplace activity on community relateddomestic violence prevention

Perceptions of workplace culture

4.4 Union survey

Online surveys for union stakeholders (local delegates and state/national offices) should also

be administered, to monitor changes in key outcome areas, and include items on the

following topics:

Introduction of clauses into industrial agreements

Factors which facilitate and hinder introduction of clauses into industrial agreements

Facilitators and barriers training, awareness programs and other initiatives (employer- andunion-initiated)

Disclosure and reporting of family and domestic violence at work

Reported impact of domestic violence on work performance and workplace

Introduction and perceived effectiveness of safety plans

Knowledge and perceived effectiveness of workplace activity on community relateddomestic violence prevention

5 Scope and Sampling

Ideally, surveys would be administered in all workplaces (a census), but resource and other

constraints will likely make this impracticable. An alternative evaluation strategy is to

administer surveys in a selection of workplaces (a sample survey), and conduct qualitativeresearch with all or some of those workplaces participating in the sample survey. Six unions,

representing large memberships, are participating in the survey being conducted for the

Domestic Violence Workplace Rights and Entitlements project, and should be invited to

participate in later surveys and qualitative data collection. The sampling strategy should be

designed to ensure that a broad representation of workplace characteristics are covered, as

well as other key variables (jurisdiction, unionisation rates, risk factors for family violence

victimisation etc).


13/13



References

Australian Bureau of Statistics (ABS) (2009), Conceptual Framework for Measuring Family

and Domestic Violence, Cat. No 4529.0, Australian Bureau of Statistics, Canberra.

Australian Institute of Family Studies (AIFS) and Social Policy Research Centre (SPRC)

(2010),Keep Them Safe: Evaluation framework, Prepared for the NSW Departmentof Premier and Cabinet. Unpublished, Sydney.

O'Leary-Kelly, A., Lean, E., Reeves, C. and Randel, J. (2008), 'Coming into the light:

Intimate partner violence and its effects at work', The Academy of Management

Perspectives (formerly The Academy of Management Executive)(AMP), 22(2), 57-72.

Perrin, N. A., Yragui, N. L., Hanson, G. C. and Glass, N. (2010). Patterns of Workplace

Supervisor Support Desired by Abused Women [electronic version].Journal of

Interpersonal Violence. Retrieved from

http://jiv.sagepub.com/content/early/2010/09/17/0886260510383025.abstract. DOI:

10.1177/0886260510383025

Swanberg, J. E., Caroline Macke and Logan, T. (2006), ' Intimate Partner Violence, Women,

and Work: Coping on the Job. Violence and Victims', Violence and Victims,, 21(5),

561-578.

Swanberg, J. E. and Ojha, M., U. (2010), State Employment Protection Statutes for Victims of

Domestic Violence as an Employment Matter, University of Kentucky: Centre for

Poverty Research, Lexington. http://www.ukcpr.org/Publications/DP2010-11.pdf.

Yragui, N., Mankowski, E., Perrin, N. and Glass, N. (2011), 'Dimensions of Support Among

Abused Women in the Workplace',American Journal of Community Psychology, 1-

12.

Dv Clauses Monitoring Evalulation Framework 2011

Documents

Transcript of Dv Clauses Monitoring Evalulation Framework 2011