6/23/2014

1

Fix Fix it or Flee itit or Flee it??Proven approaches for dealing with Proven approaches for dealing with

failing, flagging and floundering association programs failing, flagging and floundering association programs

Greg Melia, CAE @gmeliacae

Photo credit: hebedesign on flickr

Program Assessment is a lot of GRIEF

Program Assessment is a lot of GRIEF

• Goals

• Research

• Impact

• Efficiency

• Finances Credit: Angie Torres on Flickr

6/23/2014

2

Defining the Goal of the Review

– Work with key stakeholders and decision-makers

– Solicit their key questions:• What do they want to know?• What information will help their decision-making?• What do they think needs to be evaluated?

– Refine questions to be measurable and defined

Exploring Program Goals

Investigate the Key “Whys”:•Why was the program originally established?•Why has it continued to be offered?•Why has it change over time?

Review the Original Intended Approach:•Who, what, when, where, why and how?•What was implemented? •What was not?

Lessons Learned

• Verbalizing goals of the review are an important part of preparing for change

• Seek superordinate goals

• Some agreements are easier reached upfront

6/23/2014

3

Research3 kinds of data:Corpus Inscriptionum

Opinions and Descriptions

Imponderabilia of behavior

Bronislaw Malinowski

Credit: Photographie et Ethnologie: Les photographes français au XXème siècle

devant d’autres formes de cultures.

Lessons Learned

• Historical minutes and documents can be VERY informative

• Seek to understand program mutations

• Good decisions based on bad data usually give bad results

• Remember they may have been involved

Impact Measure

• Outputs = ImmediateI attended

• Outcomes = Short-termI gained knowledge

• Impacts = Long-termI served more members as a result

6/23/2014

4

Efficiency

Staff and Volunteer Time

Complaints and Comments

Marketing/Communication

Registration/Orders

Technology Photo credit: mansikka on Flickr

Lessons Learned

• How do people feel about it? Do they have suggestions to increase efficiency?

• What are the trends in terms of level of effort?

• What is cost/benefit of doing it the same way versus trying a new approach?

• Can improved efficiency save it?

Finances

• Direct fixed expenses– Incurred regardless of how many people participate,

buy, or are served (e.g., office rent)

• Variable expenses– Incurred per each additional unit or person that is

serviced (e.g., per person meal charge)

• Mixed expenses– Incurred per each additional set of units or persons

serviced (e.g., room rental, temporary help)

6/23/2014

5

Key Calculations

• Revenue – Expense = Margin– Margin per attendee/unit

– Margin per staff hour

– Margin per impact

Lessons Learned

• One full flight is more profitable than two half full ones.

• Most do not realize the true full cost.

• Understanding costs helps determine price.

• Price increases (and closing programs) take time.

Implementation • Who to involve

– Consultant?– Stakeholders– Staff– Non-users

• Who to interview– Former volunteers– Former staff– Vendors

Photo credit: kierenmccarthy.co.uk

6/23/2014

6

Communication of Results

• Review purpose of review

• Review what was done

• Arrange key findings in logical order, highlighting interpretations where appropriate

• Close with recommendations or issues to be addressed

Credit: EE.UTD.Events on Flickr

Lessons Learned

• Equivocal recommendations get equivocal results

• Decision-makers need synthesized data, not the opportunity for data analysis

• Survey data is important, but the proof is in historical performance

• Plan and communicate transitions

Thank You!Thank You!Greg Melia, CAE

gmelia@asaecenter.org@gmeliacae