RP/CISOA ConferenceApril 2009

1

PresentersWillard Hom, Dean/Director Alice van Ommeren, Research and Planning StaffLeAnn Fong-Batkin, Research and Planning Staff

2

ObjectivesTo inform researchers and IT staff about specific future

research efforts in the Chancellor’s OfficeTo gather comments related to the research agendaPreview the research agenda prior to public release

3

BackgroundBoard of Governors adopted the System Strategic Plan in

2006. http://strategicplan.cccco.edu/ Research agenda addresses goal D3 of the Strategic Plan,

Analytical Capacity for Measuring Success.Prior work on organizing research topics was in the

1990s.

4

Goal of the Research AgendaResearch agenda helps prioritize projects.Research agenda promotes coverage of topics that

concern the wide array of system stakeholders.Research agenda increases transparency.

5

Research AgendasA research agenda structures a prioritization process so

that an organization can systematically weigh factors in its consideration of research projects and activities.

Research agendas are used in a wide variety of disciplines

6

Creation of the Research AgendaTwo-day meeting in October 2008Used an external facilitator

7

ParticipantsCommunity college representatives, including chief executive

officers, chief information systems officers, and researchersAcademic SenateRP GroupDepartment of FinanceLegislative Analyst’s OfficeCal-PASSCommunity College League of CaliforniaFoundation for California Community CollegesChancellor’s Office Staff

8

ProcessDiscussed value of creating a research agendaIdentified external opportunities and challengesIdentified internal strengths and weaknessesDiscussed potential research projectsDeveloped criteria to evaluate each projectPrioritized the projects

9

Value of Our Research AgendaMain ideas guiding our discussion1.Focus Resources via Prioritization2.Manage Expectations3.Provide Leadership

10

External Opportunities*Culture of evidenceEconomyChanging role of researchersChange in technologyIncreased interest in CCs from external partiesNew leadership and new relationshipsChanging demographicsNew research opportunities*as reported by participants

11

External Challenges*Restrictions on access to dataData quality, data coverage, and research methodsResearch resources (expertise)Dynamic environment

*as reported by participants

12

Internal Strengths*System alignment with certain entitiesCollaborative approach to designing agendaAvailability of data elementsAbility to communicate and teach how to analyze and

use the data

*as reported by participants

13

Internal Weaknesses*Leadership and staff turnoverCapacityState and local “silos”Funding pressuresLinking research to instructionToo many areas to researchNo consequences/incentives for bad data

*as reported by participants

14

Potential Research Projects Considered: Strategy AStrategy A: College Awareness and Access

Financial aid study (fees, role of fin. aid, practices)Access study (Improve SEARS, GIS, time series study)Program evaluations (EOPS, Financial Aid, EAP)Distance education delivery model analysisClassification Study Other

Concurrent enrollment Noncredit to Credit Transition University of Phoenix phenomena

15

Potential Research Projects Considered: Strategy BStrategy B: Student Success and Readiness

Transfer study (disaggregate transfer population, explore/identify transfers)

Analysis of course factors (distance ed, learning communities, scheduling)

Course placement recommendation collection (tie to CB 21 revision)

16

Potential Research Projects Considered: Strategy CStrategy C: Partnerships for Economic and Workforce

DevelopmentData integration (matching supply with demand)Employment outcomes (longitudinal study)Curriculum development (improve response time)Partnership academiesCTE programs (measure costs and levels of success)

17

Potential Research Projects Considered: Strategy DStrategy D: System Effectiveness

ARCC—integrate equity/national peering/benchmarkingStudent learning objectives (impact on student success)Swirl studyStudy of professional development (what is being done and

how)

18

Potential Research Projects Considered: Strategy EStrategy E: Resource Development

Fee Policy (revenue vs. affordability; affect on student access and success; 50% law, distribution of funds, SB 361)

Cost indexProgram analysis (break even costs for certain programs)Develop inventory of existing and proposed studies

19

Potential Research Projects Considered: OthersEstablish common guidelines for conducting studiesPerformance-based fundingExpand research methods and reporting toolsSuper model for forecasting

20

Selection CriteriaIs the project “doable”? Do we have the data?Topic addresses multiple goals in the Strategic PlanWill the study’s output move the system forward?Will the study impact

Political leadershipStudentsEconomy/workforce

Can we use prior studies to leverage this study?How much new information will the study provide?

21

Prioritization ProcessWe categorized the projects into:

Quick wins (high impact, short-term timeframe, 0-12 months)

Stars (high impact, long-term timeframe (12-24 months)Building Blocks (low impact, short term timeframe, 0-12

months)Back Burners (Low impact, long-term timeframe, 12-24

months)Then, each participant voted for their choice of projects

22

Voting ProcessUsed the “facilitated decision making” method, also

known as the “10-4” method.Each participant received 10 dots. The participant placed 4 dots on the project that has the

highest priority for the individual; the remaining 6 dots are placed elsewhere.

23

Definition of Research ProjectsParticipants divided into three groups to discuss the

following for the 9 projects that received the most votes:ScopeObjectivesBenefitsAction Steps

24

And the winners are…Project 1: Course Section Factors

Objectives: Conduct program evaluations Create infrastructure Operationalize definitions and magnitude

Project 2: Course Placement RecommendationsObjectives:

Collect course placement recommendations and test scores

25

And the winners are…Project 3: Inventory of Existing Studies

Objective: Create warehouse of existing studies, including program evaluations, financial aid, internal and external studies

Project 4: GIS Data AnalysisObjective:

Enrollment management Program and service planning Bond planning analysis

26

And the winners are…Project 5: Employment Outcomes

Objectives: Classification of programs Expansion of outcome data (longer tracking)

Project 6: Evaluate CTE ProgramsObjectives:

Start with evaluation of Nursing programs Establish methodology for determining cost and performance

indicators

27

And the winners are…Project 7: Integrate equity data into ARCC

Objective: Make colleges more aware of equity issues

Project 8: Expansion of Student Attributes (SEARS Survey)Objective:

Conduct research using student attributes as related to success

28

And the winners are…Project 9: Fees, Financial Aid, and Affordability

Objective: Evaluate other states’ fees, revenue, financial aid, and

participation Information will inform state policy and budget discussions Optimize student access and success

29

Caveats and ConcernsAgenda is system level researchNeed literature reviewsDid not identify projects to build research capacityNeed to support classroom level researchProjects require technical assistanceMessage and marketing of studies is importantRigor and utility analysis in design

30

Retreat EvaluationWhat went well:

Representation of a wide variety of CC and government organizations

Process and facilitationOrganization of event

Changes for next time:Need small college representationNeed more rigorous guidelines and input about research

unit’s prioritiesNeed a mechanism to connect with external research groups

31

ApplicationsAllows us to make decisions about how the Chancellor’s

Office will allocate its scarce resourcesAchieve additional efficiency in research-related activitiesHelp external stakeholders identify projects to pursueClarification of unmet research needs

32

Limitations of the AgendaNarrow scopeStudies can be done with limited costsOmits studies that require

Collection of new dataUse of field experimentsLarge-scale studies

Extensive literature review not conducted; may have missed existing studies

Does not state how research capacity can be expanded

33

Future ActionWe will attempt to do studies as time/resources permitWe will work with external researchers to see if they can

partner with us to perform the studyWe will periodically revisit the agendaSpecific features of research agenda need refinement

34

Questions?Contact Willard Hom at (916) 327-5887 or

whom@cccco.edu

35