MM-II (PGP-1: 2013-2015)

Indian Institute of Management Lucknow

Prem Prakash Dewani (Ph. D. , IIM Ahmedabad)

Assistant Prof. IIM-Lucknow

Survey ResearchCase: The Harvard Graduate Student Housing Survey

Mode: Class Discussion & Presentation

Class Notes: by Prem Prakash Dewani

Objectives of the Case

• Identifying potential contributions of surveys and marketing research in organizations.

• Learning in context the steps necessary to develop, design, write, and implement a successful questionnaire, and effective ways to analyse and present the resulting data.

• Gaining an appreciation of contemporary challenges associated with survey research. In particular, how to structure and write engaging self-administered questionnaires, how to leverage the Internet, how to create surveys that contribute to enhancing and maintaining customer relationships.

• Learning specific technical aspects related to the measurement of critical parameters such as price sensitivity and attribute importance.

• Improving the skill of writing good survey questions.

Go through the survey questionnaire in Exhibit 1 as a respondent. Reflect on your experience:• What are your feelings, thoughts (e.g., what images come to your mind?) and

emotions as you go along? • What makes you want to stop, what drives you to continue? • What is easy to answer, what is difficult? • What are the merits of this questionnaire? • What are apparent shortcomings?

• Reflect on your own experience with housing as a graduate student. • Is there an important aspect of experience that was not captured by

the 2001 survey? • Write a question (or a block of questions) that should be added to the

survey to capture that aspect in a manner that might impact decision making. Please don’t forget to specify the format of the answer to your question (e.g., a five point scale). • Predict the average answer(s) to your question(s) and the impact on

actual decision making it might have. • Try to achieve professionalism in your wordings.

• When looking back at the 2001 survey, what news did it produce, what impact did it have?• Can you attribute this impact to specific features of the survey or of

the survey design process and circumstances?• What should be kept/removed in the 2005 survey? What could the

survey do to contribute to the Allston initiative in a useful way?• If not a survey, what else would you recommend to understand the

customer in a way that would inform long range planning in Allston (you might start by focusing on the three issues on p. 10 of the case, and then think more broadly).

“Not worth $50,000”?

• Survey is too long and boring.• Too many factual questions. They should be retrieved from somewhere else

using the Student ID number.• Self-selection bias: people who elect to respond the survey are positively pre-

disposed towards Harvard real-estate services.• Some questions were hard to answer (trade-off questions in section F)• The survey makes simplistic assumptions about what drives satisfaction (e.g.,

location and features, instead of more sophisticated human factors).• It is not open-ended and fails to create a space where consumers could voice

their aspirations.• Will prove useless to address the organizational goal of defeating the private

market.

“yes it’s worth $50,000”• Survey is thorough.• Survey focuses on straightforward data that are reliable even though they are self-

reported.• Careful process integrating results with focus groups for better understanding.• Experienced as a rewarding journey through my housing experience.• Very large number of responses.• Segmentation/cross-tabulation of results by graduate school interesting and useful

for action.• Replicable over the years for monitoring.• A good signal that the administration cares.• Harvard now has at least some numbers where there were none before.• Simple but useful insights into what students value most (e.g., apartment size

secondary as compared to location and price).

What Makes a Survey Successful?

• Statistical concerns:1. Presence of biases2. Response rate

• Experience of the respondent:3. Realism (vocabulary used, attributes evoked)4. Structure (presence of an introduction, structure that matches the

chronology of the consumer’s experience, location of factual questions, etc.)5. Incentives to complete the survey6. Likely impact on the satisfaction of the respondent

• Decision value for the manager:7. Opportunities to uncover implicit structures through

correlation analyses (in this case: correlations between graduate school belonging and housing preferences).

8. Production of simple surprising (“stunning”) statistics9. Connectedness with manager’s goals.10. Ability to compare and monitor data over time

• Compatibility between the researcher and the respondent’s language and ways of thinking.• Perception of reward attached to responding to the survey: upfront

incentives, thanking, making the questionnaire interesting with a storyline (self-examination value).• Reduce participation costs: contain length, allow for self-administration

at leisure and at home.• Trust: don’t ask questions that might beg the response • “why do you ask?”, • stay on topic, • promise to share results. • Don’t ask questions that clearly are not genuinely intended to inform you.• Have the survey sponsored by a legitimate authority

How to Write Good Questions?

• If you currently live in Harvard housing, do you think that coordinating living arrangements with a privately-owned housing renter would be more or less difficult than with HRES?

[more/less]

• If you currently live in Harvard housing, what has been the single-most frustrating aspect of your housing experience thus far?

[open-ended; to appear close to the beginning of the survey]

• How do you rate the level of interactivity and active social/academic life of students within your current housing?

[scale of 1-5]

• How do you think the design and lay-out of your current housing contribute to this answer?

[1-5]

• I would be in favour of designating a portion of on-campus housing as guaranteed on-campus housing for first-year graduate students. This means that students in these apartments would have to move out at the end of the year.

[scale of 1-5]

•What changes/improvements would you like to see made to the HRES lottery process?

[open-ended]

•Overall, how would you rate your housing experience at Harvard?

[significantly below expectations - below expectations - meet expectations -exceeds expectations - exceeds expectations substantially]

•Please rate your overall experience with this survey.[choose one on scale: extremely negative – negative – neutral – positive -

extremely positive]

• Rank the following attributes in order of providing you a sense of"community": Diversity of grad school representation, shops nearby,athletic/rec facilities, community space, comfort/modernity of apartments, safety.

[multiple choice, ranking]

What Are the Limits of Survey Research?