ICES III - Montréal, Canada Listening to Respondents for Better Results Alexander Hays Distributive...

ICES III - Montréal, Canada

Listening to Respondentsfor Better Results

Alexander HaysDistributive Trades DivisionStatistics Canada


Content

The Statistics ActSurveys

Inter-provincial Wholesale Commodity SurveyAnnual Retail Chain Survey

Response RatesBeginning the Collaborative ProcessAction PlanConsultative ResultsActions from the ConsultationRespondent ReactionContinuing ChallengesConclusion


The Statistics Act

Gives Statistics Canada the authority to collect the necessary information directly from respondents, and for business surveys, makes the reporting mandatory.

Grants Statistics Canada the authority to collect, compile, analyse, and publish information.


The Statistics Act

Considers refusal or the provision of false or misleading information a criminal offence subject to penalties.

Statistics Canada has hesitated in the past from taking legal action for non-compliance.


Statistics Canada has favoured a

collaborative approach working towards a viable

solution satisfactory to both parties.


Inter-provincial Wholesale Commodity Survey

Origin and destination by commodity survey.

Measures the value of inter-provincial trade by Canadian wholesalers.

Better understanding of provincial growth and measures of Gross Domestic Product (GDP) by province.

1990, 1996, 2001.

Preset list of 103 commodities.


Annual Retail Chain Survey

Operating and financial characteristics of domiciled retail corporate chain stores and department stores.

Better understanding of provincial economic growth and measures of GDP by province.

Retail Chain – 4 or more retail locations.

Location-based survey (individual chain stores)

Revenue and major expense items at the store location


DISCONNECT

Data Needs versus

Data Availability



Single: one province / territory, one NAICS industry

Complex: either one province / territory, multi NAICS or one NAICS, multi province / territory

Very Complex: multi province / territory and more than one NAICS industry

Year Complexity Response Rate

2001 Single 34.7

2001 Complex 40.0

2001 Very Complex 36.8



Response rates on revenue and expense variables at the store location level

YearOperating Revenue

Labour remuneration

Operating Expenses

Cost of Goods Sold

1999 57.2 36.7 38.1 38.1

2000 46.2 36.7 39.5 39.2

2001 58.0 40.0 45.6 44.3

2002 65.7 48.9 52.7 52.1

2003 55.7 43.7 46.7 47.5


Respondents willingness and ability to answerunderstand the question

understand the concept

understand the terminology used

appropriateness of the question

Question: Were the two surveys too onerous to complete?


Beginning the Process –Inter-provincial Wholesale Commodity

Survey

The 2001 survey was treated as a supplement to the Annual Wholesale Trade (Financial) Survey.

Did the supplement allow respondents the choice between completing 1 of the 2 aspects of the questionnaire under time or ease of data availability constraints?

Were the pre-set commodities not relevant to the wholesale operation in question?

Was origin and destination information not readily available from company books and records?


Beginning the Process – Annual Retail Chain Survey

Studied ways to reduce and simplify the questionnaire to increase response rates.Could model individual store level expenses and attain comparable estimates.The 3 expense variables at the store location level were removed for the 2004 reference year.Some of our clients expressed concerns.Decided to study further:

1. Study: Impact on provincial – territorial gross margins.2. Go to the respondent to find other opportunities to improvement the

questionnaire and to understand reporting concerns.

Question: How could we allay client concerns and improve response rates?


Collaborative Approach

Action Plan:Work with Questionnaire Design and Resource Centre

Mock up sample questionnaires that meet the data requirements of Statistics Canada

Test questionnaires on selected respondents


Action Plan

Inter-provincial Wholesale Commodity Survey27 one-on-one interviews.

Wholesalers operating in a variety of NAICS classifications.

Wholesalers operating from either one of Canada’s 3 largest cities.

Annual Retail Chain Survey23 one-on-one interviews.

Retailers operating in a variety of NAICS classifications.

Retailers with accounting functions operating out of Toronto or Montreal.


Consultative Results – Inter-provincial Wholesale Commodity

Survey

Origin of commodity more difficult than destination – wholesalers are sales driven.Origin – billing or shipping address?STC commodity list has no bearing on respondents product lines.Once a reporting mistake is made – tend to be repeated year after year.Reporting guides are not read.One-on-one personalized help is often needed, e.g., Enterprise Portfolio Management Program.


Consultative Results – Annual Retail Chain Survey

Due to the large number of store locations, a fully functional spreadsheet or relational database application would make it easier to complete the survey.

Respondents were happy that the 3 expense variables were removed.

Adding additional variables would require respondents to

seek help from inside their organization, e.g., systems


Actions from the Consultation


The 3 expense variables not re-added to questionnaire.

The electronic version of the questionnaire was made more user-friendly.

Inter-provincial Wholesale Commodity SurveyFully integrate the questionnaire into the Annual Wholesale Survey.

3-tier collection strategy based on company size.Mail-out / Mail-back

Telephone Assistance

On-site collection

Eliminate the commodity dimension

Conduct a pilot project due to unproven collection strategy.


Respondent Reaction


The simple reduction of the 3 expense variables turned-around many non-compliant respondents.

Improved Response Rates

Inter-provincial Wholesale Commodity SurveyPilot project, although still on-going, is showing a 60% return with some level of origin & destination.

So far 59% of most complex companies responding to origin & destination.

Telephone assistance in completing the questionnaire is helping to convert non or partial response.

Year Operating Revenue1999 57.22000 46.22001 58.02002 65.72003 56.7

2004 78.62005 74.6


Continuing Challenges


Obtaining information still requires a lot of time and effort.

Acknowledgement of need for personalized assistance, but no enterprise has agreed to on-site collection by a Statistics Canada representative.


Though response rates improved greatly, and the study on the Impact on Provincial Gross Margins showed little degradation in store level estimates, work continues in understanding the financial differences and the decision making process at the store level.

Trade-off between

Cost

Quality Burden


Conclusion

Respondents:Fewer questions = less burden

Questionnaires structured to respondent data sources.

Better understanding by Statistics Canada on respondents concerns.

Statistics CanadaBetter understanding by the respondent on the need for the data.

Better understanding of record keeping and reporting capability.

Better response rates.

Information collection will always remain a contentious issue.

Respondent’s perspective – surveys are time consuming and can be conceptually impractical.

Listening to respondents can be a win-win situation.


Listening to Respondentsfor Better Results

Questions

ICES III - Montréal, Canada Listening to Respondents for Better Results Alexander Hays Distributive...

Documents

Transcript of ICES III - Montréal, Canada Listening to Respondents for Better Results Alexander Hays Distributive...