1

Lecture 10

Observation: chapter 6

Test marketing: chapter 7

2

What is observation?

• Observation is the systematic process of recording the behavioural patterns of people, objects, and occurrences as they are witnessed.– No questioning or communicating with

people– Example: The Australian Women’s Weekly.

3

When is observation scientific?

• Observation becomes a tool for scientific inquiry when it:

– Serves a formulated research purpose

– Is planned systematically

– Is recorded systematically and related to general propositions rather than simply reflecting a set of interesting curiosities.

– Is subjected to checks or controls on validity and reliability.

4

What can be observed?

5

What can be observed?

• Used to describe a wide variety of behaviour.• Attitudes, motivations, and preferences

cannot be observed.• Observation is also generally of short

duration.

6

The nature of observation studies

• Human observation versus mechanical observation.

• Unobtrusive

• Visible observation versus hidden observation.

• Data do not have distortions, inaccuracies, or other response biases.

7

Observation of human behaviour

• Toy manufacturers use observation because children cannot express their reactions to products.– How long does the child’s attention stay with the

product?– How long until the child puts the toy down?– Are the child’s peers equally interested in the

toy?• Observation of nonverbal behaviour.

8

Observation of human behaviour

9

Direct observation

• A straightforward attempt to observe and record what naturally occurs.

• Investigator does not create an artificial situation.

• An observation form keeps observations consistent.

• Response latency can be observed– The amount of time it takes to make a

choice between two alternatives.

10

Errors associated with direct observation

• Not error–free because the observer may record events subjectively.

• Observer bias: a distortion of measurement resulting from the cognitive behaviour or actions of the witnessing observer.

• Accuracy may suffer if observer does not record every detail.

• Interpretation of observation data is another major source of error.

11

Scientifically contrived observation

• Contrived observation: investigator intervenes to create an artificial environment in order to test a hypothesis.

• Contrived situations reduce the research time spent waiting and observing a situation.

• Mystery shoppers.

12

Ethical issues in the observation of humans

• Hidden observations and respondent’s right to privacy

• Contrived observation and deception

– Entrapment

• Requires a balance: if researcher obtains permission to observe, the subject may not act in a typical manner.

13

Observation of physical objects

• Physical–trace evidence is a visible mark of some past event or occurrence. – For example, wear on library books to

determine books most read, erosion traces on floor tiles of museums to determine most popular exhibits, Campbell’s soup cans in garbage.

• This method can be accurate.

14

Content analysis• Content analysis is the systematic observation and

quantitative description of the manifest content of communication.

• Content or messages of advertisements, newspaper articles, television programs etc.,– For example, frequency of appearance of

women or minorities in mass media, whether advertisers use certain themes or appeals more than others.

15

Mechanical observation• Means of observation is mechanical

– For example, video cameras, traffic counters

• OzTAM television monitoring system for estimating national TV audiences.– Electronic boxes hooked up to television

sets to capture program choices, length of viewing time, and identity of viewer.

• Monitoring website traffic• Scanner–based research.

16

Measuring physiological reactions

• Four major categories of devices to measure physiological reactions:– Eye–tracking monitors: used to observe eye

movements– Pupilometers: used to observe and record

changes in the diameter of a subject’s pupils– Psychogalvanometer: used to measure

galvanic skin response– Voice pitch analysis: records abnormal

frequencies in the voice.

17

Chapter 7Experimental research and test marketing

18

The nature of experiments

• Conditions are controlled so that one or more independent variables can be manipulated to test a hypothesis about a dependent variable.

• Causal relationship among variables may be evaluated while eliminating all other variables.– For example, influence of brand name

identification and labels on consumers’ taste perception.

19

Issues using an experimental design

• Laboratory versus field experiments• Threats to internal and external validity that

can influence results• How can these threats be controlled?• Choice of experimental design.

20

Field and laboratory experiments

• Experiments differ in the degree of control over the research situation.

• Experimenter either creates an artificial situation that limits the influence of outside factors or deliberately manipulates a real–life situation, which allows the influence of outside factors but makes it harder to determine cause–and–effect relationships.

21

Field and laboratory experiments

• Laboratory experiment is conducted in a setting where the researcher has almost complete control.– Viewing television commercial then

allowing viewer to purchase in a simulated store environment.

– Mobile shopping van– Tachistoscope: controls the amount of time

a subject is exposed to a visual image.

22

Field and laboratory experiments

• Field experiments are conducted in a natural setting where complete control of extraneous variables is not possible.– Test markets

• Controlled store test is a hybrid between a laboratory experiment and test market.– Test products are sold in selected stores to

actual customers.

23

Basic issues in experimental design

• Manipulation of the independent variable.• Selection and measurement of the dependent

variable.• Selection and assignment of subjects.• Control over extraneous variables.

24

Manipulation of the independent variable

• Independent variable can be manipulated, changed or altered independently of any other variable.

• Hypothesised to have the causal influence.• Experimental treatments.

25

Manipulation of the independent variable

• Example of variations of advertising copy and graphic designs in marketing experiments.

26

Manipulation of the independent variable

• Experimental group: group of subjects exposed to the experimental treatment.

• Control group: group of subjects not exposed to the experimental treatment.– The two treatment groups are then compared to

determine any causal effects.• There can be several experimental treatment levels.• There can be more than one independent variable.

27

Selection and measurement of the dependent variable

• The value of a dependent variable is expected to be dependent on the experimenter’s manipulation of the independent variable.

• Selection of dependent variable is crucial.– Determines what type of answer is given to

the research question.

28

Selection and assignment of test units

• Test units are the subjects or entities whose responses to experimental treatments are observed or measured.

• Sample selection error• Random sampling error

– Repetitions of the basic experiment sometimes favour one experimental condition and sometimes the other on a chance basis.

• Randomisation and matching.

29

Control over extraneous variables

• Experimenters may strive for constancy of conditions.• Error due to order of presentation.• Blinding is used to control subjects’ knowledge of

whether or not they have been given an experimental treatment.

• Constant experimental error occurs when extraneous variables are allowed to influence the dependent variable every time the experiment is repeated.

30

Issues of experimental validity

• Internal validity refers to whether the experimental treatment was the sole cause of observed changes in the dependent variable.– If the observed results were influenced or

confounded by extraneous factors, then the experiment is not internally valid.

31

Issues of experimental validity

• External validity is the ability of an experiment to generalise beyond the experiment data to other subjects or groups in the population under study.– If the experimental situation is artificial and

does not reflect the true setting and conditions in which the investigated behaviour takes place, then the experiment is not externally valid.

32

Threats to internal validity• History: history effect and cohort effect

– For example, competitors change their marketing strategies during a test marketing experiment, two subject groups with different histories.

• Maturation: maturation effects, guinea pig effect, and Hawthorne effect– For example, day–long experimental subjects may

grow hungry, tired, or bored, thus changing the result of experiment.

– Example: subject changes behaviour in the presence of experimenter.

33

Threats to internal validity• Testing: testing effects

– For example, students exposed to the experiment the first time, may react differently the second time.

• Instrumentation: instrumentation effect– Example: change in wording of questions may

cause a change in the results of experiment.• Selection: selection effect

– Sampling bias that results from differential selection of respondents for comparison groups.

34

Threats to internal validity

• Mortality: mortality effect– Sample bias that results from the

withdrawal of some subjects from the experiment before it is completed.

• Demand characteristics– Experimental design procedures that

unintentionally suggest to subjects about the experimenter’s hypothesis.

35

Threats to external validity• Student surrogates: use of university students

as experimental subjects.– Students do not provide accurate

predictions of other populations.• Extraneous variables may have an impact on

the dependent variable, thereby distorting the experiment.– Not always possible to control everything in

marketing experiments.

36

Types of experimental designs• Experimental for one independent variable or

outcome or factorial to consider a number of causal factors– Basic experimental designs with one independent

variable and one dependent variable.– Factorial experimental designs with two or more

independent variables.• Repeated measures or not

– Subjects exposed to all experimental treatments.

37

Basic experimental designs• A single independent variable is manipulated to

measure its effect on another single dependent variable.– Complex or statistical experimental design for two

or more independent variables.• Symbolism for diagramming experimental designs:

– X: Exposure of a group to experimental treatment.– O: Observation of dependent variable.– R: Random assignment of test units.

38

Three examples of quasi–experimental designs

• One shot design: single measure is recorded after treatment is administered.

• One–group pretest–posttest design: experimental group is measured before and after treatment is administered.

• Static group design: after–only design measuring group exposed to experimental treatment and control group without exposure to treatment.

39

Three better experimental designs

• First step of true experimental design is randomisation of subject assignment.– Pretest–posttest control group design: both

experimental and control groups are measured before and after treatment administered on experimental group.

– Posttest–only control group design: after–only design measuring both experimental and control groups.

– Solomon four–group design: combines both experimental designs.

40

Time series designs

• Experiments are conducted over long periods of time to distinguish temporary and permanent changes in dependent variables.– Example: political polls tracking candidates’

popularity.

41

Complex experimental designs

• Isolate the effects of confounding extraneous variables

• Allow for manipulation of more than one independent variable.– Completely randomised design,

randomised block design, factorial design, Latin square design.

42

Completely randomised design

• Uses a random process to assign subjects to treatments to investigate the effects of only one independent variable.

43

Randomised block design

• Identifies and blocks out effects of a single extraneous variable that might affect the response of the test units.

44

Factorial designs

• Investigates the interaction of two or more independent variables on a single dependent variable.

• Main effect: the influence of a single independent variable on a dependent variable.

• Interaction effect: the influence on a dependent variable of combinations of two or more independent variables.

45

Factorial designs

• The number of treatments and the number of levels of each treatment identify the factorial design.– Example: two magazine ads tested on men

and women: 2X2

46

Factorial designs

47

Latin square design

• Balanced, two–way classification scheme that attempts to control or block out the effect of two or more extraneous factors by restricting randomisation with respect to the row and column effects.

48

Test marketing• Scientific testing and controlled experimental

procedure that provides an opportunity to measure sales or profit potential for a new product.

• Test a new marketing plan under realistic marketing conditions.– Offers the opportunity to estimate the outcomes of

alternative courses of action.– Allows management to identify and correct

weaknesses in either the product or its marketing plan before a national launch.

49

Test marketing

• An expensive research procedure.– Developing local distribution, arranging

media coverage, monitoring sales results• Many uncertainties and risks even with test

marketing.• The firm runs the risk of exposing a new

product or its plans to competitors.• Warranted only if it will save the company

money in the long run.

50

How long should a test market last?

• New product volume likely to peak out in 3 to 4 months, suggesting a number of people try new products, but many do not repeat their purchases.– Test markets should be long enough for

consumers to become aware of the product and to try it more than once.

– A test market that is too short may over–estimate sales.

51

Factors to consider in test market selection

• Population size• Demographic composition and lifestyle

considerations• Competitive situation• Media coverage and efficiency• Media isolation• Self–contained trading area• Over–used test markets• Availability of scanner data.

52

Estimating sales volume: some problems

• Over–attention• Unrealistic store conditions• Reading the competitive environment

incorrectly• Incorrect volume forecasts• Time lapse.

53

Projecting test market results

• Consumer surveys– Measure levels of change in consumer awareness

and attitudes toward the product.• Straight trend projections• Ratio of test products sales to total company sales• Market penetration X repeat–purchase rate

– Repeat–purchase rate obtained from longitudinal research.

54

Standard method versus control method of test marketing

• Standard method of test marketing has considerable external validity.

• Control method of test marketing involves a ‘mini–market test’ using forced distribution.– Retailers are paid for shelf space so that

the test marketer can be guaranteed distribution.

55

Standard method versus control method of test marketing

• Electronic test markets measures results based on scanner data combined with broadcasting systems to experiment with different ad messages.

• Simulated test markets are research laboratories in which the traditional shopping process is compressed.– Virtual–reality simulated test market attempts to

reproduce actual store atmosphere with visually appealing images on computer screen.