8/8/2019 Graph the 4

1/7

Paired Comparisons and Graph TheoryAuthor(s): Jerome B. KernanSource: Journal of Marketing Research, Vol. 4, No. 1 (Feb., 1967), pp. 67-72Published by: American Marketing AssociationStable URL: http://www.jstor.org/stable/3150167

Accessed: 15/09/2010 22:41

Your use of the JSTOR archive indicates your acceptance of JSTOR's Terms and Conditions of Use, available at

http://www.jstor.org/page/info/about/policies/terms.jsp. JSTOR's Terms and Conditions of Use provides, in part, that unless

you have obtained prior permission, you may not download an entire issue of a journal or multiple copies of articles, and you

may use content in the JSTOR archive only for your personal, non-commercial use.

Please contact the publisher regarding any further use of this work. Publisher contact information may be obtained at

http://www.jstor.org/action/showPublisher?publisherCode=ama.

Each copy of any part of a JSTOR transmission must contain the same copyright notice that appears on the screen or printed

page of such transmission.

JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of

content in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new forms

of scholarship. For more information about JSTOR, please contact support@jstor.org.

American Marketing Association is collaborating with JSTOR to digitize, preserve and extend access to

Journal of Marketing Research.

http://www.jstor.org/action/showPublisher?publisherCode=amahttp://www.jstor.org/stable/3150167?origin=JSTOR-pdfhttp://www.jstor.org/page/info/about/policies/terms.jsphttp://www.jstor.org/action/showPublisher?publisherCode=amahttp://www.jstor.org/action/showPublisher?publisherCode=amahttp://www.jstor.org/page/info/about/policies/terms.jsphttp://www.jstor.org/stable/3150167?origin=JSTOR-pdfhttp://www.jstor.org/action/showPublisher?publisherCode=ama

8/8/2019 Graph the 4

2/7

JEROME.KERNAN*

This article is a technical note on a problem frequently encountered in assessingconsumerpreferences. Intransitivitys seen insituationswhere Product a is preferredto Productb, b to c, but c to a. Inadequate preference transitivity casts suspicionon rank orders of products derived from series of paired comparisons. Thisproblem can be given graph theoretic treatment which, although it offers noabsolute solution, does make the problem clearer and emphasizes the area of the

analyst's judgment.

P a i r e d Comparisons n d G r a p h T h e o r y

Even the slowest marketing student is impressed withthe importance f discoveringonsumer astes andpreferences. he question s not whether,but how.Frequently,how one finds out about consumerprefer-ences determines what is discovered. Certain consumerattitudes may be impregnable to all but clinical ap-proaches.THE CONTROVERSY OVER PAIREDCOMPARISON TESTS

A frequent problem for the marketer is to find outhow consumers rank order a set of products or varia-tions in an attribute of a product. It is useful to know,for example, whether consumers rank brands on anabc or bac basis, or what amount of sweetness theyprefer in a soft drink. To determine such information,one of two methods might be used. When the number ofitems (brands, attribute variations, etc.) is small, con-sumers might be asked simply to rank order all itemsas a set. As the number of items increases, however, thisdirect method of ranking is doubtful on the theorythat, except at the extremes, preferences are not ade-quately discernible. An indirect ranking method thusbecomes necessary, one that requires a choice for eachpaired combination represented by the set of items. Thefamiliar technique of paired comparisons is such amethod. To effect a rank ordering of p items requiresthat a respondent make p(p - 1)/2 choices. Ranksare assigned to items in inverse relation to the numberof choices in their favor; the most frequently preferreditem is ranked first, etc. Since this method requires thatexplicit and equal consideration be given to every itemin the set (each item is considered p - 1 times), dif-

ferences in items are likely to be considerably morediscernible than in the direct ranking method.Although the method of paired comparisons offerssome obvious advantages over a direct ranking method,its superiority is far from unqualified. First of all, it ismore difficult because it is an indirect method. Next,serious questions have been raised about it [3]. Forinstance, consider this method from the standpoint ofvalidity and reliability.Validity

A research method is valid if it measures what itintends to measure. Perhaps, the severest test of validityis predictability. If a paired comparison technique givesa particularranking of brands, then that ranking shouldbe the same in the marketplace. As Blankenship [3]has argued, this technique can seldom satisfy such acriterion. Whether this implies that the technique shouldbe dropped would seem to be a matter of judgment.If not paired comparisons, what? In any event it issomewhat naive not to question the validity of themethod.Reliability as Transitivity

A research method is reliable if, under repeatedmeasurements, the measures are consistent, allowingfor random errors. There are several interpretations ofreliability; for here it means transitivity. The reasonfor this interpretation will be clarified.A transitive relation is: if a is preferred to b, and bis preferred to c, then a is preferred to c. That relia-bility is a legitimate interpretation of transitivity maybe seen as follows. As is shown later, if a consumerindicates a preference for a over b and a preference forb over c, she is implying a preference for a over c. If

* Jerome B. Kernan is associate professor of marketingad-ministration,GraduateSchool of Business,University of Texas.

Journal of Marketing Research,Vol. IV (February1967), 67-72

67

8/8/2019 Graph the 4

3/7

68 JOURNALOF MARKETINGESEARCH,EBRUARY967in actual comparison she chooses a in the set ab, b inthe set bc, and a in the set ac, there exists a reliablebasis for constructing an abc ranking for her. If sheshould choose a in the set ab, and b in the set bc, but cin the set ac, however, there is not a reliable basis forconstructing a ranking for her. Each item has beenchosen once. It might be concluded on this basis thatshe has no preference; yet on three occasions she hasmade choices indicating preference. As is shown in alater section, the intransitivity of her preferences isillogical.Since in the practical case the rankings yielded bypaired comparison tests are summaryranks, reliabilityalso involves agreement among respondent rankings.The customary procedure in reporting paired compari-

Figure1GRAPHTHEORETICEVELOPMENTF IMPLICITRELATIONSN A TRANSITIVEREFERENCE

SET,a < b < c < d < eMATRIX GRAPH

ab cd ea 0 1 0 0 0]b 0 0 1 0 0e bc0 0 0 10 =Md 0 0 0 0 1:e 0 0 0 0 0) d cabcde a

a '0 0 1 0 00010c00001=M2 e beo000o0 0 0 0 0) d c

aabcdea 0 0 0 1 0'b 0 0 0 0 1 e bc 0 0 00 0 = M,d 0 0 0 0 0)e 00000

abcdec 0 000 0 = M4d 0 0 0 0 0e 0 0 0 0 0

abcdeb 0 0 1 1 1 e bc 0 0 1 1 M+ M2d 0 0 0 0 1 M3 + M4e 0 0 0 00 d

son findings is to cite what percentage or number ofrespondents preferred a over b, b over c, etc. Thisinformation does not disclose the extent to which therankings of respondent No. 1 agree with those of No.2, with No. m. The findings typically reported areconglomerates-how, in general, respondents rankorder items. It is quite possible to have each respondentreflect perfectly transitivepreferences and yet have littleor no agreement among respondent rankings or, con-versely, to have perfect agreement among respondentrankings and have them be completely intransitive.In either case (but for different reasons), the reliabilityof the measuring instrument is doubtful. In a later partthese questions of reliabilityare covered in graph theory.

BASIC CONCEPTS OF GRAPH THEORYGraph theory, a branch of abstract mathematics,is a mixture of topology and combinatorial analysis.Its essential concern is with structural relationships.

Among its many properties [2, 10, 15], certain areessential to a treatment of the preference relation prob-lem.'A graph is a finite collection of points, some or allof which are connected by lines. If every pair of pointsis connected by a line, the graph is called complete.The points of a graph may represent product brands,product attributes, etc. The lines in a graph representrelationships among the points they connect. A graphmay be represented either in pictorial or matrix form.In matrix form, the binary 1 in the ijth cell means arelation between point i and point j. A zero means norelation.Digraphs and Their MatricesA digraph(for directed graph) is one whose relations(lines) are asymmetric. In a digraph consisting of Pointsa, b, c, the line ab means a relation from a to b andnot vice versa. In matrix form, of course, digraphs neednot be symmetric, whereas ordinary (undirected)graphs must be. Directed relations are usually indicatedby arrows on the graph, and by the binary 1 in the ijthcell of a matrix to indicate a relation from the ith rowto the jth column.Tournaments:A Special Case of Digraphs

A tournamentis a restricted class of digraph that iscomplete and asymmetric. It is apparent that round-robin competitions are tournaments, because eachteam must play every other team (complete), and theresult of each contest is one team winning (asymmetric).Ultimately, one team wins over the other teams andthey, in turn, can be assigned finishing ranks, secondthrough last. A rank order of brands, taken from a setof paired comparison tests (ties not allowed), is analo-gous. The paired comparison method, therefore, may

1For other illustrativeapplications,see [2, 7, 9, 13, 16, and17].

8/8/2019 Graph the 4

4/7

PAIRED OMPARISONS ND GRAPHTHEORY 69be interpreted in the graph theoretic concept of atournament.

A GRAPH THEORETIC INTERPRETATIONOF TRANSITIVITYIn a preceding section it was suggested that intransi-tive preference sets are unreliable preference sets. Thiscan now be demonstrated using graph theory.A ranking of items such as a < b < c < d < eimplies more relations that it makes explicit. In thisranking, in which the operator < means "is preferredto," the relations ab (a is preferred to b), bc, cd, andde are explicit. Implied, however, are ac, bd, ce, ad, be,and ae. When the implied relationships do not holdtrue empirically, intransitivity and its associated un-reliability face the researcher.In both graph and matrix form, Figure 1 shows allthe implications of the a < b < c < d < e ranking.The matrix M and its associated graph show the explicit

relations. M2, the square of M, shows the second order,implicit relations, ac, bd, and ce. M3 shows the thirdorder ones, ad and be; and M4, the fourth order, ae.Since the powers Mk reflect the number of relations(lines) of length k in the set, all implied relations canbe made explicit in at most p - 1 powers of M, givena set of p items. In this case, for example, M5 containsonly zeros, indicating that M4 has been enough toexplain all relations. The matrix [M + M2+ M3 + M4]and its digraph contain all the relations-explicit andimplicit-in the set. The digraph is complete and asym-metric, thus it is a tournament. This is to be expectedsince it shows the comparisons needed in a pairedcomparison test.The preference set in Figure 1 is transitive, and therelations it implies are not surprising. If an intransitiveset is considered-even the simplest possible kind-theimplied relations are rather disturbing. Figure 2 reflectsthe intransitive preference set a < b, b < c, c < a.The same kind of matrix operations as in Figure 1 showthat the result is an absurdity. In effect everything ispreferred to everything. (Technically, one should stopwith M2, since M3 = MP; M3 is shown only to impressthe readerwith the logical implications of intransitivity.)Neither [M + M2 + M3] nor [M + M2] qualifies as atournament because neither is asymmetric. This comesfrom the fact that by implication we have contradictorypreferences indicated-for example, ab and ba. Thesignificance of the situation in Figure 2 is what wasasserted: that intransitive preference sets are unreliablebases for constructing rank orders of items.Transitiveand Cyclic Triples

What already may have been inferred from Figures1 and 2 can now be stated. Intransitivepreference setscontain at least one cyclic triple. By cyclic triple is meantthree points in a tournament that exist in a pi < pi

8/8/2019 Graph the 4

5/7

70 JOURNALOF MARKETINGESEARCH,EBRUARY967Figure 2

GRAPHTHEORETICEVELOPMENTF IMPLICITRELATIONSN AN INTRANSITIVEPREFERENCEET,a < b;b < c; c < a

MATRIX GRAPHaabc

a '016c 00

aa

abcb 1 0 0 M2

c- b

a b cb 0 1 0 = M3

b 1 1 1=M+ MM M3c 1 1 1 c b

of p points, )max is the maximum possible number ofsuch triples.Zeta equals one when there are no cyclic triples inthe tournament and zero when there is the maximumpossible number.The coefficient of consistency may be illustrated byFigure 4. The tournament is shown in both its matrixand graphic forms. First, the total number of triplesT must be determined. For this five-point tournament,that is given by [10, p. 296]

T p(p- 1)(p- 2) 10.6Next, using the scores si associated with each point,the number of transitive triples 3 in the tournament isfound to be [10, p. 299]

S s(s - 1) 9.P 2

Since triples are either transitive or cyclic, the numberof cyclic triples 4 in the tournament is given by4= T - 3 = 1.

The maximum number of cyclic triples 4max, sincep is odd, is given by33P - _5.max - 24 5.

For the preferenceset in Figure 4, then the coefficientof consistency isI= 1 -.80.

!)maxZeta can be tested for significance,the null hypothesisbeing that a subject's preferences are assigned at ran-dom. Kendall [11, pp. 192-3] provides appropriatetables for p < 10. Otherwise, zeta has been shown [11]to follow the chi-square distribution. The observedvalue is given by

p-1 1 4 *3!p- 3)!The number of degrees of freedom, df, is

dj= p(p - 1)(p - 2)(p - 4)2The zeta value .80 in this illustration is significantat the .117 level, using Kendall's tables.It is readily apparent by an examination of the"score" column of the matrix that an element of in-

transitivity exists in the tournament depicted in Figure4. In a transitive set this would reflect a descendingorder-4-3-2-1-0. In the graph, scores are reflected bythe outdegrees of the points, that is, the number of linescoming from them. The coefficient of consistency ismerely a summary measure of this.An interesting characteristic of preference con-sistency measures should be noted. It has been shownthat the expected proportion of cyclic triples to totaltriples in any tournament is .25.4 It does not matterwhether the cyclic triples are occasioned by chance orby a respondent attempting to be inconsistent. Theextent to which a respondent can purposely distorther preferences, is (on an expected basis) no greaterthan what chance alone can produce. Hostile responsesor random responses then collectively produce the sameexpected proportion of cyclic triples. This suggeststhat attempting to distinguish between the two may beunnecessary.If the expected proportion of cyclic triples to totaltriples in a tournament is .25, the expected proportionof transitive triples is .75. This being so, perhaps a moreconservative view of empirically observed consistency

3 If p is even, ,max = (p - 4p)/24; [10, p. 300].' See [10, pp. 302-4] for a proof.

8/8/2019 Graph the 4

6/7

PAIRED OMPARISONS ND GRAPHTHEORY 71Figure3a

DETECTIONF TRANSITIVITYHROUGHTERATIVEREDUCTIONRANSITIVEASE:a < b 6 andp > 4, u can be testedfor significanceby means of x2against the null hypothesis that subjects allot theirpreferences t random.sTo test u, it is necessary o computean observedx2valueby

2 4 1 p m m -3rS m -2 \2/\2 m - 2'with degreesof freedomequalto(m 2 ) 2

In the present case, x2 = 8398 and df = 10. Thus,u = .834 is significantbeyondthe .001 level.To summarize, wo measureshave been illustrated.One of these, the coefficientof consistency,reflectsthe extent to which an individualrespondent s con-sistentin statedpreferences.The other, the coefficientof agreement, eflects he extentto which there s agree-ment among respondents n stated preferences.Thislast measure reveals nothing about individuals,onlysomethingamong them. Respondentscan agree per-fectlyon their inconsistencies.IMPLICATIONS

Theabovediscussionwouldseemto have mplicationsforboth those whouse thepaired-comparisonechniqueandthosewho mustmake decisionsbasedon its results.It has been suggested hat rank ordersthat come frompairedcomparisonsmply,butare not alwayssupportedby, transitivepreferencesets. Since intransitivitycanmake rank orders illogical, it should not be takenlightly.Graphtheorywas used to illustrate ransitivity,andtwo summarymeasureswere discussed.One of these,the coefficientof consistency,reflectsthe reliability obe associatedwith an individual'sstated preferences.The other, the coefficient of agreement, reflects thereliability (in the sense of variability) associated with agroup's stated preferences.The discussion did not cover what is perhaps a moreserious question-validity. What do paired comparisontests actually measure? To judge them uniformly for

reliabilitycould be misleading. Apparent inconsist-encies, either in or among individualpreferencesets,can result from quite plausiblefactors that are essen-tially of a valid nature.As Balinsky,Blum and Dutka[1] have noted, an apparent nconsistencymay comefrom the multivariatenature of the item being con-sidered. Since the ranking s assumedto be on a uni-variatebasis, it is tempting o conclude that reliabilityis doubtful.In fact, the subject may be using differentevaluationcriteria or the severalpaired comparisons.It is also possible to encounterwhat Gerard andShapiro [8] call a "separation."This is whereprefer-ences are found to be, say, bimodal, in spite of aunivariatelydistributed item characteristic. For ex-ample, consumersmay prefer eitherno sweetnessorheavy sweetnessto moderatesweetnessin a product.Althoughthis may appear nconsistent,any student ofmarketings awareof its common occurrence.

REFERENCES1. Benjamin alinsky, t al., "TheCoefficientf AgreementnDeterminingProductPreferences," ournal of AppliedPsychology, 5 (October 951),348-51.2. ClaudeBerge,TheTheoryof Graphs nd Its Applications,NewYork:JohnWiley&Sons,Inc.,1962.3. A. B. Blankenship,Let'sBuryPairedComparisons,"our-nalof Advertising esearch, (March 966),13-7.4. H. A. David, The Methodof PairedComparisons, ewYork:HafnerPublishing ompany, 963.5. RalphL. Day, "Systematic airedComparisonsn Pref-erenceAnalysis,"ournal f Marketing esearch, (Novem-ber1965),406-12.6. Allen L. Edwards,Techniques f AttitudeScale Construc-tion,NewYork:Appleton-Century-Crofts,nc.,1957.7. ClaudeFlament,Applications f GraphTheoryto GroupStructure, nglewoodCliffs,N. J.:Prentice-Hall,nc.,1963.8. H. B. Gerard nd H. N. Shapiro,"Determininghe Degreeof Inconsistencyn a Set of PairedComparisons,"sycho-metrika, 3 (March 958),33-46.9. FrankHararyand BenjaminLipstein,"TheDynamicsofBrandLoyalty:A MarkovianApproach,"OperationsRe-search,10(January-February962),19-40.10. - , et al., StructuralModels: An Introduction to theTheoryof DirectedGraphs,New York:John Wiley &Sons, nc.,1965.11. M. G. Kendall,RankCorrelationMethods,3rd ed. Lon-don:CharlesGriffin&Co.,Ltd.,1962.12. - and B. Babington mith,"On he Method f PairedComparisons,"iometrika,1 (March1940),324-45.13. JeromeB. Kernan,"GraphTheoreticModelsin Market-ing,"ScientificBusiness,3 (Spring1966), 331-43.14. Rosalind B. Marimont, "A New Method of Checking the

Consistencyof PrecedenceMatrices,"Journalof the Associ-ation for ComputingMachinery,6 (April 1959), 164-71.15. J. W. Moon and Leo Moser, "Bibliography on GraphTheory," Unpublished manuscript, University of Alberta,1962.16. C. E. Shannonand WarrenWeaver,The Mathematical The-ory of Communication,Urbana, Ill.: University of IllinoisPress, 1962.17. Leo Spier, "GraphTheory as a Method for Exploring Busi-ness Behavior," in J. W. McGuire, ed., InterdisciplinaryStudies in BusinessBehavior, Cincinnati,Ohio: South-West-ern Publishing Company, 1962, 70-98.Because of Kendall [11]. This sourcealso providestables forcases where m < 6 and p < 4.