Visual Persuasion: Inferring Communicative Intents of Images

Jungseock Joo1, Weixin Li1, Francis F. Steen2, and Song-Chun Zhu1

1Center for Vision, Cognition, Learning and Art,Departments of Computer Science and Statistics, UCLA

2Department of Communication Studies, UCLA{joo@cs, steen@commstds, sczhu@stat}.ucla.edu

AbstractIn this paper we introduce the novel problem of under-

standing visual persuasion. Modern mass media make ex-tensive use of images to persuade people to make commer-cial and political decisions. These effects and techniquesare widely studied in the social sciences, but behavioralstudies do not scale to massive datasets. Computer visionhas made great strides in building syntactical representa-tions of images, such as detection and identification of ob-jects. However, the pervasive use of images for commu-nicative purposes has been largely ignored. We extend thesignificant advances in syntactic analysis in computer vi-sion to the higher-level challenge of understanding the un-derlying communicative intent implied in images. We be-gin by identifying nine dimensions of persuasive intent la-tent in images of politicians, such as “socially dominant,”“energetic,” and “trustworthy,” and propose a hierarchicalmodel that builds on the layer of syntactical attributes, suchas “smile” and “waving hand,” to predict the intents pre-sented in the images. To facilitate progress, we introducea new dataset of 1,124 images of politicians labeled withground-truth intents in the form of rankings. This studydemonstrates that a systematic focus on visual persuasionopens up the field of computer vision to a new class of inves-tigations around mediated images, intersecting with mediaanalysis, psychology, and political communication.

1. IntroductionPersuasion is a core function of communication, aimed

at influencing audience beliefs, desires, and actions. Visualpersuasion leverages sophisticated technologies of imageand movie production to achieve its effects. The examplesin Fig. 1. (a) are designed to convey social judgments: thatObama is an inferior candidate to Romney, that a Mac ismore user friendly than a PC, and that Hitler is kind andtrustworthy. These claims are not stated verbally, but relyon routine visual inferences.

(b) Factual Image

(c) Persuasive Image

Syntactical Interpretation

A street scene.A black car is moving.Vehicles are parked

next to a yellow building with gray roof.

U.S. President Barack Obama talks

and smiles to 3 children in a classroom

?

He cares about children and their

education. I should vote for him.

Persuasive Interpretation

a) Mass media images with complex communicative intents

I'm a PC I'm a Mac

Figure 1. (a) A persuasive image has an underlying intention topersuade the viewer by its visuals and is widely used in mass me-dia, such as TV news, advertisement, and political campaigns. Forexample, it is a classical visual rhetoric to show politicians inter-acting with kids, arguing that they are dependable and warm. (b)Existing approaches in computer vision lead to syntactical under-standing of images to explain the scene and the objects withoutinferring the intents of images, which is absent in factual imagesin usual benchmark datasets. (c) Our paper is aimed at understand-ing the underlying intents of persuasive images.

Visual argumentation is widely used in television newsand advertisements to generate predictable social judg-ments. Why did President Obama post Fig. 1. (c) on hisWhite House Blog? The image contains no policy-relevant

1

information. It does, however, lend itself to generating a setof inferences about Obama’s character: that he loves chil-dren, that he is caring, and that he can be trusted with mak-ing the right decisions in education. Such inferences arepolitically extremely valuable for a politician, and are hardto convey verbally.

Examining the image in more detail, one can notice itcontains a suite of syntactical components to compose its in-tent: the protective gesture, steady gaze, welcoming smile,and the child smiling. Audiences see these elements andmake judgments as if they were present, yet what the imageshows is the result of professional photographers compos-ing and selecting these elements in order to create a specificimpression. Because we believe our own eyes, but knowwell that people are manipulative, we tend to be verballyskeptical and visually gullible.

In this study, we examine nine different trait dimensionsin order to characterize the communicative intents of im-ages. To infer these dimensions, we exploit 15 types ofsyntactical features – facial attributes, gestures, and scenecontexts that construct the communicative intent. Computervision research has made remarkable progress in addressingsyntactical problems; we extend these techniques to under-stand and predict large-scale patterns in the higher-level per-suasive messages that images in the media routinely convey.In summary, this study addresses the following researchquestions:

i. We define a novel problem, to infer the communica-tive intents from images, in a computational frame-work. We identify the dimensions of intent in persua-sive images and describe how they can be inferred fromsyntactical features. The complete list of intents is pre-sented in Fig. 2.

ii. We present a new dataset to study visual persuasion. Itcontains 1,124 images of 8 U.S. politicians annotatedwith the persuasive intents of 9 types as well as syntac-tical features of 15 types.

iii. Finally, to verify the impact of visual persuasion inmass media, we present a quantitative result in a casestudy that reveals a strong correlation between the vi-sual rendition of the U.S. President in mass media andpublic opinion toward him.

2. Related WorkOur paper is related to studies in computer vision on

human attribute recognition [8, 22, 16, 13], such as gen-der, race, or facial expression recognition. However, com-municative intents are distinct from traditional human at-tributes in two important ways. First, intents focus onjudgment rather than surface feature. We deploy syntac-tic interpretation to leverage surface features as interme-diate representations. In our analysis, persuasive intents

Facial Display Gesture Scene Context

SmileLook DownEye Open

Mouth Open

Hand-WaveHand-ShakeFinger-PointTouch-Head

Hug

Large CrowdDark-Background

Indoor

Emotion Personality Global

HappyAngryFearful

CompetentEnergetic

UnderstandingT

Social Power

Favorable(Pos vs. Neg)

Trustworthy

Favorability

(b) Persuasive Intents(a) Syntactical Attributes

S D E M W S F T H L D IFacial Display Gesture Scene

H A F

Emotion

C E U T S

Personality

P N

Global Favorability

(c) Information Path

Figure 2. The set of (a) syntactical attributes and (b) commu-nicative intents defined in this study. (c) Illustrative hierarchy ofintention inference. The image is first interpreted syntactically.Second, the syntactical representation is transformed to infer thecommunicative intents. Third, communicative intents in the formof emotional characteristics and personality traits are used to as-sess global favorability.

are not directly observable, but inferred from complex pat-terns involving multiple image evidence. Second, the syn-tactical feature can have specific social semantics beyondits surface, narrative, first-order meaning. For example, a“hand wave” can mean “competence” or “popularity,” while“touching face” can imply “trouble”. We seek to systemat-ically identify these underlying implicatures [10], or hid-den semantics, of the syntactic attributes, which have notbeen considered in the prior works. The distinction betweensyntactic features and communicative intents in visual com-munication parallels the distinction between literal messagemeaning and communicative intention in pragmatic theoriesof language [1].

Researchers in political science and mass media haveexamined audiences’ emotional and cognitive responses totelevised images of political leaders [17, 24] and studiedthe media’s selective use of images for persuasive purposes[2, 20, 21, 9]. This body of work has reported a series ofcorrelations between politicians’ appearance on media and

Is it a facial expression recognition problem?

Is "happy" positive and is "sad" negative?

(c) (d)

(a) (b)

Figure 3. (a-b) Communicative intents can be inferred from non-facial cues such as gestures (e.g., hand wave), or scene context(e.g., large crowds). (c-d) The intents cannot be understood co-herently by a single dimensional approach such as polarized sen-timent analysis. The annotators found the image (c) “sad” but also“comforting”, so they believe the image shows “positive favorabil-ity” toward the target, whereas the exact opposite observations arefound in the image (d).

electoral success. While their results are interesting, theyare restricted by manual analysis to a small number of im-ages or television news shows.

3. RepresentationIn this section, we identify the dimensions of commu-

nicative intents as well as the syntactical features used topredict the intents. Fig. 2 highlights the overall representa-tion of our model where the layers of hierarchy are definedaccording to the levels of abstraction. An image can be firstinterpreted at the syntactic level by many types of ‘detec-tors’ or ‘classifiers’, such as a smile classifier. The outputsof the syntactic solvers are used to infer a set of commu-nicative intents, which we aggregate to measure the globalfavorability of the image, positive or negative.

3.1. Dimensions of Communicative Intents

What are the intentional dimensions we can use to spec-ify the perception that the image conveys to viewers? Onecan imagine there exist thousands of different types of emo-tions or feelings that we can attribute to the main target ofimage. To simplify the problem, we choose a series of quan-tized dimensions of the communicative intents as follows.

i. Emotional Traits. Ekman, in his influential early workin psychology [6], has argued the human has 6 elemen-tary and ‘universally interpretable’ emotions. Masterset al. [18], have further defined 10 emotional traits intheir study on analyzing facial displays of politicians.Since this set covers a broad basis of human emotionsthat can apply to general domains including politics,we adopt these 10 emotional traits in our study.

ii. Personality Traits and Values. While the emotional di-mensions focus on the subjects’ own emotions, whatone can perceive from an image and what an imageis ultimately intended to convey requires a broaderand more general scope. For instance, ‘understanding(the others)’ is an important requirement to politiciansand many social figures and persuasive images oftenachieve this concept by showing the targets interact-ing with general public, the handicapped, or children.These traits describe more essential and canonical char-acteristics of people than the spontaneous captures ofemotions. Therefore, we consider six additional di-mensions in this category which are particularly impor-tant to the politicians.

iii. Overall Favorability. Finally, we also define the overallfavorability of a target person as an integrated measure.This can be viewed as a summarized polarity, i.e., pos-itive or negative.

Given initial dimensions, we conducted a preliminary teston a subset of our dataset to merge redundant dimensions,i.e., multiple dimensions producing very similar ratings,and obtained 9 dimensions as follows: angry, happy, fear-ful, competent, energetic, comforting, trustworthy, sociallydominant, overall favorable.

3.2. Syntactical Attributes

There are many types of visual cues from which we canpredict the communicative intents. Some cues may be re-trieved from the target person, especially from the face andgesture, and other cues from scene context and background.These cues can be viewed as the syntactical interpretationof image and they can collectively signal toward particu-lar intents at a higher and more abstract level. Specifically,we use 15 syntactical attributes categorized into 3 groups asfollows.

Facial Display. Psychological studies of emotion dis-crimination have long focused on the sophisticated humanability to read faces [6]. From the computational perspec-tive, it can be said that the facial region provides the mostdistinguishable cues to classify attributes [22]. We use 4 dif-ferent facial display types as follows: smile/frown, mouth-open/closed, eyes-open/closed, and head-down/up. To rec-ognize each display type, we first detect the face and lo-calize the facial keypoints (the centers of both eyes, mouth,and the whole face) by Intraface [25], which is a publiclyavailable software. From each keypoint, we extract HoGfeature of 4 × 4 cells at 3 different scales and then train alinear SVM for each type.

Body Cues - Gestures. Body language often provides auseful channel in non-verbal communication and similarly,certain gestures or actions can deliver or form the senti-ments about the target person. An example is “hand wave”

In which image does Barack Obama look more COMPETENT ?

Figure 4. An example question given to the annotators. Given apair of images, each annotator can respond to judge which imagehas greater intention to emphasize a certain emotion or personalityin the given dimension.

by politicians, which can be viewed as an positive action toshow an engagement between the politician and the elec-torate [4]. “Hugging” is another example which asserts asimilar function, possibly with a greater strength as it in-volves a physical contact. We define 7 types of human ges-tures frequently used in the domain of political news as fol-lows: hand wave, hand shake, finger-pointing, other-hand-gesture, touching-head, hugging, or none of these (normal).

Gesture or action recognition from images is a difficultproblem due to pose variation and occlusion. Moreover, weare interested in distinguishing subtle differences such ashand wave and finger-pointing, which might make the ex-isting pose-driven approaches ineffective. However, it goesbeyond the scope of this paper to address such challengesin detail. We adopt a simple method of 3-level spatial pyra-mids with densely sampled SIFT features encoded by thedictionary learned by K-means clustering, which has beenproposed for action recognition in the recent literature [5].

Scene Context. Finally, persuasive intents can be alsoinferred from the image background as it provides contextsand situations the target is facing. For example, a largegroup of supporters can imply strong popularity of the targetand a dark background may hint at an uncertain future of thetarget (“doomed”). We use 4 binary scene context featuresas follows: dark-background, large-crowd, indoor/outdoorscene, and national-flag. To classify each scene contexttype, we use the same method used for gesture type recog-nition, discussed above.

Once all feature type values are obtained, each image, I,can be represented by a 17 dimensional response vector atsyntactical level. We denote the response vector by f(I) =[f1(I), f2(I), ..., fp(I)] ∈ Rp, where the subscript specifiesthe feature type and p, the number of dimensions, equals 17in this paper.

4. Learning to Rank Communicative Intents4.1. Developing a scale

Relative vs. Absolute Assessment. Our goal is super-vised learning of mapping functions from the images to aseries of communicative intents (Sec. 3.1), which requiressupervision as ground-truth annotation. In many syntacti-cal problems, the outputs to predict are defined as binary

or discrete variables with predefined categories and in thiscase the absolute assessment of images from the annotatorsis suitable (i.e., fact-checking). In contrast, we require ourannotators to report their own subjective judgments basedon their perceptions and thus, we cannot simply ask themto assign an absolute score to each image since they do notshare the same reference scale. Moreover, the magnitudeof signals of the intents may be very subtle, which makesabsolute assessment even harder.

Fortunately, we observe that communicative intents, al-though ambiguous when evaluating each image individu-ally, can be much better grounded on relative scale such thatthe valence of an image in a perceptual dimension emergesfrom the comparison against the other images. Similar mo-tivations can be found in literatures of information retrievaland computer vision [12, 23, 15]. We therefore present apair of two images at a time and ask the annotators to choosewhich image depicts the higher degree of given trait, e.g.,“Which one looks more positive?”. Fig. 4 shows an exam-ple question and an image pair.

We can now introduce the notation for the intents. Givena pair of images (Ii, Ij), Ii � Ij indicates the image Ii has aproceeding order to Ij . Since each response only provides apair-wise order, we aggregate all responses to construct theglobal ranking order using HodgeRank [11] which arrangesthe entire image set in one sequential order. One motiva-tion of having the global ranking is to deal with inconsistentpair-wise annotations (A � B,B � C,C � A).

Individual vs. Universal Rank. In this study, we do notconsider the difference between individuals but solely fo-cus on the difference between the different photographs ofthe same person. The purpose of this treatment is two-fold.First, we want to rule out the annotators’ personal prefer-ence on certain politicians that may be based on historicalrecords or stereotypes (gender or age). Evidently, these arenot observable from the images. Second, our overall studyis aimed at understanding the editorial intents conferred onimages by altering specific features. However, the individ-ual factors such as gender or their original facial appearancecannot be altered by editorial tastes as they are constant.

Therefore, we restrict the pair-wise comparisons and theglobal rank to be obtained from the same subject in orderto systemically exclude the individuated factors and exist-ing political preference of human annotators in our study.For example, we do not attempt to compare an image of“Obama” with another image of “Romney”. Our treatmentis exactly orthogonal to that of [23] in which all image in-stances of one person share the same attribute value. This isbecause they model person-specific and image-invariant at-tributes such as “big-lips”, while we seek the image-specificattribute values.

4.2. Model

Our model to predict the intents - the upper structureof the overall hierarchy in Fig. 2 - builds on the frame-work of Ranking SVM [12]. While the binary SVM at-tempts to maximize the margin between the examples (sup-pert vectors) of two classes, the ranking SVM maximizesthe pair-wise margins in the order specified by the train-ing set. For each dimension of persuasive intents, we aregiven N training images and their global ranking order,D = {(i, j)|Ii � Ij}Ni,j=1. We first obtain the syntac-tical feature vector (Sec. 3.2), f(I) ∈ Rp, for each im-age. Next, our goal is to learn a linear ranking function,r(I) = 〈w, f(I)〉, with the following objective:

minimize :1

2||w||22 + C

∑ξi,j

subject to : w>f(Ii) ≥ w>f(Ij) + 1− ξi,j ,ξi,j ≥ 0,∀(i, j) ∈ D,

(1)

introducing a non-negative slack variable, ξi,j , for everypair in D. C controls the trade-off between training errorand margin maximization. We use the implementation of[3] to solve this optimization problem. Finally, since weassume the global favorability can be inferred on the basisof the other types of intentions, we train its ranking func-tion with the outputs from the second layer as well as fromsyntactical features.

Universal Model. If we train a separate model for eachperson, it is likely to produce better prediction performance,but at cost of higher complexity and limited scalability. Wetrain one unified model that can address universal charac-teristics shared by the group of different politicians. Hence,the ranking order set, D, contains the example pairs of allpeople but does not have any pair of images of different in-dividuals. This should not be confused with individual vs.universal rank discussed above; the annotation and the eval-uation still follow the individual protocol but we learn oneshared model.

5. Experiments5.1. Dataset: Persuasive Portraits of Politicians

We present a new dataset of 1,124 images of the politi-cians with the labeled communicative intents and syntac-tical features. Fig. 5 shows a few examples. While ourmethodology is applicable to general domains, we specifi-cally choose the political domain in our dataset because thisis where the media would have the biggest intention to per-suade the audience. Also, we can easily find hundreds ofdifferent photographs of the same politician. This uniqueproperty enables the media to deliberately select which im-age to present according to their own editorial tastes andnews contexts.

Specifically, we chose 8 U.S. high-profile politicians1

whose has frequently appeared in the main stream mediaand collected their photographs from many news outlets on-line. 10 undergraduate and graduate students participatedin annotation process. As discussed earlier, we let the an-notators rate the ordinal values by comparison from pairsof photographs of the same politician, from which we re-covered the global rank. We also labeled the syntactical at-tributes used in Sec. 3.2 and a bounding box to specify themain target of each image.

Annotator Agreement. It is important to verify howmuch consensus the annotators had in evaluating intents.We measured the correlations among the annotators by re-trieving an independent global ranking order for each an-notator and obtaining correlation coefficients between ev-ery two ranking orders from different annotators, which en-sured a high degree of agreement (0.647).

5.2. Predicting Communicative IntentsBaseline. Since there are no existing methods devel-

oped for our problem, we trained baseline classifiers whichtake the low-level image features and directly output theintents. Our baseline classifiers adopt 3-level SPM withdensely sampled SIFT features, which is the same methodthat we use for recognizing gesture types.

Measure. We use Kendall’s τ [14] (or Kendall rankcorrelation coefficient) as performance measure, which iscommon in ranking evaluation [12]. Given two global rank-ings (one from ground-truth and the other from predictionin our case), it measures how similar or dissimilar they areby counting pair-wise consistencies for all pairs. It is simplydefined as follow:

τ =(# of concordant pairs)− (# of discordant pairs)

# of all possible pairs.

A concordant pair means two examples in the pair are ob-served in the same order in both ranking sequences. If tworankings are identical, Kendalls’ tau equals to one. If theyare reversed, it is negative one. Since we separate the rat-ings for different politicians, we measure the accuracy foreach person and use the average value.

Given this baseline and measure, we evaluate how wellthe learned model can predict the communicative intents ofimages. Fig. 6. shows the results. First, one can see thedominant effect of the facial displays in the emotional di-mensions. Indeed, a face provides an instant delivery ofemotional states and some other dimensions such as “trust-worthy” can be also perceived and inferred from face. Atthe same time, we also observe the other cues, such as ges-ture types, can better predict certain dimensions such as“comforting” or “socially dominant”. When do we feel that

1Barack Obama(199), George W. Bush(174), Mitt Romney(154),Hillary Clinton(152), John McCain(153), Joe Biden(121), Paul Ryan(82),and Sarah Palin(89), (# of images of each person).

Most FAVORABLELeast FAVORABLE

Most COMFORTINGLeast COMFORTING

Most COMPETENTLeast COMPETENT

Most DOMINANTLeast DOMINANT

Predicted Intents

Figure 5. Example images in our dataset and their intents inferred by our model. The images of the right side are “the most” examples ingiven dimensions and plotted by “blue” curves, whereas the left side are “the least” examples plotted by “red” curves.

Emotional Personality/ValuesGlobal

Favorable Angry Happy Fearful Competent Energetic Comforting Trustworthy Powerful MEAN

Figure 6. Intents prediction performance evaluation measured by Kendall’s tau. For all dimensions, the full approach that exploits allthree types of syntactical features yields the best result. In addition, the facial display type outperforms the other cues on the emotionaldimensions while the gesture type is more discriminative for 3 among 5 dimensions of personality traits and values.

someone is comforting from the image? To quantitativelyanswer this question, we further investigate what are theparticular causes to invoke these sentiments. Fig. 8. showsthe correlation coefficient matrix between the ground-truth

syntactic features and the intent annotation. From this ma-trix, one can say, for example, perception of competencearises from combination of facial displays (smile), gestures(hand wave, hand shake), and scene context (large-crowd).

Election(11/7/2012) Election(11/7/2012)

U.S. Domestic - NYT, Washington Post European Media - Reuters, Guardian

Figure 7. Contrastive media coverage: agenda-setting and mirroring. The correlation between the computed image favorability and publicopinion can reveal different motivations of the media outlets.

Syntactical Feature Recognition. Table. 1 reports theaccuracy of our model to recognize syntactical attributes. Inparticular, some fine gesture types are very difficult to dis-tinguish: finger-pointing vs. other-hand-gesture (e.g., fistor v-shape). If we replace the intermediate-level responsevector by ground-truth attribute annotations, the intent pre-diction accuracy improves up to 0.48 (compared to 0.30 inthe fully automated model), suggesting room to improve.

5.3. Media and Public OpinionNow we consider the visual persuasion in the real-

world news stories in which the 3rd party newsmaker (e.g.,Reuters) describes the target person (e.g., US president). Inmost cases, the news stories are delivered to the audiencethrough the verbal (text) and optionally the visual (image),which complement each other: the verbal elaborates the de-tails of an event and the visual spotlights the key aspects to

Table 1. Accuracy of Syntactical Feature Recognition.Category Facial Scene Gesture

AP .744 .658 .351Frequency .392 .267 .143

Figure 8. Correlation between the ground-truth syntactic featuresand the intents.

consider.In this case study, we examine the temporal behavior of

the media in framing a particular subject by presenting thevisuals and its relation to the public opinion about the sub-ject. In particular, we choose Barack Obama, the 44th andthe current President of the U.S., as our main target becausehe has gained the biggest attention from the U.S. domes-tic media as well as international media over past 5 years.Moreover, there exist many types of polls to measure thepublic opinion about him during this period.

We processed the data as follows. First, we crawled allnewspaper articles that mention his name from 4 sources -the New York Times, Washington Post, Reuters, and TheGuardian - from 01/01/2008 to 09/30/2013. Then we dis-carded the articles without any photographs and the oneswhose photograph captions do not specify the name of thetarget. For each image, we obtained the syntactical featuresas follows. Since there is no bounding box available, wefirst identified Barack Obama’s face by the same approachtaken for recognizing the facial display types (Sec. 3.2).Once the system recognizes him, it infers the facial displaytypes from the recognized face. For gesture type, we useddeformable part model [7] to detect the person boundingbox and continued recognizing gestures. The scene contextfeature can be inferred irrespective of prescribed bound-ing box. The remaining stages are the same as our modeldiscussed and finally the system outputs the overall favor-ability of each image. At a data point (a particular date),we smoothed the prediction scores of images of the articlesaround the date within 1-month time window. To comparethis computed statistics against, we obtained the presiden-tial job approval rates from Huffington Post, aggregating2,356 polls from 86 pollsters.

Figure 7. shows the temporal evolutions of the media’spresentation (red) and the public opinion (blue). We seethat the detected overall image favorability in the imagesfrom foreign sources (Reuters and The Guardian) correlatesmore closely with the aggregate opinion polls than the im-ages from the politically most influential US newspapers(New York Times and Washington Post). The closer corre-lation (ρ = 0.756) in the foreign sources suggests that these

news outlets are passively mirroring the ups and downsof Obama’s facial expressions in public events. The graphshows him peaking at the previous election, dropping downafterwards, and then rising back up to peak some time afterhis re-election in November 2012, clearly showing the dra-matic effects of the elections and their immediate aftermath.

In contrast, the prominent domestic newspapers showless correlation overall (ρ = 0.362), and significantly asharp negative correlation with opinion polls from the re-eletion. Images used in these news media appear to take thelead in showing less favorable images of the President, fore-shadowing the drop in popularity that follows some weeksafter the election. Media scholars have argued that the massmedia play an important role in setting the public agenda[19]. These results are consistent with a large body of workin media studies and political communication, using datasources and methodologies that have previously been un-available.

6. ConclusionThis study aims to demonstrate that a systematic exami-

nation of communicative intents can yield new insights intothe meaning and persuasive impact of images, which goesfar beyond traditional classification on surface features. Wecontribute a new dataset of political images, and show howto build on an advanced syntactical analysis, and infer mul-tiple dimensions of persuasive intents. Finally, we showthat the resulting favorability judgments correlate in infor-mative ways with independent measures of public opin-ion, proposing the contrasting media behaviors of agenda-setting and mirroring. By engaging with the ubiquitous useof visual images in the mass media, computer vision canmake unique new contributions to an emerging field.

Acknowledgements. This work was supported by NSFCNS 1028381, ONR MURI N00014-10-1-0933, and NSFIIS 1018751.

References[1] J. L. Austin. How to Do Things with Words. Clarendon,

1962.[2] K. G. Barnhurst and C. A. Steele. Image-bite news the vi-

sual coverage of elections on US television, 1968-1992. TheHarvard International Journal of Press/Politics, 2(1):40–58,1997.

[3] O. Chapelle and S. S. Keerthi. Efficient algorithms forranking with SVMs. Information Retrieval, 13(3):201–215,2010.

[4] P. D. Cherulnik, K. A. Donley, T. S. R. Wiewel, and S. R.Miller. Charisma is contagious: The effect of leaders’charisma on observers’ affect. Journal of Applied Social Psy-chology, 31(10):2149–2159, 2001.

[5] V. Delaitre, I. Laptev, and J. Sivic. Recognizing human ac-tions in still images: a study of bag-of-features and part-based representations. In BMVC, 2010.

[6] P. Ekman and et al. Universals and cultural differences inthe judgments of facial expressions of emotion. Journal ofpersonality and social psychology, 53(4):712, 1987.

[7] P. F. Felzenszwalb, R. B. Girshick, D. McAllester, and D. Ra-manan. Object detection with discriminatively trained part-based models. TPAMI, 32(9):1627–1645, 2010.

[8] B. A. Golomb, D. T. Lawrence, and T. J. Sejnowski. Sexnet:A neural network identifies sex from human faces. In NIPS,1990.

[9] M. E. Grabe and E. P. Bucy. Image Bite Politics: News andthe visual framing of elections. Oxford University Press,2009.

[10] H. P. Grice. Logic and conversation. Dickenson, 1967.[11] X. Jiang, L.-H. Lim, Y. Yao, and Y. Ye. Statistical ranking

and combinatorial hodge theory. Math. Program., 2011.[12] T. Joachims. Optimizing search engines using clickthrough

data. In ACM SIGKDD, 2002.[13] J. Joo, S. Wang, and S.-C. Zhu. Human attribute recogni-

tion by rich appearance dictionary. In ICCV, pages 721–728,2013.

[14] M. Kendall. Rank Correlation Methods. Griffin, 1948.[15] A. Kovashka, D. Parikh, and K. Grauman. Whittlesearch:

Image search with relative attribute feedback. In CVPR,2012.

[16] N. Kumar, A. C. Berg, P. N. Belhumeur, and S. K. Nayar.Describable visual attributes for face verification and imagesearch. TPAMI, October 2011.

[17] J. T. Lanzetta, D. G. Sullivan, R. D. Masters, and G. J.McHugo. Emotional and cognitive responses to televised im-ages of political leaders. Mass media and political thought:An information-processing approach, pages 85–116, 1985.

[18] R. Masters, D. Sullivan, J. Lanzetta, G. Mchugo, and B. En-glis. The facial displays of leaders: Toward an ethology ofhuman politics. Journal of Social and Biological Systems,9(4):319–343, Oct. 1986.

[19] M. E. McCombs and D. L. Shaw. The agenda-setting func-tion of mass media. Public opinion quarterly, 36(2):176–187, 1972.

[20] P. Messaris and L. Abraham. The role of images in framingnews stories. In S. Reese, O. Gandy, and A. Grant, editors,Framing Public Life: Perspectives on Media and Our Un-derstanding of the Social World, Routledge CommunicationSeries. Taylor & Francis, 2001.

[21] J. E. Newhagen. The role of meaning construction in theprocess of persuasion for viewers of television images. InM. P. James Price Dillard, editor, The Persuasion Handbook:Developments in Theory and Practice. 2002.

[22] C. Padgett and G. W. Cottrell. Representing face images foremotion classification. In NIPS, 1997.

[23] D. Parikh and K. Grauman. Relative attributes. In ICCV,2011.

[24] D. G. Sullivan and R. D. Masters. “happy warriors”: Lead-ers’ facial displays, viewers’ emotions, and political support.American Journal of Political Science, 1988.

[25] X. Xiong and F. De la Torre. Supervised descent method andits applications to face alignment. CVPR, 2013.