Designing Ranking Systems for Consumer Reviews: The Economic Impact of Customer Sentiment in...
-
Upload
laurel-ray -
Category
Documents
-
view
215 -
download
1
Transcript of Designing Ranking Systems for Consumer Reviews: The Economic Impact of Customer Sentiment in...
Designing Ranking Systems for Consumer Reviews: Designing Ranking Systems for Consumer Reviews:
The Economic Impact of Customer Sentiment in Electronic MarketsThe Economic Impact of Customer Sentiment in Electronic Markets
Anindya GhoseAnindya GhosePanagiotis IpeirotisPanagiotis Ipeirotis
Stern School, New York UniversityStern School, New York University
ICEC 2007
Democratization of content creationDemocratization of content creation
Opinions in user-generated content
– The Web has changed the way that people express their views and opinions – One can express opinions on almost anything at:
• review sites, forums, reputation profiles, blogs
Opinions Matter
• Businesses: marketing intelligence, product design and service improvement.
• Firms invest a lot to find consumer opinions• Consultants, Surveys and focus groups, etc
• Individuals: interested in other’s opinions on products, services, topics, events
• Online word-of-mouth
Selected Prior ResearchSelected Prior Research
• Analyze and classify sentiments in online opinions – Pang and Lee (2004), Hu and Liu (2004), Kim and Hovy (2004), Liu, Hu and
Cheng (2005)
• Volume and valence in online reviews influences product sales
– Dellarocas et al. (2005), Chevalier and Mayzlin (2006), Liu (2006), Clemons and Gao (2006)
• Reviewer’s social information influences product sales and review helpfulness over and above volume and valence– Forman, Ghose and Wiesenfeld (2006)
What about the impact of textual sentiments on •Product sales
•Review Helpfulness
Focus of this workFocus of this work
Our QuestionsOur Questions• How sentiment in the text affects How sentiment in the text affects product demandproduct demand??
• How sentiment in the text affects How sentiment in the text affects informativenessinformativeness of reviews? of reviews?
Searching for reviews is different from general Web search.
– Reading only the review ranked at the top can be erroneous because it is only opinion of one person.
Improve the design of electronic markets
• Propose two ranking mechanisms for ranking product reviews:– a consumer-oriented ranking mechanism ranks the reviews according
to their expected helpfulness.
– a manufacturer-oriented ranking mechanism ranks the reviews
according to their expected effect on sales.
DataData
Elements of a review and reviewer profile
Rating (number)->Moderate or Extreme Review
Number of helpful votes and total votes
Review posting date
Textual content-> ‘Subjective’ or ‘Objective’ Review and ‘Readability’
Reviewer information disclosure -> Real Name, Location, etc.
Overview
• Mix of products: Audio-Video, Digital Cameras.• Sample: March 2005 - April 2006
Summary of Amazon Data
• New: List Price, Amazon price, Sales rank, Release Date.• Valence, Volume of reviews
Measuring SubjectivityMeasuring Subjectivity
• Pang and Lee (2004)-technique that identifies which sentences in a text convey objective or subjective elements.• Sentences in product description: “Objective”• Sentences in reviews: “Subjective”
• A training set with two classes of documents:– A set of objective documents that contains the product descriptions of each
of the 1,000 products in our data set.– A set of subjective documents that contains randomly retrieved reviews.
• Train a classifier (n-gram based) using Dynamic Language Model classifier from LingPipe toolkit to distinguish between subjective and objective sentences in each review.– ‘Average Probability of Subjectivity’ for a review– ‘Standard deviation of Subjectivity’ for each review
• In our context: “Subjectivity” measures deviation of review from manufacturer-provided product information
Measuring ReadabilityMeasuring Readability
“Readability” measures amount of information in the review, and difficulty of reading the provided information
Current work:1. Measured number of sentences2. Measured length of review in words and characters3. Ratio of length in characters (or words) to number of
sentences.
Ongoing work: • Use ‘Readability’ statistics of the reviews
– ColemanLiau– Flesh Kinkaid Level– Gunning– SMOG Index
Impact of Sentiments and Sales
it it it it
it it i it
log(SALESRANK ) Log( AMAZONPRICE) log(ELAPSEDDATE ) AVGPROB
DEVPROB X
1 2 3 1
4 1 1
• Product level fixed effects
• We also ran first differences model using variations in the unit of time analysis (weekly, biweekly and monthly)
• Qualitative nature of results are robust to these specifications
• Derived the stylistic characteristics of each review. • Proceed to examine the economic impact of the
subjectivity (or objectivity) of the review.
Estimates for effect on sales rankEstimates for effect on sales rank
Independent Variable
Audio-Video Digital Camera
Average Subjectivity
-1.47***
(.72)1.27
(1.24)
Deviation of Subjectivity
-0.69(1.06)
-2.91***
(1.7)
Amazon Price 1.59***
(0.3)6.2*** (0.61)
Elapsed Date 0.12(0.07)
0.28**
(0.13)
Average Rating -0.016(0.02)
-0.01(0.03)
Log (Num. Reviews)
0.6*** (0.15)
1.08***
(0.2)
Readability 0.06(0.057)
-0.15*
(0.08)
R2 0.18 0.37
Validation with Content AnalysisValidation with Content Analysis
• Asked two coders to classify 1,000 reviews (test set):– Is the review informative or not?– As influencing their purchasing decisions or not?
• Good inter-rater agreement (Kappa statistic of 0.73)
• For second question, using polychoric correlation agreement was strong (p < 0.05)
• Estimated regression coefficients using rest of reviews (training set)
• Predicted helpfulness and influence of the test reviews, using only text
Findings and ConclusionFindings and Conclusion
• Show that textual sentiment influences sales over and above the numeric and self-descriptive information that consumer reviews and reviewers display.
– Increasingly subjective reviews can increase product sales.
– A mixture of objective and subjective opinions with extreme subjective content can increase sales.
– A mixture of objective and subjective opinions with extreme subjective content can increases informativeness of reviews
– Increased `Readability’ can increases product sales and informativeness of reviews
• Our method can quickly identify reviews that are expected to be helpful to the users, and display them first.
– Improving the usefulness of the reviewing mechanism to the consumer in an electronic market.
– Manufacturers can understand which reviews are most impactful and use those product features for marketing promotions
Ongoing Work
• Combine the sentiments (positive, mixed or negative) with subjectivity analysis.– Negative reviews may increase sales if the reviews are
informative
• Incorporate the impact of all the past reviews of a reviewer.
• Examine more text-based variables, in detail (e.g. readability metrics).