Automated Linguistic Personalization of Targeted Marketing...

© 2014 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.© 2014 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.

Automated Linguistic Personalization of Targeted Marketing Messages MiningUser-generated Text on Social Media

Rishiraj Saha RoyAdobe Research Lab India

Aishwarya PadmakumarIIT Madras

Guna Prasaad JeganathanIIT Bombay

Ponnurangam KumaraguruIIIT Delhi

16-Apr-2015CICLing 2015, Cairo, Egypt1

© 2014 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.

Agenda

2

Motivation

Data

Method

Results

16-Apr-2015CICLing 2015, Cairo, Egypt


The need for personalization

3

Content

Product

MessageLayouthttp://venturebeat.com/2013/05/07/mass-marketing-vs-personalization-infographic/


Presenter

Presentation Notes

- Large commercial websites contain significant amount of textual marketing content, content created by internal or external (ad agency) copywriters - They are highly creative people, but personalization and diversification are key to engagement – several versions of same content required - Marketing resources limited, Human domain knowledge is limited - difficult to know what "clicks" with specific target preferences - What worked, what didn’t with past content - Even creative people can avoid writers’ blocks and save time with intelligent suggestions!


The power of personalization

4

ALL POWERFUL

Check out the all-new powerful line of gaming laptops from Alienware.

Now pre-loaded with a copy of the award-winning game, Portal.

IMMEDIATE EXECUTE. ROGER THAT!

THE POWER TO DO MORE.The Latitude Series makes high performance computing portable.

The best laptop in the market for high performance scientific computing. Improves the computing power by 20%.

Get the latest version of Mathematicacompletely free!

Place Order

Place Order


Presenter

Presentation Notes

Pain points: searching, copy-pasting, finding and editing relevant areas


Importance of message text


Presenter

Presentation Notes

Pain points: searching, copy-pasting, finding and editing relevant pieces


Why personalize ad message?

6

People from different demographics talk differently (Schwartz et al. 2013)

Linguistic style is an important component in advertising (Lowrey 1998)

Communicating to specific audience segments in their own linguistic styles is expected to increase engagement

with advertisements


Presenter

Presentation Notes

Syntactic, semantic patterns, clustering


Why automate message personalization?

7

Content authors or copywriters are creative individuals

Diversification and personalization key to user engagement

Several versions of same content required with limited resources


Presenter

Presentation Notes

Ask for feedback


What can we do as the first steps?

8

Draw with the Pencil tool.

Integrating platforms with flexibility.

Content management features and tools that get your work done.

Word usage patterns representative of linguistic styles [Hu et al. 2013]

Adjectives convey emotion and make the advertisement sound more urgent and exciting [Content Marketing Tips,

Econsultancy, June 2014]


Presenter

Presentation Notes

Gartner WCM – Oct 2014 - Content authoring, through browser-based templates or via conversion from a word-processing application Content management lifecycle – content need not expire soon, reuse Cost argument is difficult in business value



9

Draw more accurately with the all-new Pencil tool.

Seamlessly integrating multiple platforms with amazing flexibility.

Must-have content management features and innovative tools that get your work done really fast.

Word usage patterns representative of linguistic styles [Hu et al. 2013]

Adjectives convey emotion and make the advertisement sound more urgent and exciting [Content Marketing Tips,

Econsultancy, June 2014]


Presenter

Presentation Notes



Insertion of positive adjectives and adverbs

Preserves key content in the ad

Distributions vary between segments

Automate insertion constrained by usage patterns in segment



Presenter

Presentation Notes



Agenda

11

Motivation

Data

Method

Results


Presenter

Presentation Notes

What are things that the author wants to do? How to ensure right level of automation without affecting author satisfaction? Annotations – markup * - Not exactly a research question


Location, age, gender and occupation

Personal information access restricted

Data tagged by location and occupation

Characterizing audience segments

12

User

Attributes

Location

Age

Occupation

Gender


Presenter

Presentation Notes

Ask for prioritization, refer to earlier mention


Segment-specific data collection

13

US

UK

AU


Presenter

Presentation Notes

Made public


Different modifiers with different products

1621 Tweets collected for product

1364 Tweets with positive sentiment

562 modifiers extracted

60 ads collected for skeletons

Product-specific data collection

14

Adobe Creative Cloud


Presenter

Presentation Notes

I will not talk about all these.. Add companies from white space


Agenda

15

Motivation

Data

Method

Results


Presenter

Presentation Notes

Explore Inbound writer


Formal characterization of word usage

Probability distribution over all patterns

Efficient way of storage and representation

Model: Word pairs with parts-of-speech

Estimated from Tweet corpora

Statistical language models

16

Segment specific text corpus

Extract patterns

Compute probability distributions


Presenter

Presentation Notes

Forrester wave companies; Acquia Platform, November 2014; Adobe Experience Manager (AEM) 6.0; Ektron Content Management System (CMS) V9.1; EPiServer Cloud, November 2014; HP TeamSite 7.5; IBM Customer Experience Suite 8.5, IBM Employee Experience Suite 8.5; OpenText Web Experience Management 10.5; Oracle WebCenter Sites version 11.1.1.8; SDL Web, November 2014; Sitecore Experience Platform 8


Part-of-speech tagging

Dependency parsing

Lemmatization

Mining keyword-modifier pairs from segment-specific corpora


Presenter

Presentation Notes

Normalization: spelling correction, expanding informal contractions like 2morrow, etc.


Avoiding too many suggestions: Term weight thresholds

19

Adobe will be providing prizes.

4.728 12.871Adobe will be providing prizes.

Adobe will be definitely providing prizes.

Adobe will be faster definitely providing prizes.

𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼 𝐷𝐷𝑜𝑜𝑜𝑜𝑜𝑜𝑜𝑜𝐼𝐼𝐼𝐼𝑜𝑜 𝐹𝐹𝐼𝐼𝐼𝐼𝑟𝑟𝑜𝑜𝐼𝐼𝐼𝐼𝑜𝑜𝑟𝑟 𝑘𝑘𝐼𝐼𝑟𝑟𝑘𝑘𝑜𝑜𝐼𝐼𝑘𝑘

= log10𝑃𝑃𝐼𝐼𝑜𝑜𝑘𝑘𝑜𝑜𝑜𝑜𝑜𝑜 𝐼𝐼𝑠𝑠𝐼𝐼𝑜𝑜𝑠𝑠𝑠𝑠𝑠𝑠𝑜𝑜 𝑜𝑜𝐼𝐼𝐼𝐼𝐼𝐼𝑚𝑚𝑚𝑚𝐼𝐼𝐼𝐼

𝑃𝑃𝐼𝐼𝑜𝑜𝑘𝑘𝑜𝑜𝑜𝑜𝑜𝑜- 𝐼𝐼𝑠𝑠𝐼𝐼𝑜𝑜𝑠𝑠𝑠𝑠𝑠𝑠𝑜𝑜 𝑜𝑜𝐼𝐼𝐼𝐼𝐼𝐼𝑚𝑚𝑚𝑚𝐼𝐼𝐼𝐼 𝑘𝑘𝑠𝑠𝑜𝑜𝑤 𝑘𝑘𝐼𝐼𝑟𝑟𝑘𝑘𝑜𝑜𝐼𝐼𝑘𝑘



Handling context

20

Where to insert new words? But what about …

If you have been waiting to join eventually the club, …

This includes also the plans on marketing …

Convert your artworks to useful something …

Rules not good enough!

General trends

- Adjective before noun

- Adverb after verb


Presenter

Presentation Notes

Contract: Need not only be summarization, may be eye-catching snackable content


Point-wise mutual information

𝑃𝑃𝑃𝑃𝐼𝐼 𝑥𝑥;𝑟𝑟 = log2𝑠𝑠(𝑥𝑥,𝑟𝑟)𝑠𝑠 𝑥𝑥 𝑠𝑠(𝑟𝑟)

Probabilities from language models

PMI thresholds control significance

Left and right PMI thresholds must be satisfied

Pointwise mutual information

21

with

complex

systems

PMI > δ PMI > δ

with complex systems


Presenter

Presentation Notes

Hypothesis: Several news categories may exhibit similarity in content and structure Average words, sentences in an article To a common man, news categories – sports, business, politics. But news agencies store at a deeper granularity, and in these categories you can find patterns.


Noun phrases are tricky

22

… specific license new management free tools ….

… license management tools ….12.786 11.272 10.323

… license management tools ….

We provide license management tools.

We provide license management tools.Noun Phrase

Chunker

… license management tools …. … renewable license management tools ….


Presenter

Presentation Notes

Highlight research challenges


Finding adjectives for nouns embedded inside noun phrases

23

Use dependencies to identify the chunk-head and find an adjective for it

… handy license management tools ….





Agenda

24

Motivation

Data

Method

Results



Statistical evaluation using cross-entropy

Can approach target language models

Crowdsourced evaluation through Amazon Mechanical Turk

Machine transformed messages show syntactic and semantic coherence

Evaluation



Representative transformation

26

Ad Skeleton Transformation 1 Transformation 2

Even as the landscapecontinues to change, MAXwill remain the place tolearn about generatinggraphics content fordevices, and discoveringabout tools, developmentapproaches, and formats.Learn about licensemanagement tools and allthe things you wanted toknow!

Even as the landscape continuesto change dramatically, MAX willalways remain the first place tolearn about generating originalgraphics content for mobiledevices, and discovering abouttools, unique developmentapproaches, and formats. Learnabout valuable licensemanagement tools and all thegreatest things you wanted toknow!

(Segment: US-student)

Even as the landscapecontinues to change daily,MAX will remain the firstplace to quickly learn aboutgenerating adaptive graphicscontent for devices, anddiscovering about tools,critical developmentapproaches, and formats.Learn about handy licensemanagement tools and all thebest things you wanted toknow right!

(Segment: AU-designer)



Experimental results on cross-entropy

27

Model Unigram (Sentence) Unigram (Frequency) Bigram (Sentence) Bigram (Frequency)

SegmentAds with

dropMean %

drop in CEAds with

dropMean % drop

in CEAds with

dropMean %

drop in CEAds with

dropMean %

drop in CE

AU-designer 60/60 11.65 60/60 10.82 60/60 10.17 60/60 46.08

AU-developer 60/60 11.58 60/60 10.74 60/60 10.09 60/60 46.04

AU-manager 60/60 11.10 60/60 10.34 60/60 9.64 60/60 45.81

AU-student 60/60 11.02 60/60 10.28 60/60 9.68 60/60 45.76

UK-designer 60/60 13.71 60/60 12.89 60/60 12.00 60/60 47.34

UK-developer 60/60 12.65 60/60 11.77 60/60 11.10 60/60 46.66

UK-manager 60/60 12.39 60/60 11.64 60/60 10.90 60/60 46.59

UK-student 60/60 10.63 60/60 9.86 60/60 9.27 60/60 45.51

US-designer 60/60 11.98 60/60 11.21 60/60 10.57 60/60 46.32

US-developer 60/60 12.27 60/60 11.43 60/60 10.71 60/60 46.44

US-manager 60/60 12.18 60/60 11.27 60/60 10.58 60/60 46.36

US-student 60/60 11.05 60/60 10.23 60/60 9.61 60/60 45.73


Presenter

Presentation Notes

Call out for Analytics integration


Presented human assessors with triplets of ad

messages

Human composed message hidden among two

computer modified ones

Evaluator asked to identify human message and

score the other two

Computer modified messages get comparable

ratings as human compositions

Experimental results through crowdsourcing

28

Segment Average RatingHuman 3.91AU-designer 3.54AU-developer 3.71AU-manager 3.50AU-student 3.57UK-designer 3.50UK-developer 3.69UK-manager 3.55UK-student 3.44US-designer 3.50US-developer 3.56US-manager 3.58US-student 3.54


Presenter

Presentation Notes

We want to complement human and machine skills… mundane effort reduced


Improved techniques like synonym, hypernym, hyponym analysis

Controlled insertions and substitutions

Beyond parts of speech – Automatic transformations for:

Formal and informal styles

Statements, questions, interjections

Sentence lengths

Next steps


Presenter

Presentation Notes

Hypothesis: Several news categories may exhibit similarity in content and structure Average words, sentences in an article To a common man, news categories – sports, business, politics. But news agencies store at a deeper granularity, and in these categories you can find patterns.


Not too far away?

30

Dear Ms. Navarro Hi Gabriella

Enclosed please find Just sending along

was requested you asked for

Appropriate recommendation The right one

Additional information Know more

Contact me Just give me a ring

Most sincerely yours Take care

Samuel G. Berenz Sam


Presenter

Presentation Notes

Most impressed visually – those features are important


Questions?


Presenter

Presentation Notes

Customer data testing timelines Most impressed visually – those features are important

Automated Linguistic Personalization of Targeted Marketing...

Documents

Transcript of Automated Linguistic Personalization of Targeted Marketing...