2012 ConvergeSE: Exploring Human Identity Through Personalization and Data Mining

50
Through Personalization and Data Mining Exploring Human Identity Jonathan LeBlanc Developer Evangelist: X.commerce Twitter: @jcleblanc E-Mail: [email protected] Github: github.com/jcleblanc

description

This workshop dives into the depths of how a person can be identified online, allowing for the delivery of highly personalized experiences. This will cover a few topics, including:Building the identity foundation using authentication systems like BrowserID, PayPal Access and Facebook Connect.Understanding how cultural identity concepts like tribalism play into how people group themselves innately online.Building personality and interest profiles for users by tracking actions using keyword density scraping and categorization.

Transcript of 2012 ConvergeSE: Exploring Human Identity Through Personalization and Data Mining

Page 1: 2012 ConvergeSE: Exploring Human Identity Through Personalization and Data Mining

Through Personalization and Data Mining

Exploring Human Identity

Jonathan LeBlancDeveloper Evangelist: X.commerce

Twitter: @jcleblancE-Mail: [email protected]

Github: github.com/jcleblanc

Page 2: 2012 ConvergeSE: Exploring Human Identity Through Personalization and Data Mining

http://www.x.com http://slideshare.net/jcleblanc

The Foundations of Human Identity

Tribalism and Social Grouping

Experimental Identity Methods

The Big Bag of Social Identity Fail

What We’re Going to Cover

Page 3: 2012 ConvergeSE: Exploring Human Identity Through Personalization and Data Mining

http://www.x.com http://slideshare.net/jcleblanc

The Foundations of Human Identity

Tribalism and Social Grouping

Experimental Identity Methods

The Big Bag of Social Identity Fail

What We’re Going to Cover

Page 4: 2012 ConvergeSE: Exploring Human Identity Through Personalization and Data Mining

http://www.x.com http://slideshare.net/jcleblanc

Human Identity: User Types

Anonymous Users Registered Users

Page 5: 2012 ConvergeSE: Exploring Human Identity Through Personalization and Data Mining

http://www.x.com http://slideshare.net/jcleblanc

OpenID (…and the upcoming OpenID Connect)PayPal Access, Google, Yahoo!

OAuth (1.0a + 2.0)PayPal Access, Facebook, Twitter

BrowserIDMozilla

Human Identity: Open Identity Programming

Page 6: 2012 ConvergeSE: Exploring Human Identity Through Personalization and Data Mining

http://www.x.com http://slideshare.net/jcleblanc

Human Identity: Anonymous Users

Page 7: 2012 ConvergeSE: Exploring Human Identity Through Personalization and Data Mining

http://www.x.com http://slideshare.net/jcleblanc

There are a few common options

Human Identity: Tracking Anonymous Users

Tracking Cookie Local Storage

Page 8: 2012 ConvergeSE: Exploring Human Identity Through Personalization and Data Mining

http://www.x.com http://slideshare.net/jcleblanc

• On each page visited, track the URL

• HTML5 Local Storage as primary storage

• Cookies as secondary storage

Human Identity: Tracking Anonymous Users

Program Overview

Page 9: 2012 ConvergeSE: Exploring Human Identity Through Personalization and Data Mining

Tracking Anonymous Users with Local Storage

var storeName = "visited";if (typeof(localStorage) == 'undefined' ) { //Local Storage Not Available} else { try { var sites = localStorage.getItem(storeName); sites = (sites === null) ? window.location : sites + window.location; localStorage.setItem(storeName, sites + "|"); } catch (e) { if (e == QUOTA_EXCEEDED_ERR) { //quota exceeded } }}

Page 10: 2012 ConvergeSE: Exploring Human Identity Through Personalization and Data Mining

Tracking Anonymous Users with Cookies

function readCookie(name) { var nameEQ = name + "="; var ca = document.cookie.split(';'); for (var i = 0; i < ca.length; i++) { var c = ca[i]; while (c.charAt(0) == ' '){ c = c.substring(1, c.length) }; if (c.indexOf(nameEQ) == 0){ return c.substring(nameEQ.length, c.length); } } return null;}

Page 11: 2012 ConvergeSE: Exploring Human Identity Through Personalization and Data Mining

var storeName = "visited";if (typeof(localStorage) == "undefined" ) { var cookieVal = readCookie(storeName); var value = ((cookieVal === null) ? window.location : cookieVal + window.location); var days = 1; var date = new Date(); date.setTime(date.getTime() + (days*24*60*60*1000)); var expires = "; expires=" + date.toGMTString(); document.cookie = storeName + "=" + value + "|" + expires + "; path=/";} else { //Use Local Storage}

Tracking Anonymous Users with Cookies

Page 12: 2012 ConvergeSE: Exploring Human Identity Through Personalization and Data Mining

http://www.x.com http://slideshare.net/jcleblanc

• Remove oldest results when storage fills

• Build categorization mapping prior to storage to save space (more on this later)

Human Identity: Tracking Anonymous Users

Next Steps / Improvements

Page 13: 2012 ConvergeSE: Exploring Human Identity Through Personalization and Data Mining

http://www.x.com http://slideshare.net/jcleblanc

Human Identity: Registered Users

Page 14: 2012 ConvergeSE: Exploring Human Identity Through Personalization and Data Mining

http://www.x.com http://slideshare.net/jcleblanc

Human Identity: Identity Sources

Social (perceived) Concrete (true)

Sources of Real Identity

Page 15: 2012 ConvergeSE: Exploring Human Identity Through Personalization and Data Mining

http://www.x.com http://slideshare.net/jcleblanc

Human Identity: Concrete Identity

Page 16: 2012 ConvergeSE: Exploring Human Identity Through Personalization and Data Mining

http://www.x.com http://slideshare.net/jcleblanc

PayPal Access: OAuth 2 + Commerce

Seamless Checkout

Prospect Scores

Recommendations

Page 17: 2012 ConvergeSE: Exploring Human Identity Through Personalization and Data Mining

PayPal Access: The Common Code

<?phpdefine('KEY', 'YOUR APPLICATION ID');define('SECRET', 'YOUR APPLICATION SECRET');

define('CALLBACK_URL', 'YOUR CALLBACK PATH - TO COMPLETE.PHP');define('AUTH_ENDPOINT', 'https://identity.x.com/xidentity/resources/authorize');define('TOKEN_ENDPOINT', 'https://identity.x.com/xidentity/oauthtokenservice');define('USER_ENDPOINT', 'https://identity.x.com/xidentity/resources/profile/me');

function run_curl($url, $method = 'GET', $postvals = null){ ... }?>

Page 18: 2012 ConvergeSE: Exploring Human Identity Through Personalization and Data Mining

PayPal Access: Forwarding for Login

<?phprequire_once "common.php";

$auth_url = sprintf( "%s?scope=%s&response_type=code&redirect_uri=%s&client_id=%s", AUTHORIZATION_ENDPOINT, urlencode("https://identity.x.com/xidentity/resources/profile/me"), urlencode(CALLBACK_URL), KEY);

//forward user to PayPal auth pageheader("Location: $auth_url");?>

Page 19: 2012 ConvergeSE: Exploring Human Identity Through Personalization and Data Mining

PayPal Access: Obtaining the Access Token

<?phprequire_once "common.php";

//capture code from auth$code = $_GET["code"];

//construct POST object for access token fetch request$postvals = sprintf("client_id=%s&client_secret=%s&grant_type=authorization_code&code=%s&redirect_uri=%s", KEY, SECRET, $code, urlencode(CALLBACK_URL));

//get JSON access token object$token = json_decode(run_curl(ACCESS_TOKEN_ENDPOINT, 'POST', $postvals));

Page 20: 2012 ConvergeSE: Exploring Human Identity Through Personalization and Data Mining

PayPal Access: Using the Access Token

//construct URI to fetch profile information for current user$profile_url = sprintf("%s?oauth_token=%s", PROFILE_ENDPOINT, $token->access_token);

//fetch profile of current user$profile = run_curl($profile_url);

var_dump($profile);?>

Page 21: 2012 ConvergeSE: Exploring Human Identity Through Personalization and Data Mining

http://www.x.com http://slideshare.net/jcleblanc

PayPal Access: Using the Raw Data

Page 22: 2012 ConvergeSE: Exploring Human Identity Through Personalization and Data Mining

http://www.x.com http://slideshare.net/jcleblanc

PayPal Access: Using the Raw Data

Page 23: 2012 ConvergeSE: Exploring Human Identity Through Personalization and Data Mining

http://www.x.com http://slideshare.net/jcleblanc

The Foundations of Human Identity

Tribalism and Social Grouping

Experimental Identity Methods

The Big Bag of Social Identity Fail

What We’re Going to Cover

Page 24: 2012 ConvergeSE: Exploring Human Identity Through Personalization and Data Mining

http://www.x.com http://slideshare.net/jcleblanc

Social Grouping: It’s Not A New Thing…

Page 25: 2012 ConvergeSE: Exploring Human Identity Through Personalization and Data Mining

http://www.x.com http://slideshare.net/jcleblanc

Tribalism started as a way to keep us safe

…it has lead to some horrible parts of history

but it is also a foundation of many of our social relationships

Social Grouping: Foundation in Tribalism

Page 26: 2012 ConvergeSE: Exploring Human Identity Through Personalization and Data Mining

http://www.x.com http://slideshare.net/jcleblanc

Social Grouping: The Real Life Social Graph

Page 27: 2012 ConvergeSE: Exploring Human Identity Through Personalization and Data Mining

http://www.x.com http://slideshare.net/jcleblanc

Social Grouping: The Online Social Graph

Page 28: 2012 ConvergeSE: Exploring Human Identity Through Personalization and Data Mining

http://www.x.com http://slideshare.net/jcleblanc

Social Grouping: Group Types

Follower Type

Connection Type

Group Type

Page 29: 2012 ConvergeSE: Exploring Human Identity Through Personalization and Data Mining

http://www.x.com http://slideshare.net/jcleblanc

Social Grouping: Data Miners are Rock Stars

Page 30: 2012 ConvergeSE: Exploring Human Identity Through Personalization and Data Mining

http://www.x.com http://slideshare.net/jcleblanc

• Use all URLs from the previous program.

• Obtain content category for page.

• Categorize user interest.

Social Grouping: Group Programming Primer

Program Overview

Page 31: 2012 ConvergeSE: Exploring Human Identity Through Personalization and Data Mining

http://www.x.com http://slideshare.net/jcleblanc

Social Grouping: Group Programming Primer

Step 1: Obtain Website Content

Page 32: 2012 ConvergeSE: Exploring Human Identity Through Personalization and Data Mining

http://www.x.com http://slideshare.net/jcleblanc

Social Grouping: Group Programming Primer

Step 2: Perform Keyword Density Search

Page 33: 2012 ConvergeSE: Exploring Human Identity Through Personalization and Data Mining

http://www.x.com http://slideshare.net/jcleblanc

Social Grouping: Group Programming Primer

Step 3: Weight Keywords

Page 34: 2012 ConvergeSE: Exploring Human Identity Through Personalization and Data Mining

http://www.x.com http://slideshare.net/jcleblanc

The Foundations of Human Identity

Tribalism and Social Grouping

Experimental Identity Methods

The Big Bag of Social Identity Fail

What We’re Going to Cover

Page 35: 2012 ConvergeSE: Exploring Human Identity Through Personalization and Data Mining

http://www.x.com http://slideshare.net/jcleblanc

Experimental Identity: WebFinger

Page 36: 2012 ConvergeSE: Exploring Human Identity Through Personalization and Data Mining

http://www.x.com http://slideshare.net/jcleblanc

Step 1: Perform Discovery

curl https://gmail.com/.well-known/host-meta

Experimental Identity: WebFinger

Page 37: 2012 ConvergeSE: Exploring Human Identity Through Personalization and Data Mining

http://www.x.com http://slideshare.net/jcleblanc

Experimental Identity: WebFinger

<XRD xmlns='http://docs.oasis.open.org/ns/xri/xrd-1.0' xmlns:hm='http://host-meta.net/xrd/1.0'> <hm:Host xmlns='http://host-meta.net/xrd/1.0'>gmail.com </hm:Host> <Link rel='lrdd' template='http://www.google.com/s2/webfinger/?q={uri}'> <Title>Resource Descriptor</Title> </Link></XRD>

Page 38: 2012 ConvergeSE: Exploring Human Identity Through Personalization and Data Mining

http://www.x.com http://slideshare.net/jcleblanc

Step 2: Collect User Data

curl http://www.google.com/s2/webfinger/[email protected]

Experimental Identity: WebFinger

Page 39: 2012 ConvergeSE: Exploring Human Identity Through Personalization and Data Mining

http://www.x.com http://slideshare.net/jcleblanc

User Profilehttp://www.google.com/profiles/nakedtechnologist

Portable Contactshttp://www-opensocial.googleusercontent.com/api/people/118167121283215553793/

Experimental Identity: WebFinger

Page 40: 2012 ConvergeSE: Exploring Human Identity Through Personalization and Data Mining

http://www.x.com http://slideshare.net/jcleblanc

profileUrlidthumbnail urlurlsphotos

Experimental Identity: WebFinger

name formatted family name given name display name

Page 41: 2012 ConvergeSE: Exploring Human Identity Through Personalization and Data Mining

http://www.x.com http://slideshare.net/jcleblanc

Experimental Identity: BrowserID

Page 42: 2012 ConvergeSE: Exploring Human Identity Through Personalization and Data Mining

http://www.x.com http://slideshare.net/jcleblanc

BrowserID Source<script src="https://browserid.org/include.js" type="text/javascript"></script>

JQuery Source<script src="http://code.jquery.com/jquery.min.js" type="text/javascript"></script>

Experimental Identity: BrowserID

Page 43: 2012 ConvergeSE: Exploring Human Identity Through Personalization and Data Mining

http://www.x.com http://slideshare.net/jcleblanc

Experimental Identity: BrowserID

navigator.id.get(function(assertion) { if (assertion) { $.ajax({ url: 'https://browserid.org/verify', type: 'POST', data: 'assertion='+assertion+'&audience=jcleblanc.com', success: function(res) { console.log(res); } });});

Page 44: 2012 ConvergeSE: Exploring Human Identity Through Personalization and Data Mining

http://www.x.com http://slideshare.net/jcleblanc

{ audience: "jcleblanc.com", email: "[email protected]", expires: 1320081400987, issuer: "browserid.org", status: "okay"}

Experimental Identity: BrowserID Results

Page 45: 2012 ConvergeSE: Exploring Human Identity Through Personalization and Data Mining

http://www.x.com http://slideshare.net/jcleblanc

The Foundations of Human Identity

Tribalism and Social Grouping

Experimental Identity Methods

The Big Bag of Social Identity Fail

What We’re Going to Cover

Page 46: 2012 ConvergeSE: Exploring Human Identity Through Personalization and Data Mining

http://www.x.com http://slideshare.net/jcleblanc

“My privacy concerns are not trite. They are linked to my actual physical safety” --Harriet Jacobs (Gizmodo)

Social Identity Fail: Personal Safety

When Social Discovery Impacts Personal Safety

Page 47: 2012 ConvergeSE: Exploring Human Identity Through Personalization and Data Mining

http://www.x.com http://slideshare.net/jcleblanc

“Path Uploads Your Entire iPhone Contact List By Default” --Mark Hachman (PCMag)

Social Identity Fail: Privacy Concerns

When Making Things Easy Impairs Privacy

Page 48: 2012 ConvergeSE: Exploring Human Identity Through Personalization and Data Mining

http://www.x.com http://slideshare.net/jcleblanc

“How Target Figured Out A Teen Girl Was Pregnant Before Her Father Did” --Kashmir Hill (Forbes)

Social Identity Fail: The Fine Line

The Fine Line Between Insightful and Creepy

Page 49: 2012 ConvergeSE: Exploring Human Identity Through Personalization and Data Mining

http://www.x.com http://slideshare.net/jcleblanc

Identity is more than just a login

Have a social conscience

Find the tool that:– Has the raw data that you need– Works with your business

Identity Programming Core Concepts

Page 50: 2012 ConvergeSE: Exploring Human Identity Through Personalization and Data Mining

http://slidesha.re/convergese_id

Thanks! Any Questions?

Jonathan LeBlancDeveloper Evangelist: X.commerce

Twitter: @jcleblancE-Mail: [email protected]

Github: github.com/jcleblanc