Making Email a Useful Web App

Web 2.0 ConferenceApril 23, 2008San Francisco

Andy DenmarkTripIt, Co-Founder and VP Engineering

Email Bankruptcy

“I routinely declare email bankruptcy and simply delete my entire inbox.”

- Michael Arrington, TechCrunch“Wondering if I can get a Bear Sterns bailout for my email bankruptcy.”

- willpate on Twitter

“(He) just declared email bankruptcy. I've read those declarations enough to know it means our email tools aren't usable.”

- garrickvanburen on Twitter

So, why is email still around despite all the challenges and criticism…

?

Email Is Still The “Killer App”

Email is a natural extension of a metaphor for communication that humans have used forever

Everyone has email and the overwhelming majority of all person to person Internet communication takes place there

No matter what else people use to communicate, they all use email too

Email contains an amazing amount of context and data about our lives Our relationships

Our purchases

Our thoughts and the thoughts of people we know

What More Can We Do With Email?

Spam and mass email marketing may be contributing to the problems outlined earlier

But think of all the things we can do to make our applications better by making better use of email: Broadcasting data

Query interfaces

Data transfer

Intelligent parsing of data

Mining of social connections (“Inbox 2.0”)

Does An Email App Make Sense?

PROS Ubiquity – everyone has

email Exposure – email is on all day

and captures a lot of attention

Simplicity – easy to send email, everyone knows how to do it

Embedded identity

CONS May just not fit your service May be too complex for your

users to understand It’s challenging to properly

support all email clients, versions and formats

Building and supporting a system that sends, receives, and reads email is a pain

Prehistoric Email Apps

Domain Registration Email Sign-Up Email to:

hostmaster@internic.net From address

authentication for new sign-ups

Choose to receive your bill via email or USPS

Authentication schemes for modify/delete were CRYPT-PW and PGP

Archie File Query Email InterfaceAvailable Interfaces Text-based client Graphical client Telnet client Email

Browsing The Web Via Email

“For personal reasons, I do not browse the web from my computer. (I also have no net connection much of the time.) To look at page I send mail to a demon which runs wget and mails the page back to me. It is very efficient use of my time, but it is slow in real time.”

-- Richard Stallman (2007-12-15)

About TripIt

?

***CONFIDENTIAL*** 13

Forward Any Confirmation

Email

Automated Online Master

Itinerary

Automated Weather, Maps,

Guides…

Trip Sharing & Social Alerts

iCal & Atom Support

Email & Mobile Access

Add Meetings, Activities,

Restaurants, etc.

Why Email Made Sense For TripIt

The native format of travel data is varied and unfriendly

The travel industry isn’t moving to expose that data Partnering with big travel companies and building

APIs would take too long Users already have all their travel data in email, but

that data is “dead” Auto-account creation using the embedded identity

of an email message It has become an “instant delight” factor for users

Types Of Email App Services

Command Line Interface

DataTransfer

SemanticParsing

Broadcast

?

Types Of Email App Services

Command Line Interface

DataTransfer

SemanticParsing

Broadcast

One Way

• User requests email information to be sent to them

• Typically triggered by an action or an alert

Examples

• Mailing Lists

• Google Alerts

Example: Mailing Lists

Searchable knowledge base built as a by-product of natural human communication

Example: Google Alerts “Real time” alerts

delivered right to my desktop

Useful way to asynchronously broadcast data I’ve requested as it’s available

With linkage to the web, alerts are actionable and controllable

Types Of Email App Services

Command Line Interface

DataTransfer

SemanticParsing

Broadcast

CLI

• Users emails a command and Email App replies with a response

• Typically a standard list of commands and responses

Examples

• I Want Sandy

• TripIt

• Twitter

Example: I Want Sandy

http://iwantsandy.com/

Example: TripIt

Example: Twitter

Messaging platform with multiple UIs Web (http://twitter.com/)

SMS (40404)

IM (Jabber: twitter@twitter.com)

Facebook (http://www.facebook.com/)

FriendFeed (http://friendfeed.com/)

Desktop (e.g. Twhirl)

Email

I can tweet by email?!? Yes, it’s just another way to use the messaging platform’s CLI

Very natural, ubiquitous and free (no SMS charges)!

See: http://emailtwitter.com/ and http://twittermail.com/

Types Of Email App Services

Command Line Interface

DataTransfer

SemanticParsing

Email Forwarding

• Users forward emails and Email App recognizes and parses them

• Typically converts “unstructured” data into “structured” data making it more usable to the app and user

Examples

• Flickr

• Blogger

Broadcast

Example: Flickr

Example: Blogger

Types Of Email App Services

Command Line Interface

DataTransfer

SemanticParsing

Broadcast

Intelligent Agent

• Structured data from email is enriched by the Email App by retrieving related information from other sites

• Typically leverages “keys” such as geo-coding to capture relevant data

Examples

• TrackMyShipments

• TripIt

Example: TrackMyShipments

Example: TripIt

Timezone

Online Checkin Flight Status Seat Advice

Flight DetailsWeather

Types Of Email App Services

Command Line Interface

DataTransfer

SemanticParsing

Email Forwarding

• Users forward emails and Email App recognizes and parses them

• Typically converts “unstructured” data into “structured” data making it more usable to the app and user

Examples

• Flickr

• Blogger

Broadcast

CLI

• Users emails a command and Email App replies with a response

• Typically a standard list of commands and responses

Examples

• I Want Sandy

• TripIt

• Twitter

One Way

• User requests email information to be sent to them

• Typically triggered by an action or an alert

Examples

• Listserv

• Google Alerts

Intelligent Agent

• Structured data from email is enriched by the Email App by retrieving related information from other sites

• Typically leverages “keys” such as geo-coding to capture relevant data

Examples

• TrackMyShipments

• TripIt

Caution: Email Is Full Of Problems

CAUTION

STANDARDSCAUTIONCAUTION

SPAMCAUTION

CAUTION

PRIVACYCAUTION

CAUTION

DELIVERABILITYCAUTION

Deliverability Issues

Your reputation is attached to your IPs Monitor your logs, it’s a dynamic system Work with the ESPs, they are as much

invested in this as you are Even if you're a good citizen, there are

no guarantees email will get through Tips for getting email delivered…

No Open Relays Proper DNS Configuration (forward + reverse

resolution) SPF/DKIM Privacy Policy RFC Compliance ESP “Whitelists” Bounce Handling

CAUTION

DELIVERABILITYCAUTION

Spam Issues

No silver bullet solutions, but a suite of counter-measures… Spam assessment tools Spam Firewalls (Hardware/Software) Improper/forged headers are key factor Non-RFC Compliant Mailers (don't retry, don't try

all your MXs, etc.) DHCP/home IPs w/ no reverse DNS Blacklists (be very careful)

But all of these cause you to run the risk of catching legitimate email in the net

Spammers like to go fast, so slowing things down in an RFC compliant way can help

CAUTION

SPAMCAUTION

Privacy Issues

People are very protective of their Inbox, it’s their private space

Laws vary wildly but there’s a baseline in common sense

Make sure people can opt-out/in easily Build mechanisms that prevent you from

contacting people who don’t wish to be contacted

Listen to your communities, you’ll get some of this wrong, but you can fix it

Be transparent and responsive

CAUTION

PRIVACYCAUTION

Standards Issues

There are no standards that are strictly followed across mail clients

Pretty much every mail-related RFC has been outright violated or creatively interpreted by at least one major mail client

White space, HTML markup, attachments are all mangled and re-interpreted depending on which mail client is working on the message

If you build a mail application that reads email you will have to deal with this, it’s expensive and painful

CAUTION

STANDARDSCAUTION

Key Take-aways

Email is not dead! Treasure-trove of data waiting to be mined

At least four different types of apps Broadcast, CLI, Data transfer, Semantic parsing

Remember the gotchas: Deliverability, Spam, Privacy, Standards

If you get it right, huge payoff for you and your users

Questions

Andy DenmarkTripIt, Co-Founder and VP Engineering

Appendix

ESP Whitelists/Resources Microsoft Hotmail/MSN:

http://postmaster.msn.com/Services.aspx https://postmaster.live.com/snds/ http://postmaster.live.com/Troubleshooting.aspx http://download.microsoft.com/download/e/3/3/e3397e7c-17a6-497d-9693-78f80be272fb/

enhance_deliver.pdf http://www.senderscorecertified.com/ by ReturnPath

Yahoo http://help.yahoo.com/l/us/yahoo/mail/postmaster/

Google http://www.google.com/mail/help/bulk_mail.html

AOL http://postmaster.aol.com/

Appendix cont’d

Mail Server Setup http://domainkeys.sourceforge.net/

http://www.openspf.org/

http://www.maawg.org/about/MAAWG_Sender_BCP/MAAWG_Senders_BCP_Combine.pdf

CLI Examples http://microformats.org/wiki/picoformats

Relevant RFCs http://rfc.net/rfc2821.html (SMTP)

http://rfc.net/rfc2822.html (Mail Headers)

http://rfc.net/rfc2045.html (MIME part 1)

http://rfc.net/rfc2046.html (MIME part 2)