EMA
description
Transcript of EMA
EMA
IT Arch Meeting, 5/2010
EMA’s History
• Pitched 10/2008(!) as lightweight split testing• Received positive interest from all verticals
except Redball• Designed several components, couple of
which were built• Shelved Q109, resources diverted to Graffiti
EMA Today
• Support running tests for customers that can do the site side instrumentation
• Use DW/Redball tracking fields• Supported 5 tests over the last year• Have interest from sites that can’t handle site
side (Findarticles, Gamespot)
EMA Components
TES(server)
Client Lib(js)
TMI(Tools)
DW
DW Tracking Convention
EVA(reporting)
40% development doneComponent Percent Complete Notes
DW Tracking Spec 100%How to instrument your site for EMA evaluation
EVA Beta 80% http://c13-pts-dev40.cnet.com:3293/ema/
TES Beta 60%, Untested
User Tools 20% - Dev Stopped Here
Started using Django
Client Lib 0% Still some open issues
EVA
• Runs significance test on tracked DW data• Requires manual setup• Needs tie-in to user tools (TMI)
TES
• Prototyped using POME (Rubics’ Python stack)• Takes affiliate_id, test_id• Decides – Whether the user is in the test– Which variant user should see
• Returns payload based on test variant
TMI and Client
• No significant development for either component
• Bulk of development is here• Also, a couple outstanding design issues:– Automatic set up/tear down for tests (SQL
generation, etc.)– DW Cookie synchronization
Deployment
• Finish TMI and Client• Solve remaining design/integ issues• Documentation• Pilot partner (FA?)