Data is never accurate - and that's OK!
-
Upload
chanpreet-singh -
Category
Data & Analytics
-
view
106 -
download
0
Transcript of Data is never accurate - and that's OK!
“Errors using inadequate data are much less than those using no data at all”- Charles Babbage
The data is never accurate – and that’s OK!
By: Chanpreet Singh| twitter.com/chanpreets
Part 1: Why is data not accurate?
Part 2: How to ensure data sanity?
Part 3: How do we work with this kind of data?
May 1, 2023 Chanpreet 2
Tracking toolGoogle Analytics• Integrate an Analytics tool – like Google Analytics (GA)
May 1, 2023 Chanpreet 3
Part 1: Why is data not accurate?
May 1, 2023 Chanpreet 4
What is the meaning of Accurate
Towards care• (especially of information, measurements,
or predictions) correct in all details; exact.• “Done with care”• Digital measurement - ‘care’ unfortunately
is not enough
May 1, 2023 Chanpreet 5
Why would data not be AccurateFactors responsible• Digital Analytics programs do not
collect exact number of hits or queries• Designed to give ‘valuable insights’• Digital Analytics is not an Audit• Inherent uncertainties• Human nature
May 1, 2023 Chanpreet 6
Not designed to collect exact hits or queriesJS limitations
• Different layout engines render JS differently
May 1, 2023 Chanpreet 7
Layout Engine Browser
TridentInternet Explorer for WindowsMaxthonNetscape 8.1
Tasman Internet Explorer 5 for Macintosh
Gecko
FirefoxNetscape 6 and laterMozillaCaminoK-MeleonSeaMonkeyEpiphany 2.20 and beforeGaleon
KHTML KonquerorTkhtml Html Viewer
Layout Engine Browser
WebKit
SafariChromeiCab 4 and laterEpiphany 2.26 and laterMaxthon 3OmniWebShiiraMidori
Presto OperaiCab iCab 3 and beforeTkhtml Html Viewer
Not designed to collect exact hits or queriesCookies• Cookies
blocked or disabled by users
May 1, 2023 Chanpreet 8
Not designed to collect exact hits or queriesCookies• Cookie
deletion
May 1, 2023 Chanpreet 9
Not designed to collect exact hits or queries<noscript>• <noscript> tag
May 1, 2023 Chanpreet 10
Insights through % of data collectedDigital landscape
• Digital landscape is ever changing• Multiple platforms• Basic functional mobile phones• Networks, CDNs and other systems• Considerations of Outliers and Bots• Sample data to extrapolate performance
May 1, 2023 Chanpreet 11
Not an AuditAudit involves• Series of checks and balances• Accuracy• Standardized metrics and
methodology• Consistency of process• Transparency of results
May 1, 2023 Chanpreet 12
Inherent uncertaintiesUncertainties of Digital Analytics
• Algorithms• Models• Data itself• Outcomes
May 1, 2023 Chanpreet 13
Human NatureHuman factor at play• Humans think differently• Big organization, big teams, multiple functions• Cross functional/ team dependencies• Dependencies on partners• Lack of proper understanding
May 1, 2023 Chanpreet 14
Human NatureTracking Code• No tracking code
May 1, 2023 Chanpreet 15
Human NatureTracking Code• Exclusion/ removal of tracking code
May 1, 2023 Chanpreet 16
Human NatureHuman factor at play• Some users disable JavaScript
May 1, 2023 Chanpreet 17
Human NatureCommunication• Inter team communication gap
May 1, 2023 Chanpreet 18
Human NatureTeam conflict• Product and technology teams
May 1, 2023 Chanpreet 19
Human NatureUTM• Campaign tagging
May 1, 2023 Chanpreet 20
http://www.visitpa.com/pa-hotels-motels?utm_campaign=EVG_PA_Primary_Lodging&utm_source=google&utm_medium=cpc&utm_term=places-to-stay&utm_content=family-ad
Human NatureEvents• Event tagging
May 1, 2023 Chanpreet 21
Human NatureAccess• Access levels and profile view referred
May 1, 2023 Chanpreet 22
AccountPropertyView
Part 2: How to ensure data sanity?
May 1, 2023 Chanpreet 23
AwarenessBe Aware of differences• Discrepancies vs Differences• Different tools - different data• Sampling• Purpose of tools• Metrics used
May 1, 2023 Chanpreet 24
Different tools different data – GA vs AdWords Metrics usage and nature differs
• Clicks vs Sessions • AdWords filters invalid clicks, Analytics shows
all data• Landing page might redirect to another• Users browser preferences• Users return during the lifetime of campaign• Users return to your site via bookmarks
May 1, 2023 Chanpreet 25
Different tools different data – GA vs DFP Metrics usage and nature differs
• DFP counts ad clicks at the source; GA counts pageviews or sessions when a user hits the site
• Metrics are counted at different points in the click-referral cycle
May 1, 2023 Chanpreet 26
Different tools different data – Real time GA vs Another (Chartbeat)Common to use multiple tools• Chartbeat checks in for visitors every
few secs.• GA checks in once for visitors every
5 mins.• GA > Chartbeat = concurrent users
are high, but not staying long• GA < Chartbeat = most users stay
longer than 5 mins.
May 1, 2023 Chanpreet 27
Different tools different data – GA vs ComScore
Competition reporting• ComScore• Eliminates/ignores machine generated traffic• Measures people • Eliminates visits from multiple browsers,
multiple screens• Extrapolates numbers based on a panel users • 3 second rule
May 1, 2023 Chanpreet 28
Limitations within the tool – GA vs GA
GA Sampling• GA sampling and limitations• Standard vs Ad-Hoc reports• GA Standard vs Premium• High-cardinality dimensions• Report Query limit - 1 million rows of data• Data limits - single day/ multi day processed tables• Multi channel funnel report > 1 million conversions• Flow visualization report more than 100k sessions
May 1, 2023 Chanpreet 29
Part 3: How do we work with this kind of data?
May 1, 2023 Chanpreet 30
Best attempt to ensure better data Plan> Implement> configure> report> Interpret
• Coding• Admin Panel• Product• Interface usage and reporting• Interpretation
May 1, 2023 Chanpreet 31
Ensuring correct implementation - CodeCoding• Updated version (UA)• Correct UAID• Correct domain/ multi domain coding• Ensure code is not Commented out• Check for code Breakages• Customizations (session duration, session end)• Placement of code• Duplication of code• Conflict with another code (other tool), events
May 1, 2023 Chanpreet 32
Ensuring correct implementation - Interface
Admin Panel• Properties and Profile creation• Correct filters• Limitations/ Free_vs_Paid• Custom Dimensions & Metrics• Goals• Cross tool integrations/ agreements
(change)
May 1, 2023 Chanpreet 33
Ensuring correct implementation - Product
Product Sanity (Website/ Mobile Apps)• URL/ Screen pattern and duplications• UTM tagging• Conflict with tech and tools• Version control• Audience platforms - Desktop vs Mobile• Cache management/ deletion/
exclusions• Bots/ malware• Geo location of audience and network• FirewallsMay 1, 2023 Chanpreet 34
Ensuring correct ReportingInterface usage and reporting• Report/ data pull/ interpretation• Sampling• Time period/ timelines• Data influencing Events
• Data Cleansing - Incomplete/ inaccurate data• Interpretations Differ • Blind to our own biases• Critical points that might influence another outcome
• Working with Limitations
May 1, 2023 Chanpreet 35
Same data – different interpretation
Interpretation matters• Same data can yield wildly different
results• Redundant data stored in multiple
systems and in different formats• Data transformation is also important
May 1, 2023 Chanpreet 36
FinallyDirectional Guidance
• Digital Analytics should be about providing directional guidance
May 1, 2023 Chanpreet 37
Thank youReach me twitter/@chanpreets or email: [email protected]
May 1, 2023 Chanpreet 38