Mobile apps as one possible solution - Shara Evans, Market Clarity

Shara Evans CEO, Market Clarity Sydney, Australia (02) 9043-9100 www.marketclarity.com.au

Australian Mobile BB Network Performance

Mobile apps as one possible way to provide consumer information

ACMA Mobile Network Performance Forum 14 November 2013

(ACMA Website Version)

Australia’s Source for Telecommunications Intelligence

© Market Clarity 2013 – Slide 2 Australia’s Source for Telecommunications Intelligence

Agenda

  How do consumers find out about network performance (coverage, dropouts and slow data speed)?   Carrier maps

  Mobile speed test apps

  How are Government organisations using Mobile BB apps?   Mini Case study – US Broadband Mapping initiative

  Implications for Australia


Carrier Maps: Telstra

Source: http://www.telstra.com.au/mobile-phones/coverage-networks/our-coverage/ (Nov 2013)


Carrier Maps: Optus

Source: https://www.optus.com.au/network/mobile/coverage (Nov 2013)


Carrier Maps: Vodafone

Source: http://www.vodafone.com.au/aboutvodafone/network/checker (Nov 2013)


Australian Coverage Maps

Different types of maps make it difficult

for consumers to directly compare

carrier coverage and expected broadband

speeds

Image source: http://askjune.military.com/2013/09/roth-ira-or-roth-tsp.html


Measuring Mobile BB Speed

Image source: www.smarta.com/blog/2012/6/broadband-the-need-for-speed


Classifying Mobile BB Speeds

Source: Mobile Pulse (www.mobilepulse.com) + FCC Definition of broadband, as cited by Western Telecommunications Alliance to US Dept of Commerce, NTIA, American Recovery

and Reinvestment Act of 2009 submission (April 2009)

FCC Definition of Broadband Mobile Pulse Speed Grades


Apps Used in US BB Mapping: Manual Testing, Crowd Sourcing


More Apps


Mobile BB App Considerations

Automated versus manual

speed test

Data Usage

Monitoring

Battery Level

Monitoring

Metrics: - Up/Down

Speed - Latency

- Delay variation -  Packet loss

- Signal Strength

Problem Reporting: - Carrier - Other

Automatic or manual?

Mapping (in App or

Web Portal)

Cell, WiFi

Privacy


Mobile Apps: OS Support


Mobile Apps: Metrics

Note: Other metrics may be collected, but not shown in the app


Mobile Apps: Design Parameters


Mobile Apps: Measurement Testing Methodology – Varies by App

  Download protocol: typically HTTP over TCP

  Download/Upload file size + test length (impact on consumer mobile BB plans)

  Number of download + upload threads (depends on type of application being emulated)

  Methodology for latency measurement

  Discarded data (can skew representation of consumer experience)

  Selection of speed test server

  Server availability

  Methodology to initiate test (manual or automatic)

  Compliance with work being done in IEEE 802.16.3 Working Group on Mobile Broadband Network Performance Measurements

Some testing considerations:


The US BB Mapping Initiative

Source: www.broadbandmap.gov


The US Mobile BB Mapping Initiative



The US Mobile BB Mapping Initiative (2)



The US Mobile BB Mapping Initiative (3)



Utah 6000 Miles, 6 Carriers (2011)

Source: http://broadband.utah.gov/2011/10/13/utah-mobile-broadband-drive-test-data-available-for-download/


California: Custom Designed Android App, Drive Testing + Crowd Sourcing

Source: http://www.cpuc.ca.gov/PUC/Telco/Information+for+providing+service/Broadband+Availability+Maps.htm and http://www.broadbandmap.ca.gov/v2.1/

1,200 locations, 35,000 miles

Example Map – Average Download

Speed


Delaware: Carrier Shape Files, Excel Data, Field Verification, Speed Tests

Sources: http://www.broadband.delaware.gov/ http://www.delawarespeedtest.com/ http://www.broadband.delaware.gov/de_mobile/default.shtml


Some States + Local Govt Use Automated Crowd Source Testing: Mobile Pulse

State of Idaho – Using Mobile Pulse app on public safety and residents’ smartphones to find coverage holes

State of Nebraska – Using residents and state employees to run the Mobile Pulse national testing app for mobile wireless validation

State of Wisconsin - Through broadband mapping vendor, deploying Mobile Pulse for validation of cellular vendor maps and school wireless testing

City of Colorado Springs– Deployed on Toughbooks, iPhones and Androids to select new carrier and negotiate service improvements

Colorado Statewide Internet Portal Authority– Deployed 1600 advanced testing apps on 11 local governments across the state

State of Arizona – Using Mobile Pulse to complete coverage map and validate carrier networks

State of Alabama – Deploying to smartphones to monitor and measure different wireless networks

Source: www.mobilepulse.com


Colorado: Drive Testing, Web Speed Test + Crowd Sourcing

Source: www.mobilepulse.com

Very large data sets


The US Mobile BB Mapping Initiative: Reported v Actual Speeds



Implications for Australia

Image source: http://www.theage.com.au/national/education/the-voice/is-it-possible-to-sustain-a-healthy-society-and-a-healthy-planet-by-2050-20130903-2t2y4.html/


Concluding Thoughts

•  The US Govt provided grant funding for the States + Territories to collect and map fixed and mobile broadband performance data

•  A wide range of methodologies were used for this purpose – with the results stitched together into a national coverage map

•  Carriers are cooperating in the provision of data (GIS, Excel, other) – usually showing theoretical speeds, rather than real world averages

•  A number of States are using road testing, data purchasing and crowd source methodologies to correlate + map out real-world coverage in their area

•  Could a similar program work in Australia? I think it could…

•  Consumer privacy issues are important…

•  Consumer data usage issues are also important

•  Automated periodic testing at regular intervals (rather than manual speed tests) provides results that are consistent with a typical consumer experience

•  Would carriers contribute data? To the extent that this data is already published…

•  A number of methodology issues remain unanswered – for instance, how do you represent data sets with variable results? One way is to have large samples, and display averages. The larger the sample, the more accurately it will reflect a typical consumer experience.


Further info is available at

http://www.marketclarity.com.au

Mobile apps as one possible solution - Shara Evans, Market Clarity

Technology

Transcript of Mobile apps as one possible solution - Shara Evans, Market Clarity