Roaming Dynamics and Implications in the Cellular Ecosystem · Why Study International Roaming?...
Transcript of Roaming Dynamics and Implications in the Cellular Ecosystem · Why Study International Roaming?...
research
Roaming Dynamics and Implications in the Cellular
Ecosystem
Andra Lutu, Telefonica Research
whoami• Associate Researcher at Telefónica Research since
March 2018• Based in Barcelona, Spain• Diverse group working on networks, security and
privacy, ML/DL and HCI• Network Measurements (focus on interactions between
entities in the mobile ecosystem), Testbeds for Mobile Networks, Inter-domain routing, ML/AI for Networks
• H2020 Marie Curie Individual Fellowship (DICE: Dynamic Interconnections for the Cellular Ecosytem)
• Postdoctoral Fellow at Simula Research Laboratory from 2015 to 2018• Building the MONROE platform
• Graduated PhD in November 2014 from IMDEA Networks Institute and UC3M
2
Illustrations from @butlikemaybehttp://www.butlikemaybe.com/
whoamiDiego Perino (Telefonica Research)Alessandro Finamore (Telefonica UK) Anna Brunstrom (Karlstad Universitet)Özgu Alay (Simula Metropolitan)Cristel Pelsser (University of Strasbourg) Marcelo Bagnulo (University Carlos III of Madrid)Anna Maria Mandalari (Imperial College London)Foivos Michelinakis (Simula Metropolitan)Gorry Fairhurst (University of Aberdeen)Ana Custura (University of Aberdeen)Ali Safari Khatouni (Politecnico di Torino)Marco Mellia (Politecnico di Torino)Vaibhav Bajpai (Technische Universität München)Jörg Ott (Technische Universität München)Mohammad Rajiullah (Karlstad Universitet)Hossein Doroud (IMDEA Networks)Narseo Vallina-Rodriguez (ICSI/IMDEA Networks)Joerg Widmer (IMDEA Networks)Abbas Razaghpanah (Stony Brook University)Phillipa Gill (UMass Amherst)Fabian Bustamante (Northwestern University)Byungjin Jun (Northwestern University)
..and many others…
3
today• Motivation: Why study roaming?• Background: Roaming in the cellular ecosystem• The bigger picture: Roaming of Things• Methodology: How can we measure the
implications of roaming?• Results: Performance, content implications and
quality of experience• Reproducibility• What’s next?
4
today• Motivation: Why study roaming?• Background: Roaming in the cellular ecosystem• The bigger picture: Roaming of Things• Methodology: How can we measure the
implications of roaming?• Results: Performance, content implications and
quality of experience• Reproducibility• What’s next?
5
Roaming in the Cellular Ecosystem• Cellular networks have the built-
in feature of supporting user mobility across operators (national or international roaming)• International Roaming is driven
by new regulation (Roam like at Home in the EU), the increased mobility of users and new technologies (e.g., VoLTE) and other industry trends (IoT, M2M)
6
Why Study International Roaming? • Understand the operational reality, the business of roaming and the
ecosystem around it• Which are the technical solutions to enable international roaming that are
actually being used today?• How does roaming impact the performance and the quality of experience
for end users?
7
today• Motivation: Why study roaming?• Background: Roaming in the cellular ecosystem• The bigger picture: Roaming of Things• Methodology: How can we measure the
implications of roaming?• Results: Performance, content implications and
quality of experience• Reproducibility• What’s next?
8
The roaming interconnection ecosystem
Interconnection Network
IPX Provider 2
IPX Provider 1
Mobile Network
Operator 2
Mobile Network
Operator 1
Mobile Network
Operator 3
Fixed Network Operators
Content Provider
Internet Service
Providers
Enterprise and Financial Services
Application Service
Providers 9
The business of roaming
• Each operator has a contract with an IPX provider (for the roaming hub function)• Roaming partners no longer need to have bilateral contracts with each other
• Operators pay a service charge to the roaming hub• Inter-operator tariff (IOT) between the Home Operator (HMNO) and Visited Operator (VMNO)
IPXProvider
VMNOIPXProvider
HMNO Roaming hub Roaming hub
Service ChargeService Charge
Inter-operator tariff
10
View from an Operational Mobile Network• UK MNO with large population of devices (~30mil) • How many are roaming?
11
Inbound Roamers: Things or People?
• Using the GSMA TAC database and the APN used, we classify the devices we identify as inbound roamers
12
Majority of IoT devices are inbound roamers!
Out of all inbound roamers, only 27% are people!
Where are these devices roaming from?
13
today• Motivation: Why study roaming?• Background: Roaming in the cellular ecosystem• The bigger picture: Roaming of Things• Methodology: How can we measure the
implications of roaming?• Results: Performance, content implications and
quality of experience• Reproducibility• What’s next?
14
Why are so many things roaming?
• International Carriers (e.g., Telefónica, Orange, Syniverse, Tata Communications) leverage their extensive infrastructure and deploy M2M/IoT platforms to support IoT verticals (smart meters, connected cars, health etc.)• Roaming is a major component guaranteeing the success of M2M/IoT
platforms• SIM for Things • Single home MNO supports connected things globally
15
The roaming interconnection ecosystem
Interconnection Network
IPX Provider 2
IPX Provider 1
Mobile Network
Operator 2
Mobile Network
Operator 1
Mobile Network
Operator 3
Fixed Network Operators
Content Provider
Internet Service
Providers
Enterprise and Financial Services
Application Service
Providers 16
Operations of an M2M/IoT platform
• A view from one of the largest M2M platforms in the world enabling 4G/LTE devices world-wide
17
today• Motivation: Why study roaming?• Background: Roaming in the cellular ecosystem• The bigger picture: Roaming of Things• Methodology: How can we measure the
implications of roaming?• Results: Performance, content implications and
quality of experience• Reproducibility• What’s next?
18
Roaming People vs. Roaming Things
• Things are active for longer times, consume little data but generate a lot of signaling traffic and occupy resources meant for people
19
On average, IoT devices are active for a period 4 times longer than smartphones
IoT devices generate little to no traffic thus the VMNO does not profit from billing data communications
IoT devices generate significant signaling traffic
Roaming People vs. Roaming Things
• IoT devices are not moving as much and they depend mostly on 2G connectivity
20
Configurations for (Data) Roaming
21
MME
HSS
AAA
S-GW
P-GW
S-GW MME
HSS
AAA
P-GW
eNodeB
HMNO VMNOIPX Network
Other MNOs, FNOs, ISPs, ASPs
eNodeB
MobileDevice
Internet
Evol
ved
Uni
vers
alTe
rres
tria
l Rad
ioA
cces
s N
etw
ork
Evol
ved
Pack
etCo
re (E
PC)
Local breakoutHome-routed
R
R
R
R
IPX breakout
(1) Home-routed (2) Local breakout (3) IPX breakout
Let’s build a testbed!
Operator 1
Operator 2
Operator 3
User Access & Scheduling
Experiment Deployment
INTERNETExperimentation on Commercial Mobile
Networks
[MOBICOM17]Ö. Alay, A. Lutu, M. Peón-Quirós, V. Mancuso, T. Hirsch, K. Evensen, A. Hansen, S. Alfredsson, J. Karlsson, A. Brunstrom, A. S. Khatouni, M. Mellia and M. A. Marsan, Experience: An Open Platform for Experimentation with Commercial Mobile Broadband Networks, in Proceedings of ACM International Conference on Mobile Computing and Networking (MOBICOM), 2017. ACM.
22
Experimental Setup: MONROE-Roaming• Joint effort between UC3M, SRL, KAU,
POLITO, TUM, UoA, TEF to run synchronized measurements in 6 countries (ES, NO, SE, IT, DE, UK)
• 6 countries, 16 operators, each country has two MONROE nodes
• Norway: Telia, Telenor• Sweden: Telenor, Tre, Telia• Spain: Orange, Vodafone, Movistar• Germany: Vodafone, O2, T-Mobile• UK: EE, O2• Italy: Vodafone, TIM, WIND
23
Experimental Setup:MONROE-Roaming• 6 countries, 16 operators, each
country has one MONROE node• One measurement server per
country• Measure one operator at a time
(all nodes have the SAME SIM in the same time)
• Also, measure the visited network natively, where possible
24
25
SpainItalyUK Norway
MONROE-Roaming Experiment Scheduler
Measurement Servers and Targets
Experiment Deployment
Results
Home Country
MONROE-Roaming Nodes
Backend Servers
Vodafone DE
Germany Sweden
Hom
eSI
M
26
Visited Countries
Vodafone DE
Roam
ing
SIM
Vodafone DE Vodafone DE Vodafone DE Vodafone DE
Telia NO Vodafone UK Vodafone IT Vodafone ESVi
site
dSI
MTelenor SE
today• Motivation: Why study roaming?• Background: Roaming in the cellular ecosystem• The bigger picture: Roaming of Things• Methodology: How can we measure the
implications of roaming?• Results: Performance, content implications and
quality of experience• Reproducibility• What’s next?
27
What do we measure?
• Roaming setup and performance• Metadata for tracking the roaming partner• traceroute to discover roaming setup • Dig queries against third party service providers (e.g., ad providers)• Monroe-browsertime for web browsing quality of experience
• Performance of Cloud Service Providers• VoIP and Content Discrimination • ooniprobe web connectivity test• Traffic differentiation measurements using three applications (FaceTime,
Facebook Messenger, Whatsapp) to determine potential traffic differentiation in roaming
28
What do we measure?
• Roaming setup and performance• Metadata for tracking the roaming partner• traceroute to discover roaming setup • Dig queries against third party service providers (e.g., ad providers)• Monroe-browsertime for web browsing quality of experience
• Performance of Cloud Service Providers• VoIP and Content Discrimination • ooniprobe web connectivity test• Traffic differentiation measurements using three applications (FaceTime,
Facebook Messenger, Whatsapp) to determine potential traffic differentiation in roaming
29
Roaming Setup
• Context metadata (RAT, signal strength, visited MNO etc.) from each SIM card• Traceroute: path towards each of the measurement servers
in each country to discover roaming setup
• Result: all 16 MNOs we measure use Home Routed Roaming configuration
30
31
MME
HSS
AAA
S-GW
P-GW
S-GW MME
HSS
AAA
P-GW
eNodeB
HMNO VMNOIPX Network
Other MNOs, FNOs, ISPs, ASPs
eNodeB
MobileDevice
Internet
Evol
ved
Uni
vers
alTe
rres
tria
l Rad
ioA
cces
s N
etw
ork
Evol
ved
Pack
etCo
re (E
PC)
Local breakoutHome-routed
R
R
R
R
IPX breakout
(1) Home-routed (2) Local breakout (3) IPX breakout
Roaming Setup and Performance: Home routed roaming is the usual configuration
Roaming Setup and Performance: Delay implicationsUK User roaming in Germany
32
Roaming Setup and Performance: Delay implicationsUK User roaming in Germany
33
Roaming Setup and Performance: Delay implicationsUK User roaming in Germany
GTP tunnel is slower than native Internet
34
Delay penalty to go back to home
country
35
Roaming Setup and Performance: Delay implicationsUK User roaming in Germany
Roaming Setup and Performance: DNS implications• dig queries against third party
providers FQDNs (e.g., using ABP filter lists)• For the home user the query time
is significantly lower in average than for the other five roaming users
[MOBICOM18]A. Mandalari, A. Lutu, A. Custura, A. Safari, O. Alay, M. Bagnulo, V. Bajpai, A. Brunstrom, J. Ott, M. Mellia, G. Fairhurst (2018). Experience: Implications of Roaming in Europe. Proceedings of ACM International Conference on Mobile Computing and Networking (MOBICOM), 2018.
36
Web Browsing QoE
• 100 target webpages, 10 visits per page• 8 MNOs from SE, ES, DE, NO in roaming,
same parallel configuration as above• MONROE-Browsertime using Chrome,
mobile device, H1s – H2• Collect radio context • QoE metrics: Page Load Time (PLT),
Speed Index (SI), First Paint (FP)
• [WWW19]M. Rajiullah, A. Lutu, A. Safari Khatouni, M. Fida, M. Mellia, A. Brunstrom, O. Alay, S. Alfredsson and V. Mancuso (2019). Web Experience in Operational Mobile Networks: Lessons from Two Million Page Visits. The Web Conference 2019 (WWW’19), 13-17 May 2019, San Francisco, CA, USA
37
What do we measure?
• Roaming setup and performance• Radio metadata for tracking the roaming partner• Traceroute to discover roaming setup • dig: DNS against third party service providers (ad providers)• Monroe-browsertime for web browsing quality of experience
• Performance of Cloud Service Providers (CSPs)• VoIP and Content Discrimination • ooniprobe web connectivity test• Traffic differentiation measurements using three applications (FaceTime,
Facebook Messenger, Whatsapp) to determine potential traffic differentiation in roaming
40
How do mobile apps work?
api.facebook
.comcss.
washingt
onpost.c
om
s.amazon-adsystem.com
config.uca.cloud.unity3d.comsvastx.moatads.com
Fully Qualified Domain Name (FQDN)41
Methodology outline
api.facebook.comcss.w
ashingtonp
ost.com
s.amazon-adsystem.comconfig.uca.cloud.unity3d.comsvastx.moatads.com
Accurate App Traffic Logs
1
CSP
2
PTR ASDetection by:
Commercial MNOs
Active Measurements
• DNS lookups• Traceroute • TCP connection
time• TLS session
establishment time
Empirical Performance Analysis
3
42
TCP Performance towards CSPs (Telia SE)
• Measure performance against Cloud Service Providers (e.g., Akamai, Amazon, Google) to capture the impact of roaming• Metric: Time to First Byte (TTFB)
[INFOCOM18]F. Mickelinakis, H. Doroud, A. Razaghpanah, A. Lutu, N. Vallina-Rodriguez, P. Gill, J. Widmer (March, 2018). The Cloud that Runs the Mobile Internet: A Measurement Study of Mobile Cloud Services. IEEE INFOCOM 2018 – IEEE Conference on Computer Communications. IEEE.
43
TCP Performance towards CSPs (Telia SE)GoogleAmazon AWSAkamai
[INFOCOM18]F. Mickelinakis, H. Doroud, A. Razaghpanah, A. Lutu, N. Vallina-Rodriguez, P. Gill, J. Widmer (March, 2018). The Cloud that Runs the Mobile Internet: A Measurement Study of Mobile Cloud Services. IEEE INFOCOM 2018 – IEEE Conference on Computer Communications. IEEE. 44
today• Motivation: Why study roaming?• Background: Roaming in the cellular ecosystem• The bigger picture: Roaming of Things• Methodology: How can we measure the
implications of roaming?• Results: Performance, content implications and
quality of experience• Reproducibility• What’s next?
48
Reproducible Research
1. [MOBICOM18]A. Mandalari, A. Lutu, A. Custura, A. Safari, O. Alay, M. Bagnulo, V. Bajpai, A. Brunstrom, J. Ott, M. Mellia, G. Fairhurst (2018). Experience: Implications of Roaming in Europe. Proceedings of ACM International Conference on Mobile Computing and Networking (MOBICOM), 2018.https://www.it.uc3m.es/amandala/roaming.html
2. [INFOCOM18]F. Mickelinakis, H. Doroud, A. Razaghpanah, A. Lutu, N. Vallina-Rodriguez, P. Gill, J. Widmer (March, 2018). The Cloud that Runs the Mobile Internet: A Measurement Study of Mobile Cloud Services. IEEE Conference on Computer Communications (INFOCOM), 2018.http://wireless.networks.imdea.org/cloudmap-project
3. [WWW19]M. Rajiullah, A. Lutu, A. Safari Khatouni, M. Fida, M. Mellia, A. Brunstrom, O. Alay, S. Alfredsson and V. Mancuso (2019). Web Experience in Operational Mobile Networks: Lessons from Two Million Page Visits. The Web Conference 2019 (WWW’19), 13-17 May 2019, San Francisco, CA, USA.https://zenodo.org/record/2574157
49
Reproducible Research
Experience: The Implications of Roaming in Europe• Hyperpaper version:
https://bitbucket.org/andralutu/roam_like_at_home_hyperpaper/src/master/
• What is a hyperpaper?Alberto Dainotti, Ralph Holz, Mirja Kühlewind, Andra Lutu, Joel Sommers, and Brian Trammell. "Open collaborative hyperpapers: a call to action." ACM SIGCOMM Computer Communication Review 49, no. 1 (2019): 31-33.
https://ccronline.sigcomm.org/wp-content/uploads/2019/02/sigcomm-ccr-final285.pdf
• Check other examples of hyperpapers: On the Suitability of RTT Measurements for Geolocationhttps://github.com/britram/trilateration
50
Reproducible Research
MONROE Resources• Datasets and papers:
https://zenodo.org/communities/h2020_monroe/
• The MONROE testbeds are open, join the MONROE Alliance• https://www.monroe-project.eu/
• Marvin MONROE – a scheduling daemon https://github.com/MONROE-PROJECT/Scheduler• Experiments code available: • https://github.com/MONROE-PROJECT/Experiments
51
today• Motivation: Why study roaming?• Background: Roaming in the cellular ecosystem• The bigger picture: Roaming of Things• Methodology: How can we measure the
implications of roaming?• Results: Performance, content implications and
quality of experience• Reproducibility• What’s next?
53
• Home routed roaming comes with a delay penalty, that translates into decreased QoE for the end user • Need for enabling local breakout (or home like at
the destination)• Industry trends such as M2M/IoT will put more
stress on the roaming dynamics• IPX providers get more appealing
• Security and privacy aspects of the roaming interconnect need further investigation (3G/4G) to inform the next generation (5G) solutions
54
What’s next?
Thank you!
57
Illustrations from @butlikemaybehttp://www.butlikemaybe.com/