Creating Capacity Using Superior Routing: The Metro-Scale Mesh...

Creating Capacity Using Superior Routing: The Metro-Scale Mesh Networking Facts

A Technology WhitepaperApril, 2005

Photo courtesy of NASA Image eXchange.Image use in no way implies endorsement by NASA of any of the products, services, or materials offered by Tropos Networks, Inc.

Creating Subscriber Capacity With Superior Routing

Metro-Scale Mesh Networking DefinedTM2

A Technology Whitepaper




Introduction

Tropos Networks is the proven leader in the metro-scale Wi-Fi mesh networking market. Tropos metro-scale mesh networks deliver the highest subscriber capacity, lowest deployment and on-going costs, and easiest installation and operation in the industry.

This paper explores the real-world facts about deployed metro-scale Wi-Fi mesh networks to explain why the Tropos MetroMesh architecture, using Predictive Wireless Routing Protocol (PWRP™) and currently based on a single-radio platform, delivers more subscriber capacity per square mile than multi-band, multi-radio mesh approaches. MetroMesh delivers this subscriber capacity for one-seventh the equipment costs of multi-band systems. Tropos creates subscriber capacity through routing software by eliminating packet errors and retransmissions. Without PWRP, others must attempt to create subscriber capacity through cumbersome radio engineering resulting in systems that are expensive to purchase, install and upgrade and that are unproven on a metro-scale.

Creating Capacity Through Superior Routing: PWRP

Tropos Networks creates subscriber capacity through intelligent routing, leveraging commercial off-the-shelf (COTS) radios into a carrier-class system. Our patented PWRP dynamically selects end-to-end paths through a metro-scale mesh that offer the minimum predicted packet error rates. By dramatically reducing packet errors, PWRP avoids capacity-zapping and latency-causing retransmissions. By creating capacity in software, Tropos can rely on low-cost, easy-to-install COTS radio components. This approach delivers required capacity while keeping MetroMesh networks low cost and easy to install, maintain and upgrade.

Multi-Band, Multi-Radio Myth and Reality

Several companies have recently introduced Wi-Fi mesh networking products that employ one1 or more2 5GHz UNII band radios for inter-node communication, while using the 2.4GHz band for Wi-Fi client access. Their mesh protocols are based on routing or bridging approaches adapted from wired networks. These companies claim that:

• Multi-band, multi-radio architectures offer the best subscriber capacity. Perhaps this is true in a static, interference-free lab environment. However in real deployments, there are no static or interference-free environments. Dynamic routing intelligence creates subscriber capacity by mitigating effects of packet loss caused by dynamic interference.







• They claim that multi-band, multi-radio architectures offer best scalability. Despite more than 18 months of availability, the largest documented multi-band, multi-radio network is 20 nodes. Most links in metro-scale mesh deployments are non-line of sight with trees and buildings, and 5GHz systems require extensive link engineering. Not only is it impractical to manually engineer a large number of 5GHz links for line-of-sight, routing protocols not designed to scale to large node counts will ultimately consume all available capacity due to growth in routing overhead.

In contrast, the Tropos Networks MetroMesh™ architecture with the patented, throughput-optimized Predictive Wireless Routing Protocol (PWRP) has always operated only in the 2.4GHz band with one radio. This includes the newly announced Tropos 5210 and Tropos 3210. Today, single-radio solutions power all documented 100 node-plus mesh networks operating across metro areas.

Others attempt to create subscriber capacity through cumbersome radio engineering, paying a performance penalty as a result of not using a metro-scale optimized mesh routing protocol and a price penalty in the form of the much higher node densities required by using 5 GHz intra-mesh links, plus the cost of building multiple radios into every mesh node. These multi-band, multi-radio systems are, as of now, unproven on a metro scale.

Metro-Scale Mesh Networking Facts

From more than 150 customer deployments, Tropos Networks has learned a number of facts about metro-scale Wi-Fi mesh networks. These facts include:

• Today’s applications run on single radios• Smart routing creates capacity• Metro-scale means big networks• 5GHz at least triples node density• Multiple bands multiply costs

The remainder of this paper examines these facts in detail and spells out their implications for metro-scale Wi-Fi mesh networks.

FACT: Today’s Applications Run on Single Radio SystemsThe key performance metric for metro-scale Wi-Fi mesh networks is concurrent subscriber capacity – the total amount of bandwidth available to users at the same time. To facilitate comparison between networks, it is best measured in Mbps delivered per square mile.







Today, using 802.11b, MetroMesh networks deliver 5-6 Mbps of concurrent subscriber capacity per square mile. This exceeds the capacity required for…

• High density residential broadband access in Chaska, MN• Video surveillance in New Orleans• Traffic monitoring in Doha, Qatar• And applications in more than 150 other networks worldwide

The Tropos 5210 MetroMesh router with 802.11g boosts concurrent subscriber capacity per square mile to 10-15 Mbps. This is enough capacity to, for example, serve 300 to 500 residential broadband customers per square mile while providing 30 kbps of bandwidth per provisioned subscriber.

FACT: Smart Routing Creates CapacityRetransmissions reduce subscriber capacity when a shared airlink is used more than once to transmit the same data. There are two causes of retransmissions in a metro-scale Wi-Fi mesh network: hops and packet errors.

Hop Retransmissions - While each hop that a data packet traverses in a mesh causes a retransmission, the effect is limited because, in a well-designed metro-scale mesh network, subscribers are generally only a few hops away from an Internet connection. For example, delivering 10-15 Mbps of subscriber capacity per square mile, typical for MetroMesh networks that use 802.11g, usually requires at least two Internet connections per square mile. In deployed networks with node densities of about 20 MetroMesh routers per square mile for 802.11g speed in the U.S., two Internet connections per square mile implies that subscribers are typically 2 to 3 hops from an Internet connection.

Packet Error Retransmissions - Packet errors caused by interference, multi-path fading, etc., also cause retransmissions. However, the effect of retransmissions caused by packet errors is much greater than that caused by mesh hops because a potentially unlimited number of retransmissions can be required on error-prone links.

Maximizing subscriber capacity requires minimizing total retransmissions caused by both hops and packet errors.







Multi-band, multi-radio products reduce retransmissions caused by hops because they use different frequency bands, reducing airlink sharing. However, the legacy protocols used by multi-band products, such as link state, distance vector and spanning tree protocols, are insensitive to packet errors and do nothing to limit the resulting retransmissions, leading to severe reductions in subscriber capacity.

On the other hand, PWRP, the only commercially available throughput optimizing mesh routing protocol, dynamically selects the end-to-end path through the network that combines a low packet error rate with a hop count that minimizes total retransmissions and creates subscriber capacity. As a result, PWRP produces an operating regime where the need for retransmissions is very limited; multi-band, multi-radio products are subject to a potentially very high number of retransmissions and severely reduced subscriber capacity.

Multi-Hop Loss: The Real Story

Mesh network performance declines with the number of hops that traffic must traverse. However, much of the recent discussion in the trade press and blogs overstates the severity of this performance decline.

Multi-hop loss continues only so long as nodes share an airlink. When nodes are far enough apart that they can’t hear each other, performance no longer decreases. Based on Tropos Network’s experience in deploying metro-scale Wi-Fi mesh networks, by far the most extensive in the industry, this generally happens in three to four hops, assuming real-world node spacing (i.e., distances of one-fourth to one-third of a mile between nodes).

More importantly, much of the discussion misstates the severity of the performance decrease. Performance decreases in a 1/n (where n is the number of hops) fashion, not 1/2N, as some commentators have claimed.

The cause of multi-hop loss is the use of a shared airlink. When a client connects directly to a wired gateway, each of its messages must only use the airlink once to get from source to destination. If a client is connected to a node that is one hop away from the wired gateway, its messages must use the airlink twice to get from source to destination, and so on. For example, if a client’s messages must be transmitted four times over a shared airlink, it requires four chunks of airlink for each message and, hence, the bandwidth available to that client is 1⁄4 of the maximum supported by the airlink.5

While performance declines of up to 1/n can be attributed to multi-hop loss, some observers have cited empirical evidence that multi-hop performance declines more quickly than 1/n in single-radio systems. Their analysis is flawed because they do not properly identify the sources of the performance decline. Performance declines of up to 1/n can be attributed to multi-hop loss. The remainder of the performance decline is due to packet errors induced by interference and multi-path fading. A retransmission due to a packet error has the same impact as a retransmission due to a hop and may fool a naïve observer into thinking the multi-hop loss is greater than 1/n.

Packet Errors Cause Latency

Low, consistent latency is key for running real-time applications over networks, and PWRP reduces end-to-end latency by selecting paths with fewest packet errors. Contrary to some reports, multi-band architectures do not reduce the time it takes for a packet to traverse the network. Both single- and multi-band mesh systems are store-and-forward systems and the store-and-forward latency is the same for both types of systems. The most significant determinant of latency is the retransmission time required to compensate for packet errors. Each retransmission creates 1 ms (802.11g) to 4 ms (802.11b) of additional latency. Up to 16 retransmissions can be required per packet error. Because the mesh protocols used by multi-band, multi-radio systems are insensitive to packet errors, they can have the effect of dramatically magnifying latency due to retransmissions. In contrast, Tropos MetroMesh using PWRP minimizes latency by minimizing packet errors, thereby reducing the need for retransmissions. In a MetroMesh system, the combination of PWRP, an average latency per hop of 1 ms (802.11g) to 4 ms (802.11b), and the reduced node density of 2.4GHz systems as compared to multi-band systems that use 5GHz links translates to the lowest latency for end-user transmissions.







FACT: Metro-Scale Means Big Networks

Hundreds or thousands of nodes are required to cover metro areas. For example, covering a city of 100 square miles with a density of 20 nodes per square mile requires a network of 2000 nodes.

Protocol overhead for legacy mesh algorithms, such as link state, distance vector and spanning tree, grows as network size grows. Without routing software designed to intelligently manage protocol overhead, it can grow unchecked to as much as 20 Mbps in a 2000 node city-wide network, i.e., almost all available real throughput of an 802.11a or 802.11g network. In contrast, PWRP routing overhead remains flat at <5% of network bandwidth in a 200 or 2000 node network, preserving subscriber capacity for user applications.

FACT: 5GHz Triples Node Density

Multi-band systems require at least 3x the node density of single-radio systems that use the 2.4GHz band. Under the non-line-of sight conditions typical of metro-scale installations at the street-lamp level, the use of 5GHz radios in the UNII band introduces at least a 20dB penalty (relative to the 2.4GHz band) to the RF link budget due to trees and other obstructions.3 Because of this penalty, the node density required for optimal system performance at 5GHz is at least 3x the node density required by MetroMesh.

Optimal Use of 5GHz in Metro-Scale Mesh Networks

As a practical matter, Tropos Networks has found that most metro-scale Wi-Fi mesh systems are constrained by the capacity available at the system’s Internet connections, just as in wired last-mile solutions. So, the first step in adding more subscriber capacity is to ensure that the Internet connections support enough bandwidth for the number of subscribers and their applications.

This is a place where 5GHz solutions may make sense. In line-of-sight applications, 5 GHz radio links may be attractive due to their higher potential data rates. In fact, many MetroMesh deployments today utilize 5GHz point-to-multipoint (P2MP) wireless systems to inject Internet connections. In these installations, the MetroMesh routers attached to the P2MP backhaul radios can be considered multi-band. This approach has three distinct advantages over systems with integrated 5GHz radios. First, such a system does not burden every node with the cost of additional radios. The 5GHz radios are added only to areas of the network where additional performance is needed and strict line-of-sight is available. Second, such a system does not require additional mesh node density. Use of the 5GHz P2MP backhaul can be limited to areas where clear line-of-sight to the P2MP base station is available. Finally, use of 5GHz P2MP backhaul doesn’t merely increase throughput to a few nodes – it also increases the aggregate capacity of the metro-scale Wi-Fi mesh network.







FACT: Multiple Bands Multiply CostsMulti-band, multi-radio systems do a much better job of multiplying costs than they do of multiplying subscriber capacity. Tropos Networks thoroughly understands this fact – we developed and tested but chose not to productize a patent-pending multi-band, multi-radio architecture. During our evaluation process, we discovered that using 5 GHz connections between mesh routers is infeasible in real-world environments. We determined that the meager performance gains achieved by throwing radios operating at different frequency bands into a box cannot justify the increased node density and added unit costs, given technology available today.

This is why we continue to use a single-radio architecture with our new Tropos 5210 MetroMesh router. Today, a single-radio architecture with intelligent routing is proven to deliver the highest subscriber capacity at the lowest cost. Developing PWRP was a challenging engineering feat but it was worth it to deliver the metro-scale mesh architecture with the highest subscriber capacity at the lowest cost.

Currently, the equipment cost for a deployed multi-band system is at least 7x that of a single-radio system. Multi-band systems using 5GHz backhaul impose serious economic disadvantages on their owners because their per-node cost is higher than that of single-band, 2.4 GHz systems and they must be deployed using much higher node densities. A multi-band, multi-radio mesh node typically costs 2–3x the cost of a single-radio MetroMesh router. The higher cost arises due to the cost of multiple radios, packaging and, in some cases, large high-gain sectorized antennas to help compensate for the lower RF transmit power that accompany radios packaged in such close proximity. Coupled with a 3x greater node density, the cost of multi-band, multi-radio mesh systems cost is 7x or more that of MetroMesh systems.

For example, a Tropos MetroMesh installation requires approximately 20 MetroMesh routers per square mile to run at 802.11g speeds in the United States. The Tropos 5210 MetroMesh routers have a list price of about $3,500. The total node cost per square mile for a MetroMesh system is, therefore, about $68,000. Based on vendor statements in the trade press, analysis of multi-band link budgets and observations of multi-band deployments, we conclude that multi-band, multi-radio products with list prices of about $8,000 require approximately 60 nodes per square mile in the US. The total node cost per square mile for a multi-band system is, therefore, about $480,000.

MetroMesh™ Multi-Band

Average Node Density Required (U.S.)

20 per sq. mi. 60 per sq. mi.

Price per Node $3,400 $8,000 @ 60 per sq. mi.

Node Price per Square Mile $68,000 $480,000

MetroMesh Price Advantage 7x -







Moreover, the simple COTS radios with omni-directional antennas used in MetroMesh systems minimize move, add and change costs of metro-scale mesh networks. Multi-band, multi-radio systems with sectorized antennas require extensive link-by-link engineering to ensure the line-of-sight required by 5GHz transmissions. Trade-level installers do not have the expertise to perform this work. Skilled and expensive RF technicians must be hired to perform the error-prone chore of aligning one to three links per node in networks of hundreds or thousands of nodes. Each node can take hours or even days to install. In contrast, a trade-level installer with only a screwdriver can install a MetroMesh router in fewer than 15 minutes.

MetroMesh Delivers Higher Capacity at Lower Cost

By creating subscriber capacity through routing and making simple-to-install-and-use radios carrier-class with software, Tropos Networks MetroMesh architecture delivers the industry’s highest metro-scale Wi-Fi mesh network concurrent subscriber capacity at the lowest cost. With the current combination of PWRP and 802.11g, MetroMesh networks deliver real-world concurrent subscriber capacity of 10-15 Mbps per square mile at one-seventh the equipment cost of multi-band, multi-radio approaches.

In the coming years, MetroMesh price-performance will only get better as new developments dramatically increase concurrent subscriber capacity with constant or falling costs. Tropos Networks expects that next-generation PWRP software will multiply concurrent subscriber capacity by at least a factor of two to 20-30 Mbps per square mile, using the Tropos 5210 MetroMesh router. Beyond that, combining these PWRP enhancements with higher capacity, lower cost radios using MIMO, new Wi-Fi chipsets that cost-effectively support multi-channel transmissions in the 2.4GHz band, WiMAX and other new technologies will once again double concurrent subscriber capacity to an aggregate of 40-60 Mbps per square mile. Each phase of MetroMesh enhancement will be backwards compatible with previous phase.

Fewer Paths, More Outages

Simple omni-directional radio links maximize metro-scale mesh reliability. In addition to the noted installation disadvantages, the line-of-sight links required by 5GHz transmissions and directional/sector antennas normally used in multi-band, multi-radio systems typically limit alternate paths to a low number, generally about three. This lack of path diversity means more chance of outage and lower, fluctuating subscriber capacity. Tropos MetroMesh routers use omni-directional antennas and the 2.4 GHz band. As a result, in a typical deployment, each MetroMesh router may see 10 to 30 alternate paths, maximizing reliability and subscriber capacity.







Only MetroMesh is Proven to Scale

Tropos Networks has more than 150 deployed customer networks worldwide, many of more than 100 nodes. Despite more than 18 months of availability, the largest documented multi-band, multi-radio network is 20 nodes.4




555 Del Rey Avenue • Sunnyvale, Ca 94085phone 408.331.6800 • fax 408.331.6801

www.tropos.com • [email protected]©2005 Tropos Networks, Inc. All rights reserved. Tropos Networks, MetroMesh and Tropos Control are trademarks of Tropos Networks, Inc. All other brand or product names are the trademarks or registered trademarks of their respective holder(s). Information contained herein is subject to change without notice.

The only warranties for Tropos products and services are set forth in the express warranty statements accompanying such products and services. Nothing herein should be construed as constituting an additional warranty. Tropos shall not be liable for technical or editorial errors or omissions contained herein.

Summary

Tropos Networks is the proven leader in the metro-scale Wi-Fi mesh networking market. Rather than rely on cumbersome radio engineering, Tropos Networks creates concurrent subscriber capacity through superior routing, operating on COTS radios optimized for the metro-scale environment. This approach enables metro-scale Wi-Fi mesh networks built using Tropos products to deliver the highest subscriber capacity, lowest deployment and on-going costs, and easiest installation and operation in the industry, as demonstrated today in more than 150 customer deployments.

Endnotes1 E.g., Nortel Networks, Firetide and SkyPilot. Note that this approach does not eliminate the multi-hop performance decline. It merely eliminates the client-to-node hop from the equation.

2 E.g., BelAir Networks, MeshDynamics and Strix

3 See S. Perras, L. Bouchard, “Fading characteristics of RF signals due to foliage in frequency bands from 2 to 60 GHz,” IEEE 5th International Symposium on Wireless Personal Multimedia Communications (WPMC ’02), October 2002.

4 http://www.belairnetworks.com/about_belair/press_releases_view.cfm?p_id=32&blnArchive=1

5This subject has been well discussed in academic literature. See, for example: 1. Gavin Holland , Nitin Vaidya, Analysis of TCP performance over mobile ad hoc networks, Proceedings of the 5th annual ACM/IEEE international conference on Mobile computing and networking, p.219-230, August 15-19, 1999, Seattle, Washington, United States http://portal.acm.org/citation.cfm?id=506913&dl=ACM&coll=portal and 2. Mario Gerla, Ken Tang , Rajive Bagrodia, TCP Performance in Wireless Multi-hop Networks, Proceedings of the Second IEEE Workshop on Mobile Computer Systems and Applications, p.41, February 25-26, 1999 http://portal.acm.org/citation.cfm?id=837518&dl=ACM&coll=portal. The information is graphically presented in a presentation by Nitin Vaidya available at http://www.crhc.uiuc.edu/wireless/talks/adhoc-tutorial-2003-august.ppt – see, in particular, Slide 527.

WiMAX and MetroMesh

In January 2004, Tropos Networks announced its three-phase roadmap for integrating 802.16-2004 and 802.16e WiMAX into the MetroMesh architecture. This roadmap includes dual-mode Wi-Fi plus WiMAX systems that can serve both types of clients in unlicensed and licensed bands. The result is complete flexibility for municipalities and service providers. In this roadmap, for reasons stated in this paper, Tropos will use multi-radio, single-band designs for its mesh routers (which must be non-line-of-sight), and multi-radio, multi-band designs for its gateways (which are line-of-sight). Tropos expects to use very integrated multi-radio silicon for both applications. The result will be the same fast, low cost and simple radio designs our customers have come to expect from Tropos. For more information, see “Open Standards for Broadband Wireless Networks: Wi-Fi to WiMAX” at http://www.tropos.com/pdf/wimax_integration.pdf.

http://www.pdos.lcs.mit.edu/roofnet/sigcomm-talk.pdf

http://portal.acm.org/citation.cfm?id=837518&dl=ACM&coll=portal

http://www.crhc.uiuc.edu/wireless/talks/adhoc-tutorial-2003-august.ppt

Creating Capacity Using Superior Routing: The Metro-Scale Mesh...

Documents

Transcript of Creating Capacity Using Superior Routing: The Metro-Scale Mesh...