INTERFERENCE CHARACTERIZATION AND …Abstract Wireless sensor networks (WSNs) provide novel insights...
Transcript of INTERFERENCE CHARACTERIZATION AND …Abstract Wireless sensor networks (WSNs) provide novel insights...
INTERFERENCE CHARACTERIZATION ANDMITIGATION
INLARGE-SCALE WIRELESS SENSOR
NETWORKS
byChieh-Jan Mike Liang
A dissertation submitted to the Johns Hopkins University in conformity with therequirements for the degree of Doctor of Philosophy.
Baltimore, MarylandJanuary, 2011
© 2011 Chieh-Jan Mike LiangAll Rights Reserved
Abstract
Wireless sensor networks (WSNs) provide novel insights into our world by enabling data col-
lection at unprecedented spatial and temporal scales. Over the past decade, the WSN com-
munity has significantly improved the success rate and the efficiency of WSN deployments
through progress in networking primitives, operating systems, programming languages, and
sensor mote hardware design. However, as WSN deployments grow in scale and are embed-
ded in more places, their performance becomes increasingly susceptible to interference from
external sources, as well as poor coordination in the radio transmissions of the network’s
nodes.
This thesis is a coordinated effort to study three types of radio interference in the context
of large-scale WSNs: intra-network, external, and protocol interference. The first part of the
dissertation introduces the Typhoon and WRAP protocols that minimize interference from
concurrent transmitters; Typhoon leverages channel diversity to improve data dissemination
performance and WRAP uses a token-passing mechanism to coordinate data collection traffic
in a network. Then, the dissertation characterizes the external interference from 802.11 traf-
fic to 802.15.4 networks and introduces the BuzzBuzz protocol that uses multiple redundancy
techniques to improve the 802.15.4 packet reception ratio. The final part of the dissertation
presents ViR that multiplexes a single physical radio to satisfy requests from multiple proto-
cols or applications on the same node.
ii
Dr. Andreas Terzis Associate ProfessorAdvisor Department of Computer Science
The Johns Hopkins University
Dr. Alexander Szalay Alumni Centennial ProfessorPrimary reader Department of Physics and Astronomy
The Johns Hopkins University
Dr. Jie Liu Senior ResearcherSecondary reader Microsoft Research
Redmond, Washington
iii
Acknowledgments
The last five years have been a truly rewarding journey for me. This journey concluded a
wonderful chapter of my life, and it prepared me for the next chapter. This journey had
its ups that motivated me to keep going, and it had its downs that gave me opportunities
to grow. During this journey, I had the privilege to meet and work with some of the most
amazing people I have known.
With profound gratitude, I want to thank my advisor and mentor Andreas Terzis. I am
thankful that he believed in me when I had doubts in myself. I am thankful that he pushed
me when I did not think I could move another inch forward. I am thankful that he gave me
directions when I simply did not know what to do. I am thankful that he showed me how to
think and act like a researcher when I was, and still am, learning to be one. I am thankful
that he gave me numerous great opportunities when I did not know what future had planned
for me. I have grown significantly since I met Andreas, both personally and academically,
and I am extremely grateful for his guidance.
I want to thank Alex Szalay for being an amazing colleague in so many different ways. I
can still clearly remember what he said in an interview that changed how I look at research:
”I’m not afraid of investing a few years’ effort into something if it has a chance of paying off”.
I am also amazed that he is always full of energy and enthusiasm to take interests in new
ideas and projects. Finally, I am grateful that he was willing to provide me with the resources
and the time necessary for my projects and my PhD career.
iv
I want to thank Jie Liu at Microsoft Research for taking a big part in my PhD journey. Jie
showed me the joy of research freedom, which motivated me to pursue a research career at
an industry lab. Jie also showed me not to be afraid of standing behind one’s belief. Finally,
to this day, I am still thrilled that he was willing to fly several hours to attend my Graduate
Board Oral (GBO) exam and dissertation defense.
I am grateful to many faculty members and colleagues at Johns Hopkins University
(JHU). I want to thank Katalin Szlavecz for working with me on my first wireless sensor
network project, Life Under Your Feet. As an ecologist, Katalin gave me a different perspec-
tive on computer science. I also thank Randal Burns, Gerald Masson and John Linwood
Griffin for participating in my GBO exam and for their insightful comments.
I thank my colleagues and friends from the Hopkins interNetworking Research Group
(HiNRG). Razvan Musaloiu-E. helped tremendously when I first started in the area of wire-
less sensor networks, and he always had time when I needed someone to bounce ideas with.
Jayant Gupchup, one of the first friends I made at JHU, was always very supportive. Sandeep
Sarat and Moheeb Rajab gave me advices at the beginning of my PhD career. I am glad that
JeongGil (John) Ko decided to join the lab as he always made my day more enjoyable. I was
fortunate to work with Yin Chen and see the fruit in our project. I thank JongHyun Lim
for being a good friend and listener, and for getting me started with the exercise routine. I
enjoyed Douglas Carlson’s great personality and enthusiasm to learn. Finally, I had the plea-
sure of meeting the three newest members of the lab before I graduated: Andong Zhan, Da
Zheng, and ZaiNan (Victor) Zhou.
Ming Chang was one of my closest friends at JHU. I am glad that Ming was always there
when I wanted to talk about bad news or share good news. I treasure the friendship with
ChungWei Yen, Sheng Chien, YuTa Chen, ZhiFei Li, Raluca Musaloiu-E., Reza Curtmola,
Omar Zaidan, Scott Pitz, Lijun Xia, Taesung Kim and more. Finally, I want to thank Youn
v
Na and Aeri Cho, whom I met during a frustrating period of my PhD years, for reminding me
that there is always time to smile.
I also want to thank Srivasan Keshav at University of Waterloo for supervising me in
the Undergraduate Research Assistantship program and for introducing me to the world of
research. Bodhi Priyantha at Microsoft Research for mentoring and hosting my internship.
I especially want to express my thanks to my parents, Gladys Wang and Kevin Liang, for
their never ending encouragements and support in my education.
vi
Contents
Abstract ii
Acknowledgments iv
List of Figures x
List of Tables xiv
1 Introduction 1
1.1 Wireless Sensor Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.3 Organization of the Dissertation . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
2 Intra-network Interference in Dissemination 6
2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
2.2 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
2.2.1 Randomized Protocols . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
2.2.2 Protocols with Coordination . . . . . . . . . . . . . . . . . . . . . . . . . . 10
2.3 Typhoon . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
2.3.1 Metadata Dissemination . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
2.3.2 Data Request Handshake . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
vii
Contents Contents
2.3.3 Data Transfer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
2.3.4 Channel Switching . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
2.4 Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
2.4.1 Evaluation Metrics and Methodology . . . . . . . . . . . . . . . . . . . . . 18
2.4.2 Effect of Network Density and Size . . . . . . . . . . . . . . . . . . . . . . 20
2.4.3 Effect of Object Size . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
2.4.4 Impact of Packet Loss . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
2.4.5 Benefits of Overhearing and Channel Switching . . . . . . . . . . . . . . 27
2.4.6 Protocol Overhead . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
2.4.7 Testbed Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
3 Intra-network Interference in Collection 32
3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
3.2 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
3.2.1 The Need for Wireless Sensors . . . . . . . . . . . . . . . . . . . . . . . . 35
3.2.2 Application Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
3.2.3 Data Center RF Environment . . . . . . . . . . . . . . . . . . . . . . . . . 39
3.2.4 Genomotes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
3.3 WRAP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
3.3.1 Protocol Design Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
3.3.2 Topology Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
3.3.3 Channel Diversity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
3.3.4 Data Collection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
3.4 Protocol Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
3.4.1 Protocol Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
viii
Contents Contents
3.4.2 Application-Level Performance . . . . . . . . . . . . . . . . . . . . . . . . 60
3.5 Deployment Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
3.6 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
3.6.1 Comparison with TDMA . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
3.6.2 Optimal Sampling Intervals . . . . . . . . . . . . . . . . . . . . . . . . . . 67
3.7 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
4 External Interference 71
4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
4.2 Motivations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
4.2.1 Microsoft PDC Conference . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
4.2.2 JHU Library . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
4.3 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
4.3.1 802.15.4 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
4.3.2 802.11 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
4.4 Measuring 802.11 and 802.15.4 Interactions . . . . . . . . . . . . . . . . . . . . 79
4.4.1 Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
4.4.2 802.15.4 Packet Reception Ratio . . . . . . . . . . . . . . . . . . . . . . . 81
4.4.3 Dynamics of 802.15.4 and 802.11 Interaction . . . . . . . . . . . . . . . . 84
4.4.4 802.15.4 Bit Error Distribution . . . . . . . . . . . . . . . . . . . . . . . . 86
4.5 Protecting 802.15.4 packets from 802.11 . . . . . . . . . . . . . . . . . . . . . . . 91
4.5.1 Symmetric Region Techniques . . . . . . . . . . . . . . . . . . . . . . . . . 92
4.5.2 Asymmetric Region Techniques . . . . . . . . . . . . . . . . . . . . . . . . 97
4.6 BuzzBuzz MAC Layer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104
4.6.1 BuzzBuzz Protocol Design . . . . . . . . . . . . . . . . . . . . . . . . . . . 104
ix
Contents Contents
4.6.2 Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106
4.7 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108
4.7.1 802.11n Interference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108
4.7.2 Network-Wide Blocker . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
4.8 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
5 Protocol Interference 114
5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114
5.2 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115
5.2.1 Benefits of Using Multiple Channels . . . . . . . . . . . . . . . . . . . . . 115
5.2.2 Limitations of Current Multi-Channel Protocols . . . . . . . . . . . . . . 116
5.3 A Multi-Channel Protocol Framework . . . . . . . . . . . . . . . . . . . . . . . . 117
5.3.1 Architecture Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118
5.3.2 Component Interfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
5.3.3 Protocol Composition Example . . . . . . . . . . . . . . . . . . . . . . . . 122
5.4 Prototype Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122
5.4.1 Basic Protocol . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123
5.4.2 Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124
5.5 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125
5.5.1 Support for Low-Power MACs . . . . . . . . . . . . . . . . . . . . . . . . . 126
6 Conclusions 128
Vita 149
x
List of Figures
2.1 State transition diagram for Typhoon. State transitions are marked using thecondition/action notation in which a transition occurs when a condition is metand results in an action (or no action in case of ’-’). . . . . . . . . . . . . . . . . . 13
2.2 Pipelining pages through the network. . . . . . . . . . . . . . . . . . . . . . . . . 142.3 (a) Propagation of consecutive pages on a linear topology when only one fre-
quency channel is used. Notice that node A has to wait until time period 4 totransmit the second page in order to avoid colliding at B with node C’s trans-mission of the first page. (b) When nodes can use different frequency channelsto transmit data packets (indicated by different colors in the figure) the waittime is reduced by one time period. . . . . . . . . . . . . . . . . . . . . . . . . . . 16
2.4 Packet reception rate as a function of distance from a packet source. The path-loss exponent is 4. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
2.5 Average network completion time and average node power consumption for a20 KB object, as a function of network density. Network nodes are placed on agrid over a 180× 180-foot field. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
2.6 Node completion time for Typhoon and Deluge on a 180×180-foot field. The fieldhad 302 nodes uniformly distributed with a density of 0.028 nodes per squarefoot. A 20 KB object was initially injected at the bottom left corner of the field. 22
2.7 (a) Network completion time of a 20 KB object, as a function of the diameterchanges in 1 × n linear topology. (b) Page acquisition time, including the pagerequest phase and subsequent data transfer. The vertical lines represent 5th
and 95th quantiles. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 232.8 Completion time as the object size varies in (a) 1 × 50 sparse linear topology
where nodes can reach only their immediate neighbors, and (b) 20 × 20 gridtopology with 10-feet node spacing. . . . . . . . . . . . . . . . . . . . . . . . . . . 24
2.9 Modeled and simulated completion time of Typhoon in a 1 × 50 sparse lineartopology. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
2.10 Completion time as the inter-node distance varies in a 20× 20-node grid topology. 262.11 Probability distribution of (a) Typhoon request messages and Deluge advertise-
ments (b) Typhoon and Deluge data messages. The topology is a 10 × 10 nodegrid, with 10-feet node distances, and the object size is 20 KB. The vertical linesshow the average. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
xi
List of Figures List of Figures
2.12 The testbed floor plan shows the locations of Tmote Connect boxes, which canhave either one or two motes attached. . . . . . . . . . . . . . . . . . . . . . . . . 29
2.13 PDF of node completion time on the testbed for (a) Typhoon and (b) Deluge.The vertical line shows the network average in each case. . . . . . . . . . . . . . 30
3.1 A row of computer racks inside a data center (left) and the corresponding in-frared image representing the spatial temperature distribution (right). . . . . . 36
3.2 Temperature measured at different locations in and around an HP DL360 server.Also shown is the server’s CPU load. Internal sensors reflect the server’s work-load instead of ambient conditions. . . . . . . . . . . . . . . . . . . . . . . . . . . 36
3.3 Distribution of packet reception ratios (PRR) across all the links from a 52-node data center site survey. A large percentage of the network’s links exhibitnon-trivial loss rates. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
3.4 Boxplots of link PRR as a function of RSSI and LQI values. Boxplots show thesample minimum, first quartile, median, third quartile, and sample maximum.Links with RSSI > −75 dBm and LQI > 90 have persistently high PRR. . . . . 41
3.5 Background noise distribution across all 802.15.4 frequency channels in the2.4 GHz ISM band. Each of the circumferences is proportional to the occurrencefrequency of the corresponding RSSI level. Channels 15, 20, 25, and 26 arerelatively quiet compared to other channels. . . . . . . . . . . . . . . . . . . . . 42
3.6 Two types of Genomotes designed for RACNet. The wireless node (on the left)controls several wired nodes (on the right) to reduce the number of wirelesssensors within the same broadcast domain. . . . . . . . . . . . . . . . . . . . . . 43
3.7 Piece-wise linear approximation of PRR from LQI. The dots are the averagePRR for each LQI obtained from the site survey results. . . . . . . . . . . . . . . 46
3.8 Tree construction with channel diversity. During the channel scanning phase,node N1 joins two trees. Then, it decides to stay at channel C2, and N2 atchannel C1 eventually removed N1 from the child list. . . . . . . . . . . . . . . 49
3.9 An sample two-level binary tree. . . . . . . . . . . . . . . . . . . . . . . . . . . . 543.10 Minimum WRAP backoff slot size necessary to hear HEARTBEAT messages
from a node’s neighbors. The curve grows linearly with the neighborhood size. 593.11 Average one-round collection time and the collection tree’s sum of hops as a
function of the testbed network size. . . . . . . . . . . . . . . . . . . . . . . . . . 593.12 Boxplots of the inter-packet interval (IPI) and data yields under various sam-
pling intervals on the 62-node lab testbed. The number on top of each IPI plotshows the average. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
3.13 Per-node inter-packet interval achieved by WRAP on the 62-node testbed. Nodestransmit one packet every three seconds. Nodes are labeled according to theirlocation on the testbed. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
3.14 Boxplots of the inter-packet interval (IPI) and data yields under various net-work sizes on the data center testbed. Each node generates one packet every10 seconds. The number on top of each IPI plot shows the average. . . . . . . . 63
xii
List of Figures List of Figures
3.15 Sum of hops of the four WRAP trees in the production deployment as a functionof time. All trees stabilize after a few hours. . . . . . . . . . . . . . . . . . . . . . 64
3.16 Boxplots of daily network yield from the production deployment over a periodof 21 days. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
4.1 802.11b/g and 802.15.4 frequency channels in the 2.4 GHz ISM band. Each802.11 channel is 22 MHz wide, while 802.15.4 channels are 2 MHz wide. . . . 72
4.2 The number of sensor nodes reporting data over ten hours. Thousands of userswere connected to a co-located WiFi network during the first four hours causingdramatic interference to the sensor network. . . . . . . . . . . . . . . . . . . . . 75
4.3 Format of a 15.4 packet. Field sizes are in bytes. . . . . . . . . . . . . . . . . . . 764.4 Messages and delays defined in the 802.11 MAC protocol. Durations of packet
transmissions and time intervals depend on the 802.11 variant used. The lead-ing RTS/CTS exchange is used only for large packets. . . . . . . . . . . . . . . . 77
4.5 Setup for the garage packet reception ratio experiment. . . . . . . . . . . . . . . 834.6 Percentage of 15.4 packets correctly received, corrupted, and lost as the dis-
tance between 802.11 nodes and 15.4 nodes increases. . . . . . . . . . . . . . . . 844.7 Overlay of 802.11b and 15.4 traffic. Each vertical box corresponds to a 15.4
packet transmission, while the gray lines are RSSI measurements correspond-ing to 802.11 transmissions. When d is small, the 802.11 radio backs-off whenit senses a 15.4 transmission. As d increases, the 802.11 radio cannot detect15.4 transmissions and packets collide. . . . . . . . . . . . . . . . . . . . . . . . 85
4.8 Bit-error distribution for 15.4 packets that failed the CRC check when the in-terfering 802.11 transmitter is in the symmetric region. It is far more likelythat the front part of the 15.4 packet will be corrupted. . . . . . . . . . . . . . . 87
4.9 Detailed view of the 802.11b and 15.4 packet transmission timeline. The over-lap at the beginning of the 15.4 packet (vertical box) corresponds to a collisionwith a 802.11 packet. The 802.11 sender defers sending any additional packetsuntil the 15.4 transmission completes. . . . . . . . . . . . . . . . . . . . . . . . . 88
4.10 15.4 packet reception rate as the payload size varies. The competing 802.11sender lies within the symmetric region of the 15.4 sender. Since only bits inthe front section of the 15.4 packet are corrupted varying the packet’s lengthdoes not affect PRR. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
4.11 Bit-error distribution for 15.4 packets that failed the CRC check when the in-terfering 802.11g transmitter is in the asymmetric region. Bit errors are evenlydistributed across the whole packet. . . . . . . . . . . . . . . . . . . . . . . . . . 89
4.12 CDF of the number of bad bits in a corrupted packet. . . . . . . . . . . . . . . . 914.13 Distance n between any two corrupted bits in a 15.4 packet. Errors caused by
competing 802.11b and 802.11g senders occur in concentrated bursts. . . . . . 914.14 15.4 PRR in the presence of 802.11b interference as the TI CC2420 correlation
threshold varies. Lower threshold values do not increase PRR but lead thereceiver to incorrectly decode channel noise as packets. . . . . . . . . . . . . . . 92
xiii
List of Figures List of Figures
4.15 15.4 PRR as the preamble length varies. Due to the shorter 802.11g packets itis possible to recover all 15.4 frames by increasing the preamble’s length. . . . 93
4.16 Multi-Headers: a 15.4 packet with an additional header. . . . . . . . . . . . . . 954.17 Hamming(12,8) encoding format. Four parity bits are generated for each eight
bits of data. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 994.18 Total number of 15.4 transmissions necessary to transfer a 38-KB object over a
single hop in the presence of interference from 802.11 traffic. . . . . . . . . . . . 102
5.1 Proposed decomposition and interfaces of multi-channel protocols. . . . . . . . 118
xiv
List of Tables
2.1 Completion time under different loss environments for a 20×20-node grid topol-ogy with 10-feet node distance. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
2.2 Completion time as channel-switching and overhearing are disabled to dissem-inate a 3-KB object in a 5× 5-node grid topology with 20-feet node spacing. . . 27
4.1 Packet and interval durations for 802.11 and 802.15.4. The length of the con-tention windows (CW) for 802.15.4 corresponds to the MAC used by TinyOS.The 802.11 standard uses Binary Exponential Backoff (BEB). . . . . . . . . . . 78
4.2 Percentage of packets successfully received using the original 15.4 header orone of the additional headers. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
4.3 Percentage of corrupted packet payloads that can be recovered by two versionof the Hamming code and the RS code. . . . . . . . . . . . . . . . . . . . . . . . . 100
4.4 Execution time for TinyRS operations. The original message is 65 bytes longand the RS-encoded message contains the original 65-byte message and the30-byte parity. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
4.5 Summary of experiment results running CTP with BuzzBuzz on a 57-nodetestbed. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
4.6 802.15.4 PRR with and without the blocker near the 802.11 AP. . . . . . . . . . 110
5.1 When running both CTP and Typhoon on a 3× 3 mesh, ViR improves the CTPdelivery rate to 95%. The increase in CTP latency is due to ViR serving simul-taneous Typhoon and CTP traffic. . . . . . . . . . . . . . . . . . . . . . . . . . . . 125
xv
Chapter 1
Introduction
1.1 Wireless Sensor Networks
MIT Technology Review identified wireless sensor networks (WSNs) as one of the ten tech-
nologies that will influence our lives in the near future [59]. This distinction is due to the
observation that WSNs will provide novel insights into our world by collecting data at spatial
and temporal scales that were previously infeasible or impractical. Imagine a world with
trillions of computing devices that enable us to have a high degree of interaction with, bet-
ter knowledge of, and higher control of our surroundings. In fact, this idea is captured by
the different visions of WSNs, e.g., Smart dust [96] and Internet of Things, and it has mo-
tivated exciting applications for energy consumption awareness [32], early disaster warning
systems [4], human activity inference [53] among others.
WSNs consist of a large number of small wireless sensor devices (also known as motes).
Two major differences separate WSNs from ad-hoc wireless systems. First, the mote’s hard-
ware and software designs emphasize low power consumption, which gives motes the poten-
tial to operate unattended for long periods of time. Second, the low power hardware design
generally translates to simpler hardware circuits and smaller energy reservoir, and thus giv-
ing motes a smaller physical size than other wireless devices. Compared to a pocket WiFi
access point of 200cm3 [1], motes can be as small as 1cm3 [66]. With their small footprint,
1
Chapter 1. Introduction 1.1. Wireless Sensor Networks
motes can be embedded in places difficult to reach, such as the sub-glacial environments [64].
In the last decade, the WSN community has improved the success rate and efficiency
of sensor network deployments through the incremental development of networking primi-
tives [45], such as data collection [18] and dissemination [48]. At the same time, innovations
in operating systems and programming languages for resource-constrained embedded de-
vices have improved software reliability and ease of development [11, 39]. Finally, progress
in sensor mote hardware design has provided hardware platforms with increased computing
and storage resources. Supported by these community efforts, WSNs have been successfully
deployed in a wide range of applications from environmental monitoring [88], data center
facility management [52] to monitoring glacier movement [15]. These early successes have
motivated domain scientists to deploy sensor networks in an increasing number of applica-
tions and with an increasing number of motes.
Unfortunately, the WSN community is still not able to fully realize the decade-old visions
of unattended and ubiquitous sensor deployments. As Section 1.2 argues, one road block is
the interference from uncoordinated radio resource sharing among networks of nodes. Radio
interference either prevents WSNs to be deployed in certain areas, or it requires the admin-
istrators to reconfigure the network topology and parameters as new sources of radio inter-
ference are present. Radio interference is generally due to the broadcast nature of the radio
medium. Imagine a room where many people are talking loudly, it would be very difficult for
someone to understand what each speaker is saying.
In fact, radio is a particularly important example because of its impact on network com-
munication and mote energy consumption. Thus, I believe the next step that the community
should take is to consider the practical issues of radio interference that future WSN deploy-
ments will face.
2
Chapter 1. Introduction 1.2. Problem Statement
1.2 Problem Statement
To be ubiquitously deployed, WSNs must adapt to the environment. The environment fac-
tors in question are not necessarily conditions such as temperature, humidity, air pressure,
but a group of nodes sharing the common radio spectrum resource. Traditionally, the WSN
community has assumed a monolithic network setup for simplicity: networks consist of a
small number of nodes that have exclusive access to the spectrum resource. In reality, these
assumptions no longer hold, especially considering the trend towards more dense sensor de-
ployments and more crowded radio spectrum. Thus, new challenges have emerged to deal
with radio interference for efficient WSN communication.
We further motivate the importance of considering radio interference with the deploy-
ment experience of the Collection Tree Protocol (CTP) from Gnawali et al. [18]. CTP is a
popular data collection protocol in the WSN community. Several observations can be made
from the empirical results collected by the authors on 12 testbeds. First, due to the external
interference from co-located 802.11 networks, nodes on USC’s Tutornet testbed performed
more retransmissions than other testbeds to recover from lost packets. The external 802.11
interference also resulted in a data delivery rate of 95% on the Tutornet testbed compared to
99% on other testbeds. Second, testbeds have different node density, and thus each requires
different traffic generation rates to avoid saturating the radio channel capacity. For example,
the Tutornet testbed had a sampling interval of 30 seconds compared to 16 seconds on the
Motelab testbed at Harvard University or the Wyman Park testbed at JHU.
This thesis focuses on characterizing and mitigating three types of radio interference:
intra-network interference, external interference, and protocol interference. Intra-network
interference is the result of neighboring nodes of the same network transmitting on overlap-
ping radio frequency bands simultaneously. External interference concerns with the inter-
3
Chapter 1. Introduction 1.3. Organization of the Dissertation
action among networks with different radio technologies. Protocol interference occurs when
multiple local protocols sending conflicting commands (e.g., switching on/off) to the same
physical radio.
1.3 Organization of the Dissertation
This thesis is organized as follows.
Chapter 2 and Chapter 3 discuss intra-network interference in the context of network-
wide data dissemination and collection respectively. Chapter 2 presents the Typhoon proto-
col that leverages channel diversity to minimize intra-network interference. Experimental
results show that Typhoon can be up to three times faster than Deluge, the de facto stan-
dard for data dissemination in TinyOS, in sparse and lossy networks. Chapter 3 presents the
Wireless Reliable Acquisition Protocol (WRAP) that relies on a token-passing mechanism to
efficiently coordinate in-network data collection traffic. Results from two testbeds show that
WRAP outperforms protocols that use rate-based congestion control (RCRT) and uncoordi-
nated transmissions (CTP), especially at high network loads.
Chapter 4 focuses on external interference. First, it characterizes the behavior of 802.15.4
and 802.11 radios when both co-exist in the same radio frequency band. Under certain con-
ditions, ZigBee transmissions can trigger a nearby WiFi transmitter to back off, in which
case the header is often the only part of the ZigBee packet being corrupted. We call this
the symmetric interference region, in comparison to the asymmetric region where the ZigBee
signal is too weak to be detected by WiFi senders, but WiFi activity can uniformly corrupt
any bit in a ZigBee packet. With these observations, we design BuzzBuzz to mitigate WiFi
interference through header and payload redundancy. Multi-Headers provides header redun-
dancy giving ZigBee nodes multiple opportunities to detect incoming packets. Then, TinyRS,
a full-featured Reed Solomon library for resource-constrained devices, helps decoding pol-
4
Chapter 1. Introduction 1.3. Organization of the Dissertation
luted packet payload. On a medium-sized testbed, BuzzBuzz improves the ZigBee network
delivery rate by 70%. Furthermore, BuzzBuzz reduces ZigBee retransmissions by a factor of
three, which increases the WiFi throughput by 10%.
Finally, Chapter 5 discusses protocol interference and describes the ViR framework that
enables multiple local users on a node to share the underlying radio resource. Results of real-
istic applications synthesized from existing protocols show how ViR reduces conflicts among
protocols and packet losses.
This thesis concludes in Chapter 6.
5
Chapter 2
Intra-network Interference inDissemination
2.1 Introduction
One common end-user requirements for WSNs is the ability to reprogram the network over-
the-air after it has been deployed. In fact, as WSNs grow in network size, the time saved
from over-the-air reprogramming over manual reprogramming becomes more pronounced.
In turn, the requirement of over-the-air reprogramming generates the need to reliably dis-
seminate large objects (∼ 50-100 KB) to every node in the network. This combination of large
object sizes, 100% reliability, and network-wide distribution is not addressed by other WSN
protocols and thus requires a custom protocol. This need has been identified by numerous
researchers in the past (e.g., [2,25,41,63,85] among others).
An interesting aspect of network-wide reprogramming is the bursty traffic pattern it gen-
erates. As the network starts to disseminate an object, nodes become active and relay data
packets until all of their neighbors successfully receive the object. However, in dense net-
works, the bursty traffic pattern can cause intra-network interference if nodes in a neigh-
borhood are not coordinated carefully. In previous protocol proposals, sender arbitration is
achieved either through a randomization mechanism (e.g., use of random timers) or through
explicit coordination. However, many of these proposals do not consider energy efficiency.
6
Chapter 2. Intra-network Interference in Dissemination 2.1. Introduction
This chapter presents Typhoon, a reliable data dissemination protocol that represents a
different set of choices in the design space. Our choices are motivated by the observation that
idle listening is the major consumer of energy during dissemination. Thereby, all protocol
decisions should be geared towards minimizing the time that nodes are not transmitting or
receiving data packets (i.e., competing to request or waiting for the retransmission of a lost
packet).
Unlike previous protocols, Typhoon sends data packets via unicast. This approach allows
receivers to acknowledge the receipt of individual packets and thereby quickly recover lost
packets. While data packets are sent via unicast, interested nodes can still receive them by
snooping on the wireless medium. Through the combination of unicast transfers and snoop-
ing, Typhoon achieves the best of both worlds—prompt retransmissions and data delivery to
all the nodes in a broadcast domain through a single transmission. Dissemination latency
is also reduced by exploiting spatial reuse, through which nodes in different parts of the
network can be transmitting at the same time. We enhance spatial reuse through the com-
bination of two techniques: setting timers in a way that encourages nodes further from the
origin to propagate the object and the use of channel switching. Specifically, it has been shown
that the minimum node distance, or the number hops in a linear topology, necessary to avoid
interference among concurrent transmissions is three hops [12]. On the other hand, if nodes
switch frequency channels1 during data transfer, it is possible to reduce the distance to two
hops in many cases. Typhoon leverages this observation to reduce the object dissemination
time.
We evaluate the performance of Typhoon through a combination of simulations and exper-
iments on a testbed deployed in an office building. Performance is measured in terms of the
time required and the energy expended to deliver an object to the whole network. We vary1The IEEE 802.15.4 standard defines 16 non-overlapping channels.
7
Chapter 2. Intra-network Interference in Dissemination 2.2. Related Work
the size, diameter, and density of the network and test Typhoon using different object sizes
and loss rates to understand the effects of these factors on the protocol’s behavior. Moreover,
we compare Typhoon’s performance to that of Deluge—the de facto standard for data dissem-
ination in TinyOS [25]. Our results show that Typhoon can be up to three times faster than
Deluge in sparse and lossy networks.
This chapter has four sections. We summarize related work in the section that follows
and provide a detailed description of the Typhoon protocol in Section 2.3. In Section 2.4, we
evaluate Typhoon’s performance using Deluge as the comparison baseline.
2.2 Related Work
The problem of designing protocols for reliably disseminating large data objects has received
considerable attention in the past. One can divide existing protocols in two broad categories:
randomized protocols in which nodes compete to acquire and subsequently transmit parts of
the object, and protocols that avoid contention by scheduling node transmissions.
2.2.1 Randomized Protocols
The genealogy of the first protocol family starts with PSFQ (Push Slowly Fetch Quickly) [94],
a transport protocol for reliable delivery of objects from a sink to a group of sensor nodes in
a network. PSFQ origin nodes use the TTL-scoped broadcast to limit the scope of dissemina-
tion, and nodes use aggressive hop-by-hop retransmissions to recover any lost data. Unlike
PSFQ, Typhoon uses unicast messages to propagate objects hop-by-hop, and this approach
allows Typhoon receivers to quickly acknowledge data reception. In addition, Typhoon nodes
leverage overhearing from other nodes in the neighborhood to opportunistically receive the
data. Moreover, PSFQ uses negative acknowledgments, whereas Typhoon uses positive ac-
8
Chapter 2. Intra-network Interference in Dissemination 2.2. Related Work
knowledgments and multiple frequency channels to increase spatial reuse.
MOAP [85] transfers the complete object one hop at a time. After receiving the whole
object, a node can become a secondary source to deliver the object one-hop further away from
the origin node. The design of MOAP is driven by the desire to trade latency for reliability
and simplicity. Unlike MOAP, Typhoon uses pipelining in which nodes offer to further deliver
pages) (i.e., subsets of the object) as soon as they receive them. This approach dramatically
reduces the network completion time, defined as the time by which all nodes receive the full
object, thereby reducing energy consumption due to radio idle listening.
MNP [41] reduces download time by using pipelining and minimizes radio medium con-
tention through a sender selection algorithm. MNP achieves reliability with retransmissions
initiated by query messages from the packet source to the receiver nodes. Unlike MNP, Ty-
phoon implements opportunistic overhearing for traffic of common interest. Moreover, Ty-
phoon uses fast acknowledgments transmitted after each packet rather at the end of a page.
Finally, Typhoon uses channel switching to reduce the radio medium contention, amplifying
the benefits of spatial reuse.
Deluge [25] is currently the de facto standard for data dissemination in the TinyOS com-
munity. It uses an epidemic protocol that eventually propagates the object to all the nodes
in the network. Deluge relies on randomized Trickle timers [48] to reduce contention among
transmission requests. Objects are transmitted as sequences of fixed-size pages via broadcast
to leverage the broadcast nature of the wireless medium. NACKs trigger retransmissions of
lost messages after a full page has been transmitted. NACKs also use Trickle timers to min-
imize the probability that multiple retransmission requests will collide. While beneficial in
reducing the number of collisions, random timers can prolong the time required to propagate
the object throughout the network. Typhoon also delivers data to multiple receivers when-
ever possible. On the other hand, receivers send acknowledgments after each data message
9
Chapter 2. Intra-network Interference in Dissemination 2.2. Related Work
instead of NACKs after each block transmission. This design choice enables nodes to start
offering data to downstream destinations sooner, thereby minimizing the completion time
and thus the energy cost. This is especially important in lossy networks in which the number
of retransmissions is expected to be high. Moreover, Typhoon uses channel switching to re-
duce radio medium contention and to allow multiple concurrent transmissions over the same
broadcast domain.
2.2.2 Protocols with Coordination
Protocols of the second family follow a two-step process. First, the network distributes the
object to a subset of nodes using a fixed schedule that avoids overlapping transmissions in
the second step. Second, nodes in the initial set deliver the object within their neighborhood.
In order to minimize completion time, the initial set of nodes should be the minimum con-
nected dominating set (MCDS) of the graph induced by the wireless network [63]. Calculating
that set however is an NP-hard problem even for the unit graph connectivity model [10] and
therefore approximation algorithms are necessary. Sprinkler uses a distributed approxima-
tion algorithm to compute a connected dominated set that is an multiplicative factor larger
than the MCDS [63]. Infuse [2] follows a similar dissemination strategy and combines it with
implicit acknowledgments for reliability. Furthermore, Infuse turns off the radio of nodes not
actively participating in the dissemination and thus reducing energy consumption due to idle
listening. GARUDA [67] is a recent protocol that uses an efficient mechanism for constructing
an approximate MCDS during the first packet transfer. Moreover, GARUDA nodes publish
bitmaps indicating the packets they have received correctly. Downstream nodes use these
bitmaps to send (re)transmission requests. Unlike protocols that rely on node coordination to
prevent contention, Typhoon minimizes contention through the use of channel switching and
10
Chapter 2. Intra-network Interference in Dissemination 2.3. Typhoon
implicit synchronization. This approach does not have the overhead of building the MCDS, is
robust to node failures, and simplifies data dissemination to new nodes in the network.
2.3 Typhoon
Typhoon is designed to reliably deliver large objects, such as code binaries, to all the nodes
in a WSN. In the context of WSN, we define large objects as objects that do not fit in a typical
mote’s main memory and can be as large as 50-100 KB. Typhoon divides an object to fixed-size
pages (1 KB) and further divides pages to fixed-size packets (28 bytes in our implementation)
that can be atomically transmitted over the radio.
Even though bulk dissemination protocols are unlikely to be invoked frequently, the cost
should be minimized due to their inherent flooding nature and the need for 100% reliability ir-
respective of loss conditions. Since idle listening is one of the largest energy consumers [104],
the protocol should make every effort to “push” the object’s pages through the network as fast
as possible. In turn, this means that the protocol should attempt to leverage spatial re-use,
transmitting pages from multiple non-overlapping nodes and minimize contention that leads
to node back-offs and thereby added latency.
We note that an alternative approach would be to use duty cycling, or turning radios off
when not in use. In this case network completion time is not as crucial, because energy
consumption due to idle listening is minimized. However, we argue that duty cycling is not
appropriate for reliable bulk dissemination protocols. First, users want to reduce network
downtime due to reprogramming. Second, duty cycling introduces complexity which should
be minimized in protocols that serve a critical role to network operations.
11
Chapter 2. Intra-network Interference in Dissemination 2.3. Typhoon
2.3.1 Metadata Dissemination
We assume that the object to be disseminated is injected through an out-of-band mechanism
to a single node from which it must propagate to the network. In this regard, the first nec-
essary step is to notify the network about the existence of this new object. Typhoon uses
separate mechanisms to disseminate data objects and metadata about these objects. Meta-
data refers to information about the existence of a new object, codified into an object ID, size
and version. Nodes decide whether they should attempt to download an advertised object by
comparing the new object ID and version with those of previously retrieved objects. If a node
decides to download the new object, the number of pages is determined by dividing the object
size by the page size.
The reason for using separate mechanisms stems from the difficulty of designing a sin-
gle protocol that can efficiently serve both purposes. For example, since new nodes may join
the network at any time, the metadata dissemination protocol must always be active. This
means that, while it should quickly propagate updates to the whole network, it must mini-
mize overhead during the steady state. On the other hand, for reasons outlined above, the
data dissemination protocol should disseminate the object as fast as possible and then termi-
nate. Typhoon uses Trickle [48] to disseminate metadata.
Next, we describe what happens once nodes become aware of the existence of a new object
and attempt to retrieve it.
2.3.2 Data Request Handshake
Figure 2.1 represents Typhoon’s state transition diagram. Nodes start in the ACTIVE state
and return to this state if they have more pending pages to download. While in this state,
a node will periodically broadcast PageReq requests that contain the ID of both the object
12
Chapter 2. Intra-network Interference in Dissemination 2.3. Typhoon
ACTIVE WAIT
SNOOP RCVR
PUBMissing pages /
Tx PageReq
Accept PageReq /Tx PageOffer Accept StreamReq / -
Page not complete / -
Page not complete / -
Timeout
Tx complete or retx threshold exceeded / -
Tx completeor timeout / -
Accept PageOffer /Tx StreamReqOverhear PageOffer / -
Page not complete / -
Figure 2.1: State transition diagram for Typhoon. State transitions are marked using thecondition/action notation in which a transition occurs when a condition is met and results inan action (or no action in case of ’-’).
and the requested page. By having nodes request pages sequentially, nodes within the same
broadcast domain are more likely to be in the same state, which increases the probability of
overhearing traffic of common interest.
The broadcast period is uniformly chosen from [ta, tb] to avoid collisions among multiple
interested receivers2. For nodes that have copies of the requested page and receive a PageReq
message, each responds with an unicast PageOffer message after waiting for a random time
uniformly selected from [tc, td]. The PageOffer message includes the ID of the object as well
as the ID of the page offered. The random waiting period is used to prevent hidden-terminal
collisions among multiple potential offerers. The offerers then transition to the WAIT state
and wait for a StreamReq message. If no StreamReq arrives within Ts seconds, the offerers
return to the ACTIVE state3. Otherwise, upon receiving the unicast StreamReq message,
the offerer will transition to the PUB state and start the data transfer. That offerer returns
to the ACTIVE state after the page has been successfully downloaded or after a number
(five) of unsuccessful data packet transfers. These failures are detected because the receiver
acknowledges the receipt of individual data packets (cf. Section 2.3.3).
Conversely, a node that receives a PageOffer message matching its request, transitions2We use [ta, tb] = [400, 500] ms.3Ts is 20 ms in our implementation.
13
Chapter 2. Intra-network Interference in Dissemination 2.3. Typhoon
A B C
Data(n)
PReq(n+1) Data(n+1)
PReq(n)
POffer(n) POffer(n)
Offer cancelled
T'[15, 25] T'[15, 25]
Figure 2.2: Pipelining pages through the network.
to the RCVR state and signals the source of the PageOffer message to initiate the data
download by transmitting a unicast StreamReq message. The receiver stays in that state
while more packets from the requested page need to be retrieved and returns to the ACTIVE
state either when the whole page has been successfully downloaded or when a timeout occurs.
The second case protects the receiver against failures of the transmitting node.
Nodes that overhear a PageOffer message for a page they are missing, will transition to
the SNOOP state in which they will attempt to receive the data packets from the offered page.
While PageOffer messages are sent via unicast, interested nodes can still receive them. For
example, the CC2420 radio provides the ability to disable address filtering enabling a node
to receive all packets irrespective of their destination. Similar to the RCVR state, the node
leaves the SNOOP state when the page transfer has completed or when a timeout occurs. If
a node does not successfully overhear all the packets from a page, it discards the page.
In addition to the base scheme described above, Typhoon optimizes its use of timers to
enable the pipelining of pages through the network. We describe this optimization using the
example presented in Figure 2.2. In this scenario, node A has finished transmitting page n
to node B. In response, node B will transition to the ACTIVE state and transmit a PageReq
for page n+ 1. Node A receives this message and starts its timer to transmit the PageOffer
message. However, node C also receives the request and deduces that node B already has
page n (because pages are downloaded sequentially). C then sends its own PageReq for page
14
Chapter 2. Intra-network Interference in Dissemination 2.3. Typhoon
n to B. From the perspective of pipelining, C’s request has priority over B’s original request,
since it pushes pages further downstream. To encourage this behavior, Typhoon sets the
timer at B to fire before A’s timer4. Once B’s timer expires, it transmits a PageOffer for
page n. A overhears that offer and cancels its own PageOffer, implicitly deferring to B’s
data transmission.
2.3.3 Data Transfer
Typhoon achieves reliable transfer in the face of packet loss through the use of acknowledg-
ments (ACKs) and retransmissions. However, unlike previous protocols that use negative
acknowledgments after all packets in a page have been transmitted, Typhoon acknowledges
the receipt of individual data packets. If the sender does not receive an acknowledgment,
it retransmits the last data packet thus implementing a stop-and-wait ARQ protocol. This
approach allows the receiver to sequentially receive a page of data without any gap and thus
reduce any caching necessary on resource-constrained motes.
A node can generate these ACKs in two different ways. First, modern radios offer the abil-
ity to automatically generate hardware ACKs [89]. The benefit of this approach is reduced
latency because the hardware ACKs are generated as soon as the radio hardware correctly re-
ceives the packet. On the other hand, it is possible for an acknowledged packet to be dropped
before it reaches the application layer. In this case, hardware ACKs result in a false positive.
Fortunately, TinyOS 2 [46], on which Typhoon is developed, implements a mechanism called
software ACKs that can trigger ACKs at the system level. It is thus possible to disable the
hardware from automatically generating hardware ACKs and achieve equivalent functional-
ity using software ACKs.4In our implementation, [tc, td] = [15, 25] ms for a node that has just finished transmitted a page and [0, 10] ms
otherwise.
15
Chapter 2. Intra-network Interference in Dissemination 2.3. Typhoon
A B C D ETime
Pkt 1
Pkt 2 Pkt 1
Pkt 1
Pkt 1
1
2
3
4
(a)
A B C D ETime
Pkt 1
Pkt 2 Pkt 1
Pkt 1
1
2
3
One hopDifferentchannels
(b)
Figure 2.3: (a) Propagation of consecutive pages on a linear topology when only one frequencychannel is used. Notice that node A has to wait until time period 4 to transmit the secondpage in order to avoid colliding at B with node C’s transmission of the first page. (b) Whennodes can use different frequency channels to transmit data packets (indicated by differentcolors in the figure) the wait time is reduced by one time period.
An additional benefit of disabling hardware ACKs is that it enables overhearing of uni-
cast packets. This is because enabling hardware ACKs in the commonly-used CC2420 radio
also enables destination address filtering, in which case the radio automatically discards all
unicast frames not destined to the current node. With address filtering disabled, nodes in
the SNOOP state can still receive data packets sent to the unicast address of the node that
transmitted the StreamReq message, while the explicit receiver will generate ACKs for those
data packets.
2.3.4 Channel Switching
As we already argued, data dissemination protocols should leverage spatial re-use to acceler-
ate the propagation of pages through the network. Spatial re-use is achieved by having nodes
retransmit pages as soon as they arrive. However, as Figure 2.3 (a) demonstrates, in order
to avoid collisions due to the hidden terminal problem a node must wait for two additional
periods (a period is defined as the amount of time necessary to transmit a page) before it can
transmit the next page. On the other hand, as Figure 2.3 (b) shows, this bound can be further
reduced if nodes have the ability to transmit at different frequency channels. Channel switch-
16
Chapter 2. Intra-network Interference in Dissemination 2.3. Typhoon
ing provides another benefit in addition to accelerating the pipelining process. Because nodes
exchange PageReq and PageOffer messages on the default common channel, having data
transfers on different frequencies eliminates the danger of ongoing data transfers colliding
with these control messages.
Considering the advantages of channel switching, Typhoon incorporates it to the data
request handshake described above. Rather than using an explicit agreement protocol in
which nodes are assigned specific frequencies, Typhoon employs a randomized scheme to
select data transmission frequencies. Specifically, the publisher suggests a frequency channel
in its PageOffer message by randomly selecting from one of the possible channels (e.g.,
15 in the case of 802.15.4, since one channel is reserved for broadcast messages). If the
receiver accepts the offer, it replies with an acknowledgment (similar to the ACK used for
data packets) and switches to the suggested frequency channel. After receiving the ACK
the publisher also tunes to the new channel and the data transfer starts. Note that the
receiver transmits a StreamReq message after switching to the channel indicated in the
PageOffer message. Although the channel is randomly chosen, it is still possible to have
multiple publishers willing to serve the same receiver on the same channel. Therefore, the
StreamReq message serves as an explicit indication of the receiver’s decision. Although
nodes randomly select data transfer channels, it is possible that more than one ongoing data
transfers with overlapping radio coverage take place on the same channel. In this case,
interference can cause higher packet loss and thus retransmissions and possibly failure to
transmit the page due to the loss of multiple acknowledgments. In the second case, the
sender and/or the receiver will timeout, return to the ACTIVE state, and retry downloading
the original page.
While channel switching provides clear performance benefits, it also introduces new com-
plications. For example, Typhoon uses Trickle for metadata dissemination, and both Typhoon
17
Chapter 2. Intra-network Interference in Dissemination 2.4. Evaluation
Distance (ft)
PR
R (
%)
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●
●
●
●
●
●
●
●
●
●
●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
0 10 20 30 40 50 60 70 800
10
20
30
40
50
60
70
80
90
100
Figure 2.4: Packet reception rate as a function of distance from a packet source. The path-lossexponent is 4.
and Trickle can be active at the same time. Since Trickle is not aware of the channel changes
it will transmit over the channel selected by Typhoon. This means that if a node is trans-
ferring data on a channel other than the default one, the node’s neighbors will not be able
to receive any metadata sent via Trickle. Realizing this conflict, we implement two schemes
to minimize its effects. First, upon receiving the initial notification via Trickle, nodes wait
for a random period before they start Typhoon 5. This delay allows Trickle to propagate the
metadata downstream. Second, nodes switch to the default channel immediately after each
page transfer, thus allowing the continued dissemination of metadata.
2.4 Evaluation
2.4.1 Evaluation Metrics and Methodology
We evaluate the performance of Typhoon using simulations and experiments performed on a
testbed deployed in an office building. The results we report are based on an implementation
of Typhoon built on top of TinyOS 2 (T2) [46]. Moreover, we use the standard CSMA MAC
protocol implemented by the CC2420 radio stack in T2.5Set to [400, 500] ms in our implementation.
18
Chapter 2. Intra-network Interference in Dissemination 2.4. Evaluation
We use Deluge, the de facto standard for reliable bulk data dissemination in the TinyOS
community, as the baseline for our comparisons. Since Deluge provides no guidelines for
setting its parameters under different network conditions we use the default parameters
provided with Deluge under all cases. All simulations were carried out in TOSSIM, a discrete
event based simulator for TinyOS [47]. We leverage two of TOSSIM’s features to improve the
fidelity of our simulations. First, TOSSIM allows defining signal attenuation levels on a
per link basis. We calculate these attenuations using the log distance path loss model [74].
In this model, the path loss at distance d from the source, measured in dB, is, PL(d) =
PL(d0) + 10n log(d/d0), where n is the path-loss exponent and PL(d0) is an experimentally
measured path loss at reference distance d0. A path-loss exponent of n = 2 corresponds to
free space propagation, while n = 3, 4 model environments with reflections and refractions.
We use n = 4 for all our simulations. Figure 2.4 shows the packet reception rate at various
distances from a source node. Second, we utilize TOSSIM’s ability to emulate bursty noise
due to interference.
We quantify the performance of Typhoon through two metrics: (1) Completion time,
which captures the time necessary to disseminate an object. We measure the completion
time of individual nodes as well as the network completion time, defined as the longest node
completion time. (2) Power consumption. While completion time quantifies the level of
disruption from executing the object dissemination protocol (assuming the network’s oper-
ation is disrupted during the download), power consumption quantifies the impact of data
dissemination on the network’s lifetime.
Due to the lack of a direct mechanism for measuring power consumption in TOSSIM, we
use the indirect approach of measuring the amount of time the nodes spend transmitting,
in idle listening mode, as well as the number of packets it receives. Because the Tmote
Sky data sheet [60] publishes only the current drawn in transmit mode (17.4 mA), and in
19
Chapter 2. Intra-network Interference in Dissemination 2.4. Evaluation
idle listening mode (19.7 mA), we experimentally measured using a Tmote Sky mote [71]
the average current drawn while receiving one packet to be 21.7 mA. Note that our energy
estimates do not include the costs of reading and writing to flash. The reason is that they
represent a fixed cost which is orthogonal to the operation of the data dissemination protocol
and therefore provides no insight into the impact of different protocol design decisions.
We run each experiment five times and use the two evaluation metrics to reason about the
impact of different factors on the performance of Typhoon. Specifically, we investigate the im-
pact that network density, network size, object size, and loss rate have on data dissemination.
Moreover, we evaluate the incremental benefits of overhearing and channel switching in Ty-
phoon. Finally, we present the behavior of Typhoon in practice with results from a testbed in
an office building.
2.4.2 Effect of Network Density and Size
Network density is a critical performance factor since it affects the level of contention when
requesting and downloading pages. We first discuss the impact of network density on com-
pletion time. Figure 2.5(a) shows the effect of increasing the number of nodes per square foot
by increasing the size of an N × N node grid, deployed on a fixed 180 × 180-foot field. The
same figure also shows the average node degree, defined as the set of nodes with PRR > 0,
as network density increases. One can make two observations from this figure. First, the
performance margin between Typhoon and Deluge increases in sparse networks. This is be-
cause Deluge uses timer values that reduce the number of messages sent and increase the
probability of overhearing. However, in sparse networks, these timer values increase the idle
listening time and thus completion time. Second, Typhoon is consistently faster throughout
the density range despite its more aggressive timers. This indicates that channel switching
20
Chapter 2. Intra-network Interference in Dissemination 2.4. Evaluation
Nodes per square foot
Com
plet
ion
time
(sec
)
●
●
●
● ●
●
●
●
●
●
●
●
● ●
●
●
●
●
0.000 0.005 0.010 0.015 0.020 0.025 0.0300
20
40
60
80
100
120
0
20
40
60
80
100
120
Avg
nod
e de
gree
DelugeTyphoonAvg node degree
(a) Completion time.
Nodes per square foot
Pow
er c
onsu
mpt
ion
(mA
h) ●
●
●
● ●●
●
●
●
●
●
●
0.000 0.005 0.010 0.015 0.020 0.025 0.0300.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
DelugeTyphoon
(b) Average power consumption.
Figure 2.5: Average network completion time and average node power consumption for a20 KB object, as a function of network density. Network nodes are placed on a grid over a180× 180-foot field.
is effective in relieving channel contention.
Both Typhoon and Deluge require nodes to keep their radios on for the duration of the data
dissemination. Considering that the radio consumes a considerable energy in idle listening
state, completion time will influence energy consumption. Figure 2.5(b) verifies this intuition
as it shows that energy consumption follows closely completion time. We found that for both
protocols nodes spend less than 7% of their time transmitting further indicating that energy
cost is dominated by idle listening time. With this result in mind, we present only completion
times for the remainder of the evaluation.
Figure 2.6 illustrates the propagation time for individual nodes in a dense grid. As re-
21
Chapter 2. Intra-network Interference in Dissemination 2.4. Evaluation
35 40 45 50 55 60 65 70 75
0 5 10 15 20 25 30 0
5
10
15
20
25
30
(a) Typhoon.
65 70 75 80 85 90 95 100
0 5 10 15 20 25 30 0
5
10
15
20
25
30
(b) Deluge.
Figure 2.6: Node completion time for Typhoon and Deluge on a 180 × 180-foot field. The fieldhad 302 nodes uniformly distributed with a density of 0.028 nodes per square foot. A 20 KBobject was initially injected at the bottom left corner of the field.
ported in [25], Deluge propagates the data object faster around the edges than in the middle
of the network. The main reason is that nodes in the middle of the network have more neigh-
bors and thus higher probability of collisions. On the other hand, Typhoon generates an
uniform wavefront pattern from corner to corner by leveraging channel diversity to reduce
channel contention. Specifically, although nodes in the middle have more neighbors, the only
messages broadcasted on the default channel are the handshake messages. The probability
of collision is thus lower than Deluge.
Unlike the grid topology in which a node might receive data from different neighbors, the
linear topology limits the propagation to only one direction. It is therefore easier to study the
effects of network size on completion time using linear topologies.
A number of interesting observations can be made from Figure 2.7(a) that plots completion
time as a function of network diameter in a linear topology. First, both Typhoon and Deluge
benefit from pipelining, and the completion time does not increase at the same rate as the
number of nodes. Second, Deluge exhibits a faster increase compared to Typhoon. As the
network diameter increases, the number of neighboring nodes for some nodes also increases,
22
Chapter 2. Intra-network Interference in Dissemination 2.4. Evaluation
Network diameter (node)
Com
plet
ion
time
(sec
)
●
●
●
●●
●
●
●
●
●
0 10 20 30 40 50 600
20
40
60
80
100DelugeTyphoon
(a)
Network diameter (node)
Pag
e re
ques
t and
tran
sfer
tim
e (s
ec)
●
●●
●●
●● ● ● ●
0 10 20 30 40 50 600
1
2
3
4
5
6
7
(b)
Figure 2.7: (a) Network completion time of a 20 KB object, as a function of the diameterchanges in 1× n linear topology. (b) Page acquisition time, including the page request phaseand subsequent data transfer. The vertical lines represent 5th and 95th quantiles.
and thus the probability of contention increases. This has a larger influence on Deluge be-
cause Typhoon sends packets on the common channel only during the page request phase.
Figure 2.7(b) verifies this conjecture with the average time to request and download a single
page as the network’s diameter increases. From the similarity between the two graphs, it is
easy to see that page acquisition time dictates completion time. Furthermore, Typhoon has
approximately constant page transfer time in all cases, which suggests that the shorter page
request phase underlies the difference in completion time. Finally, Deluge exhibits larger
variability in page acquisition time, due to the varying levels of contention that different
nodes experience.
23
Chapter 2. Intra-network Interference in Dissemination 2.4. Evaluation
Data object size (KB)
Com
plet
ion
time
(sec
)●
●
●
●
●
●
●
●
●
●
0 10 20 30 40 50 60
010
020
030
0
DelugeTyphoon
(a)
Data object size (KB)
Com
plet
ion
time
(sec
)
●
●
●
●
●
●
●
●
●
●
0 10 20 30 40 50 60
010
020
030
0
DelugeTyphoon
(b)
Figure 2.8: Completion time as the object size varies in (a) 1×50 sparse linear topology wherenodes can reach only their immediate neighbors, and (b) 20 × 20 grid topology with 10-feetnode spacing.
2.4.3 Effect of Object Size
Unlike metadata dissemination protocols for which network diameter dominates completion
time, the size of the object transferred affects the completion time of bulk data dissemination
protocols. Figure 2.8 shows the impact of object size on completion time in two cases: a sparse
linear topology in which nodes can reach only their immediate neighbors, and a 20 × 20 grid
topology with 10-feet node spacing. In both cases, the completion time grows linearly with
the object size with Deluge yielding a steeper slope.
To understand the root cause of this behavior, we briefly present a model for data dissemi-
nation in sparse linear topologies. We assume ideal conditions in which pages are transferred
24
Chapter 2. Intra-network Interference in Dissemination 2.4. Evaluation
Data object size (KB)
Com
plet
ion
time
(sec
)
●
●
●
●
●
●
●
●
●
●
0 10 20 30 40 50 600
20
40
60
80
100SimulationModeled
Figure 2.9: Modeled and simulated completion time of Typhoon in a 1 × 50 sparse lineartopology.
in perfect synchrony with no collisions. In this case, the expected completion time for Typhoon
is Tt = 2(n− 1)Pt + d× Pt, where n is the number of object pages, d is the network diameter,
and Pt is the time to request and receive a page (see Figure 2.3). Given the description of
Typhoon from Section 2.4, we can estimate Pt and thus Tt. A page transfer is preceded by
the data request handshake. According to TOSSIM, each handshake exchange of a 21-byte
message followed by the ACK requires 1.68 ms to complete. Since the handshake consists of
three messages and two d back-off timers with maximum length of 25 ms each, it should take
55.04 ms. Moreover, according to TOSSIM, a page transfer requires approximately 428 ms
and thus Pt is 483.04 ms. Figure 2.9 shows the modeled and simulated completion time for
Typhoon with different object sizes. Since the modeled completion time is based on ideal
conditions, it represents the lower bound on Typhoon’s performance. At the same time, it
explains that the lower completion time that Typhoon exhibits is due to the speedup that
channel switching offers.
25
Chapter 2. Intra-network Interference in Dissemination 2.4. Evaluation
Node distance (ft)
Com
plet
ion
time
(sec
)
●●
●
● ● ●
●
●●
●
● ●●
●
0 5 10 15 20 25 30 35 40
040
8012
016
020
0DelugeTyphoon
Figure 2.10: Completion time as the inter-node distancevaries in a 20× 20-node grid topology.
Quiet Bursty lossTyphoon 54.30 73.37Deluge 80.79 241.43
Table 2.1: Completion time underdifferent loss environments for a20 × 20-node grid topology with10-feet node distance.
2.4.4 Impact of Packet Loss
Since reliability is a requirement for bulk data dissemination protocols, completion time de-
pends on how quickly lost packets are recovered. We perform two experiments to estimate
the effect of packet loss on completion time.
First, we increase the spacing between neighboring nodes in a 20× 20 grid topology. This
increase raises the path loss on the link and therefore decreases the packet reception rate
(PRR). Figure 2.10 illustrates the completion time for this experiment. It is easy to see that
Deluge performance deteriorates with distance while Typhoon is able to maintain consistent
performance. Specifically, Deluge’s completion time increases by over twofold when nodes
are 35 feet apart from each other. This is due to the fact that the PRR of the links between
neighboring nodes at this distance falls in the so-called gray region (PRR =∼ 95%, as Fig. 2.4
indicates). Extending the inter-node distance even further leads to a precipitous decrease in
PRR (∼ 30% at 40 feet), leading to an even worse performance differential.
Second, we simulate the effect of bursty losses due to interference. To do so, we use
TOSSIM noise traces collected from environments with heavy 802.11 use [44]. As Table 2.1
shows, Typhoon’s performance degrades by 48% while the completion time for Deluge in-
creases threefold. Two main reasons underlie this trend. First, Typhoon requires all data
26
Chapter 2. Intra-network Interference in Dissemination 2.4. Evaluation
Completion time (sec)Channel-switching and overhearing 6.24Channel-switching only 8.79Overhearing only 945.80None 1016.43
Table 2.2: Completion time as channel-switching and overhearing are disabled to disseminatea 3-KB object in a 5× 5-node grid topology with 20-feet node spacing.
packets to be individually acknowledged and it bases the retransmission decision on this ac-
knowledgment instead of a timer. This allows lost packets to be recovered quickly. Second,
compared to Deluge, Typhoon is more aggressive in sending packets, so the transfer moves
at a faster pace.
2.4.5 Benefits of Overhearing and Channel Switching
In order to better understand the performance benefits that channel switching and overhear-
ing offer, we selectively disable them in an experiment on a 5× 5 grid topology.
Table 2.2 presents the results of this experiment. Disabling channel switching creates a
larger performance deterioration compared to disabling overhearing. This degradation while
large is expected because Typhoon assumes that data transfers take place on a channel that
is free from interference caused by other data transfers and request handshakes. As a result,
being aggressive hurts performance in this case. On the other hand, overhearing provides
only modest improvement. The reason is that Typhoon performs opportunistic overhear-
ing, in which nodes can snoop on a page transfer only when they overheard the preceding
PageOffer message. In other words, if a node misses that message, it loses the opportunity
to overhear since the transfer happens at another channel. Moreover, if a node in the SNOOP
misses one or more packets from a page due to interference it discards the whole page. At the
same time, when overhearing is combined with channel switching, it offers ∼ 30% reduction
in completion time.
27
Chapter 2. Intra-network Interference in Dissemination 2.4. Evaluation
Number of requests
Fre
quen
cy
0 30 60 90 120 150 1800.00
0.05
0.10
0.15
0.20
0.25
0.30DelugeTyphoon
(a)
Number of data packets
Fre
quen
cy
0 500 1000 1500 2000 25000.00
0.05
0.10
0.15
0.20
0.25
0.30DelugeTyphoon
(b)
Figure 2.11: Probability distribution of (a) Typhoon request messages and Deluge advertise-ments (b) Typhoon and Deluge data messages. The topology is a 10 × 10 node grid, with10-feet node distances, and the object size is 20 KB. The vertical lines show the average.
2.4.6 Protocol Overhead
The major design goal of Typhoon is to minimize completion time. It achieves this goal by be-
ing aggressive in requesting and transmitting object pages. Figure 2.11 illustrates the results
of this aggressive behavior by comparing the per-node packet distributions for disseminating
the same object using Typhoon and Deluge.
We focus on request and data transfer messages because they constitute the majority of
traffic. Typhoon generates approximately three times more traffic than Deluge for both mes-
sage types. The reason is that, unlike Deluge, Typhoon does not have a request suppression
mechanism, so nodes broadcast requests more aggressively. Moreover, we found that 47%
28
Chapter 2. Intra-network Interference in Dissemination 2.4. Evaluation
Figure 2.12: The testbed floor plan shows the locations of Tmote Connect boxes, which canhave either one or two motes attached.
of the overhearing attempts failed (i.e., nodes had to discard the partially overheard pages).
While one can suggest based on this result that nodes should sleep instead of performing op-
portunistic overhearing, sleep scheduling introduces coordination complexity and overhead
to the protocol. Furthermore, as Section 2.4.5 shows, overhearing when used in conjunction
with channel switching, leads to ∼ 30% reduction in completion time.
As explained above, Typhoon uses the Dissemination service to publish metadata and T2
components for reading/writing to the Flash. The combined code footprint of all three com-
ponents is 14,752 bytes of ROM and 413 bytes of RAM. At the same time, the incremental
overhead of adding Typhoon to an application that uses Dissemination and the Flash compo-
nent is 3,806 bytes of ROM and 112 bytes of RAM.
2.4.7 Testbed Evaluation
We complement the simulation results in previous sections with experimental results from
a 22-node testbed. While simulations are meant to explore the behavior of the protocols
under various conditions, the testbed is used to compare their performance in a realistic
environment. Given the two different goals, we do not compare results across simulations
and the testbed. Rather, it is the relative performance of Typhoon and Deluge under the
same testing scenarios that is of interest.
29
Chapter 2. Intra-network Interference in Dissemination 2.4. Evaluation
Node completion time (sec)
Fre
quen
cy
54 56 58 60 62 64 66 68 700.0
0.1
0.2
0.3
0.4
0.5
(a) Typhoon
Node completion time (sec)
Fre
quen
cy
80 90 100 110 120 130 1400.0
0.1
0.2
0.3
0.4
0.5
(b) Deluge
Figure 2.13: PDF of node completion time on the testbed for (a) Typhoon and (b) Deluge. Thevertical line shows the network average in each case.
We test Typhoon on a 22-node testbed in an office building according to the topology in
Figure 2.12. Each node is a Tmote Sky mote [60]. Due to the shape of the building, the testbed
physically resembles a linear topology. Moreover, the center of the testbed around location
119 tends to have relatively bad connectivity to the rest of the network. Dissemination starts
by injecting a 20 KB object from location 118 on the right side of the testbed.
The average network completion time was 75.15 seconds using Typhoon and 145.57 sec-
onds with Deluge. To understand how the object propagates through the network, Figure 2.13
shows the distribution of node completion times. For both protocols, the node completion time
is divided into two groups, with one group taking longer to receive the entire object. Anal-
30
Chapter 2. Intra-network Interference in Dissemination 2.4. Evaluation
ysis of the experiment log shows that the group of slow nodes is located on the left side of
the testbed. This is due to the the poor link connectivity in the center of the testbed. For
example, in the case of Typhoon, most nodes on the left side of the testbed download pages
from location 112. However, since the link connectivity between location 112 and nodes on
the right side of the testbed was poor, location 112 becomes the bottleneck.
31
Chapter 3
Intra-network Interference in Collection
3.1 Introduction
As one of the main goals of WSNs is to report observations from the surrounding environ-
ment, data collection is a common networking primitive. In fact, data collection is one of the
most studied topics in the WSN community. Previous work on data collection has explored
approaches to improve different performance metrics, e.g., reliability, latency, and energy
consumption.
This chapter looks at the impact of intra-network interference on the performance of a
data collection protocol, especially in the setting of large-scale and dense networks where
many nodes reside in the same radio coverage area. We leverage the experience gained from
the Data Center (DC) Genome project [52] and WSN deployments in monitoring data centers
to motivate the problem.
This chapter presents the design, implementation, and evaluation of RACNet, a large-
scale sensor network for high-fidelity data center environmental monitoring. RACNet uses
custom-made Genomote sensor nodes that employ a combination of wired and wireless com-
munications to scale. As a key technical contribution of RACNet, we developed the Reliable
Data Collection Protocol (rDCP) and the Wireless Reliable Acquisition Protocol (WRAP) for
scalable data collection (§ 3.3). To tackle the challenge of intra-network interference caused
32
Chapter 3. Intra-network Interference in Collection 3.2. Background
by the contention in dense wireless networks, both rDCP and WRAP follow a simple yet
effective congestion avoidance philosophy, leveraging frequency and time multiplexing that
is conceptually different from previous congestion control approaches (e.g. [36, 65, 73]). In
particular, rDCP and WRAP use multiple IEEE 802.15.4 frequency channels simultaneously
and adaptively balance the number of nodes on each channel based on traffic load. We note
that WRAP is intrinsically different from congestion control protocols, such as RCRT [65] and
IFRC [73]. While the later ones detect and react to congestion, WRAP proactively prevents
the contention that generates congestion in the first place. Moreover, WRAP improves on
rDCP in the way data collection is coordinated, and a token passing protocol that provides an
implicit network arbitration mechanism, allowing only one active packet flow per frequency
channel.
Using results from two testbeds (§ 3.4) and an ongoing deployment at a production data
center (§ 3.5), we show that WRAP outperforms protocols that use rate-based congestion
control (RCRT) and uncoordinated transmissions (CTP), especially at high network loads.
The remainder of the chapter first presents high-level requirements for data center mon-
itoring and the challenges of using IEEE 802.15.4 wireless communications in these envi-
ronments through a site survey in Section 3.2. Section 3.3 elaborates on the design of the
RACNet reliable data collection system. We present our evaluation results in Section 3.4,
while Section 3.5 outlines results from a production data center deployment of RACNet at
Microsoft. Finally, Section 3.7 reviews related work.
3.2 Background
Data center energy consumption has attracted global attention due to the fast growth of the
IT industry and increasing concerns about carbon footprints and climate change. While ad-
vances in component design continue to decrease the power consumption of computer servers,
33
Chapter 3. Intra-network Interference in Collection 3.2. Background
one cannot overlook the energy consumed by the hosting facilities, considering that only 30%
to 60% of the total energy that a typical data center consumes powers its IT equipment. The
rest is either lost during the power delivery and conversion process, or used by environmental
control systems such as Computer Room Air Conditioning (CRAC) units, water chillers, and
(de)humidifiers [5,91].
Lack of visibility into the data center’s operating conditions is one root cause for this
low energy efficiency. As conventional wisdom dictates that IT equipment needs abundant
cooling to operate reliably, the CRAC systems in many data centers are set to very low tem-
perature set points. Furthermore, data center operators tend to further decrease the CRAC’s
temperature set points when servers issue thermal alarms because they lack the informa-
tion to accurately diagnose the problem. Thereby, high-fidelity (i.e., with rack-level spatial
granularity and sub-minute sampling rate) historical and real-time data about the environ-
mental conditions inside a data center are invaluable not only for diagnosing problems but
for improving the data center’s efficiency [9,68].
Facility operators can use the data to troubleshoot thermal alarms, make intelligent de-
cisions on rack layout and server deployments, and innovate on facility management. More
importantly, such data can be particularly useful as data centers start to employ sophisticated
cooling controls to accommodate environmental and workload changes [68]. For example, air-
side economizers bring in outside air for cooling, while dynamic server provisioning strategies
turn on or shut down a large number of servers following load fluctuations [9]. While both
techniques can reduce a data center’s total energy consumption, the results depend of the
data fidelity and quality of spatial and temporal heat distributions.
Traditional solutions for data center environmental monitoring use wired sensors [68,
78], but the high installation and configuration costs prevent the wide adoption of these
systems. Using the motherboards’ temperature sensors is also problematic, because these
34
Chapter 3. Intra-network Interference in Collection 3.2. Background
sensors reflect the servers’ activities rather than the data center’s environmental conditions,
as the results from Section 3.2.1 show. On the other hand, wireless sensor networks are ideal
for the data center monitoring task as they offer low-cost, non-intrusive, and flexible in-situ
sensing.
At the same time, the application’s requirements in terms of latency and reliability cou-
pled with the data centers’ environment pose unique challenges to wireless sensing. As our
site survey results show (§ 3.2.3), even a single data center room requires hundreds of wire-
less sensors, 50% to 65% of which can interfere with each other. Uncoordinated wireless
transmissions in this environment can lead to congestion collapse drastically reducing the
network’s usable capacity. Furthermore, packet losses are frequent, despite the high node
density, due to interference from collocated networks (e.g., WiFi) and the large number of
metallic obstacles. Previous sensor networks that did not implement end-to-end reliability
exhibited data yields of 20-60% [21, 87, 97] and thus fail to meet the requirements of data
center control and troubleshooting applications. More recent reliable collection protocols
(e.g., [36, 61, 65, 98]) employ local data caching and end-to-end retransmissions to improve
data yields but do not scale to the network sizes and densities required by data center sens-
ing.
3.2.1 The Need for Wireless Sensors
There are seemingly several options for measuring the temperature and humidity distribu-
tions inside a data center. For one, thermal images such as the one shown in Figure 3.1
visualize temperature variations over the camera’s view frame. However, continuously cap-
turing thermal images throughout the data center is prohibitively expensive. Alternatively,
modern servers have several onboard sensors that monitor the thermal conditions near key
35
Chapter 3. Intra-network Interference in Collection 3.2. Background
Figure 3.1: A row of computer racks inside a data center (left) and the corresponding infraredimage representing the spatial temperature distribution (right).
Time (minutes)
Load
(%
)
● ● ● ● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
● ●
●
● ●
●
● ●
● ● ●
● System loadCPU board tempCPU 1 temp
IO Board tempIntake air tempOutput air temp
1 6 11 16 21 26 31 36 41 46 51 560
10
20
30
40
50
60
70
80
90
0
10
20
30
40
50
60
70
80
90
Tem
pera
ture
(C
)
Figure 3.2: Temperature measured at different locations in and around an HP DL360 server.Also shown is the server’s CPU load. Internal sensors reflect the server’s workload instead ofambient conditions.
server components, such as the CPUs, disks, and I/O controllers. These sensors are used to
detect and prevent hardware failures due to overheating rather than to sense the data cen-
ter’s ambient environment. Some recent servers also have temperature sensors at the air
intake and administrators can estimate room conditions from these sensors. However, for
servers that do not have sensors at the air intake, it is difficult to accurately estimate the
room temperature and humidity from other onboard sensors. Figure 3.2 plots the tempera-
ture measured at various points along with the CPU utilization for an HP ProLiant DL360
(G3) server with two CPUs. Air intake and output temperatures are measured with external
sensors near the server’s front grill and its back cover. It is evident from this figure that in-
36
Chapter 3. Intra-network Interference in Collection 3.2. Background
ternal sensors are quickly affected by changes in the server’s workload, rather than reflecting
ambient conditions.
Intake air temperature (IAT) is important also for auditing purposes. Server manufactur-
ers and data center facility management contracts usually specify server operation conditions
in terms of IAT. For example, the HP ProLiant DL360 (G3) servers require IAT to range from
50 ◦ to 95 ◦F (10 ◦ to 35 ◦C). Therefore, it is necessary to place external sensors at regular
intervals across the servers’ air intake grills to monitor IAT.
The communication mechanism used to retrieve collected measurements is another cru-
cial aspect in the system design. Options in this case are divided in two categories: in-band
and out-of-band. In-band data collection routes measurements through the server’s operat-
ing system (OS) to the data center’s (wired) IP network. The advantage of this approach is
that the network infrastructure is (in theory) available, and the only additional hardware
necessary is relatively inexpensive USB-based sensors. However, data center networks are
in reality complex and fragile. They can be divided into several independent domains not
connected by gateways. Traversing across network boundaries can lead to serious security
violations. Finally, the in-band approach requires the host OS to be always on to perform
continuous monitoring. Doing so however would prevent shutting down unused servers to
save energy.
Out-of-band solutions use separate devices to perform the measurements and a separate
network to collect them. Self-contained devices provide higher flexibility in terms of sensor
placement, while a separate network does not interfere with data center operations. How-
ever, deploying a wired network connecting each sensing point is undesirable as it would add
thousands of network endpoints and miles of cables to an already cramped data center.
For this reason, wireless networks are the only feasible option. Moreover, networks based
on IEEE 802.15.4 radios [26] (or 15.4 for short) are more attractive compared to Bluetooth or
37
Chapter 3. Intra-network Interference in Collection 3.2. Background
WiFi radios. The key advantage is that a 15.4 network has a simpler network stack compared
to alternative solutions. This simplicity has multiple implications. First, sensing devices need
only a low-end MCU such as the MSP430 [90] thus reducing the total cost of ownership and
implementation complexity. Second, the combination of low-power 15.4 radios and low-power
MCUs leads to lower overall power consumption. The need for low-power consumption will
become apparent when we present the mechanism used to power multiple sensing devices
from the same power source.
At the same time, there are significant challenges when using 15.4 networks for data
center sensing, due to low data throughput and high packet loss rate. The maximum trans-
mission rate of a 15.4 link is 250 Kbps while effective data rates are usually much lower due
to MAC overhead and multi-hop forwarding. Furthermore, the lower transmission power1
can lead to high bit error rates especially in RF-challenging environments such as data cen-
ters. To quantitatively understand these challenges, Section 3.2.3 presents results from an
RF environment survey at a Microsoft production data center.
3.2.2 Application Requirements
Thermal and air dynamics in data centers can be complex. Figure 3.1 is a thermal image
captured by an infrared camera that allows us to gain an understanding of the underlying
spatial variability. This thermal image exposes the temperature variations that exist over
the air intakes of multiple server racks. One can observe temperature differences larger than
10 ◦F across various heights of the same rack, as well as significant differences in the tem-
perature distribution patterns across different racks. Based on this observation, we decided
to instrument at least every third rack with sensors.
In order to rapidly detect hot spots created by complex air and thermal dynamics, the1The TI CC2420 802.15.4 radio we use, transmits at 0 dBm, or 1 mW [89].
38
Chapter 3. Intra-network Interference in Collection 3.2. Background
system needs to provide sensing with high temporal as well as spatial fidelity. To illustrate
the rapid change in temperature variations, we have recorded temperature changes as much
as 10 ◦C within the five minutes immediately following server and CRAC state transitions.
Based on these observations, we selected a 30-second sampling rate for RACNet, to promptly
detect and mitigate abnormal thermal conditions. The sampling rate can be even higher
when troubleshooting hot spots or when sampling different sensors (e.g., monitoring server
power consumption).
In order to support effective cooling control and dynamic workload distribution, RACNet
must provide data yields of 95%, or higher. Ideally, these measurements need to be collected
before the next samples are generated. When this is not feasible, we still need to archive the
data so they can be used for long-term decision making.
3.2.3 Data Center RF Environment
Data centers present a radio environment different from the ones that previous sensor net-
work deployments faced. This is intuitively true as metals are the dominant materials inside
a data center. In addition to switches, servers, racks, and cables, other metallic obstacles
include cooling ducts, power distribution systems, and cable rails. Given this departure from
RF environments studied in the past (e.g., [81,106]), characterizing the data center environ-
ment is crucial to understanding the challenges it poses on reliable data collection protocols.
We performed a site survey by uniformly distributing 52 Genomotes in a production data
center spanning an area of approximately 12,000 sq-ft. The motes were placed on the top of
the racks, following a regular grid pattern with adjacent nodes approximately 35 feet from
each other. During the experiment, all nodes took turns broadcasting 1,000 128-byte packets
with an inter-packet interval of 50 ms. All nodes used the 802.15.4 frequency channel 26
39
Chapter 3. Intra-network Interference in Collection 3.2. Background
PRR (%)
Fre
quen
cy (
%)
0 10 20 30 40 50 60 70 80 90 100
010
2030
4050
Figure 3.3: Distribution of packet reception ratios (PRR) across all the links from a 52-nodedata center site survey. A large percentage of the network’s links exhibit non-trivial lossrates.
and transmitted their packets without performing any link-layer backoffs. Upon receiving a
packet, each receiver logged the Received Signal Strength Indication (RSSI), the Link Quality
Indicator (LQI), the packet sequence number, and whether the packet passed the CRC check.
We summarize the results from this survey below:
Neighborhood Size. We found that on average 50% of all the nodes are within a node’s
communication range and that a node’s neighborhood can include as many as 65% of the net-
work’s nodes. Moreover, production deployments have significantly higher neighborhood size
as they consist of hundreds of nodes deployed over the same space. Therefore, it is imperative
to devise mechanisms that minimize packet losses due to contention and interference.
Packet Loss Rate. Figure 3.3 illustrates the distribution of packet reception ratios (PRR)
over all the network links. While the majority of the links have low loss rate (i.e., < 10%),
a significant percentage of links experience high number of losses. This observation suggests
that, even in dense networks, data collection protocols must discover high-quality links and
avoid low quality links in order to build end-to-end paths with low loss rates.
Link Qualities. Both RSSI and LQI measurements have been used to estimate link qual-
ities [8,83]. RSSI measures the signal power for received packets, while LQI is related to the
chip error rate over the packet’s first eight symbols (802.15.4 radios use a Direct Sequence
40
Chapter 3. Intra-network Interference in Collection 3.2. Background
RSSI (dBm)
PR
R (
%)
−95 −90 −85 −80 −75 −70 −65 −60 −55 −50
025
5075
100
LQI
PR
R (
%)
60 65 70 75 80 85 90 95 100 105 110
025
5075
100
Figure 3.4: Boxplots of link PRR as a function of RSSI and LQI values. Boxplots show thesample minimum, first quartile, median, third quartile, and sample maximum. Links withRSSI > −75 dBm and LQI > 90 have persistently high PRR.
Spread Spectrum encoding scheme). Indeed, the results shown in Figure 3.4 indicate that
there is a strong correlation between RSSI/LQI and packet reception rates. Based on these
results, one could opt for an RSSI threshold of -75 dBm to filter out potentially weak links.
Selecting this conservative threshold removes a large number of links. Fortunately, the net-
work remains connected because each node has many neighbors with high RSSI links. Note
that RSSI readings are highly sensitive to the background RF noise, and thus the -75 dBm
RSSI threshold might not be suitable for all networks.
The site survey also revealed that approximately 3.43% of losses in the network were due
to CRC failures. However, since the 16-bit MAC-layer CRC used by the 802.15.4 standard is
relatively weak, it might not detect all corrupted packets. To understand the extent of this
potential problem, we included an additional 16-bit CRC that covers the application-level
payload of every packet. As many as 1% of the total number of packets that passed the MAC-
layer CRC failed the one at the application level. It is thereby crucial for applications such as
RACNet that require high data integrity to opt for a two-level CRC scheme.
Background RF Interference. Figure 3.5 shows the background noise distribution
measured on each of the sixteen 802.15.4 frequency channels available on the 2.4 GHz ISM
band. The measurements were collected by a mote that sampled its RSSI register at a fre-
quency of 1 kHz while no other 802.15.4 radios were active. A total of 60,000 samples were
collected on each channel. Because the data center in which the measurements were taken
41
Chapter 3. Intra-network Interference in Collection 3.2. Background
802.15.4 channels
RS
SI (
dBm
)●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
802.11 channel 1 802.11 channel 6 802.11 channel 11●
●
●
1%5%10%
11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26
−90
−70
−50
Figure 3.5: Background noise distribution across all 802.15.4 frequency channels in the2.4 GHz ISM band. Each of the circumferences is proportional to the occurrence frequency ofthe corresponding RSSI level. Channels 15, 20, 25, and 26 are relatively quiet compared toother channels.
has a considerable level of 802.11 traffic, 802.15.4 channels that overlap with 802.11 channels
experienced higher noise levels. On the other hand, 15.4 channels 15, 20, 25, and 26 are rela-
tively quiet. This motivates us to takes advantage of all the quiet channels simultaneously by
dynamically partitioning the network’s nodes over multiple collection trees, each operating
on a different frequency channel.
3.2.4 Genomotes
Some of the aforementioned challenges are partially addressed by RACNet’s hardware de-
sign. Figure 3.6 presents a pair of Genomote sensors specifically designed for RACNet by
Microsoft Research.
The wireless master node (shown on the left) and several wired sensors (one example
shown on the right) form a (wired) daisy chain to cover one side of a rack, collecting data
at different heights. This design increases sensing coverage and reduces the number of con-
tending radios in the same space, without sacrificing deployment flexibility. However, even
with the chain design, there are easily several hundred wireless master nodes within a data
center colocation facility. For the remainder of this chapter, we consider only the network
42
Chapter 3. Intra-network Interference in Collection 3.2. Background
Figure 3.6: Two types of Genomotes designed for RACNet. The wireless node (on the left)controls several wired nodes (on the right) to reduce the number of wireless sensors withinthe same broadcast domain.
among the wireless master nodes, treating the whole chain as a single node with multiple
sensors.
The master node also has a flash memory chip that caches data locally to mitigate tempo-
rary connectivity variations. The whole chain is powered by a USB port connected to a server
or a wall charger. Using a USB connection to power the whole mote chain means that un-
like many previous sensor networks, power is not a critical concern in RACNet. At the same
time, one needs to keep in mind that the USB specification regulates a maximum current of
100 mA that a USB port can provide to a foreign device. This limitation means that it would
be impossible to use a server’s USB port to power multiple (or even a single) WiFi-based
sensing devices. Finally, we note that using the same USB port to carry measurements is not
an viable option because it requires the installation of additional software on the servers –
something that is not administratively possible in our environment.
43
Chapter 3. Intra-network Interference in Collection 3.3. WRAP
3.3 WRAP
Reliable data collection lies at the center of RACNet. Over the past two years, we have de-
veloped two data collection protocols, Reliable Data Collection Protocol (rDCP) and Wireless
Reliable Acquisition Protocol (WRAP), to achieve high data reliability and high network effi-
ciency. Both rDCP and WRAP have a network layer that controls the topology and a transport
layer for data retrieval. They are unique in the way that they combine centralized and dis-
tributed decision making to achieve scalability and responsiveness. Specifically, the network
layer (§ 3.3.2) constructs collection trees across multiple channels in a distributed way. On
the other hand, the transport layer implements rDCP-specific and WRAP-specific algorithms
to prevent network congestion and reliably retrieve data from each of the network’s nodes.
Specifically, WRAP improves upon rDCP’s request-reply mechanism with a centralized token
passing mechanism to achieve a network throughput close to the network capacity.
Since WRAP supersedes rDCP, the rest of this chapter will focus the discussion on WRAP.
3.3.1 Protocol Design Overview
From an architectural perspective, WRAP’s design is at the center of the spectrum between
distributed and centralized data collection. At one end of this spectrum, nodes participating
in a distributed data collection protocol collaborate to construct a common routing tree and
independently forward data as soon as possible [18, 101]. The derived network topology can
quickly adapt to link quality changes or node failures. However, the lack of coordination can
lead to channel contention and eventually packet losses, especially under high network load.
At the other end of the design space lies the centralized approach, in which the gateway con-
trols the operation of the entire network leveraging its ample computational resources and
complete knowledge of the network topology [61, 84]. Nodes simply report their local chan-
44
Chapter 3. Intra-network Interference in Collection 3.3. WRAP
nel conditions to the gateway which in turn determines the routing paths and orchestrates
data downloads. While centralized approaches achieve high reliability and manageability,
the control traffic to and from the gateway can add significant overhead. For example, in
order to compensate for link and node failures, all nodes’ children lists must be collected fre-
quently. However, such information scales with the number of network links, which for dense
networks can grow with the square of the number of nodes.
WRAP follows a hybrid approach whereby nodes determine the routing topology in a dis-
tributed way while the gateway coordinates data transport using a centralized token passing
mechanism. Specifically, the gateway periodically generates a token that traverses the de-
rived routing tree in a depth-first manner. Only the tree node that holds the token can
transmit packets before passing the token to the next node. By allowing only a single trans-
mitter at any point in time, token passing bypasses the inter-flow contention that can lead to
congestion and packet loss. This is especially important close to the root of a dense network
whereby concurrent flows are very likely to interfere with each other. In this respect WRAP
is a congestion avoidance mechanism, unlike existing centralized [65] or distributed [73] con-
gestion control protocols. Moreover, by eliminating congestion as a possible cause of packet
loss, WRAP removes the ambiguity that complicates the response of congestion control proto-
cols to missing data. Specifically, most WSN congestion control mechanisms infer congestion
from packet losses, but lossy links can also cause packet losses. The data collection protocol
should react accordingly under these two situations. While lowering the network traffic is
helpful under network congestion, increasing the rate over lossy links can increase the odds
of packet reception.
This division of responsibilities ensures timely adaptation to link quality variations and
at the same time gives every network node a fair share of the network’s resources without
contention. RCRT is another example of a hybrid protocol that adds centralized coordination
45
Chapter 3. Intra-network Interference in Collection 3.3. WRAP
LQI
PR
R (
%)
●
●
●
●
● ●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
● ●● ●
●
●
60 65 70 75 80 85 90 95 100 105 1100
2550
7510
0
Figure 3.7: Piece-wise linear approximation of PRR from LQI. The dots are the average PRRfor each LQI obtained from the site survey results.
– the rate control information – to an otherwise distributed protocol [65]. However, imposing
a single transmission rate for the entire network is inevitably biased towards the weakest
node (i.e., the one behind the most lossy link) and artificially degrades the overall network
throughput.
3.3.2 Topology Control
The network layer maintains robust data collection trees rooted at the network’s gateways.
The mechanism’s distributed nature allows nodes to independently react to topology changes
including degraded link qualities and node failures. Since WRAP also uses the tree to deliver
downstream traffic such as requests for lost data packets, we focus on building bi-directional
trees (BiTrees) with high quality bi-directional links.
Parent Selection
Gateways initiate BiTree construction by broadcasting HEARTBEAT messages. HEART-
BEATs include fields that represent the node’s status, including its hop distance from the
root, its parent node ID, the number of children, and the path quality metric to the root. We
describe the importance of communicating the number of children in Section 3.3.2.
Upon receiving a HEARTBEAT message, a node takes the following steps. First, the
46
Chapter 3. Intra-network Interference in Collection 3.3. WRAP
incoming HEARTBEAT message needs to have an RSSI above a threshold to avoid links
with high loss rates (§ 3.2.3). Next, the node checks whether the potential parent has already
reached its maximum number of children. If not, the next step is to evaluate the path quality
to the gateway via this potential parent by computing the path expected transmission count
(PETX) as follows:
PETXj =∑l∈P
1
PRRl= PETXi +
1
PRRi,j(3.1)
where j is the current node, i is its potential parent, and P is the path from j to the gateway
via i.
To compute PETXj recursively, the PETXi is included in the HEARTBEAT messages.
However, estimating PRRi,j directly from HEARTBEATs would require multiple message
rounds. Instead, we take advantage of the Link Quality Indicator (LQI) available from radio
chips such as the TI CC2420 [89], to reduce control message overhead. Specifically, we use
the piece-wise linear approximation shown Figure 3.7 to estimate a link’s PRR based on its
LQI. We note that while the curve shown in Figure 3.7 was derived from the site survey data,
it resembles the approximation used in [8].
The node selects the upstream neighbor with the smallest PETX as its potential parent
and initiates a TREE JOIN request. The parent also estimates the link quality from this
potential child in the upstream direction before replying with a GRANT message. Otherwise,
the TREE JOIN operation times out. This two-way handshake has two benefits. First, it
serves as an explicit agreement between the parent and the child node that both have the
resources to relay messages for each other. Second, since we require a BiTree for data down-
loading, it is important to ensure the link quality in both directions, as wireless links can be
asymmetric [106].
47
Chapter 3. Intra-network Interference in Collection 3.3. WRAP
Coordinated Beaconing
As described above, nodes broadcast HEARTBEAT messages to construct and maintain Bi-
Trees. It is therefore desirable to transmit multiple HEARTBEATs in a short amount of time,
to accelerate the tree construction process. However, in large and dense networks, this can
lead to broadcast storms and severe collisions, eventually affecting the quality and stability
of the resulting tree.
A simple and low-maintenance solution would be to adopt a contention-based approach,
in which nodes contend for the radio medium. However, this approach is ill-suited for dense
networks because the large number of HEARTBEATs is likely to cause collisions and large
delays. At the same time, a TDMA-based protocol that assigns exclusive time slots to each
node within the same interference range is cumbersome as it requires time synchronization
and additional control traffic to set up the schedule.
Instead, WRAP uses a reduced contention mechanism to regulate the broadcast of HEART-
BEAT messages. Specifically, WRAP defines a time slot of length T that starts immediately
after a node P broadcasts its HEARTBEAT message. The time slot is further divided into
two uneven sections according to the number of children that P already has and the number
of additional children that P can support. The first section is reserved for the HEARTBEAT
messages sent by P ’s children, while the second is used by nodes that are not part of the tree
to initiate the handshake process with P . Nodes that receive P ’s HEARTBEAT randomly
select a time within the appropriate section to transmit their message.
While this mechanism reduces contention among children of the same parent, it does
not guarantee a collision-free neighborhood. Specifically, we do not coordinate among nodes
within the same broadcast domain that connect to different parents. Instead, we let them
contend for the medium by performing CSMA. This decision of partially collision-free sched-
48
Chapter 3. Intra-network Interference in Collection 3.3. WRAP
Node N1
Node N2
Channel C1
Node N3
Channel C2
HEARTBEAT JO
IN_REQ
JOIN_GRANT
HEARTBEAT
JOIN_GRANTJ
OIN
_REQ
HEARTBEAT
HEARTBEAT
Remove N1
from children list
Figure 3.8: Tree construction with channel diversity. During the channel scanning phase,node N1 joins two trees. Then, it decides to stay at channel C2, and N2 at channel C1eventually removed N1 from the child list.
ule alleviates the burden of the network to maintain a perfectly collision-free schedule.
3.3.3 Channel Diversity
A RACNet system may consist of hundreds of nodes within one data center. One way to in-
crease data throughput and reduce data latency is by using multiple gateways. To do so, we
take advantage of channel diversity to build multiple BiTrees rooted at different gateways,
each on a different channel frequency. Previous work has shown that simultaneous commu-
nications over two-channel-apart 802.15.4 channels do not interfere with each other [102].
This section addresses the challenges of building multi-hop BiTrees over multiple channels
in a distributed way.
Construction of Multiple BiTrees
Every RACNet gateway has a fixed channel assigned by the operator. Non-gateway nodes
start by scanning channels sequentially, looking for a tree to join. Since gateways con-
tinuously perform data collection, a node can first overhear the network traffic and decide
whether the channel potentially has a tree that it can join. In addition, a node can bound its
wait time on each channel to (little over) one HEARTBEAT time interval because gateways
49
Chapter 3. Intra-network Interference in Collection 3.3. WRAP
periodically initiate new rounds of HEARTBEAT beaconing. A node joins the first tree using
the two-way handshake mechanism described above. However, the node joins any subsequent
trees only if the estimated quality of the new path is better than the one on the current tree.
WRAP follows a transaction model when constructing BiTrees across different channels.
It is possible that a node (temporarily) joins multiple trees as it actively scans all available
channels. However, nodes in this state do not broadcast HEARTBEAT messages to recruit
children. This is to limit further disturbance in the candidate trees that the node later decides
not to join. When the scanning phase ends, the node switches to and stays in the last tree it
joined. Finally, a node’s parent takes its HEARTBEAT transmissions as an indication of its
commitment to the tree. Other candidate parents eventually time out and remove the node
from their children lists.
Nodes can significantly reduce their channel scanning time with the gateways’ help. Specif-
ically, gateways maintain the list of all channels they collectively occupy and include this
information in their HEARTBEAT messages. Therefore, after receiving one HEARTBEAT
message, nodes immediately know all available channels.
Balancing Multiple BiTrees
As nodes join and leave the network or link qualities change, the sizes of different BiTrees
can become unbalanced. We quantify the size of a tree by its sum of hops ∆, or the total
path length from each node in the tree to the root. As we will show in Section 3.4, ∆ largely
determines the overall time necessary to finish a data collection round. For this reason WRAP
uses ∆ to balance the load among all the network’s trees.
WRAP implements a distributed algorithm for balancing BiTrees. WRAP periodically
checks the ∆’s of different trees, and it initiates the channel-balancing process by sending a
START BAL message that propagates through the tree with the largest ∆. WRAP utilizes
50
Chapter 3. Intra-network Interference in Collection 3.3. WRAP
two mechanisms to avoid network instability: (i) it restricts the channel-balancing process to
the gateway with the largest ∆, and (ii) it tolerates certain amount of imbalance in ∆. Let
∆avg be the average among all trees. A gateway b∗ starts the channel-balancing process only
under the following conditions:
∆b∗ −∆avg > δ,and
b∗ = argmaxb∈B(∆b) (3.2)
where B is the set of all gateways and δ is a threshold parameter that controls the amount of
tolerable imbalance.
The START BAL message contains the probabilities for switching to each of the other
channels. Switching probabilities are defined to be higher for more under-utilized channels.
Specifically, a node connected to the tree rooted at b∗ will decide to switch out with prob-
ability Pout =∆b∗−∆avg
∆b∗. Once the node decides to leave b∗’s tree, the probability that it
switches to another tree Bi 6= b∗ is set as follows:
Pi = 0, if ∆i ≥ ∆avg (3.3)
Pi =∆avg −∆i∑
b∈B and ∆b<∆avg
(∆avg −∆b), if ∆i < ∆avg (3.4)
Intuitively, we attempt to migrate the extra nodes from gateway b∗ to underloaded gateways,
based on their degrees of under-utilization. In other words, more nodes will attempt to join
the tree with fewer nodes. Finally, if the node can not find a parent in the target channel, it
returns to its original channel.
3.3.4 Data Collection
The transport layer reliably collects data to RACNet gateways along the network’s BiTrees.
Rather than having nodes initiate data uploads asynchronously, WRAP coordinates the net-
51
Chapter 3. Intra-network Interference in Collection 3.3. WRAP
work traffic to reduce radio contention. Our first attempt to achieve this coordination is
Reliable Data Collection Protocol (rDCP), a pull-based approach in which gateways initiate
data collection by sending requests to individual network nodes. As discussed later in this
section, this approach can incur significant overhead. This observation motivated the token
passing approach adopted by WRAP.
rDCP
We start the discussion on reliable data collection by presenting rDCP. Since WRAP suc-
ceeds rDCP and improves the data collection performance by addressing rDCP’s overhead,
presenting rDCP provides the motivation and intuition to WRAP’s design.
rDCP uses a pull-based approach to coordinate data download traffic in the network. The
basic idea is for the rDCP gateways to initiate data streaming by sequentially sending source-
routing requests to each tree node, which then responds by streaming the requested data
along the reverse path. This request-reply operation introduces two sources of overhead.
The first overhead comes from the steps taken by the rDCP gateways to learn the network
topology. Since data requests from the rDCP gateway are encapsulated inside source-routing
packets, the rDCP gateways periodically go through a child list collection phase. Specifically,
the rDCP gateways collect the child list from the network hop by hop. As the tree root, the
rDCP gateways know the first-hop nodes, and they learn the second-hop nodes by retrieving
child lists from the first-hop nodes. This process repeats until the rDCP gateways query all
nodes in the network. Since the rDCP gateways cannot collect data samples while retrieving
child lists, the network goodput is reduced.
The second overhead comes from the round-trip delay in retrieving each node’s data sam-
ples. As nodes stream data samples only after receiving data requests from their gateway,
the round-trip delay reduces the network goodput by as much as 50% in the worst case.
52
Chapter 3. Intra-network Interference in Collection 3.3. WRAP
Token Passing
With the token passing mechanism, WRAP does not require the gateways to have a priori
knowledge of the network topology. Rather, it relies on the network to determine the next
node that should hold the token. Since gateways continuously retrieve data from the network,
this property also removes the overhead of having a separate phase for collecting child lists
from all nodes in the network. The basic data collection protocol works as follows.
Gateways initiate a data collection round by passing the token to the first node on their
list of children (§ 3.3.2). Each token contains an unique 32-bit token ID so that nodes know
when a new round of data collection has started. WRAP tokens traverse the tree in a depth-
first order. After receiving the token, the node passes it sequentially to all of its children
in the tree. Once all of its children have finished transmitting their measurements, the
node streams the measurements it has accumulated since the last data collection round to
the gateway. To minimize the number of packet transmissions, nodes aggregate as many
measurements as possible in one packet. In practice, up to five such measurements fit in one
maximum-size (128-byte) 802.15.4 frame. This ability to aggregate multiple measurements
to a single packet is a side benefit of the architectural decision to decouple data collection from
data generation. For reasons explained later in the section, nodes send an empty packet if
they have no new measurements. When the gateway eventually receives the token back from
the network, it scans the measurements received and recovers lost packets by requesting any
missing sequence numbers.
Passing the token in a depth-first fashion ensures that the token travels every network
edge exactly twice during each data collection round. For example, in the two-level binary
tree shown in Figure 3.9, the edge visiting order is 1, 3, 3, 4, 4, 1, 2, 5, 5, 6, 6, and 2 (12 edges
in total). If breadth-first traversal were used instead, the token would travel each edge at
53
Chapter 3. Intra-network Interference in Collection 3.3. WRAP
S
A B
C D E F
1 2
3 4 5 6
Figure 3.9: An sample two-level binary tree.
least twice, because the token has to travel back to the gateway after each tree node. In the
case of Figure 3.9, this would lead to 16 edge traversals compared to 12.
WRAP further reduces the token passing overhead through inference. First, since a par-
ent forwards all the measurements from its children, it has the opportunity to inspect the
MORE DATA field in their packets and determine when the current child has sent its last
packet. When this happens, the parent assumes that the child has released the token. The
network assumes lost token if the parent fails to receive the last packet from the children,
and the gateway would initiate token recovery process after a timeout as described in the
next paragraph. Second, since children can overhear packets sent by the parent, the parent
piggy-backs the next child node ID inside the relayed packet when it is ready to pass the
token.
Although WRAP aggressively performs link-level retransmissions, the network can still
lose the token for various reasons such as node failure. WRAP puts the burden of token
recovery on the gateway. Since nodes stream data only when they hold the token, if the
gateway’s idle timer expires while waiting for incoming data, it assumes that the token has
been lost and regenerates a token with the same ID. The unique token ID allows nodes to
differentiate an old token from a new token, and nodes that have held a token with the same
ID will immediately release it.
54
Chapter 3. Intra-network Interference in Collection 3.3. WRAP
End-to-end Reliability
WRAP implements a NACK-based, end-to-end data recovery scheme, whereby gateways re-
quest end-to-end retransmissions for missing sequence numbers. To amortize the round trip
time incurred in the data retransmission process, WRAP accumulates multiple data retrans-
mission requests destined to the same node.
Similar to rDCP, WRAP encapsulates downstream data requests inside source-routed
packets. This requires gateways to have knowledge of their tree topology. To do so, nodes
in the tree piggy-back their parent node ID to the end of the data stream that they send to
their gateway. Based on this process, a gateway can rebuild the complete tree topology at the
end of a data collection round.
To ensure data integrity, WRAP performs Cyclic Redundancy Checks (CRC) at both the
packet and the application level. We decided to add a CRC check on application payloads, be-
cause the 16-bit CRC used by 802.15.4 provides limited protection against corrupted packets
(§3.2.3). Gateways discard messages that fail either CRC.
Data Reliability and Integrity
To achieve end-to-end reliability, WRAP employs both link-level and end-to-end retransmis-
sions. Modern radios, such as the TI CC2420, can automatically generate 802.15.4 acknowl-
edgments for each successfully received packet. However, such hardware acknowledgments
can cause false positives because the system can drop acknowledged packets before they
reach the application. TinyOS 2.x, the software platform that WRAP is developed on, has a
feature called software ACKs that allows instead the application to trigger 802.15.4 acknowl-
edgments. Therefore, software ACKs can achieve equivalent functionality but with higher
confidence. In addition to using hop-by-hop software ACKs, WRAP also implements end-to-
55
Chapter 3. Intra-network Interference in Collection 3.3. WRAP
end negative acknowledgments. Specifically, after a node finishes streaming the requested
data block, the gateway scans the received data stream for missing records and sends re-
transmission requests.
Self-Paced Data Streaming
To stream data efficiently, the source node must determine the inter-packet transmission
interval that minimizes self-interference and end-to-end delay. WRAP adopts a technique
similar to the one proposed in [36] whereby a node estimates the inter-packet delay by mea-
suring the time between transmitting the last packet of a batch and the last time it overhears
the same packet forwarded by nodes upstream. To take into account the whole path, parents
propagate the local estimates downstream via the HEARTBEAT message. Then, each child
node updates its local inter-packet value to the maximum of the previous local value and the
one in the HEARTBEAT message.
Data Time Stamping
RACNet relies on a large number of sensors to perform high fidelity data center sensing. To
better correlate the measurements at different locations and generate useful results such as
heat maps, nodes must be time synchronized and sample their sensors at the same time.
WRAP synchronizes the nodes’ clocks through a mechanism that adopts techniques proposed
in the Flooding Time-Synchronization Protocol (FTSP) [55].
In more detail, WRAP assumes that gateways maintain globally synchronized clocks. This
is a reasonable assumption as protocols such as the Network Time Protocol (NTP) are readily
available in the data center. Gateways timestamp each HEARTBEAT message with the cur-
rent global time immediately before they transmit them. Upon receiving the HEARTBEAT
56
Chapter 3. Intra-network Interference in Collection 3.4. Protocol Evaluation
message, nodes create a synchronization point (i.e., a pair of global and local time stamps).
Since different nodes have different clock frequencies and drifts [55], WRAP takes multi-
ple synchronization points on each node and applies linear regression on these data points
to model the relation between the local and the global clock. Finally, when a sensor takes
measurements, it uses the computed model to convert its local time to the global time.
3.4 Protocol Evaluation
We evaluate WRAP’s design using experiments conducted on two separate testbeds. While
simulators such as TOSSIM [47] are readily available, they cannot match the full realism
that testbeds provide and can therefore lead to inaccurate or misleading conclusions.
The first testbed consists of 62 TelosB motes [71] deployed over a single floor of an office
building. While different from the Genomote used in the data center, the TelosB mote shares
the same TI CC2420 radio [89] and MSP430 microcontroller [90] with the Genomote. The
motes are connected to the building’s wired IP network through Ethernet-equipped USB
bridges. This configuration allows us to use Ethernet as the management channel to capture
detailed timing information, collect management data, and reprogram the devices.
The second testbed resides inside a data center environment of approximately 11,000 sq-
ft. While the office testbed allows us to quickly evaluate different protocol parameters (i.e.,
sampling rate), the data center testbed allows us to test WRAP’s scalability at the system’s
target environment. We varied the number of nodes in the data center and placed them
following a grid pattern similar to the one used in the RF survey from Section 3.2.3 and the
production deployment described in Section 3.5.
57
Chapter 3. Intra-network Interference in Collection 3.4. Protocol Evaluation
3.4.1 Protocol Parameters
Topology Maintenance. Settling time is defined as the time necessary for a node to select
a stable parent during the system initialization or channel-switching phases. Only after the
node has selected a stable parent can the gateway download data from it. Furthermore, the
settling time also affects when a node’s descendants join the collection tree. It is therefore
desirable to reduce this settling time while generating stable and high-quality trees.
WRAP regulates HEARTBEAT messages by having children nodes transmit during a ran-
dom time selected from their parent’s local slot of size T (§ 3.3.2). Therefore a larger T re-
duces the probability of collisions and allows nodes to evaluate more potential parents. On
the other hand, larger T values lead to longer delays until the HEARTBEAT messages prop-
agate throughout the tree.
We perform the following experiment to determine a lower bound on T as a function of
the neighborhood size. We place a variable number of nodes within one-hop distance from
a receiver. For each neighborhood size, we vary T and count the number of HEARTBEAT
messages received when each of the transmitters randomly select their HEARTBEAT trans-
mission time from [0, T ]. The receiver follows the procedure used by a node joining the tree.
Namely, for each received HEARTBEAT, it updates its current estimate of the maximum
RSSI and LQI seen thus far and the potential parent that transmitted that message.
Figure 3.10 plots the minimum value of T necessary to receive all the transmitted HEART-
BEAT messages as a function of the neighborhood size. It it evident that T grows linearly
with the number of potential parents. Administrators can set T according to the expected
neighborhood size of the deployment. Considering the size and the density of the produc-
tion network, we estimate a neighborhood of size 80. Extrapolating from the results in Fig-
ure 3.10, we would need to set T to be around 1,600 ms to hear all neighboring nodes (in
58
Chapter 3. Intra-network Interference in Collection 3.4. Protocol Evaluation
Neighborhood size
Slo
t siz
e (m
sec)
● ● ●●
●● ●
●● ●
●● ●
●
●
● ●● ●
●
●
010
020
030
040
050
0
4 6 8 10 12 14 16 18 20 22 24
Figure 3.10: Minimum WRAP backoff slot size necessary to hear HEARTBEAT messagesfrom a node’s neighbors. The curve grows linearly with the neighborhood size.
Num nodes
Sum
of h
ops
●
●
●
● ●
●
●
●
● ●
0 10 20 30 40 50 60
030
6090
120
180
150
01
23
45
67
8
Col
lect
ion
time
(sec
)
●
●
Sum of hopsCollection time
Figure 3.11: Average one-round collection time and the collection tree’s sum of hops as afunction of the testbed network size.
reality, we set T to be 800 ms to allow nodes to receive HEARTBEAT messages from 50% of
their neighbors).
Data Retrieval. WRAP gateways sequentially collect data via token passing. Thereby
the period of a token passing cycle is equal to the total tree collection time. WRAP tries
to minimize the collection time over the whole network by evenly distributing nodes over
available channels. It does so by using the sum of tree hops as its load balancing metric.
In this experiment, we test the hypothesis that the sum of tree hops can be used as a valid
proxy for the tree collection time and thus can be used to balance the network’s nodes among
59
Chapter 3. Intra-network Interference in Collection 3.4. Protocol Evaluation
the multiple trees. We estimate the duration of the data collection round, by running WRAP
on our lab testbed while varying the number of nodes from 10 to 50. Each node generates one
packet every 30 seconds. Figure 3.11 illustrates the network size, defined as the sum of all
tree hops, and the average data collection time for a single data collection round across the
whole network. It is clear that the sum of tree hops closely follows the collection time and
therefore can be used to balance the load over the different trees.
3.4.2 Application-Level Performance
Data center monitoring and control impose stringent data latency and yield requirements
on the data collection process. We evaluate how effectively WRAP meets these requirements
using two metrics: data yield, defined as the percentage of a node’s measurements that suc-
cessfully arrive at the gateway (including those that use retransmissions) and the average
inter-packet interval (IPI), defined as the time interval between the reception of two packets
with consecutive sequence numbers from the same node at the gateway. The inverse of the
inter-packet interval is a node’s goodput which is the rate by which unique data (i.e., not
including retransmissions) arrive at the gateway.
The data yield should ideally be 100% and the inter-packet interval (goodput) should be
equal to the node’s sampling interval (rate). Note that since WRAP can aggregate multiple
sensor measurements in one packet (§3.3.4), having an inter-packet interval that is higher
than the node sampling interval does not mean that the network is becoming persistently
backlogged. For example, if nodes generate one packet every 10 seconds and the inter-packet
interval is 12 seconds, then a node would have to aggregate two measurements every other
five rounds.
Furthermore, yields can fall below 100% due to lost packets. Nevertheless, achieving a
60
Chapter 3. Intra-network Interference in Collection 3.4. Protocol Evaluation
IPI (
sec)
5 6.1 5.3 3.9 5.05
15.6 2.6
4.95
71.4CTP WRAP RCRT
025
5075
100
125
5−sec 3−sec 2−sec
Yie
ld (
%)
020
4060
8010
0
5−sec 3−sec 2−sec
Figure 3.12: Boxplots of the inter-packet interval (IPI) and data yields under various samplingintervals on the 62-node lab testbed. The number on top of each IPI plot shows the average.
yield of 100% in the presence of network losses is still feasible if the gateway persistently
issues retransmission requests until it successfully receives all the data. Doing so however
can be detrimental to a node’s goodput since retransmissions consume network resources.
At the same time, transmitting too fast or not recovering from losses can lead to high inter-
packet intervals and low goodput. The challenge then for a data collection protocol is to
achieve both high data yields and low inter-packet intervals.
We compare WRAP to the Collection Tree Protocol (CTP) [18] and Rate-Controlled Re-
liable Transport (RCRT) [65]. CTP is a best-effort data collection protocol that does not
implement end-to-end retransmissions but rather relies on hop-by-hop retransmissions to
reduce packet loss. RCRT implements end-to-end reliability as well as congestion control
by controlling the senders’ transmission rates. We implemented the same application that
periodically samples a set of sensors on top of all three protocols. All of our code is written
in TinyOS 2.1 [46]. We use the default CTP version included with the TinyOS 2.1 distribu-
tion. We also ported RCRT to TinyOS 2.1 for fair comparison. In all cases, we use a single
frequency channel (26) because CTP and RCRT do not have a channel balancing capability.
Sampling Intervals. We first stress the three protocols by increasing the application’s
sampling rate. Figure 3.12 presents the three protocols’ behavior as we vary the application’s
sampling interval on the 62-node lab testbed. In each case, the experiment ran for at least 2
61
Chapter 3. Intra-network Interference in Collection 3.4. Protocol Evaluation
Node IDIP
I (se
c)
●●
●
● ● ●● ●
●●
●
● ●
● ●
●● ●
●
● ● ●
●●
● ● ●●
● ● ● ● ● ● ● ● ● ● ● ● ●
●
● ● ●●
● ● ● ● ● ● ● ●● ●
● ●
05
1015
2 7 12 17 22 27 32 37 42 47 52 57 62
Figure 3.13: Per-node inter-packet interval achieved by WRAP on the 62-node testbed. Nodestransmit one packet every three seconds. Nodes are labeled according to their location on thetestbed.
hours.
While CTP achieves low packet inter-packet intervals at low network loads, packet losses
increase drastically as the network becomes congested. RCRT reacts to this congestion by
lowering nodes’ transmission rate. We noticed that the RCRT gateway instructed the nodes
to reduce their rate to the minimum configured rate (i.e., one packet per 60 seconds) when
a 2-sec sampling interval was used. This dramatic reaction was due to the fact that the
gateway experienced multiple timeouts while waiting for nodes to acknowledge its requests
to lower their rates. Because RCRT treats such timeouts as further signs of congestion, the
gateway reacts by lowering the nodes’ rates even further. However, even this drastic reaction
did not achieve perfect yield in the case of 2-sec sampling.
While CTP ignores congestion, leading to lower yields, and RCRT reactively lowers the
nodes’ transmission rate, leading to higher inter-packet intervals, WRAP prevents congestion
from occurring in the first place by allowing only one network flow at any point in time. As the
results from Figure 3.12 suggest this strategy achieves high data yields and low inter-packet
intervals across all sampling rates.
Figure 3.13 illustrates the conditional fairness property of WRAP. Different parts of the
network can have different link quality, especially when the network physically spans a large
area. As mentioned before, WRAP tries to increase the packet reception rate with link-layer
62
Chapter 3. Intra-network Interference in Collection 3.4. Protocol Evaluation
IPI (
sec)
10 10.5 11 10 10.7 11.5 10 12.2
35.1
11.5 12.7
CTP WRAP RCRT0
3060
9012
015
0
50 nodes 75 nodes 100 nodes 150 nodes
Yie
ld (
%)
6070
8090
100
50 nodes 75 nodes 100 nodes 150 nodes
Figure 3.14: Boxplots of the inter-packet interval (IPI) and data yields under various networksizes on the data center testbed. Each node generates one packet every 10 seconds. Thenumber on top of each IPI plot shows the average.
retransmissions. However, since WRAP bounds the number of retransmissions, it is still
possible for nodes with low link quality to miss packets such as the token. The end result
is that more network capacity is allocated to nodes with better link quality. This property
is desirable as nodes with low link quality do not deteriorate the performance of the entire
network. Half of the testbed’s nodes (node 1 to 33) have lower link qualities compared to the
other half (possibly due to the building structure), and figure 3.13 shows that these nodes
have relatively higher inter-packet intervals.
Network Density. Next, we stress the protocols by increasing the network’s density. We
do so by uniformly arranging an increasing number of nodes, following a grid pattern, over
the same physical area in the data center testbed. Having more nodes in the same space
not only increases the amount of traffic that the network must deliver, but also increases
contention when node communications are not coordinated, as in the case of CTP and RCRT.
To evaluate the effects of network size on performance we fix the application sampling rate
to one packet every 10 seconds and increase the number of nodes from 50 to 150. We did
not perform the RCRT experiment with 150 nodes because the performance of the protocol
(inter-packet interval) degraded appreciably even with a network of 100 nodes. In each case,
the experiment ran for at least 2 hours.
As Figure 3.14 shows, the inter-packet interval increases slightly with the network size.
63
Chapter 3. Intra-network Interference in Collection 3.5. Deployment Results
Time (minutes)
Sum
of h
ops
0 500 1000 1500 2000 2500 3000 3500 4000
050
100
150
200
Figure 3.15: Sum of hops of the four WRAP trees in the production deployment as a functionof time. All trees stabilize after a few hours.
Interestingly, this increase was due to packet loss in the case of CTP and to the longer time
necessary to service the whole network in the case of WRAP which achieved 100% yields for
all network sizes. As in the previous set of experiments, RCRT reacted to the increased levels
of contention by reducing the nodes’ transmission rate. Specifically, for the 100-node network
the gateway set the nodes’ rate to minimum, or one packet every 60 seconds. In practice
however the inter-packet interval was even higher because even this decreased rate was not
able to prevent network losses and decreased data yields.
3.5 Deployment Results
RACNet has been deployed in several data centers. Next, we present results from a produc-
tion deployment at a 12,000 sq-ft. facility comprising 696 Genomotes, including 174 wireless
master Genomotes. The network uses up to four 802.15.4 channels. The system has been
running since mid 2008, collecting more than 2.5 million measurement records per day. Each
Genomote chain collects the following measurements every 30 seconds: three temperature
readings collected at different rack heights, one humidity measurement, and a measurement
indicating the availability of the USB power for network monitoring purposes.
Channel Balancing. Figure 3.15 illustrates WRAP’s channel-balancing behavior, using
64
Chapter 3. Intra-network Interference in Collection 3.5. Deployment Results
Time (days)
Net
wor
k Y
ield
(%
)
1 3 5 7 9 11 13 15 17 19 21
9798
9910
0
Figure 3.16: Boxplots of daily network yield from the production deployment over a period of21 days.
the sum of hops metric during the first three days of the deployment. The first part of the
figure (t < 500 min) shows significant fluctuations as the network was incrementally deployed
and tested. The second part of the figure (t > 500 min) corresponds to the phase during which
the gateways balanced the load across all four available channels. This phase ended when
the difference between the expected load across all channels and the actual load on each
channel is within 20% (cf. Section 3.3.3), and we did not observe significant variations after
this phase.
Data Yield. Figure 3.16 presents the per-node data yield over a period of three weeks
in the production data center deployment. The median yield across all days is above 99.5%,
while the lowest yield is always above 98%. This small packet loss is due to the fact that the
administrator limits the number of end-to-end retransmission requests to five before WRAP
stops the attempt to recover the packet.
Data Collection Latency. We computed the end-to-end latency as the difference be-
tween the time the data were timestamped by the node and the time they were inserted into
the back-end database. When the network was using three channels, we observed an average
latency of 16 seconds.
65
Chapter 3. Intra-network Interference in Collection 3.6. Discussion
3.6 Discussion
After the initial deployments of RACNet, additional experiments were performed to gain
insights on possible optimizations. This section briefly discusses these details and ongoing
work.
3.6.1 Comparison with TDMA
WRAP reduces radio medium contention with a token passing mechanism. However, one
can argue that maintaining a global TDMA schedule achieves the same goal. In fact, TDMA
protocols eliminate the need for the token to traverse the entire network topology during each
round of data download, and thus have lower control overhead. As our simulation results
below show, TDMA protocols have less control traffic in the expense of data delivery latency.
TDMA protocols reserve a fixed time slot for each transmitter in the network. Unfor-
tunately, if a transmitter does not have any pending data packet, its reserved time slot is
(generally) not transferable to other nodes with pending data packets. This translates to idle
and wasted time slots. On the other hand, with the token passing mechanism, nodes without
pending data packets can simply give up the token immediately. Therefore, the token passing
mechanism ensures that there are always data packets in the air to fully utilize the network
capacity.
We demonstrated this benefit of the token passing mechanism by simulating LiteTDMA [76]
with TOSSIM. LiteTDMA was designed for environmental monitoring applications with one-
hop topologies, which is similar to the network topologies we have seen at the data centers
(§ 3.2.3). The simulation ran for one hour on a star topology with 14 nodes one-hop away
from the gateway. We used the default parameters as suggested by the LiteTDMA authors 2.2Master slot duration is 50 ms. Slave slot duration is 10 ms. Each time frame has 32 slaves.
66
Chapter 3. Intra-network Interference in Collection 3.6. Discussion
The results show that LiteTDMA nodes generated a total of 3,250 messages, or 36% less than
that of WRAP. However, WRAP is able to delivery data with a latency of 40.165 ms, or 83.7%
faster than LiteTDMA.
3.6.2 Optimal Sampling Intervals
One goal of WRAP is to ensure the network throughput is close to the network capacity.
However, if the administrator sets the sampling interval too low, the nodes can collectively
saturate the available network capacity, leading to lost data due to limited buffers for back-
logged measurements. On the other hand, if the sampling interval is too high, the network
becomes underutilized. Therefore, an optimization to WRAP that we are currently consider-
ing is adaptive sampling intervals.
The first task in achieving adaptive sampling intervals is to detect when the network
becomes underutilized or over-utilized. Both cases can be detected by the gateway observing
the number of data packets it receives in each data download round. Specifically, the former
happens if the gateway does not receive any data packet, and the latter happens if individual
node transfers the maximum number of data records allowed in each download round.
The second task is to calculate the optimal sampling intervals. An naive approach is to
use the additive increase and multiplicative decrease (AIMD) mechanism used to set TCP’s
congestion window. Whenever the network is underutilized, the gateway would piggy-back
a command to the token that instructs nodes to lower their sampling intervals by a small
decrement, which is equivalent to additively increasing their sampling rate. Similarly, in
the case of over-utilization, the gateway would instruct nodes to double their sampling inter-
val. The drawback of this iterative approach is the time needed to reach stability. We are
investigating algorithms that can directly calculate the optimal sampling intervals.
67
Chapter 3. Intra-network Interference in Collection 3.7. Related Work
3.7 Related Work
Data Gathering Sensor Networks. Sensor networks have been used in several data gath-
ering applications, including environmental [21,97], habitat [34,87], and structural monitor-
ing [37, 103]. However, most prior work focuses on outdoor deployments, in which sensors
are sparsely deployed and power is the primary concern. On the other hand, RACNet has
distinctly different trade-offs. First, power consumption is no longer a determining factor.
Instead, performance issues such as data yields and latency become critical. Second, to mon-
itor large data centers at fine spatial granularities, large and dense sensor deployments are
necessary. In turn, this dramatic increase in scale leads to solutions that are qualitatively
different from those employed in past small-scale, sparse deployments.
In the last year or so, several companies have started to offer wireless sensor networks
for data center monitoring. Among them, Federspiel Controls [16] uses OEM sensors from
Dust Networks, which incorporate a frequency-hopping protocol called Time Synchronized
Mesh Protocol (TSMP) [13]. TSMP network can support up to 255 nodes with a fixed TDMA
schedule. Unfortunately, no results on the performance of TSMP are publicly available.
SynapSense [86] provides the LiveImaging solution for monitoring data center environment
conditions. Little information is known on the networking details of LiveImaging. To the best
of our knowledge, LiveImaging supports only five-minute sampling interval (i.e., ten times
slower than the desired data rate) and does not support multiple frequency channels. Both
solutions use battery powered sensors, which limit their sampling rate and system lifetime.
Data Collection Protocols. Data collection has been addressed at length in the sen-
sor network literature. A large portion of the existing work focuses on the power aspect of
the problem, aiming to minimize energy consumption through data aggregation (e.g., [54]),
ultra-low duty cycles (e.g., [7, 61]), or optimal sensor placement (e.g., [17]). In general, these
68
Chapter 3. Intra-network Interference in Collection 3.7. Related Work
systems are designed for low data rate applications with no delay requirements. WRAP faces
new challenges from the large and dense network configuration and the stringent reliability
requirements.
Werner-Allen et al. proposed a request-reply collection protocol called Fetch in the context
of their volcano monitoring project [97]. The base station first floods the network with the
request, which triggers the target node to return the data. Since data collection occurs infre-
quently, Fetch does not maintain a dissemination topology. Rather than using per-destination
requests, WRAP uses an efficient token passing mechanism to collect data from every node
in the network.
Lance is a data driven collection protocol that schedules downloads based on the value of
the data and the cost of delivery (e.g., energy) [98]. WRAP is a general-purpose data collection
protocol, focusing on reliably retrieving all the data to the gateway in a timely manner. Flush
is a reliable, single-flow transport protocol for bulk downloads in sensor networks [36]. WRAP
adopts the source rate control algorithm from Flush that minimizes the intra-flow interfer-
ence while streaming data to the gateway. However, WRAP also implements a mechanism
for maintaining a data collection tree and utilizes multiple frequency channel to adaptively
balance the load among multiple collection trees.
Meliou et al. introduced the concept of data gathering tours, whereby a network’s gateway
gathers data from a subset of the network’s nodes [57]. To do so the gateway calculates a
source route that visits all the nodes in the tour. While superficially similar to WRAP’s token
passing mechanism, data gathering tours are fundamentally different. First, while tours are
centrally planed, WRAP is a fully distributed protocol. Second, while data gathering focuses
on retrieving data from a subset of the network’s nodes, WRAP allows the collection of all the
data from very large networks.
The work closest to ours is the Rate-Controlled Reliable Transport (RCRT) [65]. However,
69
Chapter 3. Intra-network Interference in Collection 3.7. Related Work
as the results in Section 3.4 suggest, RCRT cannot scale to the size or the application data
rates necessary for data centering monitoring applications.
A number of multi-channel protocols have been proposed to address the challenges as-
sociated with high densities in sensor networks. First, several general multi-channel MAC
protocols [43, 107] assign nearby nodes to different channels to improve spatial reuse. The
frequent channel switching required in such node-based channel assignment protocols can
generate large overhead. Considering the data collection traffic pattern in our application,
we adopt a more lightweight alternative: tree-based channel assignment. Instead of assign-
ing different channels to individual nodes, we assign one channel to each spanning tree rooted
at a gateway. Channel switches occur only occasionally to re-balance the network’s load (e.g.,
when a gateway joins the network).
Recent work from Le et al. [42] and Wu et al. [102], uses channel assignment strategies
that are similar to ours. However, one relies on a centralized algorithm to assign chan-
nels [102], while the other achieves load balancing among different trees based on a control
theory approach [42]. Both mechanisms do not offer reliable data delivery. In comparison,
our approach is both distributed and reliable.
70
Chapter 4
External Interference
4.1 Introduction
We have witnessed over the last ten years a proliferation of wireless technologies that have
now become ubiquitous. Given the scarce availability of RF spectrum, many of these tech-
nologies are forced to use the same unlicensed frequency bands. For example, IEEE 802.11
(WiFi), IEEE 802.15.1 (Bluetooth) and IEEE 802.15.4 (ZigBee)1 all share the same 2.4 GHz
ISM band. Cross Technology Interference (CTI) is a consequence of this coexistence that can
lead to loss of reliability and inefficient use of the radio spectrum.
The majority of MAC protocols are designed to share the communication medium among
nodes that understand the same PHY layer. In this model, CTI is considered the same as ran-
dom background noise, despite the fact that it can cause significant performance degradation.
Even worse, system designers have little means to coordinate across networks that use dif-
ferent wireless standards since they usually belong to different administrative domains. CTI
is especially unfavorable for 802.15.4-based wireless sensor networks that have to counter
the effects of severe interference from ubiquitous 802.11 deployments.1Technically, the IEEE standards and industrial alliances are different. Since this chapter primarily considers
the PHY and MAC layer interactions among these wireless networks, we make no distinction between WiFi andIEEE 802.11, likewise between ZigBee and IEEE 802.15.4. Furthermore, to shorten the presentation, in the restof this dissertation, we will use the terms 802.15.4 and 15.4 interchangeably. We will also invoke the names of thestandard’s different variants (i.e., 802.11b/g) whenever necessary.
71
Chapter 4. External Interference 4.1. Introduction
14
247524702465246024552450244524402435243024252420241524102405
2412 2417 2422 2427 2432 2437 2442 2447 2452 2457 2462 2467 2472 2484
Zigbee Channels
WiFi Channels
11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26
1 2 3 4 5 6 7 8 9 10 11 12 13
2480
Figure 4.1: 802.11b/g and 802.15.4 frequency channels in the 2.4 GHz ISM band. Each 802.11channel is 22 MHz wide, while 802.15.4 channels are 2 MHz wide.
This result is supported by multiple previous studies that have shown that 15.4 per-
formance degrades significantly in the presence of 802.11 interference [22, 62, 82, 99]. The
unanimous pessimistic conclusion of those studies was that the only way to mitigate such
interference is for the 15.4 network to avoid the channels occupied by 802.11.
There are two ways to pursue interference avoidance: static channel assignment and dy-
namic channel assignment (i.e., channel hopping). In a static channel assignment scheme, one
assumes that the 802.11 network occupies a fixed number of channels and the 15.4 network
is provisioned to use frequency bands that are left unused by WiFi. As Figure 4.1 suggests,
this assignment leaves at most two 15.4 channels free from potential 802.11 interference.
Furthermore, static channel assignment may not work as planned due to node mobility [22]
and incremental WiFi deployments.
In dynamic channel assignment schemes, different nodes in a sensor network, or the
same node over different points in time, will use different 15.4 channels to avoid interfer-
ence from nearby WiFi sources. These mechanisms face two challenges: detecting the pres-
ence of 802.11 traffic [22, 62] and coordinating channel selection among 15.4 senders and
receivers [99]. In addition to the coordination complexity, interference avoidance mecha-
nisms leave large portions of the spectrum unused even when there is little 802.11 traffic
72
Chapter 4. External Interference 4.1. Introduction
in them. This inefficiency is especially damaging for large and dense sensor networks that
cannot support the desired application throughput using a single 15.4 channel [49,109].
Instead of trying to avoid interference from 802.11 traffic, the goal of this chapter is to
improve the coexistence of 15.4 and 802.11 networks that operate on overlapping frequency
channels. Our approach is based on insights derived from a thorough examination of the
interactions between the two radio technologies. In particular, we make several key observa-
tions that previous work has overlooked: (1) In the time domain, 802.11 packets are typically
much shorter than 15.4 packets, so they cause bursty bit errors in 15.4 packets. For exam-
ple, a 500 byte 802.11b packet lasts 400 microseconds in the air, which corresponds to about
10 bits in a 15.4 packet. (2) A large percentage of dropped 15.4 packets are due to corruptions
in the packet headers, especially when the 15.4 transmitter is close to an 802.11 transmit-
ter. (3) We experimentally found that when a 15.4 node is close to an 802.11 transmitter, a
15.4 packet can actually cause the 802.11 transmitter to back off, due to elevated channel en-
ergy. When this happens, 802.11 only corrupts the 15.4 packet header (which causes frequent
packet losses), but the remainder of the 15.4 packet is left unaffected.
Depending on how 802.11 and 802.15.4 transmitters interact, we partition the interfer-
ence domain of the two radios into symmetric and asymmetric regions. In symmetric regions,
a 15.4 transmission can cause nearby 802.11 transmitters to back off, so bit corruption hap-
pens mainly in 15.4 packet headers. We employ a simple yet effective header redundancy
mechanism, called Multi-Headers (MH), to address this packet header corruption problem.
MH prefixes a single 15.4 packet with multiple identical packet headers. The first (poten-
tially corrupted) header will cause the 802.11 transmitter to back off, ensuring that the sec-
ond header can be correctly detected by the 15.4 receiver. In asymmetric regions, the 15.4
signal is too weak to affect 802.11 behavior. In this case, we use a forward error correction
(FEC) code to correct bit errors that occur across the entire 15.4 packet. We examine Ham-
73
Chapter 4. External Interference 4.2. Motivations
ming and Reed-Solomon (RS) codes and find that RS code is particularly effective against the
bursty error patterns we observed.
We integrate these techniques into the BuzzBuzz protocol. BuzzBuzz resides at the MAC
layer and provides a general-purpose, single hop reliable delivery mechanism that can counter
the effects of 802.11 interference. We implemented BuzzBuzz in TinyOS and evaluated
its performance on a 57-node testbed under heavy WiFi interference. Specifically, we com-
pare the performance of the Collection Tree Protocol (CTP) [18] with and without BuzzBuzz.
The results show that BuzzBuzz improves the packet delivery ratio by 70% and reduces the
number of packet retransmissions by a factor of three. Reducing the number of packet re-
transmissions from the sensor network leads to less interference to the WiFi network and
ultimately higher throughput for 802.11 as well. Thus, BuzzBuzz creates a win-win situation
in a crowded spectrum space.
The rest of the chapter is organized as follows. We provide a brief overview of the WiFi
and ZigBee protocols in Section 4.3 where we also illustrate the potential for interference
between the two radio technologies. Section 4.4 presents the methodology we used to quantify
the interactions between co-located 802.11 and 15.4 networks and an in-depth analysis of the
root causes for the observed packet losses. In Section 4.5 we present techniques for protecting
15.4 packets from 802.11 senders located in the symmetric and asymmetric regions, while in
Section 4.6 we describe BuzzBuzz and evaluate its performance on a 57-node TelosB testbed.
We present related work in Section 4.8.
4.2 Motivations
We motivate the interference problem that CTI causes to sensor networks with experience
from two WSN deployments.
74
Chapter 4. External Interference 4.2. Motivations
Time (minutes)
Num
rea
chab
le n
odes
During conference After conference
0 50 100 150 200 250 300 350 400 450 500 550 6000
20
40
60
80
100
Figure 4.2: The number of sensor nodes reporting data over ten hours. Thousands of userswere connected to a co-located WiFi network during the first four hours causing dramaticinterference to the sensor network.
4.2.1 Microsoft PDC Conference
During the Microsoft PDC conference in 2008, we placed a network of 90 Genomote motes [52]
within a 140,000 square feet lecture hall. The motes ran a building temperature monitoring
application and used four 15.4 channels simultaneously. 16 Xirrus XN8 WiFi arrays were de-
ployed in the same space. Each WiFi array integrated eight access points, which collectively
used all WiFi channels across the entire space. More than 7,000 people sat in the lecture hall
and at the peak time and about 2,500 people were connected to the WiFi network. Figure 4.2
shows the number of nodes from the sensor network deployment that successfully reported
data over a ten hour period. The WiFi activity during the first four hours completely stressed
the radio bands and left more than half of the sensor nodes unreachable.
4.2.2 JHU Library
We conducted another set of experiments at the JHU library. Each floor of the library has
about six access points hosting 802.11 b/g networks. We used two TelosB motes with 15.4
radio as the sender and the receiver and separated them by 12 feet. Then, the sender trans-
mitted 500 max-size packets (i.e., 128 bytes of payload) every 75 ms to the receiver. During
the peak hour at the library (around 7 pm), the receiver recorded a PRR of about 58.8%.
75
Chapter 4. External Interference 4.3. Background
PayloadPreamble SFD Len CRC
4 1 2 1 0-125
SHR PHR PSDU
Figure 4.3: Format of a 15.4 packet. Field sizes are in bytes.
4.3 Background
In order to mitigate the impact of cross-technology interference and improve the performance
of 802.15.4 networks under 802.11 interference, we need to delve into the details of how these
radio technologies interact. We summarize their salient features in the rest of this section.
4.3.1 802.15.4 Overview
The IEEE 802.15.4 standard defines a PHY layer for low-rate wireless networks operating
in the 2.4 GHz ISM band [26]. The standard defines 16 channels within this band, each
2 MHz wide with 3 MHz inter-channel gap-bands (see Figure 4.1). According to the standard,
outgoing bytes are divided into two 4-bit symbols and each symbol is mapped to one of the
16 pseudo-random 32-chip sequences. The radio encodes these chip sequences using offset
quadrature phase shift keying (O-QPSK) with half-sine chip shaping and transmits them at
2 Mchips/s (i.e., 250 kbps).
Figure 4.3 shows the format of a 15.4 packet including the Synchronization Header (SHR)
and the PHY Header (PHR), shown in grey. The SHR header includes a 4-byte preamble
sequence (all bytes set to 0x00) and a 1-byte Start of Frame Delimiter (SFD) set to 0x7A.
The PHR includes a 1-byte Length field that describes the number of bytes in the packet’s
payload, including the 2-byte CRC. The maximum packet size is 133 bytes, including all the
headers.
76
Chapter 4. External Interference 4.3. Background
TimeSender
TimeReceiver
RTS
SIFS
Data
SIFS SIFS DIFSCTS ACK
Contention Window
Backoffslots
Frame
Figure 4.4: Messages and delays defined in the 802.11 MAC protocol. Durations of packettransmissions and time intervals depend on the 802.11 variant used. The leading RTS/CTSexchange is used only for large packets.
The MAC protocol in the 802.15.4 standard defines both beacon-enabled and non-beacon
modes. In the beacon-less mode, the standard specifies using a CSMA/CA protocol [26]. While
the CSMA/CA protocol uses binary exponential backoff, in practice the CSMA/CA protocol
implemented in TinyOS uses a fixed-length backoff interval [100].
On the receiving side, a 15.4 radio synchronizes to incoming zero-symbols and searches
for the SFD sequence to receive incoming packets. Interference and noise can corrupt the
incoming chip stream, leading to 32-chip sequences that do not match one of the 16 valid
sequences. The receiver then maps the input sequence to the valid sequence with the smallest
Hamming distance.
We note that some 802.15.4 radios (e.g., CC2420 [89]) provide a user-defined correlation
threshold that controls the maximum Hamming distance between the received 32-chip se-
quence and the valid SFD sequence that the receiver is willing to tolerate. If this threshold is
high, the received signal must closely match the ideal signal, hence the signal-to-noise ratio
should be high. If this threshold is low, the receiver allows a low signal-to-noise ratio at the
expense of potentially interpreting corrupted packets or channel noise as valid packets.
4.3.2 802.11 Overview
WiFi networks are almost ubiquitous in office buildings, homes, and even outdoors in urban
areas. Considering that 802.11b, 802.11g, and 802.11n share the same 2.4 GHz ISM band
77
Chapter 4. External Interference 4.3. Background
Parameter 802.15.4 802.11b 802.11gSIFS N/A 30 µs 10 µsDIFS N/A 50 µs 28 µsSlot time 320 µs 20 µs 9 µsInitial CW 1-32 0-31 0-31Successive CWs 1-8 BEB BEBMin length packet 352 µs 202 µs 194 µsMax length packet 4,256 µs 1,906 µs 542 µs
Table 4.1: Packet and interval durations for 802.11 and 802.15.4. The length of the contentionwindows (CW) for 802.15.4 corresponds to the MAC used by TinyOS. The 802.11 standarduses Binary Exponential Backoff (BEB).
with 802.15.4, 802.11 transmissions can interfere with co-located 802.15.4 networks. In the
U.S., only 15.4 channels 25 and 26 do not overlap with WiFi and even these channels are
covered in other parts of the world. In practice, since most WiFi networks use channels
1, 6, and 11, 15.4 channels 15 and 20 can also be interference-free [49]. The potential for
802.11 transmissions to overwhelm 15.4 receivers is amplified by the fact that 802.11 radios
transmit at 10 to 100 times higher power than 15.4 radios.
Figure 4.4, which presents the key features of the 802.11 MAC protocol through a timing
diagram, helps us understand how WiFi nodes use the wireless medium. The 802.11 standard
specifies using CSMA/CA with ACKs as the MAC protocol, optionally with the addition of
RTS/CTS packets [27]. The protocol also specifies the SIFS and DIFS intervals when nodes
should defer using the medium.
A time period, called the contention window, follows the DIFS shown in Figure 4.4. This
window is divided into slots. Nodes use a uniform random distribution to select a slot and
wait for that slot before attempting to access the medium. The node that selects the earliest
slot wins while others defer. Nodes initialize their contention window (CW) to 31 slots and
double it every time they fail to access the medium, until CW reaches a maximum size of
1023 slots.
Table 4.1 summarizes the duration of the DIFS, SIFS, and backoff slots for 802.11b and
78
Chapter 4. External Interference 4.4. Measuring 802.11 and 802.15.4 Interactions
802.11g. The table also shows the maximum and minimum packet sizes for 802.11b, 802.11g,
and 802.15.4. It is worth noting that for many 802.11b and 802.11g packets, the entire air
time is smaller than a 802.15.4 slot time. Based on the relatively small time intervals be-
tween 802.11 transmissions, one can easily see that a backlogged 802.11 sender can poten-
tially corrupt the vast majority of 802.15.4 packets. In the section that follows we experimen-
tally measure the extent of this interference.
4.4 Measuring 802.11 and 802.15.4 Interactions
We conducted a series of experiments that proceed from quantifying the macroscopic impact
of 802.11 interference to 15.4 networks, to exposing the low-level interactions between the
two networks that generate the high-level behaviors we observe. The knowledge of when
and how 15.4 packets are corrupted is used to develop a set of compensation techniques that
markedly improve the coexistence between the two radio technologies.
4.4.1 Methodology
Most of the previous work that has examined the interaction between 802.11 and 802.15.4
networks has focused on high-level metrics such as packet reception rate (PRR) [19, 77]. In
contrast, we examine the interaction between the two radio technologies by accurately de-
tecting and measuring various packet transmission events. Given the packet and interval
durations shown in Table 4.1, this task requires instruments that detect and measure RF
events that are as short as a few µs.
The RFMD ML2724 narrow band radio gives us the ability to detect RF transmissions
with the desired timing accuracy and precision [75]. The ML2724 can be tuned to a central
frequency between 2400 and 2485 MHz and generates an analog voltage on its RSSI OUT
79
Chapter 4. External Interference 4.4. Measuring 802.11 and 802.15.4 Interactions
pin that is directly proportional to the RF signal energy received in a 2 MHz frequency band
centered at the tuned frequency. Given the relative widths of the channels used by 802.11,
15.4, and the ML2724 radio, it is possible to detect 802.11 packets that collide with 15.4
transmissions without being affected by the latter transmissions. In practice, we use 15.4
channel 22, 802.11 channel 11, and set the ML2724’s center frequency to 2465.792 MHz
(equivalent of 15.4 channel 23).
We selected the ML2724 for two key reasons. First, RSSI measurements are directly
available as analog voltage outputs, which makes it possible to detect RSSI changes quickly
by sampling this signal at a high frequency. In contrast, the CC2420 radio exposes the mea-
sured RSSI value as the content of an internal register; due to the delays associated with
accessing CC2420 registers over the SPI bus, it is impossible to detect most of the 802.11 RF
events by reading this RSSI register. The second reason for using the ML2724 is its fast RSSI
response. According to the datasheet, the maximum rise and fall times of the RSSI OUT are
4.5 µs and 3 µs respectively, which make ML2724 an excellent candidate for detecting the RF
activity we are interested in.
While we use the ML2724 radio to detect 802.11 packets, we detect events related to the
transmission of 15.4 packets using the GPIO pins of TelosB motes equipped with CC2420
radios [71]. Specifically, we use the TinyOS event, CaptureSFD.captured that is invoked
at the beginning and the end of 15.4 packet transmissions respectively. This event is the
interrupt service handler that is triggered when the CC2420 radio toggles its pin at specific
points during a radio packet transmission. Under light load, when interrupts are disabled
only for short durations, we can accurately detect these CC2420 events by toggling processor
GPIO pins from within these TinyOS events. There is however a fixed offset between the
actual RF transmissions and the toggling of pins on the CC2420 radio. We measured these
offsets by observing the RSSI OUT of a properly tuned ML2724 and the GPIO pin activities of
80
Chapter 4. External Interference 4.4. Measuring 802.11 and 802.15.4 Interactions
a TelosB mote using an oscilloscope; during these measurements, we also verified that these
offsets are constant.
To accurately correlate 802.11 and 802.15.4 transmissions over time, we connected the
RSSI OUT pin of the ML2724 radio and the mote’s GPIO pins to the same Data Acquisition
(DAQ) card that samples and logs analog inputs at 1 MHz frequency [56].
4.4.2 802.15.4 Packet Reception Ratio
The goal of the first experiment is to quantify at a high level the impact of 802.11 interference
on 15.4 links. Considering that lost 15.4 frames must be retransmitted at the cost of increased
latency and energy consumption, we adopt packet reception ratio (PRR) as the pertinent
metric for this experiment.
Experiment setup. Indoor environments are most likely to house overlapping 802.11 and
15.4 networks and for this reason we performed a controlled experiment in the basement
of a parking garage. The parking garage had minimal structural obstructions, enabling us
to examine the interference under a wide range of inter-node distances. The basement also
minimized the external RF interference on our experiments; a Wi-Spy spectrum analyzer
verified that there was very low interference from other RF sources [58].
The 802.11 network in this experiment consists of an 802.11 b/g access point and a lap-
top equipped with an 802.11 wireless radio configured in infrastructure mode. This laptop
generates a stream of 1,500-byte TCP segments using the iperf tool [33]. To measure the
worst-case impact on the 15.4 network, we configure iperf to transmit as quickly as pos-
sible. Another laptop, connected to the access point through an Ethernet cable, acts as the
traffic sink for the WiFi network. To ensure that our results are not biased by the implemen-
tation details of a specific 802.11 chipset, we repeat the experiment using multiple access
81
Chapter 4. External Interference 4.4. Measuring 802.11 and 802.15.4 Interactions
points manufactured by Belkin, Linksys, Netgear, OpenMesh, and Apple, with Broadcom or
Atheros chipsets. Since all access points produced similar results, we report results collected
from only the Netgear and OpenMesh access points in the rest of this chapter.
The 802.15.4 network consists of TelosB motes equipped with TI CC24240 radios [89]
running TinyOS 2.1. A dedicated sender sends one max-size packet (i.e., 128 bytes of payload)
every 75 ms. These parameters correspond to the sending rate of a high data-rate WSN
application [49]. The sender’s transmit power is set to 0 dBm, and the packets are sent to
the broadcast address. We use multiple receivers to minimize any receiver location-specific
effects and hardware artifacts. Furthermore, the 802.15.4 receivers are configured to accept
packets with CRC errors, while the transmitted packets have a predefined byte pattern to
enable off-line bit error analysis for corrupted packets. Each receiver logs the entire packet
contents for all incoming packets by transmitting them to a dedicated PC over its serial
interface. Receivers also record whether a packet passed the CRC check. We use the set
up described in Section 4.4.1 to acquire precise timing of all 802.11 and 802.15.4 packet
transmissions. The DAQ card used for timing these transmissions is connected to the same
PC used to log 802.15.4 packets.
Figure 4.5 illustrates the relative positions of all the nodes used in this experiment. We
examine the impact of different levels of 802.11 interference on the packet reception ratio
(PRR) by varying the distance d between the 802.15.4 and 802.11 nodes. We run four sets of
experiments with d = 15, 65, 115, 170 feet. For each value of d, we repeat the experiment using
802.11b and 802.11g radios. During each run, the 15.4 sender broadcasts 2,000 packets; with
five 802.15.4 receivers, this corresponds to a total of 10,000 packet receive events under ideal
conditions.
Results. We identify three types of 15.4 packet reception events: packets that are received
correctly, packets that fail the CRC tests due to corrupted bits, and packets that are lost (i.e.,
82
Chapter 4. External Interference 4.4. Measuring 802.11 and 802.15.4 Interactions
802.15.4 Sender
802.15.4 Receivers
802.11 Sender
802.11 Receiver
d=15, 65, 115, 170 feet12 feet 12 feet
Figure 4.5: Setup for the garage packet reception ratio experiment.
transmitted but never received).
Figure 4.6 presents the relative percentages of each of these three events for different
values of d. As expected, 802.15.4 PRR is significantly reduced due to 802.11 interference,
especially when the two networks are closer to each other. As d increases, the 802.15.4 PRR
improves since the 802.11 interference becomes progressively weaker.
We observe that 802.11b traffic has a much larger impact on the overall 802.15.4 PRR
than 802.11g. Thonet et al. made a similar observation and suggested that this difference is
due to the higher transmission rate of 802.11g networks, which lowers the channel-time for
802.11 packets [92]. We also observe, especially for smaller separations and 802.11b radios,
that the number of lost packets is larger than the number of packets received with bit errors.
This observation suggests that the front part of a 802.15.4 packet (i.e., the SHR) is more
vulnerable to 802.11 interference, especially considering the relative sizes of the different
packet parts. The second observation is important since it indicates that one needs to focus
on improving the detection of (partially) corrupted frames in order to improve the overall
PRR. We return to this point in Section 4.4.4, where we present the distribution of bit errors
over a 15.4 packet.
We also found that packet transmission latency increased by as much as 13% to 40% in
the presence of 802.11g and 802.11b traffic respectively; another sign of the impact of 802.11
83
Chapter 4. External Interference 4.4. Measuring 802.11 and 802.15.4 Interactions
Distance (ft)
Per
cent
age
of p
kts
(%)
802.11b Interference
0
20
40
60
80
100
15 65 115 170
802.11g Interference
Lost pktsCorrupted pktsValid pkts
15 65 115 170
Figure 4.6: Percentage of 15.4 packets correctly received, corrupted, and lost as the distancebetween 802.11 nodes and 15.4 nodes increases.
traffic on the 15.4 network. As discussed in Section 4.3, the 15.4 radio uses a CSMA/CA
protocol where it performs a CCA check on the channel prior to each packet transmission.
Since the 15.4 radio sends a packet only if the CCA check indicates a free channel, the latency
of packet transmissions should increase in the presence of active 802.11 traffic.
Another interesting result is the impact of the 15.4 traffic on 802.11 throughput at d =
15 feet. We instrumented the 15.4 sender to transmit 10,000 128-byte packets at a rate of
75 ms, and the packet sniffer, WireShark, running on the 802.11 receiver recorded a 4%
drop in TCP throughput.
4.4.3 Dynamics of 802.15.4 and 802.11 Interaction
To better understand the dynamics between the overlapping 802.11 and 15.4 networks shown
in Figure 4.5 we examine next the RF activity logs captured during the experiments described
in the previous section.
Figure 4.7 presents a timeline of 802.11b and 15.4 traffic activity when d = 15 feet (Fig-
ure 4.7(a)) and d = 115 feet (Figure 4.7(b)). Each vertical box corresponds to a single 15.4
broadcast, while the grey region corresponds to 802.11b activity, obtained from the narrow
84
Chapter 4. External Interference 4.4. Measuring 802.11 and 802.15.4 Interactions
Time (ms)
RS
SI (
dBm
)
150 160 170 180 190 200 210 220 230 240
−80
−60
−40
−20
0
(a) d = 15 feet.
Time (ms)
RS
SI (
dBm
)
170 180 190 200 210 220 230 240 250 260
−80
−60
−40
−20
0
(b) d = 115 feet.
Figure 4.7: Overlay of 802.11b and 15.4 traffic. Each vertical box corresponds to a 15.4 packettransmission, while the gray lines are RSSI measurements corresponding to 802.11 trans-missions. When d is small, the 802.11 radio backs-off when it senses a 15.4 transmission. Asd increases, the 802.11 radio cannot detect 15.4 transmissions and packets collide.
band radio as described in Section 4.4.1.
One can see from Figure 4.7(a) that 802.11 backs-off during 802.15.4 transmissions, when
the separation between 802.11 and 802.15.4 nodes is small — an observation that contradicts
the common belief that 802.11 nodes do not back off in the presence of 15.4 traffic [72]. We
repeated the same experiment with five off-the-shelf 802.11b/g access points (from Apple,
Belkin, Linksys, Netgear, and OpenMesh), and all exhibit the same behavior.
This behavior can be attributed to the 802.11 specification that mandates performing a
clear channel assessment (CCA) prior to every data packet transmission. In other words,
the 802.11b radio in our experiment sensed the channel noise floor being above the CCA
threshold and deferred its pending transmission.
85
Chapter 4. External Interference 4.4. Measuring 802.11 and 802.15.4 Interactions
We also note that in certain cases, the 802.11 radio will not back off. First, the 802.11
specification requires that RSSI ≥ -70 dBm for the channel to be considered busy when TX
power ≤ 50 mW [27]. Second, the 802.11 specification lists three CCA modes (i.e., energy
detection, packet detection, and both) and vendors can implement one or more at their dis-
cretion. 802.11 radios that use the packet detection CCA mode will declare the channel to be
clear, since they cannot decode the overheard 15.4 packet transmission.
On the other hand, Figure 4.7(b) suggests that 802.11 does not back off when d is large.
Nevertheless, due to the relatively high transmit power of 802.11 radios, the 802.11 trans-
missions can interfere with 802.15.4 transmission event at this distance (cf. Figure 4.6).
The discrepancy between 802.11 and 15.4 transmit powers leads to two distinct inter-
ference regions. In the symmetric region, the signal from the 15.4 sender is strong enough
to trigger the CCA check on the 802.11 transmitter. On the other hand, in the asymmetric
region the 802.11 transmitter cannot detect 15.4 packets while it can still corrupt them.
We note that the 802.11 back-off behavior in the symmetric region seemingly contradicts
the packet losses observed in Figure 4.6. Specifically, in the symmetric region, where 802.11
backs off due to ongoing 802.15.4 transmissions, we expect to see lower 802.11 interference
compared to the asymmetric region. Instead, one can observe in Figure 4.6 that there is
much larger 802.11 induced interference in this region as observed from the large number
of lost and corrupted packets. The following sections examine and explain this apparently
contradictory behavior.
4.4.4 802.15.4 Bit Error Distribution
To resolve the contradiction presented in the previous section, we examine the distribution
of corrupted bits over 15.4 packets that failed the CRC check. Since we know the original
86
Chapter 4. External Interference 4.4. Measuring 802.11 and 802.15.4 Interactions
Bit position
Fre
quen
cy (
%)
0 128 256 384 512 640 768 896 1024
05
1015
20
(a) 802.11b interfering source.
Bit position
Fre
quen
cy (
%)
0 128 256 384 512 640 768 896 1024
05
1015
20
(b) 802.11g interfering source.
Figure 4.8: Bit-error distribution for 15.4 packets that failed the CRC check when the inter-fering 802.11 transmitter is in the symmetric region. It is far more likely that the front partof the 15.4 packet will be corrupted.
packet content, it is possible to identify the bits that were incorrectly decoded.
First, Figure 4.8 shows the bit error distribution in the symmetric region. It is evident
that the front section of a 802.15.4 packet has a much higher probability of incurring bit
errors compared to the rest of the packet, which has almost zero bit errors.
For now, let us sidestep the question why a collision happens in the first place and focus
on the extent of the collision itself. Considering the relative durations of the 802.11 and 15.4
packets (cf. Table 4.1), the 802.11 transmission will finish well before the full 15.4 packet has
been sent. At this point the 802.11 sender performs a CCA check, notices that the channel is
busy, and defers any subsequent (re)transmissions. We note that in Figures 4.8(a) and 4.8(b)
the extent of the corrupted packet region closely matches the duration of a single 1,500-byte
802.11 packet respectively.
87
Chapter 4. External Interference 4.4. Measuring 802.11 and 802.15.4 Interactions
Time (ms)
RS
SI (
dBm
)
226 228 230 232 234 236 238
−80
−60
−40
−20
0
Figure 4.9: Detailed view of the 802.11b and 15.4 packet transmission timeline. The overlapat the beginning of the 15.4 packet (vertical box) corresponds to a collision with a 802.11packet. The 802.11 sender defers sending any additional packets until the 15.4 transmissioncompletes.
Figure 4.9 illustrates this behavior in more detail, providing a zoomed-in view of the event
timeline surrounding the transmission of a 15.4 packet. It is clear that once the first 802.11
packet collides with the 15.4 packet the 802.11 sender defers any subsequent transmissions
until the end of the 15.4 packet. The initial transmission overlap generates the bit errors
seen in Figure 4.8.
The large number of bit errors at the beginning of 15.4 packets can also explain the large
number of missing packets as follows. As Figure 4.3 shows, the front part of the 15.4 frame
includes the SHR and PHR headers. Several problems can occur when the receiver cannot
properly decode the SHR and the PHR. First, if the receiver cannot properly decode the
Preamble or the SFD field, it will misinterpret the packet as channel noise. Second, if the
Length field is corrupted the received packet will be either incomplete (when the decoded
length field value is smaller than the actual packet length) or will contain additional junk
bits (when the decoded length field value is larger). Both cases will result in a CRC failure.
The observation that bit errors are concentrated in the front part of the packet suggests
that reducing the size of the 15.4 packet will not increase the PRR. The results shown in
Figure 4.10 confirm this hypothesis. As before, d = 15 feet, but the 15.4 sender broadcasts
packets with payload sizes 30, 50, 70, 90, and 115 bytes. Despite a three-fold increase in
88
Chapter 4. External Interference 4.4. Measuring 802.11 and 802.15.4 Interactions
Payload size (Byte)
Per
cent
age
of p
kts
(%)
802.11b Interference
0
20
40
60
80
100
30 50 70 90 115
802.11g Interference
30 50 70 90 115
LostCorruptedValid
Figure 4.10: 15.4 packet reception rate as the payload size varies. The competing 802.11sender lies within the symmetric region of the 15.4 sender. Since only bits in the front sectionof the 15.4 packet are corrupted varying the packet’s length does not affect PRR.
Bit position
Fre
quen
cy (
%)
0 128 256 384 512 640 768 896 1024
01
23
45
Figure 4.11: Bit-error distribution for 15.4 packets that failed the CRC check when the in-terfering 802.11g transmitter is in the asymmetric region. Bit errors are evenly distributedacross the whole packet.
payload size the PRR remains effectively constant.
We can now explain why 15.4 and 802.11 packets collide. Looking at Figure 4.4, the only
time that a 15.4 sender can begin its transmission is during the DIFS + Contention Window
period since it otherwise senses the channel as busy. Furthermore, the time granularity that
the 15.4 sender senses the medium is equal to the slot time (= 320 µs), and it senses the
medium for eight symbol periods (= 128 µs) before declaring the channel as idle [26]. Consid-
ering the short length of the DIFS interval and the shorter 802.11 slot time (cf. Table 4.1) it
is very likely that during the time the 15.4 sender senses the channel, the 802.11 node also
senses the channel. As a result of both nodes sensing the channel idle, they start transmitting
at the same time and subsequently collide.
89
Chapter 4. External Interference 4.4. Measuring 802.11 and 802.15.4 Interactions
Finally, Figure 4.11 shows the 802.15.4 packet bit error distribution in the asymmetric
region. Compared to results in the symmetric regions, bit errors are almost uniformly located
throughout the 15.4 packet. We note that the lack of bit errors near the end of the packet in
the figure is partly due to the corrupted length field.
The difference in the bit corruption patterns seen in the two regions implies that we need
to develop different techniques, targeting individual regions, to overcome the negative impact
of 802.11 traffic on 15.4 transmissions.
So far we have shown aggregate statistics about which bits in the 15.4 frame are more
likely to be corrupted in the two regions. We are also interested in the number of corrupted
bits and their distribution over individual 15.4 packets since this information will be useful
in designing protection mechanisms. Figure 4.12 presents the CDF of the total number of
corrupted bits in both regions for 802.11b and 802.11g senders. Interestingly, corrupted 15.4
packets received in the symmetric region have a wider spread in the number of corrupted bits,
and they are also more severely damaged than corrupted packets received in the asymmetric
region.
While Figure 4.12 presents the number of corrupted bits, Figure 4.13 presents the density
with which these bits appear over the 15.4 packet. Specifically, it plots the probability density
function of the distance n between any two corrupted bits in the packet. Small values of n
correspond to bursts of corrupted bits, while corrupted bits randomly scattered throughout
the header will result in large values of n. We can see from this figure that errors occur
in bursts, especially in asymmetric regions. This corresponds to, possibly multiple, 802.11
packets colliding with the 15.4 packet.
90
Chapter 4. External Interference 4.5. Protecting 802.15.4 packets from 802.11
Total number of bad bits
Fre
quen
cy (
%)
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●● ●● ●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ● ●● ●● ● ● ●●● ●● ●●● ●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ● ●
0 50 100 150 200 250 300 350 400 450 5000
20
40
60
80
100
●
●
Asymmetric regionSymmetric region
(a) 802.11b.
Total number of bad bits
Fre
quen
cy (
%)
●●
●
●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●●● ● ●
●
●
●
●
●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●●● ● ●
0 50 100 150 200 250 300 350 400 450 5000
20
40
60
80
100
●
●
Asymmetric regionSymmetric region
(b) 802.11g.
Figure 4.12: CDF of the number of bad bits in a corrupted packet.
n
Fre
quen
cy (
%)
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●
●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
802.11b Interference
0
10
20
30
40
50
60
0 10 20 30 40 50
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●
●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
802.11g Interference
0 10 20 30 40 50
●
●
Symmetric regionAsymmetric region
Figure 4.13: Distance n between any two corrupted bits in a 15.4 packet. Errors caused bycompeting 802.11b and 802.11g senders occur in concentrated bursts.
4.5 Protecting 802.15.4 packets from 802.11
Based on the insights from the above measurements, we design targeted redundancy mech-
anisms to compensate WiFi interference. Depending on whether the ZigBee node can push
back the WiFi transmissions, we investigate techniques for symmetric and asymmetric re-
gions respectively.
91
Chapter 4. External Interference 4.5. Protecting 802.15.4 packets from 802.11
Correlation threshold
Per
cent
age
of p
kts
(%)
802.11b Interference
0
20
40
60
80
100
120
12 14 16 18 20
802.11g Interference
12 14 16 18 20
LostCorruptedValid
Noise pkts
Figure 4.14: 15.4 PRR in the presence of 802.11b interference as the TI CC2420 correlationthreshold varies. Lower threshold values do not increase PRR but lead the receiver to incor-rectly decode channel noise as packets.
4.5.1 Symmetric Region Techniques
Section 4.4 shows that in the symmetric region corrupted bits occur at the front of a 15.4
packet. Such corrupted packets are traditionally recovered through an Automatic Repeat
Request (ARQ) protocol. For example, the Collection Tree Protocol (CTP), a widely used
protocol in the WSN community, uses hop-by-hop retransmissions aggressively [18]. Even
though ARQ improves the packet reception ratio, it does so at the expense of higher energy
consumption and lower channel throughput.
Considering that only the front section of the packet is corrupted and that a large portion
of packets may not be affected by interference, retransmitting the same packet multiple times
is intuitively inefficient. In this section, we investigate whether it is possible to overcome this
specific form of packet corruption without retransmissions. We start with two straw man
approaches, the packet detection correlation threshold and the preamble length, that provide
the desired outcome only partially. Then, we conclude with a simple, yet efficient mechanism
that does.
92
Chapter 4. External Interference 4.5. Protecting 802.15.4 packets from 802.11
Preamble length (Byte)
Per
cent
age
of p
kts
(%)
802.11b Interference
0
20
40
60
80
100
2 5 7 9 11 13 15
802.11g Interference
2 5 7 9 11 13 15
LostCorruptedValid
Figure 4.15: 15.4 PRR as the preamble length varies. Due to the shorter 802.11g packets it ispossible to recover all 15.4 frames by increasing the preamble’s length.
Correlation Threshold
As we described in Section 4.3, some 15.4 radios such as the TI CC2420 [89] provide a user-
defined correlation threshold that determines the amount of noise that the radio is willing to
tolerate when decoding incoming 32-chip sequences in searching for the SHR. Considering
that WiFi interference is a form of (non-random) noise, it is then plausible that varying the
correlation threshold will increase the PRR.
We tested the effectiveness of this technique by varying the correlation threshold of a
CC2420 receiver. In this case, the maximum correlation threshold is 32, and the default is
set to 20. While operating in the symmetric region, we varied the correlation threshold from
12 to 20 in increments of 2 and logged the received packets. Figure 4.14 presents the results of
this experiment. Decreasing the threshold does not increase the number of correctly received
packets. The only change is a negative one: increasing the instances in which the 15.4 radio
mistakenly interpreted background or WiFi-related noise as the start of valid 15.4 packets.
Preamble Length
While the 15.4 specification mandates a 4-byte preamble, radios such as the CC2420 allow the
user to set the length of the transmitted preamble up to 17 bytes. We leverage this capability
93
Chapter 4. External Interference 4.5. Protecting 802.15.4 packets from 802.11
to overcome the effects of 802.11 interference on 15.4 packets as follows. Consider an instance
in which the WiFi and 15.4 packets overlap by n 15.4 bytes. Then, if the preamble’s length
was n + 4 bytes, the receiver would be able to properly decode the preamble’s last four bytes
and receive the packet correctly.
To verify that a longer preamble does indeed reduce packet loss in the symmetric region,
we ran an experiment in which we varied the preamble length and observed the packet re-
ception ratio. Figure 4.15 illustrates the experiment’s results. As expected, increasing the
preamble length increases the PRR. Nevertheless, this technique has two drawbacks. First,
the maximum preamble length that the CC24240 radio supports is 17 bytes. This preamble
length cannot fully protect a 15.4 packet from all WiFi packets, since such transmissions can
take more than 480 µs (cf. Table 4.1). Second, since the 15.4 specification requires a four-byte
preamble, other 15.4 radios, such as the Atmel RF230 [3], do not support variable-length
preambles.
Multi-Headers
Next, we introduce Multi-Headers (MH), a light-weight sender-initiated mechanism that em-
ulates the PRR improvements due to longer preambles, but is effective against a wider range
of competing WiFi packets.
The naıve approach to emulate the effect of longer preambles is to send two packets back-
to-back. The first packet, whose sole purpose is to save the second packet from corruption,
carries a dummy payload to make 802.11 transmitters back off. The following packet can then
be correctly received with higher probability. Unfortunately, it can be difficult to transmit
back-to-back packets. For example, the smallest inter-packet interval with TelosB motes is
as high as 600 µs, considering the time to send the STXON or STXONCCA command over the
SPI bus and the 12 symbol periods that the CC2420 radio waits before each transmission.
94
Chapter 4. External Interference 4.5. Protecting 802.15.4 packets from 802.11
PayloadPreamble SFD Len
4 1 2 1 0-115-16N
CRC
Original Header MH Header
MAC
10
Preamble SFD Len
4 1 1
MAC
10
Figure 4.16: Multi-Headers: a 15.4 packet with an additional header.
Rather than transmitting multiple packets, Multi-Headers transmits multiple packet
headers back to back for one packet payload. Such transmission of consecutive headers is
feasible due to the much simpler packet decoding approach that 15.4 radios use, compared
to 802.11 radios. Specifically, 802.11b/g radios use different modulation schemes and trans-
mission bit rates for the packet header and the payload. This feature allows 802.11 radios
to detect and switch to a stronger new packet transmission in the middle of a weaker packet
transmission and overcome hidden terminal effects. Furthermore, this approach enables
backward-compatibility with older 802.11 standards. On the other hand, 15.4 radios trans-
mit the whole packet with the same modulation scheme and bit rate. Once the 15.4 receiver
correctly detects the SHR and PHR headers, it continues to decode the incoming RF signal
until it receives the number of bytes mentioned in the PHR header. This feature implies that
we can safely add multiple headers within a single 15.4 packet and the receiver will treat
them as part of the payload.
Figure 4.16 shows a legitimate 15.4 packet with one additional set of headers in the pay-
load. Given this two-header packet structure, if there are no overlapping 802.11 transmis-
sions, the receiver will detect the first SFD after the first preamble sequence and will treat
this SFD as the start of the packet; the radio treats the second packet header as a part of the
payload. On the other hand, if the first preamble bytes were corrupted due to an overlapping
802.11 transmission, the 15.4 receiver may not detect the first SFD correctly; instead, the
receiver will detect the second SFD preceded by four preamble bytes and assume the packet
95
Chapter 4. External Interference 4.5. Protecting 802.15.4 packets from 802.11
starts at the second SFD.
With multiple packet headers per packet, the length field of each header must be properly
adjusted. For example, in Figure 4.16, the length field of the second header corresponds to
the length of the actual payload, while the length field value of the first header is larger by
16 bytes. Given that the radio treats additional headers as payload, the receiver’s software
stack needs to remove any remaining headers from the payload before delivering the packet
to the user application.
The CRC in the 15.4 packets covers both the header and payload, which limits its applica-
bility in the presence of Multi-Header. Specifically, depending on the header detected by the
radio, the 15.4 header and payload will be different. To address this problem, we disable the
15.4 CRC filtering on the radio and modify the radio software stack to include a 2-byte CRC
within the packet’s payload. This CRC covers the innermost header (excluding the length
field) and the user payload.
Evaluation. We evaluated the effectiveness of Multi-Headers using the following experi-
mental setup. Five 802.11g clients were connected to the same AP in infrastructure mode to
simulate a typical office setting. These 802.11g clients sent TCP and UDP traffic as fast as
possible to the PC connected to the AP via an Ethernet cable. The 802.11g clients were dis-
tributed within the same office and the 15.4 sender was 15 ft away from four 15.4 receivers.
Activity traces of the 802.11g activity collected by the ML2724 radio verified that the 15.4
sender was within the symmetric region. The 15.4 sender broadcasted 2,000 128-byte pack-
ets at a rate of one packet per 75 ms and it filled the packets payloads with four additional
in-payload headers and a 50-byte application payload. In addition, for each header, we re-
placed the source address field in the 15.4 header with a counter that acts as a header index
for offline analysis.
Table 4.2 shows the number of successfully received packets as triggered by each addi-
96
Chapter 4. External Interference 4.5. Protecting 802.15.4 packets from 802.11
15.4 header only Additional headers1 st 2nd 3 rd
TCP 30.5% 80.0% 90.0% 91.9%UDP 28.2% 82.1% 95.0% 96.8%
Table 4.2: Percentage of packets successfully received using the original 15.4 header or oneof the additional headers.
tional header. Having even a single additional header increases PRR by 50%. The benefit of
Multi-Header diminishes as we increased the number of additional headers beyond one, sug-
gesting that two headers in total are sufficient. Achieving an effective PRR of 80% to 82.1%
(compared to the original 28%-30%) outweighs the 20% packet overhead incurred by the one
additional header. Finally, we note that we observed similar performance for 802.11b traffic.
4.5.2 Asymmetric Region Techniques
The results in Section 4.4 showed that symmetric and asymmetric regions lead to two distinct
bit error patterns. Bit errors in the asymmetric region, in contrast to the symmetric region,
are uniformly distributed across the packet. Therefore, Multi-Headers that protects only the
packet header cannot recover from these bit errors.
Packet loss due to bit errors is resolved using either ARQ or Forward Error Correction
(FEC). Although packet retransmissions are commonly used under low to moderate packet
loss, ARQ would require multiple retransmissions, inducing extra energy cost for the motes
and increased channel congestion. Therefore, FEC is more suitable to overcome packet loss
under heavy interference.
FEC augments each packet with extra information that enables the receiver to correct
some of the bit errors. Although FEC algorithms are widely used in digital communications,
selecting the right algorithm and implementing it on resource-constrained sensor nodes are
non-trivial. In this section we evaluate several FEC techniques, in terms of their error recov-
97
Chapter 4. External Interference 4.5. Protecting 802.15.4 packets from 802.11
ery performance and implementation overhead.
Error-Correction Codes
An FEC transmitter applies an Error-Correction Code (ECC) to the data to be transmitted.
The ECC transforms the message to a larger encoded message. The receiver then applies
the reverse transformation to recover the original message from the encoded message. The
redundancy in the encoded message allows the receiver to recover the original message in
the presence of a limited number of bit errors.
One example of an ECC is the linear code that the 15.4 physical layer employs to over-
come RF noise [26]. As mentioned in Section 4.3.1, the 15.4 radio maps 4-bit symbols to
32-bit chip sequences. Since the minimum Hamming distance among the sixteen predefined
chip sequences is 12, if the received chip sequence does not have errors at more than 6 chip
positions, the 15.4 radio can properly map it to the correct chip sequence. However, we argue
that the 15.4 ECC is not sufficient against external 802.11 noise, which typically lasts longer
than six chip times, or 3 µs.
Next, we evaluate two ECCs, namely the Hamming code and the Reed-Solomon code, for
correcting 802.11-induced bit errors on 802.15.4 packets.
Hamming Code
The Hamming codes enable FEC by adding extra parity bits to the message. A common
variant of the Hamming codes can detect up to two bit errors and correct one bit error in the
encoded message. In particular, we use the Hamming(12,8) code, in which four parity bits
are added to every eight bits of data to generate encoded messages that are 12 bits long. The
Hamming(12,8) code can detect and correct one bit error within the 12-bit word.
98
Chapter 4. External Interference 4.5. Protecting 802.15.4 packets from 802.11
P1 P2 D3 P4 D5 D6 D7 P8 D9 D10 D11 D12
B1 B2 B3 B4 B5 B6 B7 B8
Original Data
EncodedMessage
B11B10B9 B12
B12
B3
B7B5 B6
B9B7B5
B11B10B7B3 B6
B11
Figure 4.17: Hamming(12,8) encoding format. Four parity bits are generated for each eightbits of data.
Figure 4.17 illustrates the Hamming encoding. Here, four parity bits are computed and
inserted into the message by computing the XOR of multiple data bits. During decoding, the
parity bits are computed again and compared with the parity bit in the encoded message. The
decimal equivalent of the sum of the parity bits that do not match gives the position of the bit
error. For example, if parity bits P2 and P4 do not match, the bit D6 (2+4 = 6) is the error bit.
In practice, the data and parity bits are grouped together, so that the data byte can be easily
extracted from the 12-bit message. Since the Hamming(12,8) code can correct only single-bit
errors in each 12-bit data word, if more than one bit errors are present, the decoded message
will most likely correspond to a data word other than the original. This implies that, once a
message with unknown number of bit errors is decoded, the validity of the decoded message
has to be verified by some other techniques such as a CRC check.
We implement a Hamming(12,8) code-based FEC to recover 802.11-induced 802.15.4 packet
errors. Our implementation takes a 72-byte message and outputs a 108-byte encoded mes-
sages (we packed two 12-bit encoded code word to three bytes). In addition, we implemented
99
Chapter 4. External Interference 4.5. Protecting 802.15.4 packets from 802.11
Hamming(12,8) Hamming(12,8) RSw/ Bit interleaving w/ 30-byte parity
11b 11g 11b 11g 11b 11g15 ft 0.6% 11.7% 12.4% 57.6% 52.0% 65.2%65 ft 4.7% 19.1% 55.6% 70.4% 85.3% 85.9%
Table 4.3: Percentage of corrupted packet payloads that can be recovered by two version ofthe Hamming code and the RS code.
a table lookup-based encoder and decoder to reduce the computation overhead, and com-
puted (off line) all the possible 256 different encoded messages for the 8-bit word. Finally,
the 72-byte message contains a 2-byte CRC field to verify the message correctness after the
Hamming decoding.
Our implementation uses 754 bytes of ROM (excluding the CRC code) and 82 bytes of
RAM (excluding the data and encoded message buffers). Encoding and decoding the 108-byte
messages requires 1.4 ms and 1.8 ms respectively on a TelosB mote running at 4 MHz.
Our Hamming(12,8) implementation places each 12-bit code word in consecutive locations
in the 108-byte packet payload. Hence, all the 12 encoded bits are transmitted as consecutive
bits in the radio packet. While the Hamming(12,8) code is very efficient in terms of execution
cost and memory overhead, the first column of Table 4.3 shows that only a very small fraction
of the corrupted messages is recovered by the Hamming(12,8)-based FEC scheme. Its poor
performance is due to the burstiness of bit errors (cf. Figure 4.13), which results in multiple
bit errors within an encoded message thus preventing correct error recovery.
Next, we apply bit interleaving to make the encoded message more resilient to bursty
errors. Our implementation treats the 108-byte encoded message as a sequence of (108 × 8)
bits. For each 12-bit code word, we evenly spread each bit over the entire message (i.e., two
consecutive bits in the 12-bit message are separated by 72 bits in the interleaved message).
Interleaving is reversed during the decoding process by concatenating 12 bits, evenly spread
across the message, to get back the 12-bit encoded message.
100
Chapter 4. External Interference 4.5. Protecting 802.15.4 packets from 802.11
Encoding Decoding15-byte error 30-byte erasure no errors
36.156 ms 181.892 ms 207.824 ms 104.296 ms
Table 4.4: Execution time for TinyRS operations. The original message is 65 bytes long andthe RS-encoded message contains the original 65-byte message and the 30-byte parity.
The second column of Table 4.3 shows the error recovery performance of the bit inter-
leaved Hamming encoding. Due to the heavy use of the bit-level operations, the execution
times for encoding and decoding a 108-byte message now become 6.7 ms and 9.2 ms respec-
tively. The implementation consumes 1.4 KB of ROM and 0.1 KB of RAM space.
Reed-Solomon Code
The Reed-Solomon (RS) code is a block-based linear error-correction code with a wide range
of applications in digital communications and storage [51]. Unlike the Hamming code, the
RS code can recover from both data corruptions and erasures; the former refers to bit flipping
at unknown positions in the packet, while the latter refers to missing pieces of data at known
locations. The RS code divides a message into x blocks of user-defined size and computes
a parity of y blocks. An RS-encoded message then consists of the original message and the
computed parity. The length of the parity determines the maximum number of corrupted
and erasure blocks in the encoded message that the receiver can successfully recover from.
Specifically, for y blocks of parity, the number of erasure and corrupted blocks that the RS
code can recover from is:
2× (num corrupted blks) + 1× (num erasure blks) < y
The RS code is commonly (mis-)believed to be too computationally and memory intensive
for resource-constrained motes [31]. Previous work minimizes the implementation complex-
ity by either implementing only the encoding engine [80], or focusing on recovering from
101
Chapter 4. External Interference 4.5. Protecting 802.15.4 packets from 802.11
RS parity size (Bytes)
Num
tran
smis
sion
s (K
)
●● ● ●
●●
●
●
● ● ● ● ● ●●
●
10 15 20 25 30 35 40 45 50 55 600
1
2
3● ●802.11b interference 802.11g interference
Figure 4.18: Total number of 15.4 transmissions necessary to transfer a 38-KB object over asingle hop in the presence of interference from 802.11 traffic.
data erasures only [35]. However, since a corrupted length field can mislead the radio into
demodulating a smaller packet, we argue that it is advantageous to handle both data cor-
ruptions and erasures. To this end, we have developed a full-featured TinyOS-compatible RS
library, TinyRS, based on the work of Karn et al. [69]. The rest of this section outlines the
performance of RS code based on our TinyRS implementation.
TinyRS has a relatively small code footprint, requiring 2.9 KB of ROM and 1.4 KB of
RAM when using 8-bit block size and 30-byte parity. Table 4.4 presents results from a micro-
benchmark on the execution time in the case of a 65-byte message. The encoding time is
significantly lower than the decoding time, and the latter also depends on the state of the
received RS-encoded message. In Section 4.6, we further describe a real-world system that
attempts to reduce the impact of decoding overhead on network throughput.
The third column of Table 4.3 illustrates the benefits associated with the increasing pro-
cessing overhead: RS can successfully recover four times more packets than Hamming(12,8)
with bit interleaving, when 15.4 packets are corrupted by an 802.11b transmitter in the sym-
metric region.
Using larger parities increases the probability of recovering bits corrupted by interference,
thereby decreasing the need for retransmissions. At the same time, a larger parity decreases
the space in the 15.4 payload that can be used to deliver application data, leading to more
102
Chapter 4. External Interference 4.5. Protecting 802.15.4 packets from 802.11
packet transmissions.
To guide the decision on the parity size, we ran simulations to determine the expected
number of transmissions necessary to deliver a 38-KB object over a single hop for various
parity sizes. To do so, we first obtained the link’s effective PRR by simulating the trans-
mission of 10,000 RS-encoded packets with various parity sizes and applying the bit errors
observed in the packet traces from Figure 4.6 (68 ft). We calculated the effective PRR for each
parity size by counting both valid packets and corrupted packets that RS can successfully re-
cover. From the effective PRR we calculate the expected number of transmissions to deliver a
38-KB object, shown in Figure 4.18. The results suggest that the 30-byte parity requires the
smallest number of transmissions.
Finally, to further illustrate the effectiveness of TinyRS in reducing the expected number
of transmissions, we consider two other packet recovery strategies: packet-level and block-
level ARQ. Both approaches rely on acknowledgments to decide whether to initiate retrans-
missions. However, block-level ARQ divides a packet into sub-blocks and allows the sender
to retransmit only the corrupted blocks [20]. For simplicity, we assume that the network can
always successfully delivery acknowledgement packets. In the scenario of delivering a 38-KB
object mentioned before, packet-level ARQ would need to transmit a total of 4,409 packets,
while block-level ARQ (with 30-byte blocks) would require a total of 2,313 packets. In compar-
ison, with 30-byte parity, TinyRS transmits only 1,720 packets. However, TinyRS requires
additional energy to encode and decode payload. In the previous example, each RS-encoded
packet would use an additional 0.435 mAs for encoding and decoding packets with 15 or less
bit errors on a 4 MHz TelosB mote, or 748 mAs for 1,720 packets. This is in comparison to
1,290 mAs and 284 mAs used by the additional transmissions in packet-level and block-level
ARQ (considering both sending data packets and waiting for acknowledgments). If we con-
sider that acknowledgments could be lost, we expect the ARQ methods would use even more
103
Chapter 4. External Interference 4.6. BuzzBuzz MAC Layer
energy.
4.6 BuzzBuzz MAC Layer
The results from Sections 4.5.1 and 4.5.2 show that Multi-Headers and TinyRS improve the
802.15.4 PRR in the presence of 802.11 interference. Multi-Headers improves the detection
of packets by protecting the packet header, while TinyRS protects against bit errors in the
packet payload. This section presents BuzzBuzz, a MAC-layer solution that combines Multi-
Headers and TinyRS to improve the overall PRR under 802.11 interference. Experimental
results from a 57-node testbed show that BuzzBuzz improves the end-to-end data yield of the
CTP data collection protocol, while reducing the overall network traffic by 71%, under severe
802.11 interference.
4.6.1 BuzzBuzz Protocol Design
When designing a networking solution, it is important to identify the best architectural layer
at which it should operate. We argue that both Multi-Headers and TinyRS are best positioned
at the MAC layer due to the following reasons.
First, the MAC layer typically maintains neighborhood and link quality information that
BuzzBuzz can leverage to improve packet delivery efficiency. For example, depending on
the link quality, BuzzBuzz can decide if the overhead of running the error correction code
is justified. BuzzBuzz can also leverage the MAC layer’s knowledge of the underlying radio
header format to construct Multi-Headers headers.
Second, while running FEC only at the source and the destination nodes of a multi-hop
path reduces the computation cost, the bit errors accumulated over the intermediate hops
will reduce the likelihood of successfully decoding the packet at the destination. Therefore,
104
Chapter 4. External Interference 4.6. BuzzBuzz MAC Layer
we optimize FEC operations for the MAC layer and perform packet encoding and decoding
on every hop along the end-to-end path.
ARQ is the most widely used approach in WSNs to implement reliable delivery at the MAC
layer. In ARQ, the sender retransmits a packet if the receiver has explicitly (i.e., negative
acknowledgement) or implicitly (i.e., the lack of acknowledgement) indicated that the current
packet is lost. BuzzBuzz complements ARQ with the selective use of Multi-Headers and
TinyRS. The three techniques (i.e., Multi-Headers, TinyRS, and ARQ) present different trade-
offs. Although ARQ has the least space and computational overhead, it is not efficient in very
noisy environments. Similarly, Multi-Headers has less computational overhead than TinyRS,
but it is less effective in the asymmetric region. The main challenge that BuzzBuzz addresses
is deciding which of these techniques is appropriate given the current link quality.
One can use RSSI samples, collected over a period of time, to detect external channel
interference. Musaloiu-E. et al. proposed three aggregation functions to analyze RSSI mea-
surements in real time and detect noisy 15.4 channels [62]. Since interference levels and
channel conditions change over time, motes need to periodically sample the channel in that
approach. Periodic sampling of RSSI may not be suitable for all the systems, especially those
that use low-power duty-cycling.
Instead of periodic sampling, BuzzBuzz infers the channel quality by observing the packet
losses, or the lack of packet acknowledgments. Upon receiving a packet from the upper layer,
the BuzzBuzz sender first attempts to deliver the packet using ARQ. If the packet cannot be
successfully delivered after three attempts, the BuzzBuzz sender adds the FEC information,
and inserts one Multi-Headers header in the packet. The sender then makes three more
transmission attempts of the encoded message before giving up. We note that the BuzzBuzz
sender explicitly indicates that a packet contains the FEC parity by setting the reserved bit
7 in the Frame Control Field (FCF) of the 15.4 frame to 1. In addition, to help the receiver
105
Chapter 4. External Interference 4.6. BuzzBuzz MAC Layer
locate the application payload, the reserved bit 8 in the FCF of the last Multi-Headers header
is set to 1.
The two-phase retransmission approach has the advantage of avoiding the overhead of
Multi-Headers and TinyRS when external interference is low. On the other hand, in en-
vironments with persistent interference, performing the two-phase probing for each packet
slows down the transmission as the initial ARQ phase will most likely fail. To address this
problem, BuzzBuzz remembers the setting of the last packet transmission for 60 seconds
and applies it to the next packet transfer. The sender falls back to the naıve retransmis-
sion scheme after sending three consecutive packets that pass the Multi-Headers CRC. The
receiver piggy-backs this information in its acknowledgments.
The results from Section 4.5.2 show that TinyRS requires at least 150 ms to decode a 95-
byte RS-encoded packet. This decoding time dominates the processing delay during packet
reception. This computational overhead is incurred even when the RS-encoded packet does
not have any bit errors. BuzzBuzz uses the CRC field of the packet (cf. Section 4.5.2) to
reduce the decoding overhead when the packet is received with no bit errors. Specifically,
BuzzBuzz uses CRC to check the integrity of the payload, and attempts decoding only if the
CRC check fails. CRC can be efficiently implemented with lookup tables, and computing CRC
over 65-byte application payload takes approximately 325 µs.
4.6.2 Evaluation
To evaluate BuzzBuzz, we integrated it with the PacketLink component in TinyOS 2.1.
As mentioned in Section 4.3.1, the CSMA/CA protocol implemented in TinyOS uses a fixed-
length backoff interval. We ran experiments on a 57-node TelosB testbed deployed in an office
building. All nodes were tuned to 15.4 channel 16. We decided to use a real testbed, because
106
Chapter 4. External Interference 4.6. BuzzBuzz MAC Layer
CTP CTPw/ BuzzBuzz
Packet Delivery Rate 43.05% 73.90%Avg number pkts/s in the network 38 11% pkts not ACKed 66% 35%% pkts received due to Multi-Headers hdr N/A 10.58%% corrupted pkts recovered with RS N/A 42.69%% decrease in 802.11g throughput 14.51% 3.35%
Table 4.5: Summary of experiment results running CTP with BuzzBuzz on a 57-node testbed.
simulators such as TOSSIM do not fully simulate the 15.4 modulation/demodulation scheme
and can not match the full realism of testbeds.
Since data collection applications dominate real-world WSN deployments, we developed a
simple application that uses the Collection Tree Protocol (CTP) [18] to deliver 65-byte applica-
tion data from each node at a rate of one packet per minute. CTP relies on acknowledgments
and packet retransmissions to improve the packet delivery rate.
The 802.11 interference originated from a co-located 802.11g testbed on the same building
floor. The 802.11g testbed backbone consisted of six OpenMesh OM1P mesh routers forming
an ad-hoc mesh backbone on 802.11 channel 5, with one node acting as the gateway between
the 802.11 mesh and an Ethernet network. Three Nokia N800 Internet tablets used the
OpenMesh network to continuously send TCP traffic to a PC on the same Ethernet network
as the OpenMesh gateway.
To ensure that CTP had sufficient time to build its routing tree, we started the 802.11g
traffic 20 minutes after the start of the 15.4 nodes. Each experiment lasted two hours. We
evaluate the effectiveness of BuzzBuzz using the end-to-end CTP data delivery ratio (or good-
put), and the amount of 15.4 network traffic.
Table 4.5 presents the average results from two experiment runs. CTP with BuzzBuzz is
able to deliver 71% more packets while reducing the network traffic by two thirds. Note that
since all data packets carry a 65-byte application payload, the packet delivery rate can be
107
Chapter 4. External Interference 4.7. Discussion
translated into effective data delivery rate. The difference in the amount of network traffic
is due to the aggressive retransmissions that CTP uses for unacknowledged packets. Specif-
ically, a CTP node can attempt up to 30 retransmissions before discarding the packet. Since
the probability of packets having at least one bit corruption is high in the presence of 802.11g
traffic, many retransmissions are needed for a successful packet delivery. The effectiveness
of BuzzBuzz is also evident from the fact that it reduces the number of packets not acknowl-
edged by 50%.
The middle part of Table 4.5 presents the contributions from each of the two BuzzBuzz
components. Approximately 10% of all packets received (valid or corrupted) were due to the
Multi-Headers header being detected by the radio. With a 30-byte parity, TinyRS was able
to successfully recover approximately 42% of the corrupted packets. Finally, Table 4.5 shows
that BuzzBuzz has a lower impact on the 802.11g throughput; a 3.35% decrease as compared
to 14.56% decrease in the case of CTP. This improvement is due to the fewer 15.4 packets
generated in the case of CTP with BuzzBuzz.
4.7 Discussion
4.7.1 802.11n Interference
The 802.11n standard introduces several new features not present in 802.11b/g. However,
these features do not completely mitigate the CTI problem between 15.4 and 802.11 networks,
making BuzzBuzz still relevant when countering WiFi interference.
802.11n incorporates channel bonding, a mechanism that combines two adjacent 802.11
channels to create a wider channel for higher throughput. As discussed earlier, current 15.4
deployments typically make use of interference-free channels that do not overlap with occu-
pied 802.11 channels as a technique to avoid 802.11 interference. However, with channel
108
Chapter 4. External Interference 4.7. Discussion
bonding, many of these interference-free channels will now be under 802.11n interference.
On the other hand, with 802.11b and 802.11g traffic, we observed that higher bit rates
make WiFi more responsive when backing off during 802.15.4 transmissions. Considering
that the 802.11n standard supports even higher bit rates than 802.11b and 802.11g, we
ran experiments to investigate whether 802.11n would impose less interference on 802.15.4
transmissions. We positioned a 802.15.4 sender and three 15.4 receivers in an office with a
pair of 802.11b/g/n nodes. The experiment results show 802.11n interference imposes 9% and
3% less impact on 15.4 PRR, compared to 802.11b and 802.11g interference respectively.
4.7.2 Network-Wide Blocker
BuzzBuzz is a reactive approach. Each ZigBee node operates independently to mitigate WiFi
interference. Leveraging the observation that 802.11 nodes back off during active transmis-
sions from nearby 15.4 nodes, it is possible to design proactive solutions for dense ZigBee
networks. The idea is to use a collection of dedicated 802.11 blockers placed close to each
802.11 node. Such implementation has challenges such as the explicit coordinations among
the blockers and between blocks and communicating 15.4 nodes. We outline a straw-men
implementation below.
Each blocker is tuned to a 15.4 channel which is adjacent to the 15.4 channel used by
the data transfer, but overlaps with the 802.11 channel that interferes with the 15.4 data
channel. For example, if the data is transmitted over 15.4 channel 11, the blocker can operate
on ZigBee channel 13. Assuming proper coordination among blockers, e.g. based on time or
other scheduling mechanisms, they together can periodically block the 802.11 channel for
small periods of time, leaving 15.4 data channels free of 802.11 interference.
We ran a simple experiment to evaluate the feasibility of this solution. We repeated the
109
Chapter 4. External Interference 4.7. Discussion
802.11 TCP 802.11 UDPWithout blocker 46% 70%With blocker 72% 78%
Table 4.6: 802.15.4 PRR with and without the blocker near the 802.11 AP.
experiment in Section 4.5.1 with a single 15.4 blocker placed next to the 802.11 access point.
We examined the 802.15.4 data throughput when the communicating 802.15.4 nodes are not
explicitly synchronized with the blocker.
We observe from Table 4.6 that 15.4 throughput increases in the presence of the blocker.
This result indicates that we may be able to achieve an even higher throughput with one or
multiple blockers. However, implementing such a blocker based solution introduces two main
challenges. First, it requires a dense deployment of blockers, especially when accommodat-
ing mobile 802.11 nodes. Second, the explicit coordination among the blockers and between
blockers and communicating 15.4 nodes may add too much overhead, resulting in significant
degradation of 802.11 performance. Nevertheless, for sensor networks which co-exist with
fixed WiFi APs, such as in home, office building, or hospital environments, the performance
improvement is attractive.
Although BuzzBuzz provides substantial PRR improvement under some conditions, the
effectiveness diminishes as the distance between the 15.4 sender and the 802.11 sender in-
creases, especially in the gray region. As mentioned in Section 4.4, in the gray region, the
802.11 signal is strong enough to impact the 802.15.4 reception, but not vice versa. This is
due to the asymmetries in the transmit power of 802.11 and 802.15.4 radios. A possible solu-
tion is to place dedicated 802.15.4 nodes near 802.11 nodes that create the gray region, and
these 802.15.4 nodes (called blockers) would continuously broadcast packets to slow down
802.11 traffic. One drawback of this approach is the number of blockers needed to cover
a large area, such as the library. Since most 802.11 networks are in infrastructure mode,
110
Chapter 4. External Interference 4.8. Related Work
placing blockers near APs only should be effective.
4.8 Related Work
The WSN community has acknowledged the impact of 802.11 interference on WSN applica-
tions in various settings. Ko et al. collected empirical results in a hospital setting where they
found that running CTP on a 15.4 network that overlapped with an active 802.11 channel
decreased the end-to-end goodput by a factor of three [40]. Hauer et al. studied the impact of
802.11 interference on body sensor networks and found that the position of bit errors in 15.4
packets are temporally correlated with 802.11 traffic [22]. Hou et al. observed a 15.4 packet
loss as high as 87%, with an 802.11b sender located in between two 15.4 nodes five meters
apart [23].
The common approach to mitigate 802.11 interference on 15.4 networks is to switch the
network to channels that do not overlap with an active 802.11 channel. Musaloiu-E. and
Terzis [62] proposed a distributed channel selection mechanism that detects 802.11 interfer-
ence using periodic RSSI samples. Unfortunately, 802.11-free 15.4 channels may not always
be available. Outside of Japan, only 15.4 channels 25 and 26 do not overlap with any of
the 802.11 channels. These two channels may not be sufficient to accommodate multiple
collocated 15.4 networks, or high-throughput applications [49]. Kannan et al. proposed an
off-line strategy to quantify the level of link burstiness due to interference, known as the β-
factor, from RSSI traces [82]. 15.4 nodes then use this information to estimate the expected
duration of the interference and defer outgoing packet transmissions to reduce the retrans-
mission cost. However, with 802.11-enabled mobile devices, the 802.11 interference pattern
may change dramatically in a short time, making off-line approaches less effective. More
recently, Boano et al. [6] simulated general 2.4 GHz interference using CC2420 radios in an
effort to identify mechanisms that can improve the robustness of existing MAC protocols un-
111
Chapter 4. External Interference 4.8. Related Work
der 802.11 interference. We investigate WiFi interference with a more persistent presence
and heavy traffic. Instead of deferring, our work aims to increase the PRR of 15.4 networks
by being more resilient to 802.11 interference.
Hou et al. proposed integrating an 802.11 radio chip on a 15.4-enabled medical device [23].
Before the device transmits a 15.4 frame, it sends out an 802.11 RTS packet to reserve the
channel, preventing nearby 802.11 radios from sending traffic. BuzzBuzz does not require
any additional hardware.
While early work from the 802.11 community concluded that 15.4 radios have little impact
on 802.11 radios [24], more recent work has amended this view. Gummadi et al. reported
that 15.4 signals can lower 802.11 SNR, significantly impacting the 802.11 throughput [19].
Furthermore, Pollin et al. showed that, despite 15.4 radios exercising carrier sense, 15.4
traffic can still collide with 802.11 transmissions and lower 802.11 throughput [72]. We are
the first to examine these interactions at a finer granularity and found two distinct modes of
15.4 and 802.11 interference.
The wireless community has recently proposed methods to efficiently recover from packet
corruption. Lin et al. proposed ZipTx that uses the RS code in 802.11 networks to reduce
the retransmission costs. In addition to the RS code, BuzzBuzz tries to minimize the num-
ber of erasure packets though Multi-Headers. Instead of relying on computation-intensive
RS code, Maranello applies CRCs on blocks of the payload [20]. Then, based on the CRC
validation feedback messages, the sender retransmits only the corrupted data blocks. How-
ever, efficiently sending feedback messages on a noisy channel can be difficult. In addition,
the trace-driven simulations in Section 4.5.2 show that such a block-CRC approach may not
efficiently solve the coexistence problem between 15.4 and 802.11 networks. Jamieson et
al. proposed Partial Packet Recovery (PPR) that replicates the packet header at the end of
802.11 packets [30], but PRR requires the use of software radios and does not work on exist-
112
Chapter 4. External Interference 4.8. Related Work
ing 802.11 hardware. Finally, Bluetooth uses a 13 rate FEC to protect the packet header from
bit errors by repeating each bit three times [28]. On the other hand, Multi-Headers replicates
both the packet header and preambles to improve the detection of incoming packets.
113
Chapter 5
Protocol Interference
5.1 Introduction
Modern radios used in wireless sensor networks, such as those that implement the IEEE
802.15.4 PHY standard [26], can operate in multiple frequency bands. Previous work has
shown that this capability can be used to increase the throughput of WSNs [102], reduce
intra-network interference [50,107], and reduce the impact of external interference to WSNs [79].
Despite these benefits, the channel-diversity mechanisms proposed thus far suffer from the
following problems. First, they are implemented at different layers of the protocol stack and
results in low code reuse. Second, they make assumptions about the applications’ character-
istics. Third, they do not support multiple applications sharing a physical radio at the same
time.
In this chapter, we argue that it is time to rethink how multi-channel protocols are engi-
neered and reach a consensus on how they fit in the emerging WSN architecture [14,38,70].
Such an agreement will promote interoperability and code reuse and is likely to simplify the
development of applications that will benefit from radio channel diversity. The additional
challenge is to do this integration with minimal disruption to the existing protocol stack and
application archetypes.
We posit that multi-channel protocols can be factored into two different components: a
114
Chapter 5. Protocol Interference 5.2. Background
Channel Allocation (CA) component responsible for allocating one or more frequency channels
to upper layers and a Channel Synchronization (CS) component that determines the time
interval that a node should switch its radio to a certain channel. Network-level protocols can
include their own CA components, allowing them to express specific ways of using frequency
channels. For example, real-time network protocols can reduce data delivery latency due
to channel switching by switching all nodes over an end-to-end network path to the same
channel. On the other hand, a single CS component is implemented at the MAC layer that
exposes the same MAC-layer interface and augments it with a minimal interface for using
multiple frequency channels.
The last part of this chapter describes ViR, a straw man implementation of the proposed
components, and outlines how existing multi-channel protocols can be re-factored along these
lines. Finally, we show how the proposed architecture can be used to virtualize the radio
among multiple network layer protocols that run concurrently on the same mote.
5.2 Background
5.2.1 Benefits of Using Multiple Channels
We group multi-channel protocols into three categories based on their primary purpose for
using multiple frequency channels.
Improve network throughput. Wu et al. proposed the TMCP multi-channel tree collec-
tion protocol and observed that the network throughput doubled as the number of available
channels increased from two to eight [102]. Zhang et al. proposed the TMMAC MAC protocol
and showed that it achieved seven times the throughput of the standard 802.11 DCF protocol
in a simulated network with 40 concurrent flows when six channels were available [105].
Minimize intra-network interference. Since the radio is a shared medium, concurrent
115
Chapter 5. Protocol Interference 5.2. Background
transmissions in the same broadcast domain result in collisions and possibly packet losses.
Wu et al. presented empirical results suggesting that 802.15.4 radios have a maximum of
eight orthogonal channels1 [102]. Zhou et al. [108] and Liang et al. [50] used channel diversity
to reduce the number of conflicting transmissions. The latter work also observed that channel
diversity promotes spatial reuse and reduces the latency of network-wide dissemination.
Avoid external interference. Considering that multiple radio standards operate in the
same unlicensed frequency bands (e.g., 802.11, 802.15.4, and 802.15.1 all operate in the 2.4
GHz band), channel diversity is one method to mitigate external interference among collo-
cated networks. The WirelessHART standard offers an example of this approach [79]. It
employs a TDMA protocol, in which nodes switch their radio frequency at the start of each
time slot. The frequency selection is based on the current time slot index and a secret channel
offset shared by the two communicating nodes.
5.2.2 Limitations of Current Multi-Channel Protocols
While the discussion in Section 5.2.1 suggests that channel diversity can improve perfor-
mance, we argue that current approaches lack the flexibility and the generality to support
diverse application requirements.
Lack of Architectural Consistency. Based on the experience from developing and deploy-
ing WSN applications, an architecture for wireless sensor networks is starting to emerge [14,
38, 70]. Nevertheless, there is no consensus as to where channel switching belongs in the
architecture and how it interfaces with the rest of the protocol stack. Existing multi-channel
protocols are mostly implemented at the MAC level [43, 105, 108] or integrated to network
layer protocols [49, 50, 95, 102]. In turn, this lack of agreement impedes code reuse and pro-
tocol interoperability.1The IEEE 802.15.4 standard specifies 16 non-overlapping channels in the 2.4 GHz band.
116
Chapter 5. Protocol Interference 5.3. A Multi-Channel Protocol Framework
Restrictive Application Assumptions. Existing multi-channel protocols generally as-
sume simplistic application behaviors —data collection with constant traffic rate being the
most common one. For example, Le et al. proposed a multi-channel MAC protocol that is opti-
mized for collection and aggregation traffic patterns [43]. However, supporting an expanding
application landscape requires a multi-channel framework that can support multiple traffic
patterns and QoS levels.
Monolithic Applications. After a decade of active development, open-source implementa-
tions of many WSN protocols are publicly available. This availability has enabled the devel-
opment of applications that compose multiple network protocols to implement their logic. For
example, one could combine a data collection protocol with a dissemination protocol to imple-
ment an environmental monitoring application with dynamic retasking [93,97]. However, as
Choi et al. pointed out, applications running multiple protocols can experience inter-protocol
conflicts [29]. For example, two protocols running on the same node can independently decide
to switch to two different channels, leading to packet losses for one of the protocols. Instead,
multi-channel protocols should support multiple concurrent upper-level protocols and auto-
matically resolve such conflicts (e.g., by switching between the two channels).
5.3 A Multi-Channel Protocol Framework
Section 5.2.2 argues that we need a consistent way to integrate frequency diversity to the
WSN architecture. Doing so requires addressing three challenges. First, we need to iden-
tify and group tightly coupled functions that multi-channel protocols require into reusable
components. Second, we should structure and position these components in ways that lever-
age the functionalities of existing architecture components. Finally, define a minimum set of
interfaces that can be used to implement different network protocols and end-to-end applica-
tions.
117
Chapter 5. Protocol Interference 5.3. A Multi-Channel Protocol Framework
Network Protocol 2
Channel AllocationNetwork Protocol 1
Channel
ReservationFreqHopping
Seq
Packet
Timer
Send /
Receive /
LowPowerListening
ChannelUtil
Monitor
Channel
PollerC
LowPowerListening
ReceivePreamble
Sender
Channel Synchronization
Radio Core
Network
Layer
MAC
Layer
ListenerCSenderC
Figure 5.1: Proposed decomposition and interfaces of multi-channel protocols.
5.3.1 Architecture Overview
We propose to decompose multi-channel protocols into a channel allocation (CA) compo-
nent and a channel synchronization (CS) component. The goal of the CA component is
to assign one or more channels to the network’s nodes in a way that optimizes application-
specific metrics such as network throughput. The CS component controls when radios should
switch to each of the CA-selected channels so that nodes can communicate with each other.
The CS component should base these scheduling decisions on data delivery metrics such as
latency and energy usage.
Contrary to existing work that implements multi-channel protocols at either the network
or the MAC layer, we propose a cross-layer approach that positions the CA component at the
network layer and the CS component at the MAC layer.
We allow each network protocol to implement its own CA component for two reasons.
First, since different network layer protocol have different sets of requirements, a common
CA component would not be suitable for all use cases. Second, because the CA component
is part of the protocol it has access to the protocol’s node state and information exchanged
118
Chapter 5. Protocol Interference 5.3. A Multi-Channel Protocol Framework
among protocol peers.
On the other hand, there is a single CS component per MAC protocol that interacts with
all CA components at the network layer. This design has two advantages. As the single
point of entry for all radio control and I/O requests, the CS component has the visibility
on radio requests to resolve possible inter-protocol conflicts. In addition, the CS component
can leverage the MAC layer functionalities to deliver messages to next-hop neighbors. Such
functionalities include neighborhood state maintenance and link-level acknowledgments.
The decomposition of multi-channel protocols to CA and CS components is similar to the
separation of routing and forwarding planes in the Internet; the CA component and the rout-
ing plane make network-wide decisions, while the CS component, similar to the forwarding
plane, implement the mechanisms necessary to forward packets to the next hop.
Figure 5.1 illustrates the position of the proposed CA and CS components in the WSN
architecture. The section that follows describes the interfaces these components expose and
use.
5.3.2 Component Interfaces
The MAC layer provides three basic services to upper layers: transmit a packet, receive a
packet, and turn the radio on and off. The CS component minimally extends this narrow
interface to allow upper layers to leverage the benefits of multiple radio channels. At the
same time, the existence of the CS component is transparent to legacy protocols that are
agnostic to the availability of multiple radio frequencies.
We divide the new CS interfaces to those that the CA component uses (§5.3.2) and those
that network layer protocols can use to improve their performance (§5.3.2). We note that
while we use nesC to describe TinyOS-compliant interfaces, they are not specific to TinyOS.
119
Chapter 5. Protocol Interference 5.3. A Multi-Channel Protocol Framework
Interfaces for the CA Component
Channel Reservation. The ChannelReservation interface offers two services. First,
CA components can issue the numChannels() command to retrieve the number of available
radio channels. Second, the reserveChannel() command allows a CA component to reserve
a channel on the local node. Since the CS component intercepts all radio control requests, if a
CA component attempts to switch to a previously reserved channel, the request will fail and
return the appropriate error code.
interface ChannelReservation {command uint8_t numChannels();command error_t reserveChannel(uint8_t channel);
}
Channel Utilization. The ChannelUtilMonitor interface reports the utilization of a
channel. Although the interface leaves the definition of utilization open, common choices in-
clude the number of network-layer protocols that intend to use a particular channel and the
number of packets that have been sent and received on a channel. The interface mandates
the getUtilization() command to return an integer between 0 and 255, corresponding to
the lowest and the highest utilization respectively.
interface ChannelUtilMonitor {command uint8_t getUtilization(uint8_t channel);
}
Interfaces for Network Layer Protocols
Frequency Hopping Sequence. The CS component switches the radio channel according
to an internal schedule to handle radio transmit/receive requests across different channels.
The FreqHoppingSeq interface allows protocols at the network layer to query for the next
120
Chapter 5. Protocol Interference 5.3. A Multi-Channel Protocol Framework
time that the CS component switches the radio to a particular channel. This information can
be useful for setting protocol timeout values, especially since multiplexing a radio essentially
reduces the effective bandwidth for each protocol.
interface FreqHoppingSeq {command uint32_t nextTime(uint8_t channel);
}
Timers. Existing multi-channel mechanisms generally follow an eventual delivery service
model due to the delay that channel synchronization introduces. Moreover, they do not pro-
vide upper layers the ability to express the urgency of their packet transmission requests. For
example, a time synchronization protocol could indicate that its timing beacons need urgent
transmission.
To meet these requirements, the CS component introduces the PacketTimers interface
with commands that set per-packet timing attributes. First, we borrow the urgent bit concept
from the SP architecture [70]. The urgent bit notifies the CS component that a radio request
should have a higher priority and be processed before others. Second, we allow upper layers
to set service duration timers for their packet transmissions. When one of these timers fires,
the MAC layer should discard the corresponding transmission request and signal the proper
return error code to the network layer.
interface PacketTimers {command void setUrgentBit(message_t* pkt);command void clrUrgentBit(message_t* pkt);command void setServiceDuration(message_t* pkt, uint32_t timeout);command uint32_t getServiceDuration(message_t* pkt);
}
121
Chapter 5. Protocol Interference 5.4. Prototype Implementation
5.3.3 Protocol Composition Example
Next, we demonstrate how multi-channel protocols can be developed using the interfaces
proposed in the previous sections. We do so through Typhoon, a network-wide dissemination
protocol that uses a dedicated control channel to initiate and negotiate the channels used for
subsequent data transfers (cf. Chapter 2).
During bootstrapping, the CA component uses the ChannelReservation interface to re-
serve the dedicated control channel and to query the total number of available radio channels.
Typhoon nodes then advertise a summary of objects that they have. These advertisements
allow nodes to detect any stale objects within their local neighborhood and initiate requests
to retrieve new version of the objects.
To retrieve an object from neighbor x, node y initiates the handshake by sending a request
message over the common control channel. Node x then queries its CA component to get a
new channel for the data transfer. The CA component on node x can select a random channel,
or a relatively free channel by querying the ChannelUtilMonitor interface for each chan-
nel’s utilization. Node x finishes the handshake by responding to y with the new channel
number. Finally, both nodes call the MAC layer API (e.g., the CC2420Config interface in
TinyOS) to switch to the new channel for the data transfer. The CS module intercepts these
requests as it exposes this interface (see Fig.5.1). After the data transfer completes, x and y
return to the control channel.
5.4 Prototype Implementation
We now describe ViR, a straw man implementation of the channel synchronization compo-
nent. While other implementations are possible, we use ViR to show how the proposed
architecture can support multiple concurrent network layer protocols with different radio
122
Chapter 5. Protocol Interference 5.4. Prototype Implementation
frequency usage patterns.
5.4.1 Basic Protocol
Being the channel synchronization component, ViR sits above all MAC layer components and
intercepts I/O and radio control requests (e.g., requests to switch channels) from the network
layer. The ViR uses this information to multiplex the physical radio across the different
network layer protocols.
ViR divides time into equal-length slots, with nodes occupying one of the active channels
during each time slot. A channel is active if at least one network layer protocol has previously
requested to switch to that channel. We note that while all nodes use the same slot duration,
they do not have to synchronize their schedules.
When a node has no pending outgoing packets, it cycles through all active channels in
ascending order. Incoming packets are immediately delivered to the intended protocol. Nodes
broadcast a CH SCHEDULE packet at the beginning of each slot to help neighbors learn
their channel schedule. CH SCHEDULE packets carry timing offsets that indicate the next
time a node is scheduled to be on each of the active channels.
One change is necessary to this basic scheme to handle outgoing packets. Specifically, a
node can deviate from the predefined channel schedule in order to rendezvous and transmit
packets to a neighbor. The probability of this deviation depends on local node’s confidence
that the intended neighbor will be on the desired channel during the next time slot. For
instance, the age of the cached CH SCHEDULE packets from a neighbor can influence this
confidence. However, if a node x frequently deviates from its predefined channel schedule, it
becomes difficult for other nodes to rendezvous with node x. For this reason, nodes stay on
their scheduled channel during the next slot with probability p = 0.25. Nodes switch to one
123
Chapter 5. Protocol Interference 5.4. Prototype Implementation
of the N channels with pending outgoing packets with probability (1− p)/N .
After deciding the channel for the next time slot, ViR switches the physical radio chan-
nel and then it repeatedly sends any pending packets intended for the current channel one
by one. Similar to the sender-initiated packet delivery approach, ViR stops transmitting a
packet once the intended receiver acknowledges its reception, or when the packet exceeds its
lifetime (the PacketTimers interface in Section 5.3.2 can be used to assign packet transmis-
sion deadlines).
Optimizations are possible on top of this basic scheme. If a node has pending packets
for several neighbors, it can attempt delivery using a round-robin schedule to avoid head of
line blocking by a node that is late switching to the current channel. Furthermore, if a node
knows when a neighbor will enter the channel (e.g., from CH SCHEDULE messages) it can
postpone the transmission of packet(s) intended to that neighbor.
5.4.2 Evaluation
The primary goal of the channel synchronization component is to deliver packets from mul-
tiple network layer protocols with different radio frequency usage patterns. In this section,
we evaluate how well the ViR implementation of the CS component meets this requirement.
Specifically, we explore a scenario in which CTP [18], the standard data collection protocol in
TinyOS, operates concurrently with Typhoon, a multi-channel network dissemination proto-
col [50].
We implemented ViR in TinyOS 2.1 and used the TOSSIM simulator to measure the pro-
tocols’ performance. The evaluation results were collected with the default TinyOS MAC
without low-power listening (LPL). We modified TOSSIM to simulate a physical layer that
supports multiple channels. We simulated a 3× 3 node grid in which the top left node acts as
124
Chapter 5. Protocol Interference 5.5. Discussion
CTP delivery rate (%) CTP avg. latency (ms)CTP only 99.7 56.8Without ViR 52.3 642.8With ViR 95.4 979.2
Table 5.1: When running both CTP and Typhoon on a 3× 3 mesh, ViR improves the CTP de-livery rate to 95%. The increase in CTP latency is due to ViR serving simultaneous Typhoonand CTP traffic.
a base station. Each node is able to communicate with its immediate grid neighbors. All CTP
nodes generate one packet every second. In addition to being a CTP sink, the base station
initiates a network-wide dissemination of a 50KB object using Typhoon.
Table 5.1 summarizes the experimental results. When both protocols are active, ViR can
approximately double CTP’s delivery rate. We used the experiment’s traces to understand
the cause of this performance deterioration. We found that Typhoon’s channel-switching be-
havior segmented the network paths that CTP used to deliver packets to the root. Note that
if it is the CTP sender that switches to a different channel, all the alternate parents that
CTP previously maintains to improve data delivery will not help. ViR mitigates this problem
by exchanging frequency hopping sequences among neighbors and scheduling transmissions
when both the sender and the intended receiver are on the desired channel. Since multiplex-
ing the radio effectively reduces the available bandwidth for each protocol, the end-to-end
latency of CTP packets should increase when both protocols are active. The results from
Table 5.1 validate this.
5.5 Discussion
So far this chapter has described how the CS module can multiplex requests from one or
more network layer protocols that have conflicting requests for the radio such as changing
the radio frequency. It is interesting to note that the CS component has properties that allow
it to function as a virtualization layer and virtualize the physical radio among all the network
125
Chapter 5. Protocol Interference 5.5. Discussion
protocols that run on the same mote. Specifically, the CS component offers the following
properties:
Traffic Isolation. Concurrent traffic flows on the same radio channel can experience col-
lisions leading to energy waste and performance degradation. Choi et al. [29] cited a case
in which the bursty traffic from Deluge [25] changed MintRoute’s perception on link quali-
ties [101]. Using the mechanisms proposed in this chapter, one can de-conflict traffic flows by
placing them over orthogonal channels.
Process Isolation. As the single entry point for all radio requests from network protocols,
the CS component can track protocol-specific radio states and ensure per-protocol state con-
sistency. These states include the frequency channel, node address, transmit power level,
and radio power state. Furthermore, the CS component ensures that state conflicts among
protocols will not cause failures.
Resource Management. The CS component manages the physical radio using its global
knowledge of protocol-specific radio states. For example, it can shut down the physical radio
after all consumers have indicated no further intention in using the radio.
5.5.1 Support for Low-Power MACs
Duty-cycling a radio is desirable since it reduces idle listening and conserves energy. Inte-
grating duty-cycling and channel switching effectively multiplexes the radio in two different
dimensions: time and frequency. We outline a method to achieve this integration.
We leverage the MAC Layer Architecture (MLA), a component-based architecture for
duty-cycling MAC layers [38]. Since there are similarities between the operations of the CS
component and duty-cycling MACs, the CS component can reuse functionalities in the MLA.
The MLA defines three high-level components: ChannelPollerC, SenderC, and ListenerC.
126
Chapter 5. Protocol Interference 5.5. Discussion
ChannelPollerC maintains the radio power state by both following the duty-cycling sched-
ule and sampling the radio channel for activity. SenderC and ListenerC are concerned with
one-hop packet delivery by using link-level mechanisms such as preambles and retransmis-
sions. As Figure 5.1 illustrates, we position the CS component above ChannelPollerC so
that minimum changes are required to the MLA. In addition, the CS component can delegate
one-hop packet delivery and radio power state control to the MLA’s components.
However, two changes are necessary to the protocol described in Section 5.4.1, for the CS
component to work above the MLA. Namely, the ViR slot size should be equal to one duty
cycle period and the ViR slots should be aligned with the start of each duty cycle period.
These requirements imply that each ViR slot occupies one channel and one duty cycle period.
Since the CS component delegates the radio’s on/off state to the duty-cycling MAC below, we
add call-back events to notify the CS component of the radio’s state changes.
127
Chapter 6
Conclusions
The WSN community has come a long way in the past decade. For WSN deployments in the
next decade, radio resource sharing is a pressing issue that the WSN community needs to ad-
dress. In particular, as the number of radio users in a given space increases, the lack of radio
resource coordination can result in significant interference and thus low communication effi-
ciency. This dissertation aims to characterize and mitigate three types of radio interference:
intra-network, external, and protocol interference.
The first part of the dissertation looks at intra-network interference, or interference due
to nodes in a neighborhood transmitting on the same radio frequency band simultaneously.
Typhoon improves data dissemination performance through the use of channel diversity and
opportunistic overhearing. Simulation results of sparse and lossy networks show that Ty-
phoon can offer a threefold speed-up over Deluge, the current de facto standard for data
dissemination in the TinyOS community. WRAP improves data collection performance in
large-scale networks through centralized traffic coordination with a token-passing mecha-
nism. Results from a production data center testbed show that WRAP outperforms protocols
that use rate-based congestion control (RCRT) and uncoordinated transmissions (CTP). As
the network loads increase with network density, WRAP has a higher data delivery ratio and
lower inter-packet interval than RCRT and CTP.
128
Chapter 6. Conclusions
Next, this dissertation considers external interference, or the interaction among networks
with different radio technologies. This part of the dissertation aims to devise mechanisms to
improve the coexistence of 802.15.4 and 802.11 networks sharing the same radio frequency
bands. We observed that, despite the lower transmission power, 802.15.4 radios can still
trigger 802.11 senders’ CCA check under certain conditions. This observation forms the basis
of Multi-Headers that improves the receivers’ detection rate of incoming 802.15.4 frames. In
cases where the 802.15.4 signal is too weak to be detected by 802.11 senders, we leverage
error correction codes (ECC) to recover corrupted packets to minimize the retransmission
costs. We collectively term these two solutions BuzzBuzz. Results from a medium-sized
office testbed show that BuzzBuzz can improve the CTP application goodput on the 802.15.4
testbed by 30% and the 802.11 network throughput by 11%.
The last part of the dissertation considers protocol interference that arises from multiple
applications on the same node sending conflicting commands to the same physical radio. Our
contributions include proposing a framework and interfaces for implementing multi-channel
protocols. We show how ViR, our implementation of the framework, can virtualize and mul-
tiplex a single physical radio. Simulations of two channel-conflicting protocols demonstrate
that ViR can lower the impact on the network data delivery ratio.
Finally, I would like to express my thoughts on the contributions presented in this dis-
sertation. Looking back at the effort in studying radio interference, I find it intriguing that
the mitigation approaches described here are all software-based solutions. Looking forward,
I think an interesting direction in this research area is to consider what capabilities that
the next radio for WSNs would offer. In fact, after talking with experts in different area of
computer science, I started to question various aspects of the current 802.15.4 radio specifica-
tion. An example is whether the effectiveness of the the linear code mandated by the 802.15.4
standard to overcome RF noise actually justifies the overhead. As software is designed with
129
Chapter 6. Conclusions
considerations of the underlying hardware capabilities, I believe that evolution in the radio
hardware will allow the WSN community to more effectively tackle deployment challenges in
the future.
130
Bibliography
[1] Apple Computer Inc. Apple Airport Express. Available at http://www.apple.com/
airportexpress.
[2] Mahesh (Umamaheswaran) Arumugam. Infuse: A TDMA Based Reprogramming Ser-
vice for Sensor Networks. In SenSys ’04: Proceedings of the 2nd international con-
ference on Embedded networked sensor systems, pages 281–282, New York, NY, USA,
2004. ACM.
[3] Atmel Corporation. Low Power 2.4 GHz Transceiver for ZigBee, IEEE 802.15.4, 6LoW-
PAN, RF4CE and ISM applications. Available at http://www.atmel.com/dyn/
resources/prod_documents/doc5131.pdf, 2009.
[4] Elizabeth A. Basha, Sai Ravela, and Daniela Rus. Model-based Monitoring for Early
Warning Flood Detection. In Proceedings of the 6th ACM conference on Embedded
network sensor systems, SenSys ’08, pages 295–308, New York, NY, USA, 2008. ACM.
[5] Christian L. Belady. In the Data Center, Power and Cooling Costs More Than the IT
Equipment It Supports. ElectronicsCooling magazine, 3(1), February 2007.
[6] Carlo Alberto Boano, Thiemo Voigt, Nicolas Tsiftes, Luca Mottola, Kay Romer, and
Marco Antonio Zuniga. Making Sensornet MAC Protocols Robust Against Interference.
In EWSN ’10: Proceedings of the 7th European conference on Wireless sensor networks,
2010.
131
Bibliography Bibliography
[7] Nicolas Burri, Pascal von Rickenbach, and Roger Wattenhofer. Dozer: Ultra-low Power
Data Gathering in Sensor Networks. In IPSN ’07: Proceedings of the 6th international
conference on Information processing in sensor networks, pages 450–459, New York,
NY, USA, 2007. ACM.
[8] Bor-rong Chen, Kiran-Kumar Muniswamy-Reddy, and Matt Welsh. Ad-hoc Multicast
Routing on Resource-Limited Sensor Nodes. In REALMAN ’06: Proceedings of the 2nd
international workshop on Multi-hop ad hoc networks: from theory to reality, pages
87–94, New York, NY, USA, 2006. ACM.
[9] Gong Chen, Wenbo He, Jie Liu, Suman Nath, Leonidas Rigas, Lin Xiao, and Feng Zhao.
Energy-Aware Server Provisioning and Load Dispatching for Connection-Intensive In-
ternet Services. In NSDI ’08: Proceedings of the 5th USENIX Symposium on Networked
Systems Design and Implementation, 2008.
[10] Brent Clark, Charles Colbourn, and David Johnson. Unit Disk Graphs. Discrete Math-
ematics, 86:165–177, 1990.
[11] Nathan Cooprider, Will Archer, Eric Eide, David Gay, and John Regehr. Efficient Mem-
ory Safety for TinyOS. In SenSys ’07: Proceedings of the 5th international conference on
Embedded networked sensor systems, pages 205–218, New York, NY, USA, 2007. ACM.
[12] Douglas S. J. De Couto, Daniel Aguayo, John Bicket, and Robert Morris. A High-
Throughput Path Metric for Multi-hop Wireless Routing. In MobiCom ’03: Proceedings
of the 9th annual international conference on Mobile computing and networking, pages
134–146, 2003.
[13] Dust Networks, Inc. Time Synchronized Mesh Protocol. Available at http://www.
dustnetworks.com/docs/TSMP_Whitepaper.pdf, 2006.
132
Bibliography Bibliography
[14] Cheng Tien Ee, Rodrigo Fonseca, Sukun Kim, Daekyeong Moon, Arsalan Tavakoli,
David Culler, Scott Shenker, and Ion Stoica. A Modular Network Layer for Sensornets.
In OSDI ’06: Proceedings of the 7th USENIX Symposium on Operating Systems Design
and Implementation, 2006.
[15] Ahmed Elsaify, Paritosh Padhy, Kirk Martinez, and Gang Zou. Gwmac: A TDMA Based
MAC Protocol for a Glacial Sensor Network. In PE-WASUN ’07: Proceedings of the 4th
ACM workshop on Performance evaluation of wireless ad hoc, sensor,and ubiquitous
networks, pages 54–61, New York, NY, USA, 2007. ACM.
[16] Federspiel Controls. Federspiel Controls. http://www.federspielcontrols.com.
[17] Deepak Ganesan, Razvan Cristescu, and Baltasar Beferull-Lozano. Power-Efficient
Sensor Placement and Transmission Structure for Data Gathering Under Distortion
Constraints. In IPSN ’04: Proceedings of the 3rd international symposium on Informa-
tion processing in sensor networks, pages 142–150, New York, NY, USA, 2004. ACM.
[18] Omprakash Gnawali, Rodrigo Fonseca, Kyle Jamieson, David Moss, and Philip Levis.
Collection Tree Protocol. In SenSys ’09: Proceedings of the 7th ACM Conference on
Embedded Networked Sensor Systems, pages 1–14, New York, NY, USA, 2009. ACM.
[19] Ramakrishna Gummadi, David Wetherall, Ben Greenstein, and Srinivasan Seshan.
Understanding and Mitigating the Impact of RF Interference on 802.11 Networks. In
SIGCOMM ’07: Proceedings of the 2007 conference on Applications, technologies, archi-
tectures, and protocols for computer communications, pages 385–396, New York, NY,
USA, 2007. ACM.
[20] Bo Han, Aaron Schulman, Francesco Gringoli, Neil Spring, Bobby Bhattacharjee,
Lorenzo Nava, Lusheng Ji, Seungjoon Lee, and Robert Miller. Maranello: Practical
133
Bibliography Bibliography
Partial Packet Recovery For 802.11. In NSDI’10: Proceedings of the 7th USENIX con-
ference on Networked systems design and implementation, pages 14–14, Berkeley, CA,
USA, 2010. USENIX Association.
[21] Carl Hartung, Richard Han, Carl Seielstad, and Saxon Holbrook. FireWxNet: A Multi-
Tiered Portable Wireless System for Monitoring Weather Conditions in Wildland Fire
Environments. In MobiSys ’06: Proceedings of the 4th international conference on Mo-
bile systems, applications and services, 2006.
[22] Jan-Hinrich Hauer, Vlado Handziski, and Adam Wolisz. Experimental Study of the
Impact of WLAN Interference on IEEE 802.15.4 Body Area Networks. In EWSN ’09:
Proceedings of the 6th European Conference on Wireless Sensor Networks, pages 17–32,
Berlin, Heidelberg, 2009. Springer-Verlag.
[23] James Hou, Benjamin Chang, Dae-Ki Cho, and Mario Gerla. Minimizing 802.11 In-
terference On ZigBee Medical Sensors. In BodyNets ’09: Proceedings of the Fourth
International Conference on Body Area Networks, pages 1–8, ICST, Brussels, Belgium,
Belgium, 2009. ICST (Institute for Computer Sciences, Social-Informatics and Telecom-
munications Engineering).
[24] Ivan Howitt and Jose Gutierrez. IEEE 802.15.4 Low Rate - Wireless Personal Area
Network Coexistence Issues. In WCNC ’03: Proceedings of the IEEE Wireless Commu-
nications and Networking, volume 3, pages 1481–1486, March 2003.
[25] Jonathan W. Hui and David Culler. The Dynamic Behavior of a Data Dissemination
Protocol for Network Programming at Scale. In SenSys ’04: Proceedings of the 2nd
international conference on Embedded networked sensor systems, pages 81–94, New
York, NY, USA, 2004. ACM.
134
Bibliography Bibliography
[26] IEEE Computer Society. 802.15.4: Wireless Medium Access Control (MAC) and Phys-
ical Layer (PHY) Specifications for Low-Rate Wireless Personal Area Networks (LR-
WPANs). Available at http://standards.ieee.org/getieee802/download/
802.15.4-2003.pdf.
[27] IEEE Computer Society. IEEE Standard for Information Technology - Telecommuni-
cations and information exchange between systems - Local and metropolitan area net-
works - Specific requirements Part 11: Wireless LAN Medium Access Control (MAC)
and Physical Layer (PHY) Specifications. Available at: http://standards.ieee.
org/getieee802/download/802.11-2007.pdf.
[28] IEEE Computer Society. IEEE Standard for Information Technology - Telecom-
munications and information exchange between systems - Local and metropolitan
area networks - Specific requirements Part 15.1: Wireless medium access control
(MAC) and physical layer (PHY) specifications for wireless personal area networks
(WPANs). Available at: http://standards.ieee.org/getieee802/download/
802.15.1-2005_part1.pdf.
[29] Jung Il Choi, Maria A. Kazandjieva, Mayank Jain, and Philip Levis. The Case for a
Network Protocol Isolation Layer. In SenSys ’09: Proceedings of the 7th ACM Confer-
ence on Embedded Networked Sensor Systems, pages 267–280, New York, NY, USA,
2009. ACM.
[30] Kyle Jamieson and Hari Balakrishnan. PPR: Partial Packet Recovery for Wireless Net-
works. In SIGCOMM ’07: Proceedings of the 2007 conference on Applications, technolo-
gies, architectures, and protocols for computer communications, pages 409–420, New
York, NY, USA, 2007. ACM.
135
Bibliography Bibliography
[31] Jaein Jeong and Cheng-Tien Ee. Forward Error Correction in Sensor Networks. In
WWSN ’07: Proceedings of the First International Workshop on Wireless Sensor Net-
works, 2007.
[32] Xiaofan Jiang, Stephen Dawson-Haggerty, Prabal Dutta, and David Culler. Design
and Implementation of A High-Fidelity AC Metering Network. In Proceedings of the
2009 International Conference on Information Processing in Sensor Networks, IPSN
’09, pages 253–264, Washington, DC, USA, 2009. IEEE Computer Society.
[33] Jon Dugan and Mitch. Iperf. Available at http://sourceforge.net/projects/
iperf/.
[34] Philo Juang, Hidekazu Oki, Yong Wang, Margaret Martonosi, Li Shiuan Peh, and
Daniel Rubenstein. Energy-Efficient Computing for Wildlife Tracking: Design Trade-
offs and Early Experiences with ZebraNet. SIGARCH Comput. Archit. News, 30(5):96–
107, 2002.
[35] Sukun Kim, Rodrigo Fonseca, and David Culler. Reliable transfer on wireless sensor
networks. In SECON ’04: Proceedings of the First Annual IEEE Communications Soci-
ety Conference on Sensor and Ad Hoc Communications and Networks, pages 449–459,
October 2004.
[36] Sukun Kim, Rodrigo Fonseca, Prabal Dutta, Arsalan Tavakoli, David Culler, Philip
Levis, Scott Shenker, and Ion Stoica. Flush: A Reliable Bulk Transport Protocol for
Multihop Wireless Networks. In SenSys ’07: Proceedings of the 5th international con-
ference on Embedded networked sensor systems, pages 351–365, New York, NY, USA,
2007. ACM.
136
Bibliography Bibliography
[37] Sukun Kim, Shamim Pakzad, David Culler, James Demmel, Gregory Fenves, Steve
Glaser, and Martin Turon. Wireless Sensor Networks for Structural Health Moni-
toring. In SenSys ’06: Proceedings of the 4th international conference on Embedded
networked sensor systems, pages 427–428, New York, NY, USA, 2006. ACM.
[38] Kevin Klues, Gregory Hackmann, Octav Chipara, and Chenyang Lu. A Component-
Based Architecture for Power-Efficient Media Access Control in Wireless Sensor Net-
works. In SenSys ’07: Proceedings of the 5th international conference on Embedded
networked sensor systems, pages 59–72, New York, NY, USA, 2007. ACM.
[39] Kevin Klues, Chieh-Jan Mike Liang, Jeongyeup Paek, Razvan Musaloiu-E., Philip
Levis, Andreas Terzis, and Ramesh Govindan. TOSThreads: Thread-safe and Non-
invasive Preemption in TinyOS. In SenSys ’09: Proceedings of the 7th ACM Conference
on Embedded Networked Sensor Systems, pages 127–140, New York, NY, USA, 2009.
ACM.
[40] JeongGil Ko, Tia Gao, and Andreas Terzis. Empirical Study Of A Medical Sensor Ap-
plication In An Urban Emergency Department. In BodyNets ’09: Proceedings of the
Fourth International Conference on Body Area Networks, pages 1–8, ICST, Brussels,
Belgium, Belgium, 2009. ICST (Institute for Computer Sciences, Social-Informatics
and Telecommunications Engineering).
[41] Sandeep S. Kulkarni and Limin Wang. MNP: Multihop Network Reprogramming Ser-
vice for Sensor Networks. In ICDCS ’05: Proceedings of the 25th IEEE International
Conference on Distributed Computing Systems, pages 7–16, Washington, DC, USA,
2005. IEEE Computer Society.
137
Bibliography Bibliography
[42] Hieu Khac Le, Dan Henriksson, and Tarek Abdelzaher. A Control Theory Approach
to Throughput Optimization in Multi-channel Collection Sensor Networks. In IPSN
’07: Proceedings of the 6th international conference on Information processing in sensor
networks, pages 31–40, New York, NY, USA, 2007. ACM.
[43] Hieu Khac Le, Dan Henriksson, and Tarek Abdelzaher. A Practical Multi-Channel
Medium Access Control Protocol for Wireless Sensor Networks. In IPSN ’08: Proceed-
ings of the 7th international conference on Information processing in sensor networks,
2008.
[44] HyungJune Lee, Alberto Cerpa, and Philip Levis. Improving Wireless Simulation
Through Noise Modeling. In IPSN ’07: Proceedings of the 6th international confer-
ence on Information processing in sensor networks, pages 21–30, New York, NY, USA,
2007. ACM.
[45] Philip Levis, Eric Brewer, David Culler, David Gay, Samuel Madden, Neil Patel, Joe Po-
lastre, Scott Shenker, Robert Szewczyk, and Alec Woo. The Emergence of A Networking
Primitive in Wireless Sensor Networks. Commun. ACM, 51(7):99–106, 2008.
[46] Philip Levis, David Gay, Vlado Handziski, Jan-Heinrich Hauer, Ben Greenstein, Mar-
tin Turon, Jonathan Hui, Kevin Klues, Robert Szewczyk Cory Sharp, Joseph Polastre,
Philip Buonadonna, Lama Nachman, Gilman Tolle, David Culler, and Adam Wolisz.
T2: A Second Generation OS For Embedded Sensor Networks. Technical Report TKN-
05-007, Telecommunication Networks Group, Technische Universitat Berlin, 2005.
[47] Philip Levis, Nelson Lee, Matt Welsh, and David Culler. TOSSIM: Accurate and Scal-
able Simulation of Entire TinyOS Applications. In SenSys ’03: Proceedings of the 1st
138
Bibliography Bibliography
international conference on Embedded networked sensor systems, pages 126–137, New
York, NY, USA, 2003. ACM.
[48] Philip Levis, Neil Patel, David Culler, and Scott Shenker. Trickle: A Self-regulating
Algorithm for Code Propagation and Maintenance in Wireless Sensor Networks. In
NSDI’04: Proceedings of the 1st conference on Symposium on Networked Systems De-
sign and Implementation, pages 2–2, Berkeley, CA, USA, 2004. USENIX Association.
[49] Chieh-Jan Mike Liang, Jie Liu, Liqian Luo, Andreas Terzis, and Feng Zhao. RACNet:
A High-Fidelity Data Center Sensing Network. In SenSys ’09: Proceedings of the 7th
ACM Conference on Embedded Networked Sensor Systems, pages 15–28, New York, NY,
USA, 2009. ACM.
[50] Chieh-Jan Mike Liang, Razvan Musaloiu-E., and Andreas Terzis. Typhoon: A Reliable
Data Dissemination Protocol for Wireless Sensor Networks. In EWSN ’08: Proceedings
of the 5th European conference on Wireless sensor networks, pages 268–285, Berlin,
Heidelberg, 2008. Springer-Verlag.
[51] Shu Lin and Daniel J. Costello. Error Control Coding: Fundamentals and Applications.
Prentice-Hall, 1983. TUB-HH 2413-469 3.
[52] Jie Liu, Bodhi Priyantha, Feng Zhao, Chieh-Jan Mike Liang, Qiang Wang, and Sean
James. Towards Fine-Grained Data Center Cooling Monitoring using Sensor Nets.
In HotEmNets’08: Proceedings of the 5th Workshop on Embedded Networked Sensors,
2008.
[53] Dimitrios Lymberopoulos, Thiago Teixeira, and Andreas Savvides. Detecting Patterns
for Assisted Living Using Sensor Networks: A Case Study. In Proceedings of the 2007
139
Bibliography Bibliography
International Conference on Sensor Technologies and Applications, SENSORCOMM
’07, pages 590–596, Washington, DC, USA, 2007. IEEE Computer Society.
[54] Samuel Madden, Michael J. Franklin, Joseph M. Hellerstein, and Wei Hong. TAG:
A Tiny AGgregation Service for Ad-hoc Sensor Networks. SIGOPS Oper. Syst. Rev.,
36(SI):131–146, 2002.
[55] Miklos Maroti, Branislav Kusy, Gyula Simon, and Akos Ledeczi. The Flooding Time
Synchronization Protocol. In SenSys ’04: Proceedings of the 2nd international confer-
ence on Embedded networked sensor systems, pages 39–49, New York, NY, USA, 2004.
ACM.
[56] Measurement Computing Corp. USB-2523: USB-Based 16 SE/8 DI Multifunc-
tion Measurement and Control Board. Available at http://www.mccdaq.com/
usb-data-acquisition/USB-2523.aspx, 2009.
[57] Alexandra Meliou, David Chu, Joseph Hellerstein, Carlos Guestrin, and Wei Hong.
Data Gathering Tours in Sensor Networks. In IPSN ’06: Proceedings of the 5th inter-
national conference on Information processing in sensor networks, pages 43–50, New
York, NY, USA, 2006. ACM.
[58] MetaGeek. Wi-Spy USB Spectrum Analyzers. Available at http://www.metageek.
net/products/wi-spy.
[59] MIT Technology Review. 10 Emerging Technologies That Will Change the World. Avail-
able at http://www.technologyreview.com/Infotech/13060/page2/, Febru-
ary 2003.
[60] Moteiv Corporation. Tmote Sky Datasheet. Available at http://sentilla.com/
files/pdf/eol/tmote-sky-datasheet.pdf.
140
Bibliography Bibliography
[61] Razvan Musaloiu-E., Chieh-Jan Mike Liang, and Andreas Terzis. Koala: Ultra-Low
Power Data Retrieval in Wireless Sensor Networks. In IPSN ’08: Proceedings of the
7th international conference on Information processing in sensor networks, pages 421–
432, Washington, DC, USA, 2008. IEEE Computer Society.
[62] Razvan Musaloiu-E. and Andreas Terzis. Minimising the Effect of WiFi Interference
in 802.15.4 Wireless Sensor Networks. Int. J. Sen. Netw., 3(1):43–54, 2007.
[63] Vinayak Naik, Anish Arora, Prasun Sinha, and Hongwei Zhang. Sprinkler: A Reliable
and Energy Efficient Data Dissemination Service for Wireless Embedded Devices. In
RTSS ’05: Proceedings of the 26th IEEE International Real-Time Systems Symposium,
pages 277–286, Washington, DC, USA, 2005. IEEE Computer Society.
[64] Paritosh Padhy, Kirk Martinez, Alistair Riddoch, H. L . Royan Ong, and Jane K. Hart.
Glacial environment monitoring using sensor networks. In RealWSN ’05: Workshop on
Real-World Wireless Sensor Networks, 2005.
[65] Jeongyeup Paek and Ramesh Govindan. RCRT: Rate-Controlled Reliable Transport
for Wireless Sensor Networks. In SenSys ’07: Proceedings of the 5th international con-
ference on Embedded networked sensor systems, pages 305–319, New York, NY, USA,
2007. ACM.
[66] Chulsung Park, Jinfeng Liu, and Pai H. Chou. Eco: an Ultra-Compact Low-Power
Wireless Sensor Node for Real-Time Motion Monitoring. In Proceedings of The Fourth
International Conference on Information Processing in Sensor Networks, IPSN ’05,
pages 398–403, 2005.
141
Bibliography Bibliography
[67] Seung-Jong Park, Ramanuja Vedantham, Raghupathy Sivakumar, and Ian F. Akyildiz.
GARUDA: Achieving Effective Reliability for Downstream Communication in Wireless
Sensor Networks. IEEE Transactions on Mobile Computing, 7(2):214–230, 2008.
[68] Chandrakant D. Patel, Cullen E. Bash, Ratnesh Sharma, Monem Beitelmal, and Rich
Friedrich. Smart Cooling of Data Centers. In Proceedings of International Electronic
Packaging Technical Conference and Exhibition, Maui, Hawaii, June 2003.
[69] Phil Karn. Reed-Solomon Coding/Decoding Package v1.0. Available at http://www.
piclist.com/tecHREF/method/error/rs-gp-pk-uoh-199609/index.htm,
1996.
[70] Joseph Polastre, Jonathan Hui, Philip Levis, Jerry Zhao, David Culler, Scott Shenker,
and Ion Stoica. A Unifying Link Abstraction for Wireless Sensor Networks. In SenSys
’05: Proceedings of the 3rd international conference on Embedded networked sensor
systems, pages 76–89, New York, NY, USA, 2005. ACM.
[71] Joseph Polastre, Robert Szewczyk, and David Culler. Telos: Enabling Ultra-Low Power
Wireless Research. In IPSN ’05: Proceedings of the 4th international symposium on
Information processing in sensor networks, page 48, Piscataway, NJ, USA, 2005. IEEE
Press.
[72] Sofie Pollin, Ian Tan, Bill Hodge, Carl Chun, and Ahmad Bahai. Harmful Coexis-
tence Between 802.15.4 and 802.11: A Measurement-based Study. In Cognitive Radio
Oriented Wireless Networks and Communications, 2008. CrownCom 2008. 3rd Interna-
tional Conference on, pages 1–6, May 2008.
142
Bibliography Bibliography
[73] Sumit Rangwala, Ramakrishna Gummadi, Ramesh Govindan, and Konstantinos Psou-
nis. Interference-Aware Fair Rate Control in Wireless Sensor Networks. SIGCOMM
Comput. Commun. Rev., 36(4):63–74, 2006.
[74] Theodore S. Rappaport. Wireless Communications: Principles & Practices. Prentice
Hall, 1996.
[75] RF Micro Devices, Inc. ML2724 2.4 GHz Low-IF 1.5 MBps FSK Transceiver. Available
at http://www.rfmd.com/CS/Documents/ML2724_ML2724SPACEDatasheet.
pdf, December 2005.
[76] L. Selavo, A. Wood, Q. Cao, T. Sookoor, H. Liu, A. Srinivasan, Y. Wu, W. Kang,
J. Stankovic, D. Young, and J. Porter. LUSTER: Wireless Sensor Network for Envi-
ronmental Research. In Proceedings of the 5th international conference on Embedded
networked sensor systems, SenSys ’07, pages 103–116, New York, NY, USA, 2007. ACM.
[77] Soo Young Shin, Hong Seong Park, and Wook Hyun Kwon. Mutual Interference Analy-
sis of IEEE 802.15.4 and IEEE 802.11b. Computer Networks, 51(12):3338–3353, 2007.
[78] Smart Works. Smart Works. Available at http://www.smart-works.com.
[79] Jianping Song, Song Han, Aloysius K. Mok, Deji Chen, Mike Lucas, Mark Nixon, and
Wally Pratt. WirelessHART: Applying Wireless Technology in Real-Time Industrial
Process Control. In RTAS ’08: Proceedings of the 14th Real-Time and Embedded Tech-
nology and Applications Symposium, pages 377–386, April 2008.
[80] Henoc Soude, Max Agueh, and Jean Mehat. Towards An Optimal Reed Solomon Codes
Selection for Sensor Networks: A Study Case Using TmoteSky. In PE-WASUN ’09:
Proceedings of the 6th ACM symposium on Performance evaluation of wireless ad hoc,
sensor, and ubiquitous networks, pages 165–166, New York, NY, USA, 2009. ACM.
143
Bibliography Bibliography
[81] Kannan Srinivasan, Prabal Dutta, Arsalan Tavakoli, and Philip Levis. Some Implica-
tions of Low Power Wireless to IP Networking. In HotNets-V: Proceedings of the Fifth
Workshop on Hot Topics in Networks, Nov 2006.
[82] Kannan Srinivasan, Maria A. Kazandjieva, Saatvik Agarwal, and Philip Levis. The
β-factor: Measuring Wireless Link Burstiness. In SenSys ’08: Proceedings of the 6th
ACM conference on Embedded network sensor systems, pages 29–42, New York, NY,
USA, 2008. ACM.
[83] Kannan Srinivasan and Philip Levis. RSSI is Under Appreciated. In EmNets ’06:
Proceedings of the 3rd Workshop on Embedded Networked Sensors, May 2006.
[84] Thanos Stathopoulos, Lewis Girod, John Heidemann, and Deborah Estrin. Mote Herd-
ing for Tiered Wireless Sensor Networks. Technical Report CENS-TR-58, University of
California, Los Angeles, Center for Embedded Networked Computing, December 2005.
[85] Thanos Stathopoulos, John Heidemann, and Deborah Estrin. A Remote Code Update
Mechanism for Wireless Sensor Networks. Technical Report CENS-TR-30, University
of California, Los Angeles, Center for Embedded Networked Computing, November
2003.
[86] SynapSense Corporation. LiveImaging: Wireless Instrumentation Solutions. Available
at http://www.synapsense.com/, 2008.
[87] Robert Szewczyk, Alan Mainwaring, Joseph Polastre, John Anderson, and David
Culler. An Analysis of a Large Scale Habitat Monitoring Application. In SenSys ’04:
Proceedings of the 2nd international conference on Embedded networked sensor sys-
tems, pages 214–226, New York, NY, USA, 2004. ACM.
144
Bibliography Bibliography
[88] Andreas Terzis, Razvan Musaloiu-E., Joshua Cogan, Katalin Szlavecz, Alexander Sza-
lay, Jim Gray, Stuart Ozer, Chieh-Jan Mike Liang, Jayant Gupchup, and Randal Burns.
Wireless Sensor Networks for Soil Science. Int. J. Sen. Netw., 7(1/2):53–70, 2010.
[89] Texas Instruments Inc. CC2420: Single-Chip 2.4 GHz IEEE 802.15.4 Compliant and
ZigBee Ready RF Transceiver. Available at http://focus.ti.com/docs/prod/
folders/print/cc2420.html.
[90] Texas Instruments Inc. MSP430x1xx Family User’s Guide (Rev. F). Available at http:
//www.ti.com/litv/pdf/slau049f, 2006.
[91] The Green Grid. The Green Grid Data Center Power Efficiency Metrics: PUE
and DCiE. Available at http://www.thegreengrid.org/gg_content/TGG_Data_
Center_Power_Efficiency_Metrics_PUE_and_DCiE.pdf, 2007.
[92] Gilles Thonet, Patrick Allard-Jacquin, and Pierre Colle. ZigBee - WiFi Coexistence:
White Paper and Test Report. Technical report, Schneider Electrics, 2008.
[93] Gilman Tolle and David Culler. Design of an Application-Cooperative Management
System for Wireless Sensor Networks. In EWSN ’05: Proceedings of the 2nd European
conference on Wireless sensor networks, 2005.
[94] Chieh-Yih Wan, Andrew T. Campbell, and Lakshman Krishnamurthy. PSFQ: A Reli-
able Transport Protocol for Wireless Sensor Networks. In WSNA ’02: Proceedings of the
1st ACM international workshop on Wireless sensor networks and applications, pages
1–11, New York, NY, USA, 2002. ACM.
[95] Xiaodong Wang, Xiaorui Wang, Xing Fu, Guoliang Xing, and Nitish Jha. Flow-Based
Real-Time Communication in Multi-Channel Wireless Sensor Networks. In EWSN ’09:
145
Bibliography Bibliography
Proceedings of the 6th European Conference on Wireless Sensor Networks, pages 33–52,
Berlin, Heidelberg, 2009. Springer-Verlag.
[96] Brett Warneke, Matt Last, Brian Liebowitz, and Kristofer S. J. Pister. Smart Dust:
Communicating with a Cubic-Millimeter Computer. Computer, 34(1):44–51, 2001.
[97] Geoff Werner-Allen, Konrad Lorincz, Jeff Johnson, Jonathan Lees, and Matt Welsh.
Fidelity and Yield in a Volcano Monitoring Sensor Network. In OSDI ’06: Proceedings
of the 7th symposium on Operating systems design and implementation, pages 381–396,
Berkeley, CA, USA, 2006. USENIX Association.
[98] Geoffrey Werner-Allen, Stephen Dawson-Haggerty, and Matt Welsh. Lance: Optimiz-
ing High-Resolution Signal Collection in Wireless Sensor Networks. In SenSys ’08:
Proceedings of the 6th ACM conference on Embedded network sensor systems, pages
169–182, New York, NY, USA, 2008. ACM.
[99] Chulho Won, Jong-Hoon Youn, Hesham Ali, Hamis Sharif, and Jitender Deogun. Adap-
tive Radio Channel Allocation for Supporting Coexistence of 802.15.4 and 802.11b. In
VTC ’05: Proceedings of the 62nd IEEE Semiannual Vehicular Technology Conference,
2005.
[100] Alec Woo and David E. Culler. A Transmission Control Scheme for Media Access in
Sensor Networks. In MobiCom ’01: Proceedings of the 7th annual international confer-
ence on Mobile computing and networking, pages 221–235, New York, NY, USA, 2001.
ACM.
[101] Alec Woo, Terence Tong, and David Culler. Taming the Underlying Challenges of Re-
liable Multihop Routing in Sensor Networks. In SenSys ’03: Proceedings of the 1st
146
Bibliography Bibliography
international conference on Embedded networked sensor systems, pages 14–27, New
York, NY, USA, 2003. ACM.
[102] Yafeng Wu, John Stankovic, Tian He, and Shan Lin. Realistic and Efficient Multi-
Channel Communications in Wireless Sensor Networks. In INFOCOM ’08: Proceedings
the 27th IEEE International Conference on Computer Communications, April 2008.
[103] Ning Xu, Sumit Rangwala, Krishna Kant Chintalapudi, Deepak Ganesan, Alan Broad,
Ramesh Govindan, and Deborah Estrin. A Wireless Sensor Network For Structural
Monitoring. In SenSys ’04: Proceedings of the 2nd international conference on Embed-
ded networked sensor systems, pages 13–24, New York, NY, USA, 2004. ACM.
[104] Wei Ye, John Heidemann, and Deborah Estrin. An Energy-Efficient MAC Protocol for
Wireless Sensor Networks. In INFOCOM ’02: Proceedings of the 21st IEEE Interna-
tional Conference on Computer Communications, volume 3, pages 1567–1576, 2002.
[105] Jingbin Zhang, Gang Zhou, Chengdu Huang, S.H. Son, and J.A. Stankovic. TM-
MAC: An Energy Efficient Multi-Channel MAC Protocol for Ad Hoc Networks. In ICC
’07: Proceedings of the international conference on Communications, pages 3554–3561,
June 2007.
[106] Jerry Zhao and Ramesh Govindan. Understanding Packet Delivery Performance in
Dense Wireless Sensor Networks. In SenSys ’03: Proceedings of the 1st international
conference on Embedded networked sensor systems, pages 1–13, New York, NY, USA,
2003. ACM.
[107] Gang Zhou, Chengdu Huang, Ting Yan, Tian He, John Stankovic, and Tarek Abdelza-
her. MMSN: Multi-Frequency Media Access Control for Wireless Sensor Networks. In
147
Bibliography Bibliography
INFOCOM ’06: Proceedings of the 25th IEEE International Conference on Computer
Communications, 2006.
[108] Gang Zhou, Chengdu Huang, Ting Yan, Tian He, John Stankovic, and Tarek Abdelza-
her. MMSN: Multi-Frequency Media Access Control for Wireless Sensor Networks. In
INFOCOM ’06: Proceedings of the 25th IEEE International Conference on Computer
Communications, pages 1–13, April 2006.
[109] Gang Zhou, Yafeng Wu, Ting Yan, Tian He, Chengdu Huang, John A. Stankovic, and
Tarek F. Abdelzaher. A Multifrequency MAC Specially Designed for Wireless Sensor
Network Applications. ACM Trans. Embed. Comput. Syst., 9(4):1–41, 2010.
148
Vita
Chieh-Jan Mike Liang received his Bachelor in Mathematics from University of Waterloo,
Canada, in 2005. In Fall 2005, he began his master study at the Department of Computer
Science at the Johns Hopkins University. And, he subsequently enrolled in the Ph.D. program
in 2007 and joined the Hopkins interNetworking Research Group (HiNRG). His research
focus is on building large-scale wireless sensor networks - specifically, research problems
related to robust sensor network architecture and radio interference mitigation.
Starting January 2011, Mike continues his research career at Microsoft Research Asia.
149