1

Wi-Fi Conferencing System: Locating Array Efficient experiment parameter screeningIngrid Moerman, Eli De Poorter, Michael Mehari (iMinds)Violet Syrotiuk, Randy Compton, Charles Colbourn (Arizona state university)

2

Overview

• Theoretical background• Applying these concepts in FIRE

3

Screen Experiments

• A wireless system is characterized by a large amount of input parameters (factors) each with different configurable levels or settings.

• The purpose of a screening experiment is to identify the significant factors and interactions between them that influence the overall performance

4

Screen Experiments

• An experiment is given by an N × k array• The k columns correspond to the factors

• Each factor Fi, 1 ≤ i ≤ k has a set of levels Li

• Each of the N rows corresponds to a test in which each factor Fi is set to a level in Li

• Example• 2 factors F1 & F2

• Tx power, 20 levels (0 -> 20 dBm)

• Transmission rate, 5 levels (6, 9, 12, 24, 36 Mbps)

• 5 experiments

(2 columns, k=2)

(5 rows, N=5)

5

Coverage

• To measure a t-way interaction it must be covered• For example, if t=2, then all pairs of (factor, level)

combinations must occur in the experiment

• Covering arrays (CA) are designed to provide coverage

• A CA(N;t,k,v) is an N × k array where each N × t subarray contains all ordered t-sets

6

Covering Arrays

• An example covering array in which each 6 × 2 subarray contains all ordered (binary) pairs

0 1 1 1 1

1 0 1 0 0

0 1 0 0 0

1 0 0 1 1

0 0 0 0 1

1 1 0 1 0

CA(6;2,5,2)

7

Location is more than Coverage• However, the ability to locate factors and

interactions is more than just coverage• A locating array must uniquely identify the rows in

which significant t-way interactions are covered• The number of weak or faulty interactions must

be limited (not known a priori)

8

Locating Arrays – How Large?

• How does the array size N grow as a function of k for fixed v, t, and d?

• Number of factors: k

• Number of levels: v

• “Size” of interaction: t

• Maximum number of faulty interactions d

• There are many many sets of selection tuples to distinguish

When t and d are fixed, and the number of factors k increases, the number of tests grows as a logarithmic function of k

9

Overview

• Theoretical background• Applying these concepts in FIRE

10

From theory to experiment

• The locating arrays have been used successfully in simulations [1] to identify parameter interactions

•  Reducing the design points from 1043 to only 421

• But is it also feasible on realistic systems?

[1] Abraham N. Aldaco, Charles J. Colbourn, Violet R. Syrotiuk: Locating Arrays: A New Experimental Design for Screening Complex Engineered Systems. ACM SIGOPS Operating Systems Review - Special Issue on Repeatability and Sharing of Experimental Artifacts, 49(1): 31-40 (2015)

11

1 transmitter42 receivers

Experiment set-up

13

# Factor Description Range

1 band Frequency band (2.4, 5) GHz

2 channel Frequency channel 1, 6, 11, 36, 40, 44

3 rate Transmission bitrate (6, 9, 12, 24, 36) Mbps

4 txpower Transmission power 20 dBm

5 mtu Maximum Transmission unit (256, 512, 1024, 1280, 1500) bytes

6 txqueuelen Transmission Queue length (10, 50, 100, 500, 1000) bytes

7 qdisc Queuing discipline pfifo, bfifo, pfifo_fast

8 ipfrag_low_thresh IP fragmentation lower threshold (0.25 0.5 0.75 1)

9 ipfrag_high_thresh IP fragmentation higher threshold (16384, 65536, 262144, 1048576, 4194304) bytes

10 udp_rmem_min Minimum UDP receive memory (0.019231 0.1 0.5)

11 rmem_default Default receive memory (0 0.25 0.5 0.75 1)

12 rmem_max Maximum receive memory (2304, 10418, 47105, 212992) bytes

13 udp_wmem_min Minimum UDP write memory (0.019231 0.1 0.5)

PHY IP UDP APP

Factors and Ranges

14

# Factor Description Range

14 wmem_default Default write memory (0 0.25 0.5 0.75 1)

15 wmem_max Maximum write memory (4608, 16537, 59349, 212992) bytes

16 udp_mem_min Minimum UDP memory (0.25 0.5 0.75)

17 udp_mem_pressure Pressure UDP memory (0 0.33338 0.5 0.75 1)

18 udp_mem_max Maximum UDP memory (95, 949, 9490, 94896) bytes

19 ROHC RObust Header Compression OFF

20 codec Audio codec type Opus, speex

21 codecBitrate Audio codec bitrate (8 16 24 32) kbps

22 FrameLengthAggregation Audio frame length aggregation (20 40 60) msec

23 interferenceCOR Interference Channel Occupancy Ratio 10%, 50%, 100%

24 sensing Background sensing activitity ON, OFF

PHY IP UDP APP

Factors and Ranges

15

Total number of experiments (with 24 factors)l = 6*5*5*5*5*3*4*5*3*4*4*3*5*4*3*5*4*2*4*3*3*2l = 5.59872×10¹² experimentsl = 20 M years (if 2 min. / experiment)l

Locating Array experimentsl = 109 experimentsl = less then 4 hours

Design space Range

16

Transmission Exposure Measures the electromagnetic exposure as felt by a receiver in SAR

(Specific Absorption Rate) units

Mean Opinion Score (MOS) Measures the audio quality of the speaker after the encoding and

transmission path

Experiment Outputs

17

Experimentation Challenges

• Interference generation using USRP does not work consistently

• Clients loose connection to Access point (speaker) when TxPower is reduced

• There used to be a bug inside the Speex decoder• Continuously bringing the Wi-Fi interface ON and OFF

sometime makes a node non-responding• Applying the LA factors on the system sometimes fail• Since the experiment takes very long time, watching

for a failed experiment is time taking• Singe output criteria required reports from multiple

sources (some devices might fail, other might not)• Variable outcomes (time dependent behavior)

THANK YOU FOR YOUR TIME

?Questions