101

111

dfads

Using Feedback in MANETs: a Control Perspective

Todd P. Colemancolemant@illinois.eduUniversity of Illinois

DARPA ITMANET

Current Uses of Feedback

Theory•Feedback modeled noiseless•Point-to-point: capacity unchanged •Significantly improved error exponents•Reduction in complexity

•MANETs: Enlargement of capacity region

Current Uses of Feedback

PracticeFeedback is noisy, used primarily for•Robustness to channel uncertainty•Estimation of channel parameters•ARQ-style communication w/ erasures

Current Uses of Feedback

PracticeFeedback is noisy, used primarily for•Robustness to channel uncertainty•Estimation of channel parameters•ARQ-style communication w/ erasures

But: Burnashev-style “forward error correction+ARQ” schemes are extremely fragile w/ noisy feedback (Kim, Lapidoth, Weissman 07)

•Instantiate network feedback control algorithms for MANETs•Develop iterative practical schemes for noisy feedback?•Coding w/ feedback over statistically unknown channels?•Develop fundamental limits of error exponents with feedback w/ fixed block length

Applicability of Feedback in MANETs

dfads

101

111

0 0.25 0.50 0.75 1.00

00 01 10 11

0 1

....011010]1,0[ W

Communication w/ Noiseless Feedback

0 0.25 0.50 0.75 1.00

00 01 10 11

0 1

....011010]1,0[ W

Communication w/ Noiseless Feedback

Given an encoder’s Tx strategy, decoding is almost trivial (Baye’s rule)

0 0.25 0.50 0.75 1.00

00 01 10 11

0 1

....011010]1,0[ W

Communication w/ Noiseless Feedback

Given an encoder’s Tx strategy, decoding is almost trivial (Baye’s rule)How do we select a (recursive) encoder

strategy for an arbitrary memoryless channel?

A Control Interpretation of the Dynamics of the PosteriorColeman ’09: “A Stochastic Control Approach to ‘Posterior Matching’-style Feedback Communication Schemes”

A Control Interpretation of the Dynamics of the PosteriorColeman ’09: “A Stochastic Control Approach to ‘Posterior Matching’-style Feedback Communication Schemes”

Fk-1

Controller

Z-1

P(Fk|Fk-1, uk)uk Fk

reference signalFw

*

A Control Interpretation of the Dynamics of the PosteriorColeman ’09: “A Stochastic Control Viewpoint on ‘Posterior Matching’-style Feedback Communication Schemes”

Xk

Fk

Fk+1

Fw*

D(F w* ||F k+1)

D(F w* ||F k)

Reward at any stage k is the reduction in

“distance” to target

Stochastic Control: RewardColeman ’09

Maximum Long-Term Average RewardColeman ’09

Maximum Long-Term Average Reward

(1),(2) hold w/ equality if:• a) Y’s all independent• b) Each Xi drawn

according to P*(x)

Coleman ’09

Maximum Long-Term Average Reward

(1),(2) hold w/ equality if:• a) Y’s all independent• b) Each Xi drawn

according to P*(x)

• Horstein ’63 (BSC)• Schalwijk-Kailath ’66 (AWGN)• Shayevitz-Feder ‘07, ‘08 (DMC)

Coleman ’09

The Posterior Matching Scheme: an Optimal Solution

• Next input indep of everything decoder has seen so far, with capacity-achieving marginal distribution

• No forward error correction. Adapt on the fly.

Coleman ’09

Posterior matching scheme

The Posterior Matching Scheme: an Optimal SolutionColeman ’09

• Next input indep of everything decoder has seen so far, with capacity-achieving marginal distribution

• No forward error correction. Adapt on the fly.

Posterior matching scheme

0 1

0 1

0 1

Implications for Demonstrating Achievable Rates

0 1

1

Coleman ’09

Lyapunov Function

0 1

0 1

Posterior matching scheme:

Coleman ’09

Lyapunov Function (cont’d)

0 1

0 1

0 1

1

Coleman ’09

ControlTheory

InformationTheory

Symbiotic Relationship

Converse Thms Give Upper Bounds on Average Long-Term Rewards for StochasticControl Problem

Coleman ’09: “A Stochastic Control Viewpoint on ‘Posterior Matching’-style Feedback Communication Schemes”

ControlTheory

InformationTheory

Symbiotic Relationship

Converse Thms Give Upper Bounds on Average Long-Term Rewards for StochasticControl Problem

KL Divergence Lyapunov functions guarantee all rates achievable

Coleman ’09: “A Stochastic Control Viewpoint on ‘Posterior Matching’-style Feedback Communication Schemes”

Research Results with This Methodology•Interpret feedback communication encoder design as stochastic control of posterior towards certainty•Converse theorems specify fundamental performance bounds on a stochastic control problem related to controlling posterior.• An optimal policy implies the existence of a Lyapunov function, which is in essence a KL divergence •Lyapunov function directly implies achievability for all R < C Coleman ’09

Research Results with This Methodology

Gorantla and Coleman ‘09: Encoders that achieve El Gamal 78: “Physically degraded broadcast channels w/ feedback“ capacity region in an iterative fashion w/ low complexity

•Interpret feedback communication encoder design as stochastic control of posterior towards certainty•Converse theorems specify fundamental performance bounds on a stochastic control problem related to controlling posterior.• An optimal policy implies the existence of a Lyapunov function, which is in essence a KL divergence •Lyapunov function directly implies achievability for all R < C Coleman ’09

New Important Directions this Approach Enables

ControlTheory

Information

Theory

•Develop iterative low-complexity encoders/decoders for noisy feedback? Partially Observed Markov Decision Process

101

111

New Important Directions this Approach Enables

ControlTheory

Information

Theory

•Develop iterative low-complexity encoders/decoders for noisy feedback? Partially Observed Markov Decision Process•Optimal coding w/ feedback over statistically unknown channels? Reinforcement learning from control literature

101

111

New Important Directions this Approach Enables

ControlTheory

Information

Theory

•Develop iterative low-complexity encoders/decoders for noisy feedback? Partially Observed Markov Decision Process•Optimal coding w/ feedback over statistically unknown channels? Reinforcement learning from control literature•Develop fundamental limits of error exponents with feedback w/ fixed block length Lyapunov function enables a fundamental Martingale condition

101

111

New Important Directions this Approach Enables

ControlTheory

Information

Theory

•Develop iterative low-complexity encoders/decoders for noisy feedback? Partially Observed Markov Decision Process•Optimal coding w/ feedback over statistically unknown channels? Reinforcement learning from control literature•Develop fundamental limits of error exponents with feedback w/ fixed block length Lyapunov function enables a fundamental Martingale condition•Also: stochastic control approach provides a rubric to check tightness of converses via structure of optimal solution

101

111