Partial Ordering and Stochastic Resonance for Discrete ... Iasi 2012.pdfPartial Ordering and...
Transcript of Partial Ordering and Stochastic Resonance for Discrete ... Iasi 2012.pdfPartial Ordering and...
Partial Ordering and Stochastic
Resonance for Discrete Memoryless Channels
Dr. Paul Cotae
University of the District of Columbia, Washington DC [email protected]
1 Technical University of Iasi 05/24/2012
Outline Introduction, Motivation and State of the Art
(2,3) DMC Mutual Information and Capacity
(2,3) DMC and particular cases
Partial Ordering of BSC/SE Stochastic Resonance in BSC/SE
Stochastic Resonance and Capacity Bounds
Conclusions
2 Technical University of Iasi 05/24/2012
Introduction Information theory associated with communication channels Relations between the general (2,3)DMC, the BAC/SE, the BSC/SE,
the BSC, and the BEC
Partial ordering for the BSC/SE under specific noise element constraints
Stochastic Resonance effect on the BSC/SE
Optimal noise power level required to obtain a maximum capacity channel for a given threshold decision
3
Technical University of Iasi 05/24/2012
Technical University of Iasi 05/24/2012 4
Underwater Sensor Network Architecture
Motivation Which DMC, for a given set of resources and noise constraints, is “better”
for communication?
How to use noise to increase the capacity of a communication channel?
This comparison and increase in channel capacity are necessary for wireless sensor communication links at the physical layer operating in very noisy environment and very limited computational power.
Stochastic Resonance observed in many non-linear systems.
Consider Helgert and Pinsker limits capacity bounds
5 Technical University of Iasi 05/24/2012
Communication Channel
Technical University of Iasi 05/24/2012 6
Continuous Channel Discrete Channel
Entropy, Mutual Information and Capacity
7 Technical University of Iasi 05/24/2012
H(X/Y) H(Y/X) I(X,Y)
Partial Ordering
Technical University of Iasi 05/24/2012 8
N1
N2
N3
N4
S
C(N1,S)
C(N2,S)
C(N3,S)
C(N4,S)
Stochastic Resonance(SR) SR phenomenon is a non-linear effect wherein a communication system can transmit the information with improved efficiency in the presence of additive noise.
9 Technical University of Iasi 05/24/2012
N1
N2
N3
N4
S
C(N1,S)
C(N2,S)
C(N3,S)
C(N4,S)
State of the Art
Technical University of Iasi 05/24/2012 10
.
( ) ( )
2
( ) ( ) ( ) ( )
2
( ) ( ) log 1 2
( , )
log 2 2
h a h ba b
ah b bh a bh a ah ba b a b
ah b bh a a ba b
C a b
a b
−−
− −− −
− + + ≠ −
=
+ ≠
1( , )
1a a a a
M a bb b b b
− = = −
2 2( ) log (1 ) log (1 )h x x x x x= − − − −
Technical University of Iasi 05/24/2012 11
Capacity of the (2,2) DMC
Partial Ordering of (2,2) DMCs
Technical University of Iasi 05/24/2012 12
Technical University of Iasi 05/24/2012 13
SR in (2,2) DMC
(2,3) DMC
14 Technical University of Iasi 05/24/2012
(2,3) DMC Mutual Information and Capacity
15 Technical University of Iasi 05/24/2012
H(X/Y) H(Y/X) I(X,Y)
Binary Asymmetric Channel with Symmetric Erasure(BAC/SE) Capacity
and
16 Technical University of Iasi 05/24/2012
using the Kuhn-Tucker condition
Binary Symmetric Channel with Symmetric Erasure (BSC/SE)Capacity
Technical University of Iasi 05/24/2012 17
BSC and BEC
Technical University of Iasi 05/24/2012 18
Technical University of Iasi 05/24/2012 19
Partial Ordering of BSC/SE
Technical University of Iasi 05/24/2012 20
Partial Ordering of BSC/SE (cont.)
Technical University of Iasi 05/24/2012 21
Partial Ordering of BSC/SE: Result-I
Technical University of Iasi 05/24/2012 22
23
Majorization
1 2( , ,..., )na a a=a 1 2( , ,..., )nb b b=b
a b
1 1
k ki i
i ia b↓ ↓
= =≤∑ ∑ 1,2,..., 1k n= −
1 1
n ni i
i ia b↓ ↓
= ==∑ ∑
Technical University of Iasi 05/24/2012
Partial Ordering of BSC/SE (cont.)
Technical University of Iasi 05/24/2012 24
a+b constant, and a1<a2, then (a1,b1) is majorized by (a2,b2)
Technical University of Iasi 05/24/2012 25
Schur Convexity: Result-II-a
( ) ( )ϕ ϕ⇒ ≤a b a b
Theorem 4.2.3: If a+b is constant, then the BSC/SE is Schur convex
Schur Convexity: Result-II-b
Technical University of Iasi 05/24/2012 26
Theorem 4.2.4: for a - b constant, if a1>a2, then C(a1,b1)<C(a2,b2)
Technical University of Iasi 05/24/2012 27
Partial Ordering of BSC/SE: Summary
SR in BSC/SE
28 Technical University of Iasi 05/24/2012
Review of Gaussian, Laplace, Cauchy noise
Technical University of Iasi 05/24/2012 29
SR in BSC/SE
30 Technical University of Iasi 05/24/2012
Technical University of Iasi 05/24/2012 31
SR in BSC/SE: Results
SR in BSC/SE-Gaussian Noise
32 Technical University of Iasi 05/24/2012
SR in BSC/SE- Gaussian Noise
33 Technical University of Iasi 05/24/2012
Forbidden Interval
Technical University of Iasi 05/24/2012 34
SR and Channel Capacity Bounds Pinsker Lower (L) and Helgert Upper (U)bounds
For n=3
For the (2,n) DMC
35 Technical University of Iasi 05/24/2012
SR and Pinsker Bound
36 Technical University of Iasi 05/24/2012
SR and Helgert Bound
37 Technical University of Iasi 05/24/2012
SR and Channel Capacity Bounds
38 Technical University of Iasi 05/24/2012
Optimal Values for Threshold and Noise Level
critical points of the capacity bounds
39 Technical University of Iasi 05/24/2012
Optimal Values for Threshold and Noise Level
40 Technical University of Iasi 05/24/2012
Conclusion We analyzed the general (2,3) DMC and established the relations between the BAC/SE, BSC/SE,
BSC and the BEC.
We provided a partial ordering for the BSC/SE under specific conditional probability between channel input and channel outputs constraints using directional derivative and majorization theory.
We studied the threshold based stochastic resonance behavior of binary-input ternary-output DMC in presence of Gaussian, Laplace and Cauchy noise.
We demonstrated that in presence of the noise with even probability distribution function the general (2,3) DMC become the BSC/SE.
We used the lower and upper capacity bounds to illustrated the stochastic resonance. We derived analytically the optimum noise power level depending on the threshold level that will
maximize their capacity.
41 Technical University of Iasi 05/24/2012
References C.E Shannon and W.W Weaver, “The Mathematical Theory of Communication,” University of Illinois press, Urbana, IL, 1949. Thomas M. Cover and Joy A. Thomas,” Elements of Information Theory,” Second Edition, John Wiley and Sons, 2006. Paul Cotae “Blind Doppler Estimation and Compensation in the Underwater Communications” Technical report submitted to the Naval
Research Laboratories, Washington DC, Sept 2009. François Chapeau-Blondeau,” Noise Enhanced Capacity via Stochastic Resonance in an Asymmetric Binary Channel,” Physical Review
E, 55(2):2016-2019, 1997. François Chapeau-Blondeau, ”Periodic and Aperiodic Stochastic Resonance with Output Signal-to-noise Ratio Exceeding that at the
Input,” International Journal of Bifurcation and Chaos, vol.9, No1:267-272, 1999. K. Chatzikokolaskis and K. Martin ,”A monotonicity principle for information theory”,Electronic Notes in Theoretical computer science
218(2008)111-129. Paul Cotae and Ira S. Moskowitz,” On the Partial Ordering of the Discrete Memoryless Channels Arising in Cooperative Sensor
Networks,” 4th IFIP International Conference on New Technologies, Mobility and Security, NTMS 2011, pp.1-6, Paris, France, February 2011.
H.J. Helgert,”A Partial Ordering of Discrete Memoryless Channels”, IEEE Transactions on Information Theory , vol. IT-3,no.3, pp.360-365,July 1967.
Ira S. Moskowitz, Paul Cotae, Pedro N. Safier, and Daniel L. Kang,” Capacity Bounds and Stochastic Resonance for Binary Input Binary Output Channels,” Proc. of the IEEE Computing, Communications & Applications conference, ComComAP 2012, Hong Kong,pp.61-66,Jan. 2012.
François Chapeau-Blondeau, ”Stochastic Resonance and the Benefit of Noise in Nonlinear Systems,” Noise, Oscillators and Algebraic Randomness – From Noise in Communication Systems to Number Theory. pp. 137-155; M. Planat, ed., Lecture Notes in Physics, Vol. 550, Springer (Berlin) 2000.
Paul Cotae, Ira S. Moskowitz and Myong H. Kang, ”Eigenvalue Characterization of the Capacity of Discrete Memoryless Channels with Invertible Channel Matrices, “Proceedings IEEE 44th Annual Conference on Information Sciences and Systems-CISS 2010, Princeton University,pp.1-6, March 2010.
Ira S. Moskowitz,” Approximations for the Capacity of Binary Input Discrete Memoryless Channels,” In Proceedings 44th Annual Conf. on Information Science and Systems, CISS 2010,Princeton ,NJ,USA, March 2010.
42 Technical University of Iasi 05/24/2012
References(cont.) Ira S. Moskowitz,” An Approximation of the Capacity of a Simple Channel,” 43rd Annual Conf. on Information Science and Systems, CISS
2009,Baltimore ,MD,USA, June 2009. Paul Cotae and T.C. Yang “A cyclostationary blind Doppler estimation method for underwater acoustic communications using direct-
sequence spread spectrum signals,” 8th International conf. on Communications, COMM 2010, Bucharest, July 2010. Ira S. Moskowitz and Paul Cotae,” Channel Capacity Behavior for Simple Models of Optical Fiber Communication,” 8th International conf.
on Communications, COMM 2010, Bucharest, pp.1-6, July 2010. Roland Kamdem, Paul Cotae and Ira S. Moskowitz, ”Threshold Based Stochastic Resonance for the Binary-Input Ternary-Output Discrete
Memoryless Channel,” accepted at 7th IASTED International Conf. on Communication, Internet and Information Technology, CIIT 2012,Baltimore, May 2012.
François Chapeau-Blondeau,” Stochastic Resonance and the Benefit of Noise in Nonlinear Systems,” Noise, Oscillators and Algebraic Randomness – From Noise in Communication Systems to Number Theory. pp. 137-155; M. Planat, ed., Lecture Notes in Physics, Vol. 550, Springer (Berlin) 2000.
Sanya Mitaim, Bart Kosko,”Adaptive Stochastic Resonance in Noisy Neurons Based on Mutual Information ,” IEEE Trans on Neural Networks Vol.15,No.6, Nov 2004.
Mark Damian McDonnell, “Theoretical Aspects of Stochastic Signal Quantization and Suprathreshold Stochastic Resonance,” PhD thesis, The University of Adelaide, Australia, Feb.2006.
Bart Kosko, Sanya Mitiam, Ashok Patel and Mark M. Wilde, ”Applications of Forbidden Interval Theorem in Stochastic Resonance, ” In Applications of Nonlinear Dynamics ,Understanding Complex Systems, pages 71-89.Springer-Verlag, 2009.
Ira S. Moskowitz, Paul Cotae, Pedro N. Safier,”Algebraic Information Theory and Stochastic Resonance for Binary-Input Binary-Output Channels ,” Proceedings IEEE 46th Annual Conference on Information Sciences and Systems-CISS 2012, Princeton University, March 17-19, pp.1-6, March 2012.
Paul Cotae, et. all ”Design and Application of a Distributed and Scalable Underwater Wireless Sensor Acoustic Networks” 2009 Annual ASEE Global Colloquium on Engineering Education, Budapest, Hungary, Oct. 12-15, 2009.
R. A. Silverman,”On Binary Channels and their Cascades,” IRE Trans. Information Theory, Vol.1,no.,pp.19-27,Dec 1995. C.E. Shannon, ”A note on a partial ordering for communication channels, ”Information and control,Vol.1, pp.390-397,1958. H.J. Helgert, “A Partial Ordering of Discrete Memoryless Channel,” IEEE Transactions on Information Theory, vol.1, no.3, pp.360-365,
July 1967. A.W. Marshall and I. Olkin, Inequalities: Theory of Majorization and its applications .New York: Academic, 1979. Patrick M. Fitzpatrick “Advanced Calculus,” American Mathematical Society, 2009.
Technical University of Iasi 05/24/2012 43