Thrust #3 Active Information Exploitation for Resource ...
Transcript of Thrust #3 Active Information Exploitation for Resource ...
![Page 1: Thrust #3 Active Information Exploitation for Resource ...](https://reader031.fdocuments.net/reader031/viewer/2022012915/61c57306bf267d0504547f68/html5/thumbnails/1.jpg)
Kickoff Meeting 13 October 2011
Value
of
Information
Second-year Review 9 September 2013
ARO MURI on Value-centered Information Theory for
Adaptive Learning, Inference, Tracking, and Exploitation VOI
Thrust #3
Active Information Exploitation
for Resource Management
Information-geometric Ideas in Sensor
Management and Information Flow
Quantification
Doug Cochran
![Page 2: Thrust #3 Active Information Exploitation for Resource ...](https://reader031.fdocuments.net/reader031/viewer/2022012915/61c57306bf267d0504547f68/html5/thumbnails/2.jpg)
Kickoff Meeting 13 October 2011
Value
of
Information VOI
Second-year Review 9 September 2013
Synopsis of Research Activities
• Information-geometric trajectory planning
• Update on work from last year
• Introduction to a related collaborative topic started
this year
• Quantification of information flow
• Value of information sharing
![Page 3: Thrust #3 Active Information Exploitation for Resource ...](https://reader031.fdocuments.net/reader031/viewer/2022012915/61c57306bf267d0504547f68/html5/thumbnails/3.jpg)
Kickoff Meeting 13 October 2011
Value
of
Information VOI
Second-year Review 9 September 2013
Information-geometric Trajectory Planning*
Sensor Model
• In estimation of a parameter q in a smooth parameter
manifold M, the effect of a particular sensing action is to
produce a log-likelihood l(q)=log p(x|q)
• The Fisher information
induces a Riemannian metric on M
lldldF pp
2
)|()|( qqqqqq EE
The effect of choosing a sensing action / sensor
configuration is to select a Riemannian metric for the
parameter manifold M
*Joint work with S. Howard and W. Moran
![Page 4: Thrust #3 Active Information Exploitation for Resource ...](https://reader031.fdocuments.net/reader031/viewer/2022012915/61c57306bf267d0504547f68/html5/thumbnails/4.jpg)
Kickoff Meeting 13 October 2011
Value
of
Information VOI
Second-year Review 9 September 2013
Information-geometric Trajectory Planning
Sensor Manifold
• The collection of all Riemannian metrics on a smooth manifold M is itself an infinite-dimensional (weak) Riemannian manifold M(M)
• The geodesic structure of M has been studied outside the context of
information geometry (Gil-Medrano & Michor, 1991)
• If the sensor configuration is parameterized by a smooth “sensor
manifold” S, a particular configuration s in S gives rise to a particular
Riemannian metric gs on M
• The mapping F taking s to gs
will be assumed to be smooth and
one-to-one (immersion is enough)
• Although M is infinite-dimensional,
the trajectory planning takes place in
a finite-dimensional sub-manifold,
and…
• This sub-manifold inherits a metric structure from M
![Page 5: Thrust #3 Active Information Exploitation for Resource ...](https://reader031.fdocuments.net/reader031/viewer/2022012915/61c57306bf267d0504547f68/html5/thumbnails/5.jpg)
Kickoff Meeting 13 October 2011
Value
of
Information VOI
Second-year Review 9 September 2013
Information-geometric Trajectory Planning
Geodesics
• Geodesics g in M minimize
• A variational approach was used by Gil-Medrano and Michor to obtain a
differential equation for these curves
• Classical differential geometric quantities (e.g., Christoffel symbols) appear in
these formulae
dtdFggggE ttM
g )())()((Tr2
1 1
1
0
1 q
This year: Showed how to constrain the geodesic derivation to the sub-
manifold F(S) to obtain geodesic sensor configuration trajectories in S
Last year: Showed divergences on M derived from KL (Fisher) and from
symmetrized mutual information (Shannon) both give rise to the same Riemannian metric on M; i.e., the Gil Medrano-Michor metric
![Page 6: Thrust #3 Active Information Exploitation for Resource ...](https://reader031.fdocuments.net/reader031/viewer/2022012915/61c57306bf267d0504547f68/html5/thumbnails/6.jpg)
Kickoff Meeting 13 October 2011
Value
of
Information VOI
Second-year Review 9 September 2013
Information-geometric Trajectory Planning Bearings-only emitter localization example
• One fixed and one mobile sensor platform
(e.g., UAV) with bearings-only sensors seek to
localize a stationary emitter
• Noise is independent von Mises on each
sensor
• Target prior is Gaussian; covariance is
updated with each measurement along the
moving sensor trajectory
• Two mobile sensor platforms with different
initial positions and constrained speeds
• Gaussian target prior
• Geometrically determined platform motion
reduces error covariance determinant
measurements are collected
![Page 7: Thrust #3 Active Information Exploitation for Resource ...](https://reader031.fdocuments.net/reader031/viewer/2022012915/61c57306bf267d0504547f68/html5/thumbnails/7.jpg)
Kickoff Meeting 13 October 2011
Value
of
Information VOI
Second-year Review 9 September 2013
Navigation in the Multinomial Family*
• Consider the family F of multinomial distributions on K outcomes and
N=1 trial • p ϵ F is characterized by a parameter vector q=(q1,…, qK) with
each qk ≥ 0 and S qk = 1; i.e., q ϵ D, the standard simplicial face
in RN
• Equivalently, F is parameterized by
where S is the part of the unit sphere in the non-negative orthant
• These parameters are still suitable for N>1 trial, so information
analysis does not change
• Consider a curve q(t), t ϵ[0,1] in D; the Fisher information for
estimating q(t) is
Sqqq K ),...,(~1
)(
1)()()(
)(
1)(
2
1
2
1 tqtq
dt
dtqtq
dt
d
tqtI
k
K
k
kk
K
k
k
k
*Joint work with A. Hero and B. Sadler
![Page 8: Thrust #3 Active Information Exploitation for Resource ...](https://reader031.fdocuments.net/reader031/viewer/2022012915/61c57306bf267d0504547f68/html5/thumbnails/8.jpg)
Kickoff Meeting 13 October 2011
Value
of
Information VOI
Second-year Review 9 September 2013
Navigation in the Multinomial Family Equivalence of Hellinger and Information Distances
• On the other hand, the squared differential path length element in S
is
• So the information distance between the density at q(0) and the one
at q(t) is the shortest distance between points on the unit sphere;
i.e., a great circle distance
• The Hellinger distance (an f-divergence) is given by
)(4
1)(
)(
1
4
1)()(~
2
1
2
1
2
tItqdt
d
tqtq
dt
dtq
dt
dk
K
k k
K
k
k
)(~),0(~arccos),( )()0( tqqppd tqqI
)(~),0(~22),( )()0( tqqppd tqqH
In this family, Hellinger distance (global) is monotonically related to
information distance, which is differentially navigable by Fisher information
![Page 9: Thrust #3 Active Information Exploitation for Resource ...](https://reader031.fdocuments.net/reader031/viewer/2022012915/61c57306bf267d0504547f68/html5/thumbnails/9.jpg)
Kickoff Meeting 13 October 2011
Value
of
Information VOI
Second-year Review 9 September 2013
Quantification of Information Flow*
De Bruijn’s Identity on Rn
• Given a probability density p on Rn, its intrinsic Fisher
information is
and its entropy is
• If p evolves according to
then
dxppdpddxppFnn RR
)loglog()log( 2
02
1 2
p
t
p
nR
dxppH log
)(Tr2
1F
t
H
*Joint work with S. Howard and W. Moran
![Page 10: Thrust #3 Active Information Exploitation for Resource ...](https://reader031.fdocuments.net/reader031/viewer/2022012915/61c57306bf267d0504547f68/html5/thumbnails/10.jpg)
Kickoff Meeting 13 October 2011
Value
of
Information VOI
Second-year Review 9 September 2013
Quantification of Information Flow
Intrinsic Fisher Information on Rn
• Suppose we wish to estimate the location parameter
q ϵ Rn from a datum having pdf p(x-q)
• The Fisher information at q =0 is
i.e., the intrinsic Fisher information of p describes how
well p can be distinguished from a small translation of
itself
dxxpxpdxxpxpFnn RR
)()(log)()(log 22
qq qq
I.V. Arnold (1966) pointed out the Euler equations for an inviscid
incompressible fluid on a Riemannian manifold M arise from considering
geodesic flows on SDiff(M), the Lie group of volume preserving
diffeomorphisms on M. Does this view carry over to information dynamics?
![Page 11: Thrust #3 Active Information Exploitation for Resource ...](https://reader031.fdocuments.net/reader031/viewer/2022012915/61c57306bf267d0504547f68/html5/thumbnails/11.jpg)
Kickoff Meeting 13 October 2011
Value
of
Information VOI
Second-year Review 9 September 2013
Quantification of Information Flow K-L Divergence on a Riemannian Manifold
• Data space: Orientable Riemannian n-manifold X with
metric g and invariant volume form m
• Density: p:XR+ with ∫ pm = 1 • Consider the parametric family of densities
• How well p°f1 can be distinguished from p°f2 given data
xϵX is described by the K-L divergence
D : Sdiff(X) × Sdiff(X) R
with
)}(SDiff|)|({ Xpp fff
mff
fff 1
2
121 log)(
p
p
pppD
X
![Page 12: Thrust #3 Active Information Exploitation for Resource ...](https://reader031.fdocuments.net/reader031/viewer/2022012915/61c57306bf267d0504547f68/html5/thumbnails/12.jpg)
Kickoff Meeting 13 October 2011
Value
of
Information VOI
Second-year Review 9 September 2013
Quantification of Information Flow Intrinsic Fisher Information on a Riemannian Manifold
• Differentiating D, one obtains an expression for Fisher
information at f ϵ Sdiff(X) and correspondingly an
intrinsic Fisher information for p
and a scalar intrinsic Fisher information
• Suppose p(t,x) satisfies t p – Dp = 0 and define the entropy
density by ht = pt log pt m. Then there is a de Bruijn identity on X for entropy Ht
mppFX
log2
id
t
X
tttt fhH
dppdppfXX
* log)log(Tr 2 m
![Page 13: Thrust #3 Active Information Exploitation for Resource ...](https://reader031.fdocuments.net/reader031/viewer/2022012915/61c57306bf267d0504547f68/html5/thumbnails/13.jpg)
Kickoff Meeting 13 October 2011
Value
of
Information VOI
Second-year Review 9 September 2013
Value of Information Sharing in Networks Synopsis / Update
• Last year, results were reported quantifying the value of individual
edges in a network graph for achieving global registration in the
presence of noise • The measures take the form of a “local Fisher information”
• This year, the value of information sharing was investigated in the
context of detecting an uncharacterized emitter using a network of
receivers
![Page 14: Thrust #3 Active Information Exploitation for Resource ...](https://reader031.fdocuments.net/reader031/viewer/2022012915/61c57306bf267d0504547f68/html5/thumbnails/14.jpg)
Kickoff Meeting 13 October 2011
Value
of
Information VOI
Second-year Review 9 September 2013
Research Directions
• Develop further information-geometric characterizations
of VoI and apply them in sensor management and
adaptive information collection
• Consider how understanding of intrinsic Fisher
information and information diffusion can be used in
modeling the evolution of uncertainty in unobserved
systems
• Establish a theoretical framework that supports
experimental results that indicate the value of network
connectivity (e.g., actual data sharing vs. maximum
entropy proxies for shared data)