Characterization and Calibration of Acoustic Radar System ...
Position Calibration of Acoustic Sensors and Actuators
description
Transcript of Position Calibration of Acoustic Sensors and Actuators
![Page 1: Position Calibration of Acoustic Sensors and Actuators](https://reader036.fdocuments.net/reader036/viewer/2022062316/56813a8b550346895da287bd/html5/thumbnails/1.jpg)
Position Calibration of Acoustic Sensors and Actuators on Distributed General Purpose Computing Platforms
Vikas Chandrakant Raykar | University of Maryland, CollegePark
![Page 2: Position Calibration of Acoustic Sensors and Actuators](https://reader036.fdocuments.net/reader036/viewer/2022062316/56813a8b550346895da287bd/html5/thumbnails/2.jpg)
Motivation
Many multimedia applications are emerging which use multiple audio/video sensors and actuators.
Microphones
Cameras
Speakers
Displays
Dis
trib
uted
C
aptu
re
Dis
trib
uted
R
ende
ring
Other Applications
Number Crunching
Cur
rent
The
sis
![Page 3: Position Calibration of Acoustic Sensors and Actuators](https://reader036.fdocuments.net/reader036/viewer/2022062316/56813a8b550346895da287bd/html5/thumbnails/3.jpg)
X
What can you do with multiple microphones…
Speaker localization and tracking.
Beamforming or Spatial filtering.
![Page 4: Position Calibration of Acoustic Sensors and Actuators](https://reader036.fdocuments.net/reader036/viewer/2022062316/56813a8b550346895da287bd/html5/thumbnails/4.jpg)
Some Applications…
Audio/Video Surveillance
Smart ConferenceRoomsAudio/Image Based
Rendering
Meeting RecordingSource separation and
Dereverberation
Speech Recognition
Hands free voice communication
Speaker Localizationand tracking
Multichannel speech Enhancement
MultiChannel echoCancellation
Novel Interactive audio Visual Interfaces
![Page 5: Position Calibration of Acoustic Sensors and Actuators](https://reader036.fdocuments.net/reader036/viewer/2022062316/56813a8b550346895da287bd/html5/thumbnails/5.jpg)
More Motivation…
Current work has focused on setting up all the sensors and actuators on a single dedicated computing platform.
Dedicated infrastructure required in terms of the sensors, multi-channel interface cards and computing power.
On the other hand
Computing devices such as laptops, PDAs, tablets, cellular phones,and camcorders have become pervasive.
Audio/video sensors on different laptops can be used to form a distributed network of sensors.
![Page 6: Position Calibration of Acoustic Sensors and Actuators](https://reader036.fdocuments.net/reader036/viewer/2022062316/56813a8b550346895da287bd/html5/thumbnails/6.jpg)
![Page 7: Position Calibration of Acoustic Sensors and Actuators](https://reader036.fdocuments.net/reader036/viewer/2022062316/56813a8b550346895da287bd/html5/thumbnails/7.jpg)
![Page 8: Position Calibration of Acoustic Sensors and Actuators](https://reader036.fdocuments.net/reader036/viewer/2022062316/56813a8b550346895da287bd/html5/thumbnails/8.jpg)
Common TIME and SPACE
Put all the distributed audio/visual input/output capabilities of all the laptops into a common TIME and SPACE.
This thesis deals with common SPACE i.e estimate the 3D positions of the sensors and actuators.
Why common SPACE
Most array processing algorithms require that precise positions of microphones be known.
Painful, tedious and imprecise to do a manual measurement.
![Page 9: Position Calibration of Acoustic Sensors and Actuators](https://reader036.fdocuments.net/reader036/viewer/2022062316/56813a8b550346895da287bd/html5/thumbnails/9.jpg)
This thesis is about..
X
YZ
![Page 10: Position Calibration of Acoustic Sensors and Actuators](https://reader036.fdocuments.net/reader036/viewer/2022062316/56813a8b550346895da287bd/html5/thumbnails/10.jpg)
If we know the positions of speakers….
If distances are not exact
If we have more speakers
X
Y
?
Solve in the least squaresense
![Page 11: Position Calibration of Acoustic Sensors and Actuators](https://reader036.fdocuments.net/reader036/viewer/2022062316/56813a8b550346895da287bd/html5/thumbnails/11.jpg)
If positions of speakers unknown…
Consider M Microphones and S speakers.
What can we measure?Distance between each speaker and all microphones.
Or Time Of Flight (TOF)
MxS TOF matrix
Assume TOF corrupted by Gaussian noise.
Can derive the ML estimate.
Calibration signal
![Page 12: Position Calibration of Acoustic Sensors and Actuators](https://reader036.fdocuments.net/reader036/viewer/2022062316/56813a8b550346895da287bd/html5/thumbnails/12.jpg)
Nonlinear Least Squares..More formally can
derive the ML estimateusing a Gaussian
Noise model
Find the coordinates which minimizes this
![Page 13: Position Calibration of Acoustic Sensors and Actuators](https://reader036.fdocuments.net/reader036/viewer/2022062316/56813a8b550346895da287bd/html5/thumbnails/13.jpg)
Maximum Likelihood (ML) Estimate..
we can define a noise modeland derive the ML estimate i.e. maximize the likelihood ratio
Gaussian noise
If noise is Gaussianand independentML is same asLeast squares
![Page 14: Position Calibration of Acoustic Sensors and Actuators](https://reader036.fdocuments.net/reader036/viewer/2022062316/56813a8b550346895da287bd/html5/thumbnails/14.jpg)
Reference Coordinate SystemReference Coordinate system | Multiple Global minima
X axis
Positive Y axis
OriginSimilarly in 3D
1.Fix origin (0,0,0)
2.Fix X axis
(x1,0,0)
3.Fix Y axis
(x2,y2,0)
4.Fix positive Z axis
x1,x2,y2>0
Which to choose? Later…
![Page 15: Position Calibration of Acoustic Sensors and Actuators](https://reader036.fdocuments.net/reader036/viewer/2022062316/56813a8b550346895da287bd/html5/thumbnails/15.jpg)
On a synchronized platform all is well..
![Page 16: Position Calibration of Acoustic Sensors and Actuators](https://reader036.fdocuments.net/reader036/viewer/2022062316/56813a8b550346895da287bd/html5/thumbnails/16.jpg)
However On a Distributed system..
![Page 17: Position Calibration of Acoustic Sensors and Actuators](https://reader036.fdocuments.net/reader036/viewer/2022062316/56813a8b550346895da287bd/html5/thumbnails/17.jpg)
The journey of an audio sample..
NetworkThis laptop wants to play a calibration signal on the other laptop.
Play comand in software.
When will the sound be actually played out fromThe loudspeaker.
Operating system Multimedia/multistream applications
Audio/video I/O devices I/O bus
![Page 18: Position Calibration of Acoustic Sensors and Actuators](https://reader036.fdocuments.net/reader036/viewer/2022062316/56813a8b550346895da287bd/html5/thumbnails/18.jpg)
t
t
jtsSignal Emitted by source j
Signal Received by microphone i
ijFOT ˆ
itmijTOF
Capture Started
Playback Started
Time Origin
On a Distributed system..
![Page 19: Position Calibration of Acoustic Sensors and Actuators](https://reader036.fdocuments.net/reader036/viewer/2022062316/56813a8b550346895da287bd/html5/thumbnails/19.jpg)
Joint Estimation..
Speaker Emission Start Times
S
Microphone Capture Start Times
M -1Assume tm_1=0
Microphone and speakerCoordinates
3(M+S)-6
MS TOF Measurements
Totally
4M+4S-7 parameters to estimates
MS observations
Can reduce the number of parameters
![Page 20: Position Calibration of Acoustic Sensors and Actuators](https://reader036.fdocuments.net/reader036/viewer/2022062316/56813a8b550346895da287bd/html5/thumbnails/20.jpg)
Use Time Difference of Arrival (TDOA)..
Formulation same as above but less number of parameters.
![Page 21: Position Calibration of Acoustic Sensors and Actuators](https://reader036.fdocuments.net/reader036/viewer/2022062316/56813a8b550346895da287bd/html5/thumbnails/21.jpg)
Assuming M=S=K Minimum K required..
![Page 22: Position Calibration of Acoustic Sensors and Actuators](https://reader036.fdocuments.net/reader036/viewer/2022062316/56813a8b550346895da287bd/html5/thumbnails/22.jpg)
Nonlinear least squares..
Levenberg Marquadrat method
Function of a large number of parameters
Unless we have a good initial guess may not convergeto the minima.
Approximate initial guess required.
![Page 23: Position Calibration of Acoustic Sensors and Actuators](https://reader036.fdocuments.net/reader036/viewer/2022062316/56813a8b550346895da287bd/html5/thumbnails/23.jpg)
Closed form Solution..
Say if we are given all pairwise distances between N points can we get the coordinates.
1 2 3 41 X X X X2 X X X X3 X X X X4 X X X X
![Page 24: Position Calibration of Acoustic Sensors and Actuators](https://reader036.fdocuments.net/reader036/viewer/2022062316/56813a8b550346895da287bd/html5/thumbnails/24.jpg)
Classical Metric Multi Dimensional Scaling
dot product matrixSymmetric positive definiterank 3
Given B can you get X ?....Singular Value Decomposition
Same as Principal component Analysis
But we can measureOnly the pairwise distance matrix
![Page 25: Position Calibration of Acoustic Sensors and Actuators](https://reader036.fdocuments.net/reader036/viewer/2022062316/56813a8b550346895da287bd/html5/thumbnails/25.jpg)
How to get dot product from the pairwise distance matrix…
k
ijd
kjd
kid
i
j
![Page 26: Position Calibration of Acoustic Sensors and Actuators](https://reader036.fdocuments.net/reader036/viewer/2022062316/56813a8b550346895da287bd/html5/thumbnails/26.jpg)
Centroid as the origin…Later shift it to our
orignal reference
Slightly perturb each location of GPCinto two to get the initial guess for the microphone and speaker coordinates
![Page 27: Position Calibration of Acoustic Sensors and Actuators](https://reader036.fdocuments.net/reader036/viewer/2022062316/56813a8b550346895da287bd/html5/thumbnails/27.jpg)
Example of MDS…
![Page 28: Position Calibration of Acoustic Sensors and Actuators](https://reader036.fdocuments.net/reader036/viewer/2022062316/56813a8b550346895da287bd/html5/thumbnails/28.jpg)
• Instead of pairwise distances we can use pairwise “dissimilarities”.
• When the distances are Euclidean MDS is equivalent to PCA.
• Eg. Face recognition, wine tasting
• Can get the significant cognitive dimensions.
MDS is more general..
![Page 29: Position Calibration of Acoustic Sensors and Actuators](https://reader036.fdocuments.net/reader036/viewer/2022062316/56813a8b550346895da287bd/html5/thumbnails/29.jpg)
Can we use MDS..Two problems
s1 s2 s3 s4 m1 m2 m3 m4
s1 ? ? ? ? X X X X
s2 ? ? ? ? X X X X
s3 ? ? ? ? X X X X
s4 ? ? ? ? X X X X
m1 X X X X ? ? ? ?
m2 X X X X ? ? ? ?
m3 X X X X ? ? ? ?
m4 X X X X ? ? ? ?
1. We do not have the complete pairwise distances
2. Measured distances Include the effect of lack of synchronization
UNKNOWN
UNKNOWN
![Page 30: Position Calibration of Acoustic Sensors and Actuators](https://reader036.fdocuments.net/reader036/viewer/2022062316/56813a8b550346895da287bd/html5/thumbnails/30.jpg)
Clustering approximation…
![Page 31: Position Calibration of Acoustic Sensors and Actuators](https://reader036.fdocuments.net/reader036/viewer/2022062316/56813a8b550346895da287bd/html5/thumbnails/31.jpg)
Clustering approximation…
j i
j j
i j
i i
![Page 32: Position Calibration of Acoustic Sensors and Actuators](https://reader036.fdocuments.net/reader036/viewer/2022062316/56813a8b550346895da287bd/html5/thumbnails/32.jpg)
Finally the complete algorithm…
ApproxDistance matrixbetween GPCs
Approxts
Approx tm
Clustering
Approximation
Dot product matrix
Dimension and coordinate system
MDS to get approx GPC locations
perturb
TOF matrix
Approx. microphone and speaker
locations
TDOA basedNonlinear
minimization
Microphone and speakerlocations tm
![Page 33: Position Calibration of Acoustic Sensors and Actuators](https://reader036.fdocuments.net/reader036/viewer/2022062316/56813a8b550346895da287bd/html5/thumbnails/33.jpg)
Sample result in 2D…
![Page 34: Position Calibration of Acoustic Sensors and Actuators](https://reader036.fdocuments.net/reader036/viewer/2022062316/56813a8b550346895da287bd/html5/thumbnails/34.jpg)
Algorithm Performance… •The performance of our algorithm depends on
•Noise Variance in the estimated distances.•Number of microphones and speakers.•Microphone and speaker geometry
•One way to study the dependence is to do a lot of monte carlo simulations.
•Else can derive the covariance matrix and bias of the estimator.
•The ML estimate is implicitly defined as the minimum of a certain error function.
•Cannot get an exact analytical expression for the mean and variance.
• Or given a noise model can derive bounds on how worst can our algortihm perform.
•The Cramer Rao bound.
![Page 35: Position Calibration of Acoustic Sensors and Actuators](https://reader036.fdocuments.net/reader036/viewer/2022062316/56813a8b550346895da287bd/html5/thumbnails/35.jpg)
Can use implicit function theorem and Taylors series expansion to get approximate expressions for bias and variance.
•J A Fessler. Mean and variance of implicitly defined biased estimators (such as penalized maximum likelihood): Applications to tomography. IEEE Tr. Im. Proc., 5(3):493-506, 1996. •Amit Roy Chowdhury and Rama Chellappa, "Statistical Bias and the Accuracy of 3D Reconstruction from Video", Submitted to International Journal of Computer Vision
Using first order taylors series expansion
Jacobian
Rank Deficit..remove theKnown parameters
Estimator Variance…
![Page 36: Position Calibration of Acoustic Sensors and Actuators](https://reader036.fdocuments.net/reader036/viewer/2022062316/56813a8b550346895da287bd/html5/thumbnails/36.jpg)
Gives the lower bound on the variance of any unbiased estimator.
Does not depends on the estimator. Just the data and the noise model.
Basically tells us to what extent the noise limits our performance i.e. you cannot get a variance lesser than the CR bound.
Jacobian
Rank Deficit..remove theKnown parameters
![Page 37: Position Calibration of Acoustic Sensors and Actuators](https://reader036.fdocuments.net/reader036/viewer/2022062316/56813a8b550346895da287bd/html5/thumbnails/37.jpg)
Different Estimators..
![Page 38: Position Calibration of Acoustic Sensors and Actuators](https://reader036.fdocuments.net/reader036/viewer/2022062316/56813a8b550346895da287bd/html5/thumbnails/38.jpg)
Number of sensors matter…
![Page 39: Position Calibration of Acoustic Sensors and Actuators](https://reader036.fdocuments.net/reader036/viewer/2022062316/56813a8b550346895da287bd/html5/thumbnails/39.jpg)
Number of sensors matter…
![Page 40: Position Calibration of Acoustic Sensors and Actuators](https://reader036.fdocuments.net/reader036/viewer/2022062316/56813a8b550346895da287bd/html5/thumbnails/40.jpg)
Geometry also matters…
![Page 41: Position Calibration of Acoustic Sensors and Actuators](https://reader036.fdocuments.net/reader036/viewer/2022062316/56813a8b550346895da287bd/html5/thumbnails/41.jpg)
Geometry also matters…
![Page 42: Position Calibration of Acoustic Sensors and Actuators](https://reader036.fdocuments.net/reader036/viewer/2022062316/56813a8b550346895da287bd/html5/thumbnails/42.jpg)
![Page 43: Position Calibration of Acoustic Sensors and Actuators](https://reader036.fdocuments.net/reader036/viewer/2022062316/56813a8b550346895da287bd/html5/thumbnails/43.jpg)
Calibration Signal…
![Page 44: Position Calibration of Acoustic Sensors and Actuators](https://reader036.fdocuments.net/reader036/viewer/2022062316/56813a8b550346895da287bd/html5/thumbnails/44.jpg)
• Compute the cross-correlation between the signals received at the two microphones.
• The location of the peak in the cross correlation gives an estimate of the delay.
• Task complicated due to two reasons 1.Background noise. 2.Channel multi-path due to room reverberations.• Use Generalized Cross Correlation(GCC).
• W(w) is the weighting function. • PHAT(Phase Transform) Weighting
Time Delay Estimation…
![Page 45: Position Calibration of Acoustic Sensors and Actuators](https://reader036.fdocuments.net/reader036/viewer/2022062316/56813a8b550346895da287bd/html5/thumbnails/45.jpg)
Time Delay Estimation…
![Page 46: Position Calibration of Acoustic Sensors and Actuators](https://reader036.fdocuments.net/reader036/viewer/2022062316/56813a8b550346895da287bd/html5/thumbnails/46.jpg)
Synchronized setup | bias 0.08 cm sigma 3.8 cm
Mic 3
Mic 1
Mic 2
Mic 4
Speaker 1
Spea
ker
4Spea
ker
2
Speaker 3
X
Z
Roo
m L
engt
h =
4.22
m
Room Width = 2.55 m
Room Height = 2.03 m
![Page 47: Position Calibration of Acoustic Sensors and Actuators](https://reader036.fdocuments.net/reader036/viewer/2022062316/56813a8b550346895da287bd/html5/thumbnails/47.jpg)
Distributed Setup…
Initialization phase Scan the network and find the number of GPC’s and the UPnP services available
MasterGPC 1 GPC 2 GPC M
•GPC 1 (Speaker) GPC 2 (Mic)•Calibration signal parameters
TOA Computation
TOATOA matrix
Position estimation
Play Calibration Signal
Play ML Sequence
![Page 48: Position Calibration of Acoustic Sensors and Actuators](https://reader036.fdocuments.net/reader036/viewer/2022062316/56813a8b550346895da287bd/html5/thumbnails/48.jpg)
Experimental results using real data
![Page 49: Position Calibration of Acoustic Sensors and Actuators](https://reader036.fdocuments.net/reader036/viewer/2022062316/56813a8b550346895da287bd/html5/thumbnails/49.jpg)
Related Previous work…
J. M. Sachar, H. F. Silverman, and W. R. Patterson III. Position calibration oflarge-aperture microphone arrays. ICASSP 2002
Y. Rockah and P. M. Schultheiss. Array shape calibration using sources in unknownlocations Part II: Near-field sources and estimator implementation. IEEE Trans. Acoust.,Speech, Signal Processing, ASSP-35(6):724-735, June 1987.
J. Weiss and B. Friedlander. Array shape calibration using sources in unknow locations a maximum-likelihood approach. IEEE Trans. Acoust., Speech, Signal Processing , 37(12):1958-1966, December 1989.
R. Moses, D. Krishnamurthy, and R. Patterson. A self-localization method for wirelesssensor networks. Eurasip Journal on Applied Signal Processing Special Issue on SensorNetworks, 2003(4):348{358, March 2003.
index.htm
![Page 50: Position Calibration of Acoustic Sensors and Actuators](https://reader036.fdocuments.net/reader036/viewer/2022062316/56813a8b550346895da287bd/html5/thumbnails/50.jpg)
Our Contributions…
•Novel setup for array processing.
•Position calibration in a distributed scenario.
•Closed form solution for the non-linear minimization routine.
•Expression for the mean and variance of the esimators.
•Study the effect of sensor geometry.
![Page 51: Position Calibration of Acoustic Sensors and Actuators](https://reader036.fdocuments.net/reader036/viewer/2022062316/56813a8b550346895da287bd/html5/thumbnails/51.jpg)
Acknowledgements…
• Dr. Ramani Duraiswami and Prof. Rama Chellappa
• Prof. Yegnanarayana
• Dr. Igor Kozintsev and Dr. Rainer Lienhart, Intel Research
• Prof. Min Wu and Prof. Shihab Shamma
• Prof. Larry Davis
![Page 52: Position Calibration of Acoustic Sensors and Actuators](https://reader036.fdocuments.net/reader036/viewer/2022062316/56813a8b550346895da287bd/html5/thumbnails/52.jpg)
Thank You ! | Questions ?