Audio Location Accurate Low-Cost Location Sensing James Scott Intel Research Cambridge Boris...

23
Audio Location Accurate Low-Cost Location Sensing James Scott Intel Research Cambridge Boris Dragovic Intern in 2004 at Intel Research Cambridge Studying for PhD at University of Cambridge

Transcript of Audio Location Accurate Low-Cost Location Sensing James Scott Intel Research Cambridge Boris...

Page 1: Audio Location Accurate Low-Cost Location Sensing James Scott Intel Research Cambridge Boris Dragovic Intern in 2004 at Intel Research Cambridge Studying.

Audio Location Accurate Low-Cost Location Sensing

James ScottIntel Research Cambridge

Boris DragovicIntern in 2004 at Intel Research Cambridge

Studying for PhD at University of Cambridge

Page 2: Audio Location Accurate Low-Cost Location Sensing James Scott Intel Research Cambridge Boris Dragovic Intern in 2004 at Intel Research Cambridge Studying.

Background

Fine-grained location systems have been built using ultrasound e.g. Bats (AT&T), Cricket (MIT) Achieving ~3cm 3D accuracy 95% of the time

Many end-user devices have audible-range I/O built in, but few have ultrasound

Can we use integrated/off-the-shelf audio hardware for location?

Page 3: Audio Location Accurate Low-Cost Location Sensing James Scott Intel Research Cambridge Boris Dragovic Intern in 2004 at Intel Research Cambridge Studying.

Audio-based location withoff-the-shelf hardware

Many txSpeakers in environment as txMobile phones or PDAs as rxPrivacy-preserving

Many rxComputer microphones as rxCan use mobile devices for txBUT can also use human sounds for tx

Page 4: Audio Location Accurate Low-Cost Location Sensing James Scott Intel Research Cambridge Boris Dragovic Intern in 2004 at Intel Research Cambridge Studying.

Locating human sounds

Need coverage from at least 4 micsUnknowns: X,Y,Z,t (t=time of sound) More is better since occlusion happens

Users do not need special “tag”No per-user setup requiredLowers costs and increases simplicity

User identity is not providedMany apps do not need identityAnonymity is good for privacyCould fuse with identity e.g. from RFID

Page 5: Audio Location Accurate Low-Cost Location Sensing James Scott Intel Research Cambridge Boris Dragovic Intern in 2004 at Intel Research Cambridge Studying.

Aims of prototype

Fine-grained location sensing with hardware accessible to end users

What accuracy can we obtain for locating human sounds, e.g. finger clicking, hand clapping?

Application area: 3D user interfaces using human sounds

Page 6: Audio Location Accurate Low-Cost Location Sensing James Scott Intel Research Cambridge Boris Dragovic Intern in 2004 at Intel Research Cambridge Studying.

Prototype

Use standard PC Add 6 PCI sound

cards and 6 mics Total cost of sound

hardware ~£100 From dabs.com

Fedora Core 2 Linux distribution

Java software

Page 7: Audio Location Accurate Low-Cost Location Sensing James Scott Intel Research Cambridge Boris Dragovic Intern in 2004 at Intel Research Cambridge Studying.

SignalDetection

System Architecture

PositioningTimingSignalDetection

SignalDetection

Page 8: Audio Location Accurate Low-Cost Location Sensing James Scott Intel Research Cambridge Boris Dragovic Intern in 2004 at Intel Research Cambridge Studying.

Signal Detection

Problem: identify the same part of the same sound in audio streams from multiple mics

Amplitude-threshold algorithm Keep track of current noise floor Mark sample as “significant” when amplitude is

is at least F times noise floor. (F ≈ 2.5) Properties

Very good at detecting sharp sounds Equally important: ignores other sounds Robust to noisy environments and cheap mics

Page 9: Audio Location Accurate Low-Cost Location Sensing James Scott Intel Research Cambridge Boris Dragovic Intern in 2004 at Intel Research Cambridge Studying.

Timing

Need time sync for sound streams 1ms error ≈ 30cm in space

Problem: Linux/Java introduce delays Buffering and scheduling result in variable

delays of >1ms Solution: hacked sound driver

Timestamp taken at interrupt made available to Java app (via /proc)

Does not account for interrupt delay Around 200 lines of C code

Page 10: Audio Location Accurate Low-Cost Location Sensing James Scott Intel Research Cambridge Boris Dragovic Intern in 2004 at Intel Research Cambridge Studying.

Positioning

Survey of microphone positions is currently done manually See orthogonal work on self-surveying

Use well-studied Levenberg-Marquardt technique to find 3D position (and sound generation time)

Page 11: Audio Location Accurate Low-Cost Location Sensing James Scott Intel Research Cambridge Boris Dragovic Intern in 2004 at Intel Research Cambridge Studying.

1D evaluation

First evaluated 1D performance for relative distance

Use two mics and a 6x7 grid of test points

20 hand claps and 20 finger clicks at each point

Microphones

Y

X

60cm

60cm

Page 12: Audio Location Accurate Low-Cost Location Sensing James Scott Intel Research Cambridge Boris Dragovic Intern in 2004 at Intel Research Cambridge Studying.

1D results: hand clapping

Page 13: Audio Location Accurate Low-Cost Location Sensing James Scott Intel Research Cambridge Boris Dragovic Intern in 2004 at Intel Research Cambridge Studying.

1D results: finger clicking

Page 14: Audio Location Accurate Low-Cost Location Sensing James Scott Intel Research Cambridge Boris Dragovic Intern in 2004 at Intel Research Cambridge Studying.

Implications of 1D results

Our mics are usable ~60º either side of axis Our mics have a maximum range of 4m

Drops to ~2m in very noisy conditions

Implications for deployment Density of microphones required to sense

location in a space

Finger clicking has median 1D error <5cm At least some of this due to human error

Page 15: Audio Location Accurate Low-Cost Location Sensing James Scott Intel Research Cambridge Boris Dragovic Intern in 2004 at Intel Research Cambridge Studying.

3D experiment setup

20 finger clicks at 4x4x3 test points on 60cm grid Total clicks: ~1000. Very sore fingers.

Microphones at 2 heights, and much more spread in X,Y than in Z This might be typical for real deployments

Y

X

60cm

60cm60cm

Key:Microphone at 60cm high

Microphone at 120cm high

Page 16: Audio Location Accurate Low-Cost Location Sensing James Scott Intel Research Cambridge Boris Dragovic Intern in 2004 at Intel Research Cambridge Studying.

Lollipops!

Page 17: Audio Location Accurate Low-Cost Location Sensing James Scott Intel Research Cambridge Boris Dragovic Intern in 2004 at Intel Research Cambridge Studying.

3D distance error

Page 18: Audio Location Accurate Low-Cost Location Sensing James Scott Intel Research Cambridge Boris Dragovic Intern in 2004 at Intel Research Cambridge Studying.

What do I think it’s good for?

3D user interfacesWhen I click here in future, do thisExtend computer input beyond desk/lap

Situated interfacesAdd a light switch by the bedRemote control without a losable device

Inspiration: SPIRIT (AT&T) which allows Bats to be used as 3D pointers

Page 19: Audio Location Accurate Low-Cost Location Sensing James Scott Intel Research Cambridge Boris Dragovic Intern in 2004 at Intel Research Cambridge Studying.

What do you think it’s good for?

Accessible user interfacesElderly, disabled

Activity inferencingFusion of location with sound recognition

Performance artSpotlights follow sounds

Tracking planes in an air show!Well, maybe not…

Page 20: Audio Location Accurate Low-Cost Location Sensing James Scott Intel Research Cambridge Boris Dragovic Intern in 2004 at Intel Research Cambridge Studying.

Visualisation

To help deploy and demo it UbiComp, Mobisys

Allows placement of mics creation of

“buttons” in 3D By mouse or finger

Used to create an mp3 player demo

Page 21: Audio Location Accurate Low-Cost Location Sensing James Scott Intel Research Cambridge Boris Dragovic Intern in 2004 at Intel Research Cambridge Studying.

Demo video – Accuracy

Page 22: Audio Location Accurate Low-Cost Location Sensing James Scott Intel Research Cambridge Boris Dragovic Intern in 2004 at Intel Research Cambridge Studying.

Demo video – User Interface

Page 23: Audio Location Accurate Low-Cost Location Sensing James Scott Intel Research Cambridge Boris Dragovic Intern in 2004 at Intel Research Cambridge Studying.

Conclusions

3D location sensing for under £100 of consumer sound peripherals

Accuracy: better than 28cm (3D) for 90% of finger clicks Improves to 10cm for 2D and repeated clicks

Sound-based user interfaces

Happy to provide source and specs [email protected]