Ice ss2013
-
Upload
jun-hu -
Category
Technology
-
view
441 -
download
1
Transcript of Ice ss2013
![Page 1: Ice ss2013](https://reader030.fdocuments.net/reader030/viewer/2022020110/554a1deab4c905825d8b5571/html5/thumbnails/1.jpg)
ICE Summer School, July 2013
Visual Sensor Networks
Bernhard Rinner
Institut für Vernetzte und Eingebettete Systeme
![Page 2: Ice ss2013](https://reader030.fdocuments.net/reader030/viewer/2022020110/554a1deab4c905825d8b5571/html5/thumbnails/2.jpg)
B. Rinner. 2
Agenda
Sensor Networks
Smart Cameras
Visual Sensor Networks
![Page 3: Ice ss2013](https://reader030.fdocuments.net/reader030/viewer/2022020110/554a1deab4c905825d8b5571/html5/thumbnails/3.jpg)
B. Rinner 3 3 3
Introduction to Sensor Networks
![Page 4: Ice ss2013](https://reader030.fdocuments.net/reader030/viewer/2022020110/554a1deab4c905825d8b5571/html5/thumbnails/4.jpg)
B. Rinner 4
Wireless Sensor Networks (WSNs)
• Networks of typically small, battery-powered, wireless devices, (“sensor nodes”, “motes”) – On-board processing,
– Communication, and
– Sensing capabilities.
Sensors
Processor
Radio
Storage
P O W E R
Sensor node schematics [© Oracle Labs]
![Page 5: Ice ss2013](https://reader030.fdocuments.net/reader030/viewer/2022020110/554a1deab4c905825d8b5571/html5/thumbnails/5.jpg)
B. Rinner 5
Sensor Node Platforms
• From research prototypes to commercial products
The Vision „Smart Dust“ UC Berkeley late 1990‘s
Commercial Products „Mote-on-a Chip“ Dust Networks, 2010
Research Prototypes „Mica-2“ Crossbow 2004
![Page 6: Ice ss2013](https://reader030.fdocuments.net/reader030/viewer/2022020110/554a1deab4c905825d8b5571/html5/thumbnails/6.jpg)
B. Rinner 6
Some Applications of Sensor Networks
• Health
• Structural Monitoring
• Agriculture
• Environmental
[(c) University of Ghent] [Kim et al. ACM SenSys, 2006
[AgriNet] [M. Welsh, Harvard 2007]
![Page 7: Ice ss2013](https://reader030.fdocuments.net/reader030/viewer/2022020110/554a1deab4c905825d8b5571/html5/thumbnails/7.jpg)
B. Rinner 7
Communication is Key
• Wireless communication is an enabling technology – Eases deployment
– Enables mobility
– Increases flexibility
– Reduces costs
• Communicate on demand (ad hoc, spontaneous) with dynamic infrastructure – Nodes organize themselves into network
– Data is transferred via multiple hops
source destination
![Page 8: Ice ss2013](https://reader030.fdocuments.net/reader030/viewer/2022020110/554a1deab4c905825d8b5571/html5/thumbnails/8.jpg)
B. Rinner 8
But also …
• Advances in sensor technology – Micro electro-mechanical system
(MEMS) revolutionized sensing
– Integration of mechanical and electrical components on single chip
– Example: 3D accelerometer (in your cell phone)
• Embedded processors and integration – Moore’s Law still valid
– Trade-off between processing performance and power consumption
[© SensorDynamics]
[© ARM]
![Page 9: Ice ss2013](https://reader030.fdocuments.net/reader030/viewer/2022020110/554a1deab4c905825d8b5571/html5/thumbnails/9.jpg)
B. Rinner 9 9 9
The “Energy Problem”
• Major sources of energy consumption – Sensing, computing, communication
– High temporal variation
• Energy is the scarce resource for WSN. Several challenges – What energy reservoirs to exploit?
Constraints: availability, max. power, size, …
– How to distribute power over the network? Energy provider and consumer might be dislocated.
– How to control the distribution? “The proper amount of energy in the right place at the right time”
• Sensor networks have always been a “green” technology!
![Page 10: Ice ss2013](https://reader030.fdocuments.net/reader030/viewer/2022020110/554a1deab4c905825d8b5571/html5/thumbnails/10.jpg)
B. Rinner 10
Alternative Energy Reservoirs
• Maybe micro-heat engines – Exploit MEMS technology
to build „internal combustion engines“
– Expected power: 10-20 W
– Still in early research/development phase
• Or harvest energy from environment – Organic semiconductors for exploiting indoor ambient light
– Thin film batteries for storing energy
– EnHANTs : energy harvesting networked active tags
[Handbook of Sensor Networks, Wiley]
[Columbia University, CLUE]
![Page 11: Ice ss2013](https://reader030.fdocuments.net/reader030/viewer/2022020110/554a1deab4c905825d8b5571/html5/thumbnails/11.jpg)
B. Rinner. 11 11 11
Smart Cameras
![Page 12: Ice ss2013](https://reader030.fdocuments.net/reader030/viewer/2022020110/554a1deab4c905825d8b5571/html5/thumbnails/12.jpg)
B. Rinner. 12
Principle of Smart Cameras
• Smart cameras combine – sensing,
– processing and
– communication
in a single embedded device
• perform image and video analysis in real-time closely located at the sensor and transfer only the results
• collaborate with other cameras in the network
TrustEYE.M4 prototype on top of RaspberryPI
![Page 13: Ice ss2013](https://reader030.fdocuments.net/reader030/viewer/2022020110/554a1deab4c905825d8b5571/html5/thumbnails/13.jpg)
B. Rinner. 13
Differences to traditional Cameras
Traditional Camera – Optics and sensor
– Electronics
– Interfaces
delivers data in form of (encoded) images and videos, respectively
Smart Camera – Optics and sensor
– Onboard computer
– Interfaces
delivers abstracted image data and is configurable and programmable
Sensor
Electronics
Image enhancement/ Compression
Image Video
Sensor
Embedded Computer
Image analysis
„Events“
Programming Configuration
Light Light
![Page 14: Ice ss2013](https://reader030.fdocuments.net/reader030/viewer/2022020110/554a1deab4c905825d8b5571/html5/thumbnails/14.jpg)
B. Rinner. 14
SmartCams look for important things
• Examples for abstracted image data – compressed images and videos
– features
– detected events
© CMU
![Page 15: Ice ss2013](https://reader030.fdocuments.net/reader030/viewer/2022020110/554a1deab4c905825d8b5571/html5/thumbnails/15.jpg)
B. Rinner 15
Be aware of scarce Resources
• Major resource limitations – Processing power
– Communication bandwidth
– Onboard memory
– Energy
• Various Prototypes (with decreasing performance)
Sony XCISX100C/XP x86 VIA Eden ULV @ 1 GHz
TrustEYE.M4 ARM Cortex@ 168MHz
SLR Engineering Atom Z530@ 1.6 GHz
CITRIC PXA 270@ 13-640MHz
[Rinner et al. The Evolution from Single to Pervasive Smart Cameras. Proc. ICDSC 2008]
![Page 16: Ice ss2013](https://reader030.fdocuments.net/reader030/viewer/2022020110/554a1deab4c905825d8b5571/html5/thumbnails/16.jpg)
B. Rinner 16 16 16
Characteristics of Visual Sensor Networks
![Page 17: Ice ss2013](https://reader030.fdocuments.net/reader030/viewer/2022020110/554a1deab4c905825d8b5571/html5/thumbnails/17.jpg)
B. Rinner 17
Video Surveillance Network
• 3rd generation – all-digital systems
• 3+ generation – smart cameras
– surveillance tasks run on-site on smart cameras, e.g., • video compression traffic statistics
• accident detection wrong-way drivers
• stationary vehicles (tunnels) vehicle tracking
• 1st and 2nd generation – primarily analog frontends
– backend systems are digital
[Regazzoni, Ramesh, Foresti. Special Issue on Video Communications, Processing and Understanding for Third Generation Surveillance Systems. Proceedings of the IEEE. October 2001]
![Page 18: Ice ss2013](https://reader030.fdocuments.net/reader030/viewer/2022020110/554a1deab4c905825d8b5571/html5/thumbnails/18.jpg)
B. Rinner 18
Video Surveillance Network (2)
• Even third generation networks rely on “heavy” infrastructure. – Camera nodes: sensor, onboard processing (encryption)
– Network: hierarchically structured, wired, large bandwidth
– Energy: dedicated supply
• Surveillance networks typically consist of large number of cameras
• Processing in network is fixed; (compressed) data is streamed to control center
![Page 19: Ice ss2013](https://reader030.fdocuments.net/reader030/viewer/2022020110/554a1deab4c905825d8b5571/html5/thumbnails/19.jpg)
B. Rinner 19
Characteristics of VSN
• Visual sensor networks lie somewhere in between wireless sensor networks (WSN) and multi-camera/surveillance networks.
• VSN have unique characteristics (wrt. traditional WSN)
• Resource limitations – Need to process and transfer large amounts of data
– Energy and bandwidth
• On-board processing (cp. Smart cameras) – Challenging vision algorithms
– Adaptive behavior
[Soro et al. A Survey of Visual Sensor Networks. Advances in Multimedia 2009]
![Page 20: Ice ss2013](https://reader030.fdocuments.net/reader030/viewer/2022020110/554a1deab4c905825d8b5571/html5/thumbnails/20.jpg)
B. Rinner 20
Characteristics of VSN (2)
• Real-time operation – Most applications require real-time analysis (camera to user)
• Location and orientation information (spatial calibration) – Absolute or relative coordinates and orientations
– (Multi-)camera calibration
• Time Synchronization (temporal calibration)
• Data Storage – Access to historic data necessary, eg., frame buffer, detected events
– Stored data may be discarded over time
• Autonomous Camera Collaboration – cp. Distributed smart cameras (DSCs)
![Page 21: Ice ss2013](https://reader030.fdocuments.net/reader030/viewer/2022020110/554a1deab4c905825d8b5571/html5/thumbnails/21.jpg)
B. Rinner. 21
(Selected) VSN Problems
• Sensor Placement – Eg., dynamic setting of PTZ parameters
• Clustering, cluster head election – Eg., what cameras should “work together”, who is the “leader”
• Synchronization and calibration – Eg., establish temporal and spatial correlation
• Data (and energy) distribution – Eg., when and what data to exchange
• …
![Page 22: Ice ss2013](https://reader030.fdocuments.net/reader030/viewer/2022020110/554a1deab4c905825d8b5571/html5/thumbnails/22.jpg)
B. Rinner. 22 22 22
Coordinating Resources in
Visual Sensor Networks
![Page 23: Ice ss2013](https://reader030.fdocuments.net/reader030/viewer/2022020110/554a1deab4c905825d8b5571/html5/thumbnails/23.jpg)
B. Rinner 23
Configuring Smart Camera Networks
• Smart camera networks process data onboard can modify their functionality/execute actions during runtime to reflect changes – to the state of the environment
– to the user criteria
• A configuration describes what is processed/executed where; specified by – Description of camera network (including the available actions/tasks)
– Specification of the objective
• We study configuration methods to use scarce resources in these networks more efficiently
![Page 24: Ice ss2013](https://reader030.fdocuments.net/reader030/viewer/2022020110/554a1deab4c905825d8b5571/html5/thumbnails/24.jpg)
B. Rinner 24
Configuration Problem (example)
• Configuring a camera network – Select a set of cameras to monitor an area of interest
– Set the sensor (frame rate, resolution, PTZ) to achieve QoS
– Assign monitoring functions to cameras
– Optimize wrt. multiple criteria
– Dependent on dynamics of environment
s1
s2 s3
s4
s5
t1
t2
t3
p1, p2
p3
p4, p5
![Page 25: Ice ss2013](https://reader030.fdocuments.net/reader030/viewer/2022020110/554a1deab4c905825d8b5571/html5/thumbnails/25.jpg)
B. Rinner 25
Configuration Design Space
Design space for configuration methods is given by:
• Dynamics of environment (static vs. mobile observation points)
• Configuration algorithm (centralized vs. distributed)
• Tasks and sensors (homogeneous/heterogeneous; static/mobile cameras)
• A priori knowledge (complete vs. no knowledge of environment/VSN)
• Various alternatives for solving this optimization problem, eg. – Centralized configuration algorithms
– Distributed configuration algorithms
focus on resource-aware approaches
![Page 26: Ice ss2013](https://reader030.fdocuments.net/reader030/viewer/2022020110/554a1deab4c905825d8b5571/html5/thumbnails/26.jpg)
B. Rinner 26
Centralized Configuration with EA
• Approximation with evolutionary algorithm satisfying all requirements along multiple criteria (eg., energy, data, QoS)
• Smart Camera Network – Set of cameras at known position with fixed FoV
– Sensor configurations (frame rate, resolution)
• Observation Area – Static set of observation points with monitoring activity a
at required QoS (pot, fps)
• Monitoring tasks – Assign procedures for achieving
– Required resources for
},...{ 1 nSSS =
[Dieber, Micheloni, Rinner. Resource-Aware Coverage and Task Assignment in Visual Sensor Networks IEEE Transactions on Circuits and Systems for Video Technology, Aug 2011]
},...,{ 1 ki ddD =
},...,{ 1 mttT =
},...,{; 1 apaa ppPAa =∈),,(),( iiiii emcdPr →
![Page 27: Ice ss2013](https://reader030.fdocuments.net/reader030/viewer/2022020110/554a1deab4c905825d8b5571/html5/thumbnails/27.jpg)
B. Rinner 27
Self-aware Configuration
• Adopted from proprioceptive computing systems – use proprioceptive sensors to monitor “one self”
(concept from psychology, robotics/prosthetics, …, fiction)
– reason about their behavior (self-awareness)
– effectively and autonomously adapt their behavior to changing conditions (self-expression)
• Demonstrate autonomous multi-object tracking in camera network – Exploit single camera object detector & tracker
– Perform camera handover
– Learn camera topology
![Page 28: Ice ss2013](https://reader030.fdocuments.net/reader030/viewer/2022020110/554a1deab4c905825d8b5571/html5/thumbnails/28.jpg)
B. Rinner 28
Bid C4
Virtual Market-based Handover
• Initialize auctions for exchanging tracking responsibilities – Cameras act as self-interested agents, i.e., maximize their own utility
– Selling camera (where object is leaving FOV) opens the auction
– Other cameras return bids with price corresponding to “tracking” confidence
– Camera with highest bid continues tracking; trading based on Vickrey auction
Camera 1 Camera 2
Camera 3
Camera 4
Init auction
Bid C3
Fully distributed approach no a-priori topology knowledge required
![Page 29: Ice ss2013](https://reader030.fdocuments.net/reader030/viewer/2022020110/554a1deab4c905825d8b5571/html5/thumbnails/29.jpg)
B. Rinner 29
Market-based Tracking Handover
• Utility function (each camera)
rpjvcOUiOj
ijjii +−Φ⋅⋅= ∑∈
)]([)(
tracking decision visibility confidence payments made payments received
Simulation green: tracking yellow: shared FOV red: trading (handover)
![Page 30: Ice ss2013](https://reader030.fdocuments.net/reader030/viewer/2022020110/554a1deab4c905825d8b5571/html5/thumbnails/30.jpg)
B. Rinner 30
Tracking Performance
• Tradeoff between utility and communication effort
Scenario 1 (5 cameras, few objects) Scenario 2 (15 cameras, many objects)
• Emerging Pareto front [Esterle et al. Socio-Economic Vision Graph Generation and Handover in Distributed Smart Camera Networks. ACM Trans. Sensor Networks. 2013]
![Page 31: Ice ss2013](https://reader030.fdocuments.net/reader030/viewer/2022020110/554a1deab4c905825d8b5571/html5/thumbnails/31.jpg)
B. Rinner 31
Learn Neighborhood Relationships
• Gaining knowledge about the network topology (vision graph) by exploiting the trading activities
• Temporal evolution of the vision graph
![Page 32: Ice ss2013](https://reader030.fdocuments.net/reader030/viewer/2022020110/554a1deab4c905825d8b5571/html5/thumbnails/32.jpg)
B. Rinner 32
Learning Heterogeneous Strategies
• Heterogeneous strategies at cameras may improve Pareto front
• Adapt camera behaviour by online learning using bandit solvers
Homogeneous vs. heterogeneous handover strategies (offline)
Online learning strategies with different bandit solvers
[Lewis et al. Learning to be different: Heterogeneity and Efficiency in Distributed Smart Camera Networks. In Proc. IEEE SASO. 2013]
![Page 33: Ice ss2013](https://reader030.fdocuments.net/reader030/viewer/2022020110/554a1deab4c905825d8b5571/html5/thumbnails/33.jpg)
B. Rinner 33 33 33
Applications
![Page 34: Ice ss2013](https://reader030.fdocuments.net/reader030/viewer/2022020110/554a1deab4c905825d8b5571/html5/thumbnails/34.jpg)
B. Rinner 34 34 34
#1 Trustworthy Cameras
• Smart cameras – Highly capable embedded systems (on-board video analysis)
– Large software stacks
– Networked devices using closed (CCTV) and public networks
• Applications no longer only in public but also in private areas (assisted living, home monitoring, …)
• Protection of sensitive image data – Protection against manipulation (e.g., enforcement applications;
evidence at court)
– Privacy of monitored people
![Page 35: Ice ss2013](https://reader030.fdocuments.net/reader030/viewer/2022020110/554a1deab4c905825d8b5571/html5/thumbnails/35.jpg)
B. Rinner 35 35 35
Goals and Assumptions
• We present a system level approach that addresses the following security issues: – Integrity: detect manipulation of image and video data
– Authenticity: provide evidence about the origin of image and videos
– Confidentiality: make sure that privacy sensitive image data cannot be accessed by an unauthorized party
– Multi-level Access Control: support different abstraction levels and enforce access control for confidential data
• Security and privacy protection as inherent features of the camera
• Considered attack types: only software attacks [Winkler, Rinner. Securing Embedded Smart Cameras with Trusted Computing.
EURASIP Journal on Wireless Communications and Networking, 2011 ]
![Page 36: Ice ss2013](https://reader030.fdocuments.net/reader030/viewer/2022020110/554a1deab4c905825d8b5571/html5/thumbnails/36.jpg)
Approach
• Bringing of Trusted Computing concepts into cameras • Trusted Platform Modules (TPMs) are well defined, readily
available and cheap
• TC is an open industry standard • TPMs are available from many manufacturers
B. Rinner 36
![Page 37: Ice ss2013](https://reader030.fdocuments.net/reader030/viewer/2022020110/554a1deab4c905825d8b5571/html5/thumbnails/37.jpg)
Hardware Security Anchor
37
• Trusted Platform Module (TPM) at a glance – Secure storage for cryptographic keys – Data encryption, digital signatures – System status monitoring and reporting (measurement + attestation) – Unique platform ID
Security Chip (TPM) Image Sensor CPU RAM
Bootloader
Operating System (e.g., Embedded Linux)
Software Libraries and Middleware
Image Processing and Analysis Communication …
Software
Hardware
B. Rinner
![Page 38: Ice ss2013](https://reader030.fdocuments.net/reader030/viewer/2022020110/554a1deab4c905825d8b5571/html5/thumbnails/38.jpg)
Implemented Security Features
38
• Trusted boot where camera software stack is “measured” and the status is securely reported to operator
• Integrity and authenticity guarantees using non-migratable, TPM-protected RSA keys
• Freshness/timestamping for outgoing images via TPM-protected tick (counter) sessions
B. Rinner
![Page 39: Ice ss2013](https://reader030.fdocuments.net/reader030/viewer/2022020110/554a1deab4c905825d8b5571/html5/thumbnails/39.jpg)
B. Rinner 39 39 39
Hardware Prototype
• TI OMAP 3530 CPU: ARM @ 480MHz and
DSP @ 430MHz
• 256MB RAM, SD-Card as mass storage
• VGA color image sensor
• wireless: 802.11b/g WiFi
and 802.15.4 (XBee)
• LAN via USB (primarily used for debugging)
• Atmel hardware TPM
on I2C bus
![Page 40: Ice ss2013](https://reader030.fdocuments.net/reader030/viewer/2022020110/554a1deab4c905825d8b5571/html5/thumbnails/40.jpg)
Privacy Protection Approaches
40
• Protection as an inherent feature of the camera
• Object-based protection: Identification of sensitive data (e.g., human faces)
• Data abstraction and obfuscation
• Global protection techniques: Uniform protection of entire
frames (insensitive to misdetections of computer vision)
B. Rinner
![Page 41: Ice ss2013](https://reader030.fdocuments.net/reader030/viewer/2022020110/554a1deab4c905825d8b5571/html5/thumbnails/41.jpg)
Multi-Level Protection
41
• Video stream contains sub streams • Every sub stream is encrypted
– Hardware-bound cryptographic keys
• Recovery of identities only via four eyes principle
Video Stream Smart Camera Sub Streams
B. Rinner
![Page 42: Ice ss2013](https://reader030.fdocuments.net/reader030/viewer/2022020110/554a1deab4c905825d8b5571/html5/thumbnails/42.jpg)
High-Level Processing Flow
42 B. Rinner
![Page 43: Ice ss2013](https://reader030.fdocuments.net/reader030/viewer/2022020110/554a1deab4c905825d8b5571/html5/thumbnails/43.jpg)
B. Rinner 43 43 43
Privacy-aware Camera Networks
• What about users (i.e., monitored people)? • users usually do not care much about integrity, authenticity of time
stamping
• users (hopefully!) care about confidentiality and privacy!
• Question 1: How can we increase privacy awareness?
• Question 2: How can we demonstrate that (our) cameras protect the privacy of users?
![Page 44: Ice ss2013](https://reader030.fdocuments.net/reader030/viewer/2022020110/554a1deab4c905825d8b5571/html5/thumbnails/44.jpg)
B. Rinner 44 44 44
Raising Privacy Awareness
• Let users know if there are cameras in their environment
• Use user's handheld (e.g., smart phone) for location-based notifications
![Page 45: Ice ss2013](https://reader030.fdocuments.net/reader030/viewer/2022020110/554a1deab4c905825d8b5571/html5/thumbnails/45.jpg)
User Feedback
• Goal: Trustworthy feedback to monitored persons about camera’s privacy protection
• Visual communication for authentication – Direct line of sight – Intuitive way to select intended camera
• Operator discloses applications to TrustCenter
T. Winkler and B. Rinner, “User Centric Privacy Awareness in Video Surveillance,” Multimedia Systems Journal, vol. 18, no. 2, pp. 99–121, 2012.
B. Rinner
![Page 46: Ice ss2013](https://reader030.fdocuments.net/reader030/viewer/2022020110/554a1deab4c905825d8b5571/html5/thumbnails/46.jpg)
Attestation Report
46 B. Rinner B. Rinner
![Page 47: Ice ss2013](https://reader030.fdocuments.net/reader030/viewer/2022020110/554a1deab4c905825d8b5571/html5/thumbnails/47.jpg)
B. Rinner 47
#2 Aerial Cameras for Disaster Mgm.
• Develop autonomous multi-UAV system for aerial reconnaissance
• Up-to-date aerial overview images are helpful in many situations: “Google Earth with up-to-date images in high resolution”
• Small-scale quadcopter platform with onboard sensors and computation
• GPS receiver for autonomous waypoint flights
• Generic framework not bound to specific UAV
![Page 48: Ice ss2013](https://reader030.fdocuments.net/reader030/viewer/2022020110/554a1deab4c905825d8b5571/html5/thumbnails/48.jpg)
B. Rinner 48
Key Challenges
• Increase autonomy – Control and coordination of multiple UAVs
– High-level interaction with user
• Provide prompt response to user – Provide preliminary results fast and improve over time
• Deal with strong resource limitations – Flight time, payload, computation and communication
– Limited sensing capabilities
![Page 49: Ice ss2013](https://reader030.fdocuments.net/reader030/viewer/2022020110/554a1deab4c905825d8b5571/html5/thumbnails/49.jpg)
B. Rinner 49
Autonomous UAV Operation
Mission Planning Flight
Real-World
Simulator
Image Analysis
scenario specification
waypoints
Single/multiple UAV
captured image/video
stiching, detection
user interface
![Page 50: Ice ss2013](https://reader030.fdocuments.net/reader030/viewer/2022020110/554a1deab4c905825d8b5571/html5/thumbnails/50.jpg)
B. Rinner 50
Key Questions
• How to generate and update movement routes for the UAVs? – Achieve multiple optimization goals
– Deal with changes in the environment
• How to setup a wireless UAV network? – Provide networking coverage
• How to generate the mosaic image? – Apply incremental image stiching
– Combine RGB and thermal images
• System integration and demonstration
![Page 51: Ice ss2013](https://reader030.fdocuments.net/reader030/viewer/2022020110/554a1deab4c905825d8b5571/html5/thumbnails/51.jpg)
51
Demonstration Video
B. Rinner
![Page 52: Ice ss2013](https://reader030.fdocuments.net/reader030/viewer/2022020110/554a1deab4c905825d8b5571/html5/thumbnails/52.jpg)
B. Rinner. SCVSN Tutorial (Chapter 3) 52 52 52
Research Directions of Visual Sensor Networks
![Page 53: Ice ss2013](https://reader030.fdocuments.net/reader030/viewer/2022020110/554a1deab4c905825d8b5571/html5/thumbnails/53.jpg)
B. Rinner. 53
#1: Architecture
• Low-power (high performance) camera nodes – Dedicated platforms: vision processors, PCBs, systems
– Many examples: CITRIC, NXP
• Visual/Multimedia Sensor Networks – Topology and (multi-tier) architecture
– Multi-radio communication
• Dynamic Power Management – For sensing, processing and communication
How to design resource-aware nodes and networks
![Page 54: Ice ss2013](https://reader030.fdocuments.net/reader030/viewer/2022020110/554a1deab4c905825d8b5571/html5/thumbnails/54.jpg)
B. Rinner. 54
#2: Networking
• Ad hoc, p2p communication over wireless channels – Providing RT and QoS
– Eventing and/or streaming
• Dynamic resource management – (local) computation, compression, communication, etc.
– Degree of autonomy: dynamic, adaptive, self-organizing
– Fault tolerance, scalability
– Network-level software, middleware
How to process and transfer data in the network
![Page 55: Ice ss2013](https://reader030.fdocuments.net/reader030/viewer/2022020110/554a1deab4c905825d8b5571/html5/thumbnails/55.jpg)
B. Rinner 55
#3: Deployment, Operation, Maintenance
• Development support for applications – Model/simulate the application (function, resources, QoS)
– Reuse/exchange of software/libraries
– Software updates, debugging etc.
• Autonomous calibration and scene adaption – Avoid manual procedures
– Adapt to different scenes and settings
• Network configuration
Consider the entire life cycle of the camera network
![Page 56: Ice ss2013](https://reader030.fdocuments.net/reader030/viewer/2022020110/554a1deab4c905825d8b5571/html5/thumbnails/56.jpg)
B. Rinner. 56
#4: Distributed Sensing & Processing
• Sensor placement, calibration & selection – Optimization problem
– Distributed approaches eg., consensus, game theory, multi-agent systems
• Compressive Sensing
• Collaborative data analysis – Multi-view, multi-temporal, multi-modal
– Sensor fusion
• Online/real-time processing – Can not effort to store large amounts of data
Where to place sensors and analyze the data
![Page 57: Ice ss2013](https://reader030.fdocuments.net/reader030/viewer/2022020110/554a1deab4c905825d8b5571/html5/thumbnails/57.jpg)
B. Rinner 57
#5: Mobility
• Mobile cameras are ubiquitous – PTZ, vehicles, robotics etc.
– Mobile phones
• Advanced vision algorithms – Ego motion, online calibration
– Closed-loop control, active vision
How to exploit networks of mobile cameras
![Page 58: Ice ss2013](https://reader030.fdocuments.net/reader030/viewer/2022020110/554a1deab4c905825d8b5571/html5/thumbnails/58.jpg)
B. Rinner 58
#6: Usability
• Ease of deployment, maintenance – Self-* functionality
– “Smart cameras for dumb people”
• Privacy and Security – Trust of the user
– Control the privacy setting
• Interaction with the camera network
How to provide useful services to people
![Page 59: Ice ss2013](https://reader030.fdocuments.net/reader030/viewer/2022020110/554a1deab4c905825d8b5571/html5/thumbnails/59.jpg)
B. Rinner 59
#7: Applications
• Demonstrations – Large scale networks eg., for surveillance
– Small scale networks eg., for entertainment, home environments
– Only single camera application?
• Market opportunities
• Killer Application
What applications can (only) be solved by DSC
![Page 60: Ice ss2013](https://reader030.fdocuments.net/reader030/viewer/2022020110/554a1deab4c905825d8b5571/html5/thumbnails/60.jpg)
B. Rinner. SCVSN Tutorial (Chapter 3) 60
Summary
• VSNs exploit various advantages of distributed camera sensors such as increased coverage, redundancy and 3D information.
• Distributed cameras impose various challenges such as huge amount of data, required infrastructure and (network) topology.
• VSN have unique characteristics (wireless sensor networks vs. surveillance camera networks)
• Current research addresses signal processing, communications, architecture and middleware issues.
![Page 61: Ice ss2013](https://reader030.fdocuments.net/reader030/viewer/2022020110/554a1deab4c905825d8b5571/html5/thumbnails/61.jpg)
B. Rinner 61
Acknowledgements & Further Info
Pervasive Computing @ AAU UAV Research http://pervasive.aau.at http://uav.aau.at
• Tutorial site Most recent course material is available at
http://pervasive.aau.at/S5-tutorial