Post on 09-Jun-2015
description
Kohta Kiyokawa, Shinya Yamamoto †, Naoki Shibata ‡,
Keiichi Yasumoto and Minoru Ito
Nara Institute of Science and Technology, † Tokyo University of Science Yamaguchi, ‡ Shiga University
February 23, MMSys2010
• Propose framework “UbiREMOTE” for remotely controlling appliances with 3D virtual space I/F– Intuitive operation for anyone – Intuitive monitoring of target spaces where appliances exist– Smooth operation on mobile terminals
• Implement and evaluate the framework– Runs smoothly on mobile terminals– Has better usability than text-based
remote controller
Overview
2
Outline
3
1. Background and Goals
2. UbiREMOTE Framework
3. Proposed Techniques
4. Evaluation
5. Conclusions and Future Works
Background Several frameworks for controlling appliances
through network are available UPnP, DLNA, OSGi, ECHONET, etc
These frameworks enable remote control of appliances Examples
warm/cool the living room before arriving home check if he/she switched off an appliance (e.g.,
heater) and turn it off remotely
4
5
• Several existing services– E.g., “JIG multi remocon” by jig.jp– Operate multiple appliances
through a cell phone– User I/F is based on figures/icons
and texts
• Drawbacks– Usability rapidly reduces when operating many similar
kind of appliances at different places (rooms)• Difficult to find target appliance from many
figures/texts on screen• UI is different from dedicated remote controllers
Existing services and drawbacks
© jig.jp co.ltd http://remocon.jig.jp/
Our goals
• Remote controller service that allows users to remotely control networked appliances in their home by– Intuitive operation from a remote place– Intuitive monitoring of target spaces
• Current states of appliances (on/off, …etc)• Conditions (temperature, humidity, …etc) of rooms
– Smooth operation by mobile terminals
6
Outline
7
1. Background and Goals
2. UbiREMOTE Framework
3. Proposed Techniques
4. Evaluation
5. Conclusions and Future Works
Smooth operation on mobile terminals
Intuitive monitoring of target spaces
8
Challenges and ApproachesIntuitive operation
Reproduce building with many networked appliances (target space) in 3D virtual space Imitate the real remote controller for each appliance by
using control window Show states of
appliances by visual effects
9
Challenges and ApproachesIntuitive operation
There are too many 3D objects in buildingPoor 3D capability on mobile deviceWe introduce lightweight 3D rendering
mechanism
Smooth operation on mobile terminals
Reproduce building with many networked appliances (target space) in 3D virtual space
10
Challenges and ApproachesIntuitive operation
Gather data by sensors deployed in each roomDetection/reflection of appliance positions
Lightweight 3D rendering mechanism
Smooth operation on mobile terminals
Intuitive monitoring of target spaces
Reproduce building with many networked appliances (target space) in 3D virtual space
11
Challenges and ApproachesIntuitive operation
12
Intuitive UI based on 3D virtual space
Appliance (lamp) in real space is turned on
Monitor real space conditionsMove viewpoint
25.0℃
Select target appliance (lamp), and turn it on
Target environment
13
Home server
Remote controller terminal
Sensors
Control
•Reflect information in virtual space
•Allow intuitive operations for appliances
Appliances
User
Wireless communication
3D Virtual Space (UI)
Operation
Synchronization
Target Real Space
Communication
Outline
14
1. Background and Goals
2. UbiREMOTE Framework
3. Proposed Techniques– Lightweight (LW) 3D Rendering– Reflecting Real Space Info into Virtual Space
4. Evaluation
5. Conclusions and Future Works
Why we need LW 3D rendering?• Motivation
– Render 3D objects on mobile terminals– However, there are so many 3D objects in home, and
mobile device’s 3D rendering capability is not sufficient
• Requirements– Smooth viewpoint movement– Quick response when operating an appliance– Sufficient quality of rendered image
15
16
Straightforward approach• Thin Client Method
– A powerful server renders 3D image, captures a still image from a specified viewpoint, and sends the image to a client
• Drawback– Large operation delay (response time) is produced
4. Drawing image2. Renderingand capturingstill image
3. Sending captured image
1. Operation
Server Client
Some time lag produced
17
Tradeoff for 3D rendering
short
Client renders all objects
Client renders all objects
Thin C
lient Method
Thin C
lient Method
Response time
3D renderingheavy light
long
• Our approach– Propose compromise between two methods– Utilize a fact that users are likely to operate
appliances near viewpoint
18
• Surround viewpoint with a cube, where 6 still images are captured for virtual space and put on its faces
• Client renders only objects inside the cube
Proposed LW 3D rendering technique
These objects are drawn as background images (rendered by server)
These objects inside the cube are rendered by client
Viewpoint
VP
Cube
Updating cube when viewpoint moves
Set a sphere of radius R with the same center as cube
While viewpoint (VP) stays within the sphere
No update of cube When VP goes out of sphere
Prefetch BG images for
new cube, and replace
current cube by new oneR
BG image
20
Message sequence for cube update
viewpoint movement
Viewpointmovement
ServerClient
New center pos for cube
BG images for new cube
User going out of sphere
Rendering BG images
Updating cube
Updating cube
User going out of sphere RendereingBG images
Outline
21
1. Background and Goals
2. UbiREMOTE Framework
3. Proposed Techniques– Lightweight (LW) 3D Rendering– Reflecting Real Space Info into Virtual Space
4. Evaluation
5. Conclusions and Future Works
22
• Difference between virtual and real spaces causes the following situations
• A user remotely operated a heater in living room (on the screen), but it actually had been moved to children room
• Approach to avoid above situations– Show some real space information in virtual
space• Temperature, humidity, etc• Position of appliances / furniture
Why need to reflect real space info?
23
Reflecting state change in real space into virtual space
Home server
Wireless communication
Communication
Virtual space
living roomTemp:25CHum:60 %
Sensors at each room detect temperature change, movement of appliance/ furniture, etc
Sensors at each room detect temperature change, movement of appliance/ furniture, etc
Deploy sensors at each room, and update information when there is a big change in sensor values
Detection/reflection of appliance positions
• Attach to each appliance a sensor node equipped with– Acceleration sensors for detecting movement– ZigBee communication capability for estimation of
new position
• Use existing ZigBee based localization techniques to estimate new position of appliance
24
Outline
25
1. Background and Goals
2. UbiREMOTE Framework
3. Proposed Techniques
4. Evaluation
5. Conclusions and Future Works
26
Experimental evaluation
• Implemented UbiREMOTE prototype on tablet PC • Evaluated performance
– Smoothness : Frame rates and response time– Usability : Questionnaire for User satisfaction in remote operation
TVElectric
Fan
Ceiling Light
Stand Light
CD Player
Floor plan and position of viewpoint
Controller of viewpoint
Controller foran appliance
27
Exp1 : Rendering speed
will achieve practical rendering speed on mobile terminal
Environment : Tablet PC
# objects (# objects inside cube)
# polygonsFrame rate (fps)
Render all objects
20 200000 1
Proposed 20 (1) 108 276
Proposed 20 (2) 2100 262
Proposed 20 (3) 4100 250
OS: Windows XP tablet PC editionCPU: Pentium M 1.2GHzMemory: 512MBGraphics Chip: Mobile Intel ®915GM
Exp2 : Response time (system configuration)
28
Mobile Terminal( Tablet PC )
Home Server
UPnPTCP/IP
3G cellular
network /WiFi
Electric Fan
Light
Heater
infrared Remote Controller
Exp2 : Response time (Result)
Reasonable response time for practical use 29
UbiREMOTE
appliances (a)WLAN (b)CellularElectric
fanON 0.94 3.22
OFF 0.92 3.14Light ON 0.73 2.98
OFF 0.61 3.13Heater ON 0.93 3.18
OFF 0.84 3.30average 0.828 3.141
30
Exp3:Usability evaluation by questionnaire
• 6 testees (graduate students)– operated 4 appliances by Remote controller that has
text-based interface and UbiREMOTE, – Return questionnaires.
ComparedText-based
remote controller
• It is easy to monitor appliances/conditions with UbiREMOTE31
サンプル数:24
サンプル数:24
■ Very Good■ Good■ Not good■ Bad
Q1: Is operation comfortable?
Q2:Can monitor conditions of appliances?
Text-based
Exp3 : Result of usability evaluation by questionnaire(1)
6 testees×4 appliances= 24samples
32
• Q3 : Did you feel something wrong in UbiREMOTE?
Exp3 : Result of usability evaluation by questionnaire(2)
Proposed LW 3D rendering does not spoiluser feeling and comfort in operation
3333
Video Demonstration
34
Conclusion and Future work• Conclusion
– UbiREMOTE framework• Intuitive and unified GUI for remotely controlling appliances
– Techniques to realize UbiREMOTE• LW 3D rendering based on cubic virtual space• Automatic reflection of real space information
– Performance evaluation• Achieved great improvement of 3D rendering performance• Usability was much better than Text-based interface
• Future work– Implementation on iPhone and Android– Usability evaluation by more detailed scenario
35