Setup and Configuration of NeTD(Networked Tiled Display)-based ...

Gwangju Institute of Science and Technology

2010년 협업환경 및 가시화 기술 워크숍

Setup and Configuration of NeTD(Networked Tiled Display)-based Unified Col

laboration Environment

Namgon Kim and Sang Woo Han

{ngkim, swhan}@nm.gist.ac.kr

2010/ 11/ 10

Networked Media Laboratory

Gwangju Institute of Science and Technology (GIST)


Contents

! Visual Sharing and Our Experiences

! Unified Collaboration Environment


PART 1: VISUAL SHARING AND OUR EXPERIENCES


Multiparty Visual Sharing (Concept)


Visual Sharing with Networked Tiled Display

! Concurrently share the visualization from multiple high-resolution media of participating sites

! Useful for multi-site interactive collaboration ! Each site can have different resource capability (e.g., displa

y resolution, network bandwidth, computing power).


SAGE Visualcasting with Networked Tiled Display


NeTD-based Video Presence (Access Grid Compatible)


Interactive Medical Contents Visualization

Display clusterGigabit switch

Display controller

Raw frame

Networked tiled-display # of monitors : 8Resolution : 2560 * 1600 * 4 * 2 : 10240 * 3200

Display control message

Display control message

Uncompressed streams

Storage

Brain imaging Application

RAW image data (from captured viewport)

RAW image data (from image file)

Graphic card

Brain mesh (3D graphic)

MRI image overwrapped byEIT image(pre-computed image)

3D mouse

Network interface

Interaction message(P-x,y,z)


3D-extension of Visual Sharing


Toward Smart Collaboration for u-Health (SMeet: 2006-2008)


SMeet Collaboration Environment for u-Health


SMeet Prototypes

12


MediaXAggregate Manager

NetOpenAggregate Manager

Administrators

TB Management

Server

TBExperiment

Control Server

NetOpen RA & MediaX RA

. . .

Slice #1 Slice #2 Slice #3

Composite service #1 Composite service #2 Composite service #3

Tiled display service

Transcoding service

Video tiling service

Video composition serviceLive capturing service

Media Experiment via Service Composition

Mobile display service

Display service

Live capturing service

Caching service

On-demand content service

Experimenters

Experiment Description #1

Experiment Description #2

...

Local area network

Media-Centric Service-Oriented Testbed for Visual Sharing


Workflow-Driven Service Composition for Visual Sharing

fps ≤ THfps

MediaX SAGE Monitoring

Service

MediaX DXT Video Encoder

Service

MediaX VLC MPEG2 Video

Encoder Service

MediaX SAGE MPEG2 Video

Decoder Service

MediaX Networked

Display Service

Stitching of MediaX computing services via a NetOpen networking service

Control message (via experiment control plane)

no

yes


Decoder Service


Encoder Service

NeTD

Display type?single

monitor

User input

Condition

Placement of MediaX computing service(s)


User Interfaces for Visual Sharing

DXTHD1080i@30fps, 270Mbps(GIST)

DXTHD1080i@14fps,120Mbps(CNU)

WebUI OMX UI

Interaction Flow control Configuration Monitoring

Monitoring UINetOpen UI

MPEG2 HD1080i@30fps, 25Mbps

(POSTECH)


MediaX Computing Resource Aggregate

Adminstrator ExperimenterTB Management

ServerTB Experiment Control Server

Adaptors

10G

4x3 Tiled Display

2x2 Tiled Display Controller

2x2 Tiled DisplayUncompressedHDV Producers

HD camera

HDV Producers4x3 Tiled Display

Controller

1G

1G

1G

10G1G

1G

1G

1G

1G

1G

1G

1G

1G

1G

1G

1G

1G

1G

1G

1G

HD camera

HD camera

HD camera UI

UIUsers

Users


NetOpen Virtualized and Programmable Networking Resource Aggregate


Service Composition Process


Media-Centric Adaptive Service Composition Experiments with MediaX and NetOpen RA


PART 2: UNIFIED COLLABORATION ENVIRONMENT


Contents

! Collaboration Solutions ! Skype, Access Grid, ConferenceXP, Polyco

m ! NeTD (Networked Tiled Display)

! SAGE ! Unified Meeting Space

! Audio Integration ! NeTD-centric Display Integration


Collaboration Solutions

! Skype ! 음성통화에 중점을 둔 소프트웨어 솔루션

! Conference XP ! Microsoft에서 개발한 솔루션으로 음성, 영상 뿐만 아니라

파워포인트 공유 등 원격 협업을 위한 다양한 기능을 제공

! Access Grid ! ANL (Argonne National Lab.)을 중심으로 개발된 솔루션으

로 다수의 머신을 이용하여 하나의 협업 공간을 구성 가능한 확장성과 개방성을 지닌 솔루션

! Polycom ! 상용 솔루션 중 대표적인 솔루션으로 H.323기반으로 동작


Skype

! Single machine ! Only support webcam

s ! Generally for persona

l use ! Advantage

! Easy to use ! Better audio quality

! Disadvantage ! Small size group com

munication without video support


Conference XP

! Single-machine architecture capable of scaling to handle large, group-to-group-to-group conference scenarios

! The conferencing architecture supports: ! Full screen video at 30 fps, w

ith 250 ms latency ! FireWire cameras to enable

high quality, efficient video capture

! Five way conferencing < 2 Mb/s


Access Grid

! Support composing multiple machines for constructing a node ! Controller, Displayer, Video capture,

Audio capture

! Generally equipped with echo canceller for room configuration

! Advantage ! Flexible architecture ! Support large group size

! Disadvantage ! Difficult to configure ! Requires a skillful operator


Polycom

! Commercial product for remote collaboration

! Generally provide hardware set with specific capability

! Advantage ! Easy to use ! Customer service

! Disadvantage ! Expensive ! Limit in the number of particip

ants


Unified Collaboration Environment

! 통합형 협업 환경 ! 다양한 원격 협업 솔루션들을 동일한 입출력 인터페이스를 통

해 활용할 수 있도록 지원하는 미팅 공간


Requirements

! 상용 협업 솔루션과의 경제적인 연계를 지원

! 각 협업 솔루션간에 공유할 수 있는 장치를 공유하는 방법을 제공 ! Audio Interface

¡ 하나의 마이크/스피커를 이용해서 모든 협업 솔루션을 이용

! Display Interface ¡ 하나의 디스플레이 장치를 이용해서 모든 협업

솔루션을 가시화


Audio Integration

! 오디오 입/출력 인터페이스 통합 ! PC 오디오

! Microphone: 마이크 입력부분 ! Speaker: 스피커 출력부분

! Polycom 등의 상용 제품의 오디오 ! Line in: 외부 장치의 오디오 신호를 받아들이는 부분 ! Line out: 제품의 오디오 신호를 외부 장치로 보내는 부분

! 오디오 믹서 ! 입력된 오디오 신호를 혼합해서 출력해주는 장치

¡  협업 환경을 위한 오디오 통합을 위해서는 오디오 믹서가 2개 이상 (마이크용 1 + 스피커용 1) 의 출력을 지원해야 한다

! 에코켄슬러 ! 다수의 사용자가 참여하는 협업 환경을 위해서는 헤드셋을 대

신해 에코켄슬러에 연결된 마이크와 스피커를 이용


Line connection - Audio Integration

! 에코 켄슬러의 마이크 입력은 오디오 믹서를 통해 PC와 Polycom으로 분배

! PC와 Polycom의 스피커 출력은 오디오 믹서를 통해 에코 켄슬러의 line in으로 연결


Audio Mixer Configuration - Audio Integration

(SUB OUT) Line in of Echo canceller

(MAIN OUT) Mic. in to PC and Polycom Mic. in of Echo ca

nceller Speaker out of Polycom and PC


Display Integration

! Display Integration ! 하나의 디스플레이 장치를 이용해서 모든 협업 솔루션을

가시화

! NeTD-centric Display Integration ! NeTD를 중심으로 Display Integration을 실현


Networked Tiled Display (NeTD) ! Networked Tiled Display (NeTD)

! Network-centered visualization architecture ! SAGE (Scalable Adaptive Graphics Environment)

¡  A graphics streaming architecture for supporting collaborative scientific visualization environments

¡  Pixel stream ¡  http://www.sagecommons.org/

! CGLX (Cross-Platform Cluster Graphics Library) ¡  Allows users to run their OpenGL applications on a tiled-display system ¡  Graphic stream ¡  http://vis.ucsd.edu/~cglx/

SAGE CGLX


SAGE

! Scalable Adaptive Graphics Environment (SAGE) ! Specialized middleware for real-time streaming of extremel

y high-resolution graphics and high-definition video.

http://www.evl.uic.edu/cavern/sage/


Framework of SAGE


SAGE Applications - #1

! Scalable Video Consumer (SVC) ! Application for supporting MPEG2 data through I

EEE1394, files on disk or network interface

! JUXTAVIEW ! Cluster-based application for viewing ultra-high-r

esolution images on scalable tiled displays

! Bitplay ! Player for HD animations


SAGE Applications - #2

! VNC VIEWER ! Regular VNC viewer program modified to serve as a proxy

between a VNC server (of any size and pixel depth)

! Windows Desktop Sharing Application (WSHARE) ! Application based on software for sharing desktop screen

! Scalable Video Tool (SVT) ! Application based on windows for transmitting uncompress

ed HD video through HDMI


SAGE Installation - compile

! http://code.google.com/p/sage-graphics/wiki/SAGE

! Download SAGE ! http://www.evl.uic.edu/cavern/sage/download.php

! Install libraries ! QUANTA, SDL, Freeglut, Portaudio for SAGE ! Python, wxPython, numarray for SAGE UI ! ImageMagick, libmpeg2 for SAGE Applications

! Compile ! Execute ‘make;make install’ in the sage directory (NOT s

age/src)


SAGE Installation - configuration

! Go to the “sage/bin” directory ! fsManager.conf (ex: APPENDIX A)

•  fsManager: name and IP address of the machine on which FreeSpace Manager runs •  systemPort: port for SAGE system message channel •  uiPort: port for SAGE UI message channel •  conManager: IP address and port of connection manager •  tileConfiguration: name of tile configuration file •  receiverSyncPort: port of sync connection on displaying side •  receiverStreamPort: port used for streaming on displaying side



! stdtile-1.conf (ex: APPENDIX B) •  Dimensions: number of columns and rows of tiled display •  Mullions: width(inches) of top, down, left and right mullions •  Resolution: screen resolution of each tile •  PPI: pixels per inch of each tile •  Machines: number of display nodes which drive tiled display •  for each “DisplayNode”: - Name: name of each display node (not critical) - IP: ip address of each display node - Monitors: number of tiles which each node drivers



! Environment Variables in .bashrc

! SSH NO-PASSWORD between FSManager and SAGE Receiver (APPENDIX C)

•  SAGE_DIRECTORY: to the directory where you installed SAGE ex) export SAGE_DIRECTORY=/home/username/sage •  PATH: needs to include $SAGE_DIRECTORY/bin ex) export PATH=$SAGE_DIRECTORY/bin:$PATH

•  LD_LIBRARY_PATH: needs to include $SAGE_DIRECTORY:lib ex) export LD_LIBRARY_PATH=$SAGE_DIRECTORY/lib:$LD_LIBRARY_PATH


Execute SAGE

! Start SAGE ! To start SAGE go to the sage/bin ! Execute the ‘fsManager’

! Run a test application (Atlantis) ! Execute ‘atlantis’ in sage/bin directory


Operation of SAGE

! Console ! Execute ‘uiConsole’ in sage/bin directory

•  move: Move the windows of an application ex) move app_id dx dy •  resize: Resize the window of an application ex) resize app_id left right bottom top •  bg: Change background color ex) bg Red Green Blue •  admin: Get administrative information and status ex) admin


Operation of SAGE

! GUI ! Execute ‘sageui.py’ in sage/ui directory


Display Interfaces

! PC 디스플레이 ! RGB 혹은 DVI, HDMI 출력을 해당 인터페이스를 제공하는 모

니터에 연결해 디스플레이 ! Polycom 디스플레이

! 일반적으로 TV 를 통해 디스플레이 ! Video output:

¡  1 x S-Video; 4-pin mini DIN (Main monitor) ¡  1 x Composite; RCA/Phono (Main monitor) ¡  1 x S-Video (Second monitor) ¡  1 x Composite; RCA/Phono (VCR)


PC to NeTD– Display Integration

! VNC (Virtual Network Computing) ! X 윈도우 환경을 사용할 수 있는 원격 접속

인터페이스

! MS의 윈도우의 터미널 서비스와 유사

! 클라이언트와 서버 프로그램으로 나뉨

! MS 윈도우, Linux, Mac 등의 환경을 지원

! SAGE의 VNCViewer를 이용해 모든 머신의 출력을 타일드 디스플레이 상에 디스플레이


Polycom to PC - Display Integration

! PC에서 Polycom Video output capture ! Polycom의 비디오 출력 단자의 출력을 Video

capture card의 입력과 RCA 케이블로 연결 ! Video capture card

¡ Supports NTSC/PAL/SECAM video with resolutions up to 768x576 at 30fps

¡ Works with monochrome and color NTSC/PAL/SECAM cameras

¡ http://www.accessgrid.org/hardware/video/cards


Unified Collaboration Environment Configuration


Unified Collaboration Environment Configuration - Display Integration

PC Display Integration

Polycom Display Integration


SAGE Visualcasting

! Support distant collaboration with multiple endpoints ! All participants interact with one another, as well as with their d

ata ! Various display configuration at each endpoint

! Increasing the complexity of the pixel routing problem ! Independent application layout at each endpoint ! Dynamic changes in the number of applications and endpoints


SAGE Visualcasting Architecture


SAGE Visualcasting Example

CENIC CalREN-XD Workshop Sept., 2008


Thank you! Send Inquiry to

{swhan,ngkim}@nm.gist.ac.kr http://nm.gist.ac.kr

Gwangju Institute of Science & Technology


APPENDIX A

! fsManager.conf fsManager SMODController2 210.125.84.80 systemPort 20002 uiPort 20001 trackPort 20003 conManager sage.sl.startap.net 15557 tileConfiguration stdtile-1.conf receiverSyncPort 12000 receiverStreamPort 22000 receiverBufSize 500 fullScreen 1


APPENDIX B

! stdtile-1.conf TileDisplay Dimensions 2 1 Mullions 0.625 0.625 0.625 0.625 Resolution 2560 1600 PPI 90 Machines 8 DisplayNode Name display1 IP 210.125.84.81 Monitors 1 (0,0) DisplayNode Name display2 IP 210.125.84.82 Monitors 1 (1,0)

Setup and Configuration of NeTD(Networked Tiled Display)-based ...

Documents

Transcript of Setup and Configuration of NeTD(Networked Tiled Display)-based ...