Improving cultural walk-through experiences using spherical panorama based navigation

Improving cultural walk-‐through experiences using spherical panorama based naviga9on

Andrea Ferracani, Lea Landucci, Pietro Pala and Alberto Del Bimbo Università di Firenze (IT) -‐ MICC

ACM Mul9media 2014, demo session, November 3-‐7, Orlando, Florida

The applica9on goal is to provide a lightweight environment to be deployed in a web browser for the interac(ve naviga(on and browsing of an ordered sequence of spherical images.


The first of the two key features of the system is to enable a smooth transi(on between the adjacent panoramas through an algorithm of sub-‐scene matching. ��


Image Ik Image Ik+1

The iden9fica9on of the rectangular sub-‐region of an image Ik that best matches the content of Ik+1 in correspondence to the inward direc9on is accomplished by minimiza9on of the following cost func(on with respect to the three variables (x, y, η) (x0, y0, η0) = arg[minx,y,η D(f(x, y, η), Ik+1)] where: -‐ f(x, y, η) = Crop(Ik, x, x + ηwk+1, y, y + ηhk+1) -‐ wk and hk are the width and height in pixels of the image Ik; -‐ Crop(I,x,∆x,y,∆y) returns the rectangular sub-‐region of the image I; -‐ D(In,Im) returns the dissimilarity between the content of two images In and Im; -‐ (xo, yo, ηo) defines the coordinates of the boYom-‐leZ vertex of the sub-‐region Po = (xo , yo) and the zoom factor Zo = 1/ηo to be applied to the sub-‐region of Ik in order to match the size of Ik+1. ��


5

Some results: in the first column images of the user current view are presented; in the second the best matching sub-‐region of the current image with the op9mal zoom factor for the transi9on is shown; finally, in the third column the next panorama is visualised. ��


We experimentally observed that computa9on of the dissimilarity func9on D(In,Im) through the distance of the image histograms provi-‐ des higher effec(veness compared to solu9ons based on scale invariant local keypoint descriptors such as SIFT.

The second key feature of the system is an innova(ve interac(on metaphor for a beYer frui9on of mul9media materials associated to interac(ve hotspots related to Point Of Interests (POIs). These are distributed along a cultural walk-‐through and include 3D models, PDFs, galleries of images/videos and indoor panoramas ��

Visualisa9on of a browsable panorama in the 3D space


-‐ The interface is built using HTML5, CSS3, javascript and WebGL; -‐ naviga9on among spherical images is obtained replacing a texture mapped into a sphere

taking into account the op9mal zoom factor es9mated by the sub-‐matching module; -‐ POIs infos and associated mul9media materials are shown in floa(ng panels in the 3D space

overcoming the limita9on of standard interfaces that use 2D light-‐boxes zoom factor computed by the sub-‐scene matching module.

��

Visualiza9on of an interac9ve 3D object of a church in the 3D space


Hotspots’ associated interac9ve mul9media

material can be accessed by a circular menu

represented by four icons for the differente

types: PDF, image and video gallery, 3D

object, nested panorama.

The system automa9cally provides the best

visual arrangement of mul9media materials

with respect to the user point-‐of-‐view and

allows the user to drag the hotspots in order

to customise the disposi9on of elements in

the 3D space.

Mini-‐maps are provided allowing contextual

awareness.

��

��

��

Hot-‐spost can be draggled in 3D space

Interac9ve mini-‐map showing hot-‐spots and user posi9on and orienta9on


The demo will show a navigable route based on spherical photos which starts from the current Via della Ninna near the Uffizi Gallery, to the church of San Leonardo in Arcetri in Florence (IT), giving the possibility to experience an important, though liYle-‐known mul9media virtual i9nerary of cultural and historical heritage. ��


The church of San Leonardo in Arcetri

The famous romanesque pulpit inside San Leonardo

The remains of the church of San Pier Scheraggio in Via della Ninna

We are also working on a demo to allow users to navigate the immersive interface through a gestures using MicrosoZ Kinect ��


Improving cultural walk-through experiences using spherical panorama based navigation

Documents

Transcript of Improving cultural walk-through experiences using spherical panorama based navigation