1. Interactive Palimpsests Dr Jerry Fishenden Senior Research Fellow, Centre for Creative Computing Bath Spa University j.fishenden@bathspa.ac.uk

2. objective to explore and refine computational techniques and interfaces that enable users to better explore and interact with the past of place visually and aurally IEEE ISCC 2014: interactive palimpsests / Jerry Fishenden 3. n-tier navigation IEEE ISCC 2014: interactive palimpsests / Jerry Fishenden 4. user experience testing 5. initial visual navigation controls palimpsest navigator lens IEEE ISCC 2014: interactive palimpsests / Jerry Fishenden 6. palimpsest slider control IEEE ISCC 2014: interactive palimpsests / Jerry Fishenden 7. 2nd generation techniques the cursor as palimpsest navigator lens dynamically re-sizeable lens IEEE ISCC 2014: interactive palimpsests / Jerry Fishenden 8. examples IEEE ISCC 2014: interactive palimpsests / Jerry Fishenden http://voetek.com/palimpsests/dmusquaremile/lens.html http://voetek.com/palimpsests/dmusquaremile/slider.html 9. IEEE ISCC 2014: interactive palimpsests / Jerry Fishenden 10. if (test == 0) { int SensorDistance = e.Value; Palimpsest_Opacity = 1 - ((float)SensorDistance / 100); //near=past, far=present image1.Dispatcher.BeginInvoke(DispatcherPriority.Normal, (Action)(() => { image1.Opacity = Palimpsest_Opacity; })); } IEEE ISCC 2014: interactive palimpsests / Jerry Fishenden 11. IEEE ISCC 2014: interactive palimpsests / Jerry Fishenden 12. HandCursorElement.Visibility = Visibility.Visible; DepthImagePoint point = this._KinectDevice.CoordinateMapper.MapSkeletonPointToDepthPoint(hand.Position, this._KinectDevice.DepthStream.Format); point.X = (int) ((point.X * LayoutRoot.ActualWidth/_KinectDevice.DepthStream.FrameWidth) - (HandCursorElement.ActualWidth / 2.0)); point.Y = (int)((point.Y * LayoutRoot.ActualHeight / _KinectDevice.DepthStream.FrameHeight) - (HandCursorElement.ActualHeight / 2.0)); Canvas.SetLeft(HandCursorElement, point.X); Canvas.SetTop(HandCursorElement, point.Y); txtStatus.Text = "X is: " + point.X; image1.Opacity = point.X / LayoutRoot.ActualWidth; IEEE ISCC 2014: interactive palimpsests / Jerry Fishenden 13. video IEEE ISCC 2014: interactive palimpsests / Jerry Fishenden 14. IEEE ISCC 2014: interactive palimpsests / Jerry Fishenden 15. HandCursorElement.Visibility = Visibility.Visible; DepthImagePoint point = this._KinectDevice.CoordinateMapper.MapSkeletonPointToDepthPoint(hand.Position, this._KinectDevice.DepthStream.Format); point.X = (int) ((point.X * LayoutRoot.ActualWidth/_KinectDevice.DepthStream.FrameWidth) - (HandCursorElement.ActualWidth / 2.0)); point.Y = (int)((point.Y * LayoutRoot.ActualHeight / _KinectDevice.DepthStream.FrameHeight) - (HandCursorElement.ActualHeight / 2.0)); Canvas.SetLeft(HandCursorElement, point.X); Canvas.SetTop(HandCursorElement, point.Y); txtStatus.Text = "X is: " + point.X; bigScene.Visibility = Visibility.Visible; Ellipse.Center = new Point(point.X, point.Y); IEEE ISCC 2014: interactive palimpsests / Jerry Fishenden 16. private void CreateGrammars(RecognizerInfo ri) { //user oral instruction format is {use lens/slider}, {location London/Leicester} var technique = new Choices(); technique.Add("lens"); technique.Add("slide"); technique.Add("London"); technique.Add("Leicester"); var create = new Choices(); create.Add("location"); create.Add("use"); var gb = new GrammarBuilder(); gb.Culture = ri.Culture; gb.Append(create); gb.AppendWildcard(); gb.Append(technique); var g = new Grammar(gb); _sre.LoadGrammar(g); } IEEE ISCC 2014: interactive palimpsests / Jerry Fishenden 17. video IEEE ISCC 2014: interactive palimpsests / Jerry Fishenden 18. futures tracking the distance of a user from the sensor: growing or shrinking the size of the lens depending on whether someone's hand moves closer to or further from the screen addition of audio that will come to the foreground as the lens passes over particular visual items for example, emphasising the sound of children playing as the lens passes over children in the images, or emphasising the sound of horses hooves when passing over a horse and cart in the earlier underlying image competitive strategies for handling multiple users in front of the Kinect / screen continuing user experience testing, feedback and refinement IEEE ISCC 2014: interactive palimpsests / Jerry Fishenden 19. recent work IEEE ISCC 2014: interactive palimpsests / Jerry Fishenden 20. IEEE ISCC 2014: interactive palimpsests / Jerry Fishenden 21. references / reading Fishenden, J and Hugill, A. Palimpsests of Time and Place. 2011. IEEE COMPSAC Webb, J and Ashley, J. Beginning Kinect Programming with the Microsoft Kinect SDK. 2012. Apress Phidgets http://www.phidgets.com research site at http://fishenden.com IEEE ISCC 2014: interactive palimpsests / Jerry Fishenden 22. Interactive Palimpsests Dr Jerry Fishenden Senior Research Fellow, Centre for Creative Computing Bath Spa University j.fishenden@bathspa.ac.uk