Taking a Leap Forward With JavaFX

Taking A Leap Forward With JavaFXSimon Ritter, Oracle CorporationGerrit Grunwald, Canoo Engineering AGJohan Vos, LodgonJosé Pereda, Universidad de Valladolid


The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.


Program Agenda

The Man Machine Interface

Leap Motion Controller

JavaFX Basics and 3D

Leap Motion Java API

Demos


The Man Machine Interface


How It All Started


Progress was made…


Multi-touch has become popular


Gaming Has Driven Several Interfaces


Now It’s About Gestures


The Leap Motion Controller


The Basics

Small device (80 x 30 x 12mm) USB connection

– No external power required

Multiple OS support– Windows

– Mac OSX

– Linux

Low cost: $80


The Technology

Array of infra-red sensors– Can be susceptible to bright light

Proprietary motion detection algorithm– Tracks to 0.01mm resolution

– The secret sauce

1-2% CPU load No GPU requirement


The Details

The Leap Motion sensor detects hands, fingers and tools Data captured as frames continuously Listener handles events from frames Controller is the connection between device and application Gesture recognition must be enabled through the controller


Leap Motion Java APICo-ordinate System

Right hand Cartesian co-ordinate system


60cm range 150° view angle (left/right) 120° view angle (front/back)

Field Of View


Gestures

Predefined gestures– Circle

– Swipe

– Key tap (downward movement: y-axis)

– Screen tap (forward movement: z-axis)

Turn, twist, tilt, make a fist– Use motion factors from frame

Translation, rotation axis, rotation angle, scale factor


JavaFX Basics and 3D


JavaFX: The New Way To Build Java UIs


Scene Graph

Directed Acyclic Graph Parents and children Representation of the GUI components


Binding

Creates a dependency between a property and a changeable value

High level API– Easy to use

– Covers most common situations

Low level API– Allows for more complex interactions

– Optimised for fast execution and small footprint


Properties

Basis for high level binding API Concrete types for all primitives, String and Object

– DoubleProperty, StringProperty, etc

Simple API– bind / unbind– bindBidirectional / unbindBidirectional– isBound


Timeline Based Animations

Timeline– Modifies values of variables specified in KeyFrames

KeyFrame: specifies that a variable should have– A particular value at a particular time

KeyValue: Value to be interpolated for an interval


Animated Transitions

Pre-defined, single-purpose animations– Fade, Path, Pause, Rotate, Scale, Translate

– Can specify to, from and by values

Container transitions– Parallel, sequential

– Can be nested arbitarily

Transitions and Timelines share ancestary– A Timeline can be added to a Parallel / Sequential transition


JavaFX And The Third Dimension

Basic collection of 3D shapes– Box

– Cylinder

– Sphere

– MeshView (everything else)

javafx.scene.shape

Shapes



PhongMaterial– Way to cover a 3D object in a colour or image

– Uses interpolation to smooth polygon effects

Surfaces



How to illuminate the scene javafx.scene.effect.LightBase AmbientLight

– A light source that seems to come from all directions

PointLight– An attenuated light source that has a fixed point in space and radiates light

equally away from itself in all directions

Lighting



Where the scene is viewed from PerspectiveCamera

– Field of view is configurable (default is 30°)

ParallelCamera– Renders a scene without perspective correction

Cameras


Leap Motion Java API


Interaction Principles

Frame

Application Code

Listener

GUI Node

GUI Node

GUI Node

Controller

LeapMotion


Basic Approach

Create Controller Register Listener

– Subclass to implement specific functionality

onFrame callback method in Listener called by Controller Or you can use polling Frame contains all data

– Hand position, orientation

– Fingers

– Pointer position, orientation


Frame By Frame

Set of hand and finger tracking data detected at a point in time Hand provides:

– Direction of palm

– List of visible Fingers

Finger (which is a subclass of Pointable) provides:– Direction

– Tip position and velocity

– Length, width

– Time visible


Handling Gestures

Enable the gestures you want to use Recognised gesture data is added to the Frame

GestureList gl = frame.gestures();

for (int i = 0; i < gl.count(); i++) { Gesture g = gl.get(i); if (g.type == TYPE_SWIPE) SwipeGesture sw = new SwipeGesture(g);

...


Conclusions and More Information


Conclusions

Leap Motion adds a great new way to interact with applications Java support makes integration with existing applications simple

– Clean, straightforward API

– Simple gesture recognition

Use your imagination!


Further Information

www.leapmotion.com www.oracle.com/javafx jperedadnr.blogspot.co.uk blogs.oracle.com/speakjava


Demos


Graphic Section Divider

Taking a Leap Forward With JavaFX

Technology

Transcript of Taking a Leap Forward With JavaFX