Advancing the UI — Part 1: Look, Motion, and Gestures
-
Upload
samsung-developers -
Category
Technology
-
view
1.529 -
download
3
description
Transcript of Advancing the UI — Part 1: Look, Motion, and Gestures
#SDC13
Advancing the UI – Part I:Look, Motion, and Gesture
Anna SchallerSr. Manager, Developer RelationsSamsung Media Solutions Center America

Creating Moments of Delight

What’s Next for the UI?
Mobile SDK– Look – Gestures– Motion– MultiWindow– Visual View
Part I
Part II
SAMSUNG DEVELOPERS CONFERENCE
Look

A “Look” at features
Writing Buddy– Hand writing recognition extension to UI elements
Air Button– Quick access menu for various arrays of data
Smart Clip– Clipboard capture with metadata recognition
Pointer Icon– Extends icon for hover pointer to user-defined
images

Architecturally speaking……
View System modified to support extensions to Android UI elements– New default behaviors (Writing Buddy enabled on all
editable fields)– New packaged library with added functionality to enhance
your UI
APPLICATION FRAMEWORK
Activity Man-ager
Content Provider
Window Man-ager
View System Slook Impl
Telephony Man-ager
Resource Man-ager
Notification Man-ager
…..

Look implementation
SlookJAR
APPLICATION FRAMEWORK
View System
APPLICATION
SlookImplement
MyBeautifulApp
WritingBuddyImpl AirButtonImpl SmartClipImpl HoverPointIconImplSlookImpl
SlookWritingBuddy SlookAirButton SlookSmartClip SlookHoverPointIconSlook
Application
SDK Module
Implement
1.Request(Call Public API)2.Check Feature3.Check version
5.Request(Call Internal API)
7.Response
6.Response

Samsung Mobile SDK Libraries
New APIs to check for presence and properties of technology

@Overridepublic void onCreate(Bundle savedInstanceState) {
Slook slook = new Slook(); try { slook.initialize(this); } catch (SsdkUnsupportedException e) {
// notify user device does not support return;
} if (slook.isFeatureEnabled(Slook.AIRBUTTON) if (slook.isFeatureEnabled(Slook.SPEN_HOVER_ICON) if (slook.isFeatureEnabled(Slook.SMARTCLIP) if (slook.isFeatureEnabled(Slook.WRITINGBUDDY) {
Slook<class> variable = new Slook<class> }
Implementing Look Features

Writing Buddy
On by default for editable fields (EditText, DatePicker) Disable system wide with Direct pen input turned off Disable on app-by-app basis through look library

Writing Buddy Extensions
SLookWritingBuddy class– 2 constructors: ViewGroup and EditText
Support for text and images– 2 Listeners with callbacks

Writing Buddy Styles
EditText (text only) (default)
EditText (text and image toggle)
ViewGroup

SlookWritingBuddy mWritingBuddy;EditText editInput;ImageView mOutputImageView;
public void onCreate(. . . ) {
editInput = (EditText) findViewById(R.id.edit_input);mOutputImageView = (ImageView) findViewById(R.id.image_output);
mWritingBuddy = new SlookWritingBuddy(editInput);
mWritingBuddy.setImageWritingListener( new SlookWritingBuddy.ImageWritingListener() {
public void onImageReceived(Bitmap arg0) {mOutputImageView.setBackground(
new BitmapDrawable(getResources(), arg0));
}});
Implementing Writing Buddy – Text and Image

public void onCreate(. . . ) { FrameLayout fl = (FrameLayout) findViewById(R.id.input);
mOutputTextView = (TextView) findViewById(R.id.text_output); mWritingBuddy = new SlookWritingBuddy(fl);
mWritingBuddy.setTextWritingListener(new SlookWritingBuddy.TextWritingListener() {
public void onTextReceived(CharSequence arg0) { mOutputTextView.setText(arg0); } });
<LinearLayout><FrameLayout>
<SomeView></FrameLayout><TextView>
</LinearLayout>
Implementing Writing Buddy – ViewGroup

// Turn recognition on for numeric or alphanumericmType = SlookWritingBuddy.TYPE_EDITOR_NUMBER;mType = SlookWritingBuddy.TYPE_EDITOR_TEXT;mWritingBuddy.setEditorType(mType);
// enable or disable Writing BuddymWritingBuddy.setEnabled(mIsEnabled); // true or false
Writing Buddy Miscellaneous Features

Air Button
Activated with S Pen hover + side button Defaults to “Air Command” system wide Override behavior inside app
– com.samsung.android.sdk.look.airbutton Implemented with an ArrayAdapter
– Array of “data” attached to view– Array data can be text, images, contacts, video, audio

Data Types
Images
ContactsMenus

protected void onCreate(Bundle savedInstanceState) {setContentView(R.layout.activity_airbutton_simple);
Button btnText = (Button) findViewById(R.id.btn_text); createTextListWidgetFromView(btnText);} public SlookAirButton createTextListWidgetFromView(View v) { SlookAirButton airButtonWidget = new SlookAirButton(
v, // ButtongetAdapterStringList(), // Adapter SlookAirButton.UI_TYPE_LIST); // Data type
airButtonWidget.setItemSelectListener(mCallback); airButtonWidget.setPosition(0, 50); return airButtonWidget;}
Air Button -- Text

public SlookAirButtonAdapter getAdapterStringList() {ArrayList<AirButtonItem> stringList =
new ArrayList<AirButtonItem>();
stringList.add(new AirButtonItem(null, “Come here at 5:00", null));
stringList.add(new AirButtonItem(null, "Why?", null));stringList.add(new AirButtonItem(null,
"I wanna watch movie", null));stringList.add(new AirButtonItem(null,
"Ok. No problem", null));
return new SlookAirButtonAdapter(stringList);}
Creating the list of Air Button Strings

new SlookAirButton(View parentview, SlookAirButtonAdapter adapter,int uiType)
where uiType is: SlookAirButton.UI_TYPE_LIST // use with text and media arrays SlookAirButton.UI_TYPE_MENU // use with menu arrays
// ArrayAdapter ItemsAirButtonItem(Drawable image, // R.drawable.image or null,
String description, // Text or nullObject data) // null
new AirButtonItem(null, “text”, null) // text onlynew AirButtonItem(R.drawable.png, “text”, null) // image and text
Data Types

get/setGravity(int g) // within viewSlookAirButton.GRAVITY_LEFTSlookAirButton.GRAVITY_RIGHTSlookAirButton.GRAVITY_TOPSlookAirButton.GRAVITY_BOTTOM
get/setDirection(int d) // for adapterSlookAirButton.DIRECTION_UPPERSlookAirButton.DIRECTION_LOWERSlookAirButton.DIRECTION_RIGHTSlookAirButton.DIRECTION_LEFT
get/setPosition(int x, int y) // coordinates
Positioning the List
y
x

AirButton -- Gravity versus Direction

// lets user scroll through listssetScrollEnabled(boolean enable)
// animates the elements in the list when it openssetBounceEffectEnabled(boolean enable)
// animates the list when hovering over elementsstart/stopAnimationIcon(MotionEvent event)
ACTION_HOVER_ENTER | ACTION_HOVER_EXIT
More . . .

Smart Clip

Smart Clip Features
Default behavior– Captures the text and images in ImageView and TextView
Extensions with Look– Meta-data can now be included with the image clip– Meta-data includes URLs, strings, as well as the launch
intent for your activity– Meta data can be Static or Dynamic

public void onCreate(. . .) {
TextView tv = (TextView) findViewById(R.id.text_static);SlookSmartClip sc = new SlookSmartClip(tv);sc.clearAllMetaTag();
sc.addMetaTag(new SlookSmartClipMetaTag (SlookSmartClipMetaTag.TAG_TYPE_URL,
"http://www.samsung.com"));
sc.addMetaTag(new SlookSmartClipMetaTag (SlookSmartClipMetaTag.TAG_TYPE_PLAIN_TEXT, "This is android textview."));
}
public void someRoutine() {sendData();
}
Simple Use Case -- Scrapbook

public void sendData() { Intent intent = new Intent(); intent.setComponent(new
ComponentName("com.samsung.android.app.pinboard", "com.samsung.android.app.pinboard.ui.PinboardActivity")); try { startActivity(intent); } catch (ActivityNotFoundException e) { Toast.makeText(SmartClipActivity.this, ”Scrapbook application is not installed.”,
Toast.LENGTH_SHORT).show(); }}
Simple Use Case – Scrapbook (con’t)

Pointer Icon
Writing Buddy icon overrides pointer icon– Cannot redefine hover icon for these
fields without disabling writing buddy
Set pointer icon for the View you want it to appear with – Can be ViewGroup or View

Pointer Icon – Adding in codepublic class MainActivity extends Activity {
SlookPointerIcon mPointerIcon = new SlookPointerIcon();
@Overrideprotected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);setContentView(R.layout.activity_main);
RelativeLayout panel = (RelativeLayout) findViewById(R.id.container);
mPointerIcon.setHoverIcon (panel, getResources().
getDrawable(R.drawable.x_pointer_icon));
}
SAMSUNG DEVELOPERS CONFERENCE
LOOK Demo
SAMSUNG DEVELOPERS CONFERENCE
Gesture

What is Gesture?
Gesture allows you to use events generated by user’s hand movements in your application
Recognizes hand movements using data from the gesture sensor to the right of the device speaker
Requirements:– Gesture sensor– Android 4.3 (Jelly Bean API level
18) or higher

Gesture Package

public void onCreate() {private Sgesture mGesture;private SgestureHand mHand;
mGesture = new Sgesture();
try {mGesture.initialize(this);
} catch (IllegalArgumentException e) {//Error Handling
} catch (SsdkUnsupportedException e) {//Error Handling
}
if(mGesture.isFeatureEnabled(Sgesture.TYPE_HAND_PRIMITIVE)){mHand = new SgestureHand(Looper.getMainLooper(), mGesture);mHand.start(Sgesture.TYPE_HAND_PRIMITIVE, mChangeListener);
}
}
Implementation -- Check for support of feature

private final SgestureHand.ChangeListener mChangeListener = new SgestureHand.ChangeListener() {
@Overridepublic void onChanged(Info info) {
Log.d(TAG, “Hand : angle = “ + info.getAngle());// do something with data
}};
public void onStop() {mGestureHand.stop();
}
Register Listener
0º / DOWN
180º / UP
90º / RIGHT270º / LEFT

Reporting Angles
SAMSUNG DEVELOPERS CONFERENCE
Motion

What is Motion?
Package allows you to take advantage of movement data coming in from sensors for both “calling” motion and “forward” motion (walking or running)

“Calling” motion
Requirements:– Proximity sensor, acceleration sensor,
and a gyro sensor– Android 4.3 (Jelly Bean API level 18) or
higher
Recognizes the motion of bringing device up to your ear When you read a message on your device and bring the
device up to your ear, the call motion event is triggered, and your application calls the phone number that sent the message.

“Forward” motion
Pedometer data generated when walking/running with device (steps, calories, speed, distance, and step status)
Requirements:– Acceleration sensor, pressure sensor– If no pressure sensor, number of
up/down steps is not recognized– Android 4.3 (Jelly Bean API level 18) or
higher

Motion Library
Call data
Forward data

Implementation -- Check for support of featurepublic void onCreate() {
private Smotion mMotion;private SmotionCall mCall; // SmotionPedometer
mMotion = new Smotion();
try {mMotion.initialize(this);
} catch (IllegalArgumentException e) {//Error Handling
} catch (SsdkUnsupportedException e) {//Error Handling
}
if(mMotion.isFeatureEnabled(Smotion.TYPE_CALL)){ // TYPE_PEDOMETER mCall = new SmotionCall(Looper.getMainLooper(),mMotion);
mCall.start(Smotion.TYPE_CALL, mChangeListener);
}}

private SmotionCall.ChangeListener mChangeListener = new SmotionCall.ChangeListener() {
@Overridepublic void onChanged(Info info) {
int position = info.getCallPosition(); switch (position){
case SmotionCall.POSITION_LEFT:Log.d(TAG, "Call motion happened! : left");break;
case SmotionCall.POSITION_RIGHT:Log.d(TAG, "Call motion happened! :
Right");break;
default:break;
}}
};
Register Listener – “Call” Motion

private SmotionPedometer.ChangeListener mChangeListener = new SmotionPedometer.ChangeListener() {
@Overridepublic void onChanged(Info info) {
Log.d(TAG, “Pedometer : calorie = “ + info.getCalorie());Log.d(TAG, “Pedometer : distance = “ +
info.getDistance());Log.d(TAG, “Pedometer : speed = “ + info.getSpeed());Log.d(TAG, “Pedometer : step = “ +
info.getCount(SmotionPedometer.Info.COUNT_TOTAL));Log.d(TAG, “Pedometer : status = “ + info.getStatus());
}};
Register Listener – “Forward” Motion

Sensing the data
SDK
SensorHub
Pedometerhappened 1 step 1 step 1 step 1 step 1 step 1 step 1 step 1 step 1 step
…
…
…Screen off
Application
…

Getting the latest Pedometer data
Screen : OFF
Screen : ONgetInfo()
Application SDK
The latest pedometer info.
getInfo()
null
SDK just knows the pedometer information
before screen is off.It can’t know whether the data is the latest
data or not.So in this case it returns
null.
mPedometer.getInfo(); mPedometer.updateInfo();
Screen : OFF
Application SDK
updateInfo()
The latestpedometer info
SensorHub
Request
The latest dataYou can receive the latest pedometer info through ChangeListener though
screen is off.

What’s next…….
Advancing the UI -- Part II: MultiWindow and Visual View– Tuesday 9:00– California West
Thank You!

License NoticesExcept where noted, sample source code written by Samsung Electronics Co. Ltd and provided to you is licensed as described below. Copyright © 2010-2013, Samsung Electronics Co. Ltd. All rights reserved except as otherwise explicitly indicated. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials
provided with the distribution. Neither the name of the Samsung Electronics Co. Ltd nor the names of its contributors may be used to endorse or promote products derived from this software without specific
prior written permission. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
Other source code displayed in this presentation may be offered under other licenses. Apache 2.0 Copyright © 2010, Android Open Source Project. All rights reserved unless otherwise explicitly indicated. Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0. Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. Creative Commons 3.0 Attribution License Portions of this presentation are reproduced from work created and shared by Google (http://code.google.com/policies.html) and used according to terms described in the Creative Commons 3.0 Attribution License (http://creativecommons.org/licenses/by/3.0/).