BREATHING LIFE INTO THE CANVAS

#droidconzgTomislav Homan

Tomislav Homan, Five

INTRO

Intro● Why custom views?

● Not everything can be solved with standard views

● We want to draw directly onto the Canvas

● Graphs and diagrams

● External data from sensors and mic (equalizer)

● ….

Couple of advices 1 / 3 - Initialize paints early

public final class StaticGraph extends View {

public StaticGraph(final Context context) { super(context); init();}

public StaticGraph(final Context context, final AttributeSet attrs) { super(context, attrs); init();}

public StaticGraph(final Context context, final AttributeSet attrs, final int defStyleAttr) { super(context, attrs, defStyleAttr); init();}

…..

Couple of advices 1 / 3 - Initialize paints early

private void init() { axisPaint = new Paint(Paint.ANTI_ALIAS_FLAG); axisPaint.setColor(Color.BLACK); axisPaint.setStyle(Paint.Style.STROKE); axisPaint.setStrokeWidth(4.0f);…..}

Couple of advices 2 / 3 - Memorize all the measures necessary to draw early

private PointF xAxisStart;private PointF xAxisEnd;

private PointF yAxisStart;private PointF yAxisEnd;……..@Overrideprotected void onSizeChanged(final int width, final int height, final int oldWidth, final int oldHeight) { super.onSizeChanged(width, height, oldWidth, oldHeight);

calculateAxis(width, height); calculateDataPoints(width, height);}

Couple of advices 3 / 3 - Use onDraw only to draw

@Overrideprotected void onDraw(final Canvas canvas) { super.onDraw(canvas);

canvas.drawLine(xAxisStart.x, xAxisStart.y, xAxisEnd.x, xAxisEnd.y, axisPaint); canvas.drawLine(yAxisStart.x, yAxisStart.y, yAxisEnd.x, yAxisEnd.y, axisPaint); …. canvas.drawPath(graphPath, graphPaint);}

ANIMATING CUSTOM VIEWS

● Every view is a set of states

● State can be represented as a point in a state space

● Animation is a change of state through time

A bit of philosophy

Animating custom views

Animating custom views

Let’s start with simple example - just a dot

● State contains only two pieces of information, X and Y position

● We change X and Y position through time

Animating custom viewsThe recipe

● Determine important constants

● Initialize paints and other expensive objects

● (Re)calculate size dependent stuff on size changed

● Implement main loop

● Calculate state

● Draw

Determine important constants

private static final long UI_REFRESH_RATE = 60L; // fps

private static final long ANIMATION_REFRESHING_INTERVAL = TimeUnit.SECONDS.toMillis(1L) / UI_REFRESH_RATE; // millis

private static final long ANIMATION_DURATION_IN_MILLIS = 1500L; // millis

private static final long NUMBER_OF_FRAMES = ANIMATION_DURATION_IN_MILLIS / ANIMATION_REFRESHING_INTERVAL;

Animating custom views

Determine important constants

● For animation that lasts 1500 milliseconds in framerate of 60 fps...

● We should refresh the screen every cca 16 milliseconds

● And we have cca 94 frames

Initialize paints and other expensive objects - business as usual

private void init() { dotPaint = new Paint(Paint.ANTI_ALIAS_FLAG); dotPaint.setColor(Color.RED); dotPaint.setStyle(Paint.Style.FILL); dotPaint.setStrokeWidth(1.0f);

endPointPaint = new Paint(Paint.ANTI_ALIAS_FLAG); endPointPaint.setColor(Color.GREEN); endPointPaint.setStyle(Paint.Style.FILL); endPointPaint.setStrokeWidth(1.0f);}

(Re)calculate size dependent stuff on size changed

@Overrideprotected void onSizeChanged(final int width, final int height, final int oldWidth, final int oldHeight) { super.onSizeChanged(width, height, oldWidth, oldHeight);

startPoint = new PointF(width / 4.0f, height * 3.0f / 4.0f); endPoint = new PointF(width * 3.0f / 4.0f, height / 4.0f);

….}

Implement main loop

private final Handler uiHandler = new Handler(Looper.getMainLooper());

private void startAnimating() { calculateFrames(); uiHandler.post(invalidateUI);}

private void stopAnimating() { uiHandler.removeCallbacks(invalidateUI);}

Implement main loop

private Runnable invalidateUI = new Runnable() {

@Override public void run() { if (hasFrameToDraw()) { invalidate(); uiHandler.postDelayed(this, ANIMATION_REFRESHING_INTERVAL); } else { isAnimating = false; } }};

Calculate state

● Create frames array

● Determine step by which state changes

● Increase positions by step

Animating custom views

Calculate state

private void calculateFrames() {

frames = new PointF[NUMBER_OF_FRAMES + 1]; …. float x = animationStartPoint.x; float y = animationStartPoint.y; for (int i = 0; i < NUMBER_OF_FRAMES; i++) { frames[i] = new PointF(x, y); x += xStep; y += yStep; }

frames[frames.length - 1] = new PointF(animationEndPoint.x, animationEndPoint.y);

currentFrame = 0;}

Animating custom viewsDraw

● Now piece of cake

● Draw static stuff

● Take and draw current frame

● Increase the counter

Draw

@Overrideprotected void onDraw(final Canvas canvas) { super.onDraw(canvas);

drawDot(canvas, startPoint, endPointPaint); drawDot(canvas, endPoint, endPointPaint);

final PointF currentPoint = frames[currentFrame]; drawDot(canvas, currentPoint, dotPaint);

currentFrame++;}

Now we want to animate the graph from one state to another

Animating custom viewsNow we to animate the graph from one state to another

Recipe is the same, state more complicated

Dot state:

private PointF[] frames;

Graph state:

private PointF[][] framesDataPoints;private float[] framesAxisZoom;private int[] framesColor;

EASING IN AND OUT

Easing in and out

● Easing in - accelerating from the origin

● Easing out - decelerating to the destination

● Accelerate, hit the inflection point, decelerate to the destination

● Again - dot as an example

Easing in and outEasing out (deceleration)● Differences while calculating frames● Replace linear trajectory with quadratic● The step that we used in first animation isn’t valid anymore

float x = animationStartPoint.x;float y = animationStartPoint.y;for (int i = 0; i < NUMBER_OF_FRAMES; i++) { frames[i] = new PointF(x, y); x += xStep; y += yStep;}

A bit of high school math….

….gives us the following formula:

Xi = (- L / N^2) * i^2 + (2 * L / N) * i

● Xi - position (state) for the ith frame

● L - length of the dot trajectory

● N - number of frames

● i - order of the frame

The rest of the recipe is same:

● Calculation phase modified to use previous formula

final float aX = -pathLengthX / (NUMBER_OF_FRAMES * NUMBER_OF_FRAMES);final float bX = 2 * pathLengthX / NUMBER_OF_FRAMES;

final float aY = -pathLengthY / (NUMBER_OF_FRAMES * NUMBER_OF_FRAMES);final float bY = 2 * pathLengthY / NUMBER_OF_FRAMES;

for (int i = 0; i < NUMBER_OF_FRAMES; i++) {

final float x = calculateFunction(aX, bX, i, animationStartPoint.x);final float y = calculateFunction(aY, bY, i, animationStartPoint.y);

frames[i] = new PointF(x, y);}

private float calculateFunction(final float a, final float b, final int i, final float origin) { return a * i * i + b * i + origin;}

Easing in (acceleration)

● Same approach

● Different starting conditions - initial velocity zero

● Renders a bit different formula

Acceleration and deceleration in the same time

● Things more complicated (but not a lot)

● Use cubic formula instead of quadratic

● Again some high school math - sorry :(

The magic formula:

Xi = (- 2 * L / N^3) * i^3 + (3 * L) / N^2 * i^2

● Xi - position (state) for the ith frame

● L - length of the dot trajectory

● N - number of frames

● i - order of the frame

● Same as quadratic interpolation, slightly different constants and powers

Easing in and outGraph example

● Again: Use same formulas, but on a more complicated state

DYNAMIC FRAME CALCULATION

Dynamic frame calculationActually 2 approaches for calculating state

● Pre-calculate all the frames (states) - we did this

● Calculate the next frame from the current one dynamically

Dynamic frame calculationPre-calculate all the frames (states)

● All the processing done at the beginning of the animation

● Everything is deterministic and known in advance

● Easy to determine when to stop the animation

● Con: takes more space - 94 positions in our example

Dynamic frame calculationDynamic state calculation

● Calculate the new state from the previous one every loop iteration

● Something like a mini game engine / physics simulator

● Wastes far less space

● Behaviour more realistic

● Con: if calculation is heavy frames could drop

● Respect sacred window of 16 (or less) milliseconds

Dynamic frame calculation

● First example - a dot that bounces off the walls

● Never-ending animation - duration isn’t determined

● Consequently we don’t know number of frames up-

front

● Perfect for using dynamic frame calculation

● Twist in our recipe

Dynamic frame calculationThe recipe

● Determine important constants - the same

● Initialize paints and other expensive objects - the same

● (Re)calculate size dependent stuff on size changed - the same

● Implement main loop - move frame calculation to drawing phase

● Calculate state - different

● Draw - almost the same

Implement the main loop

private final Handler uiHandler = new Handler(Looper.getMainLooper());

private void startAnimating() { calculateFrames(); uiHandler.post(invalidateUI);}

private void stopAnimating() { uiHandler.removeCallbacks(invalidateUI);}

Implement the main loop

private Runnable invalidateUI = new Runnable() {

@Override public void run() { if (hasFrameToDraw()) { invalidate(); uiHandler.postDelayed(this, ANIMATION_REFRESHING_INTERVAL); } else { isAnimating = false; } }};

Implement the main loop

private void startAnimating() { uiHandler.post(invalidateUI);}

private void stopAnimating() { uiHandler.removeCallbacks(invalidateUI);}

private Runnable invalidateUI = new Runnable() { @Override public void run() { invalidate(); uiHandler.postDelayed(this, ANIMATION_REFRESHING_INTERVAL); }};

Draw

@Overrideprotected void onDraw(final Canvas canvas) { super.onDraw(canvas);

drawDot(canvas, startPoint, endPointPaint); drawDot(canvas, endPoint, endPointPaint);

final PointF currentPoint = frames[currentFrame]; drawDot(canvas, currentPoint, dotPaint);

currentFrame++;}

Draw

private PointF currentPosition;

@Overrideprotected void onDraw(final Canvas canvas) { super.onDraw(canvas);

canvas.drawRect(topLeft.x, topLeft.y, bottomRight.x, bottomRight.y, wallsPaint); drawDot(canvas, currentPosition, dotPaint);

if (isAnimating) { updateWorld(); }}

Calculate state

private void updateWorld() { final float dx = currentVelocity.x; // * dt final float dy = currentVelocity.y; // * dt

currentPosition.set(currentPosition.x + dx, currentPosition.y + dy);

if (hitRightWall()) { currentVelocity.x = -currentVelocity.x; currentPosition.set(topRight.x - WALL_THICKNESS, currentPosition.y); } //Same for every wall}

private boolean hitRightWall() { return currentPosition.x >= topRight.x - WALL_THICKNESS;}

Dynamic frame calculationAdd gravity to previous example

● Just a couple of lines more

private void updateWorld() {

final float dvx = GRAVITY.x; final float dvy = GRAVITY.y;

currentVelocity.set(currentVelocity.x + dvx, currentVelocity.y + dvy);

final float dx = currentVelocity.x; // * dt final float dy = currentVelocity.y; // * dt

currentPosition.set(currentPosition.x + dx, currentPosition.y + dy);…..}

Springs

● Define a circle of given radius

● Define couple of control points with random

distance from the circle

● Let control points spring around the circle

private void updateWorld() { final int angleStep = 360 / NUMBER_OD_CONTROL_POINTS; for (int i = 0; i < controlPoints.length; i++) {

final PointF point = controlPoints[i]; final PointF velocity = controlPointsVelocities[i];

final PointF springCenter = CoordinateUtils.fromPolar((int) radius, i * angleStep, centerPosition); final float forceX = -SPRING_CONSTANT * (point.x - springCenter.x); final float forceY = -SPRING_CONSTANT * (point.y - springCenter.y);

final float dvx = forceX; final float dvy = forceY; velocity.set(velocity.x + dvx, velocity.y + dvy);

final float dx = velocity.x; final float dy = velocity.y;

point.set(point.x + dx, point.y + dy); }}

Dynamic frame calculation

Usefulness of those animations

● Not very useful per se

● Use springs to snap the objects from one position to another

● Use gravity to collapse the scene

● You can simulate other scene properties instead of position such as color,

scale, etc...

ANIMATING EXTERNAL INPUT

Animating external input● It sometimes happens that your state doesn’t depend only on internal

factors, but also on external

● For example equalizer

● Input is sound in fft (fast Fourier transform) data form

● Run data through the “pipeline” of transformations to get something that

you can draw

● The recipe is similar to the precalculation style, but animation isn’t

triggered by button push, but with new sound data arrival

Main loop - just invalidating in 60 fps

private void startAnimating() { uiHandler.post(invalidateUI);}

private void stopAnimating() { uiHandler.removeCallbacks(invalidateUI);}

private Runnable invalidateUI = new Runnable() { @Override public void run() { invalidate(); uiHandler.postDelayed(this, ANIMATION_REFRESHING_INTERVAL); }};

Data input

private static final int SOUND_CAPTURE_RATE = 20; // Hz

private void startCapturingAudioSamples(int audioSessionId) { visualizer = new Visualizer(audioSessionId); visualizer.setCaptureSize(Visualizer.getCaptureSizeRange()[1]); visualizer.setDataCaptureListener(new Visualizer.OnDataCaptureListener() {

@Override public void onWaveFormDataCapture(Visualizer visualizer, byte[] waveform, int samplingRate) { }

@Override public void onFftDataCapture(Visualizer visualizer, byte[] fft, int samplingRate) { calculateData(fft); } }, SOUND_CAPTURE_RATE * 1000, false, true); visualizer.setEnabled(true); }

Triggered 20 times in a second

Transforming data

private void calculateData(byte[] bytes) {

final int[] truncatedData = truncateData(bytes); final int[] magnitudes = calculateMagnitudes(truncatedData); final int[] outerScaledData = scaleData(magnitudes, OUTER_SCALE_TARGET); final int[] innerScaledData = scaleData(magnitudes, INNER_SCALE_TARGET); final int[] outerAveragedData = averageData(outerScaledData); final int[] innerAveragedData = averageData(innerScaledData);

this.outerPoints = calculateContours(outerPoints, outerAveragedData, OUTER_OFFSET, true); this.innerPoints = calculateContours(innerPoints, innerAveragedData, INNER_OFFSET, false);

currentFrame = 0;}

This is now drawable

Animating external input

Important!!! - interpolation

● Data arrives 20 times a second

● We want to draw 60 times a second

● We have to “make up” - interpolate 3 frames

Interpolation

private PointF[][] calculateContours(final PointF[][] currentData, final int[] averagedData, final int offset, final boolean goOutwards) {……. fillWithLinearyInterpolatedFrames(newFrames);……. return newFrames;}

private void fillWithLinearyInterpolatedFrames(final PointF[][] data) { for (int j = 0; j < NUMBER_OF_SAMPLES; j++) { final PointF targetPoint = data[NUMBER_OF_INTERPOLATED_FRAMES - 1][j]; final PointF originPoint = data[0][j]; final double deltaX = (targetPoint.x - originPoint.x) / NUMBER_OF_INTERPOLATED_FRAMES; final double deltaY = (targetPoint.y - originPoint.y) / NUMBER_OF_INTERPOLATED_FRAMES;

for (int i = 1; i < NUMBER_OF_INTERPOLATED_FRAMES - 1; i++) { data[i][j] = new PointF((float) (originPoint.x + i * deltaX), (float) (originPoint.y + i * deltaY)); } }

for (int i = 1; i < NUMBER_OF_INTERPOLATED_FRAMES - 1; i++) { data[i][NUMBER_OF_SAMPLES] = data[i][0]; }}

Drawing - nothing unusual

@Override protected void onDraw(Canvas canvas) { super.onDraw(canvas);

drawContour(canvas, outerPoints, currentFrame, outerPaint); canvas.drawCircle(center.x, center.y, radius, middlePaint); drawContour(canvas, innerPoints, currentFrame, innerPaint);

currentFrame++; if (currentFrame >= NUMBER_OF_INTERPOLATED_FRAMES) { currentFrame = NUMBER_OF_INTERPOLATED_FRAMES - 1; } }

All together

20 Hz

Visualizer

Average Scale Filter Interpolate 60 Hz

onDraw

CONCLUSION

Conclusion

● Animation is change of state through time

● State can be anything from color to position

● Target 60 (or higher) fps main loop, beware of frame drops

● Pre-calculate whole frameset or calculate frame by frame

● Take it slow, make demo app, increment step by step

● Use code examples as a starting point and inform me where are memory

leaks :)

QA

QATo save myself from having to answer embarrassing questions face to face

● Have you measured how much does it suck life out of battery? - No, but

we’ve noticed it does

● Why don’t you use OpenGL or something. - It’s next level, this is first

approximation

● What about object animators - Same amount of code, took me same

amount of time, more expensive, less flexible if you know what are you

doing. Can’t apply to the equalizer. It is more OO approach though.

Presentation: SpeakerDeck

AssetsSource code: https://bitbucket.org/fiveminutes/homan-demo/

http://five.agency/about/careers/