Adapting Chronotron Speed Changer to Windows Phone 8.1

Meet the App

Application Architecture

Outlook After a Quick Code Review

Universal Apps with Visual Studio

Implementation Approach ◦ User Interface and Navigation ◦ Custom Controls ◦ ViewModel ◦ Native Components

Project Ne10

Other Considerations

Conclusion

Chronotron Speed Changer is an audio/video speed changer app. It allows you to manipulate tempo and pitch independently during playback, i.e. change the tempo of audio and video files without affecting pitch (time stretching) and/or change pitch without affecting tempo (key change).

http://www.chronotron.com

User Interface ViewModel

Native MF Components

The implementation is based on the MVVM pattern.

The audio/video processing algorithms are implemented using native Media Foundation components.

The easy ◦ Simple UI: one XAML page, one Settings flyout

The challenging ◦ Three custom controls

◦ Native components

Each project in the solution is split in three: ◦ A Windows Store Project containing code specific to

the Windows Store app

◦ A Windows Phone Project containing code specific to the Windows Phone app

◦ A Shared Project containing code shared across the above two projects

Specific or Shared code? ◦ User Interface and Navigation

◦ Custom Controls

◦ ViewModel

◦ Native Components

◦ Other

Implementation Approach: Specific Code ◦ Limited screen real state

◦ Major differences in navigation approach and UI behavior (e.g. FileOpenPicker)

◦ Some XAML code could be copied over from the Windows Store project, though

Implementation Approach: Shared Code ◦ Minor differences in XAML templates - not all

theme resources are available Change the control templates to use other resources and

styles

◦ Minor differences in call sequence – e.g. on Windows Phone property binding may happen before OnApplyTemplate

Program defensively to accommodate for both XAML implementations

No #ifdef were required

Implementation Approach: Shared Code ◦ Written in C#

◦ Only one “#ifdef” construct was needed

StorageFile.GetThumbnailAsync always fails on Windows Phone 8.1

◦ Part of the application settings viewmodel is specific in order to allow different default settings on WP

Implementation Approach: Shared Code ◦ Compiling the native code for Windows Phone was

relatively straightforward

The actual processing algorithms were already developed in portable C++; x86-specific optimizations had been implemented as an extension

Media Foundation is available on WP 8.1

A few memory management APIs aren't available on WP (e.g. CreateFileMappingFromApp), so a few #ifdefs were required

Real-time performance was the main area of work ◦ The good

VC++ does a good job at vectorizing simple loops

◦ The not-so-good Memory constraints

Solution: Limit memory-hungry functionality, like reverse playback

Trigonometric functions in the MSVC Runtime Library are slow on ARM Solution: Use lookup tables and other techniques

Video processing on the CPU is prohibitively slow Solution: Use DXGI for video processing (also for the benefit of the

Windows Store app!)

The Windows Store version relies heavily on x86 SIMD optimization, in particular for the FFT Solution: Rely on Project Ne10. What is it?

A library of well-known algorithms optimized for the ARM architecture Developed primarily for Android, but can be compiled with VC++ with

minimal changes Free for non-commercial and commercial usage Available at http://projectne10.org

Number of Samples My own FFT written in C Ne10 FFT using NEON intrinsics

2048 0.176ms 0.159ms

4096 0.393ms 0.349ms

Example: 32-bit floating point real FFT/IFFT runtime on Qualcomm Snapdragon 400, average over 10K calls:

The 10-12% improvement is very welcome considering the app performs up to 3 FFT per channel per frame!

The Windows Store does not support specifying a trial period for Windows Phone apps ◦ A trial mode offering limited functionality had to be

implemented

Testing ◦ Tested on a Nokia Lumia 630

The Windows Store app could be effectively used as starting point to develop the Windows Phone app ◦ Most of the code could be shared (with a few caveats), even native code!

Meet the new app