UplinQ - enhance qualcomm® snapdragon™ audio using android audio ap_is
-
Upload
satya-harish -
Category
Internet
-
view
214 -
download
5
Transcript of UplinQ - enhance qualcomm® snapdragon™ audio using android audio ap_is
1
Enhance Qualcomm® Snapdragon™ audio using Android Audio APIs
Harikishan Desineni, Director of Engineering Qualcomm Innovation Center, Inc.
Qualcomm Snapdragon is a product of Qualcomm Technologies, Inc.
3
Agenda
Session overview
1 How Qualcomm
Technologies incorporated the
digital signal processor (DSP)
2 Snapdragon Audio
SDK
3 Open DSP program
4 Demo
5
4
Includes the following sound effects:
− Equalizer (EQ) – Adjusts the sound level of certain frequency ranges by use of audio filters to improve the quality of overall perceived sounds
− Virtualizer – Virtualizes audio channels to compensate for the mismatch between the loudspeaker setup used when producing the audio, and the audio output device used when the audio is played back by the end user
− Reverb – Simulates the reverberation that results from reflections from surrounding walls and objects in an environment to make the sound more live and real
− BassBoost – Dynamically boosts low-frequency content of an audio signal to achieve better listening experience on bass for a headset device
How Qualcomm® Snapdragon™ Audio+ uses the DSP
Qualcomm Snapdragon Audio+ is a product of Qualcomm Technologies, Inc.
5
Media playback block diagram
Surface Flinger Audio Flinger
MDP/MDDI
Media Player
Media Player Service
StageFright
User space Java
User space Native
Kernel space
DSP
Video OMX Component
Audio OMX Component
Video HW
PCM-audio
Audio HAL
Audio HW
Media Server
Audio Drivers Display Drivers Video Drivers
6
Audio framework: effects offloaded to DSP
Media Player
Media Player Service
Awesome Player
Audio Track AudioFlinger
Qualcomm® Hexagon™
AudioEffects
Software AudioEffect
BassBosst
Virtualizer
ReverB
X
Qualcomm Hexagon is a product of Qualcomm Technologies, Inc.
7
Snapdragon Audio SDK
8
Snapdragon Audio SDK Fluence™ Pro EC NS detection Snapdragon SDK for Android: Device Info
The Device Info feature set is designed to allow an application to detect if the device it is running on has a Snapdragon processor. This insight lets you make informed decisions about how to take full advantage of features enabled by Snapdragon processors.
Using Device Info is designed to let you tap into the following hardware features to help improve the performance of your applications:
− Is Snapdragon This feature is designed to allow the application to detect at runtime if the device has a Snapdragon processor
− Echo Cancellation/Noise Suppression
Echo Cancellation/Noise Suppression is designed to foster better audio communication quality at lower power by allowing full duplex mode during VOIP calls. These features are specifically tuned to each device, and use technology available on normal mobile voice calls so you can offload CPU/software-based echo cancellation and noise suppression.
Fluence Pro is a product of Qualcomm Technologies, Inc.
9
Snapdragon Audio SDK com.qualcomm.snapdragon.sdk.deviceinfo
Class Description
DeviceInfo The DeviceInfo API provides a mechanism to query a device about the presence of certain functionalities.
Class Description
DeviceInfo.Features The list of features that can be detected with the function isFeatureSupported(Feature, Context)
Sample code is subject to SDK license agreement. Source: http://developer.qualcomm.com
10
Snapdragon Audio SDK
public static final DeviceInfo.Features FEATURE_FLUENCE
Noise Cancellation Technology
− If isFeatureSupported returns true for this feature, then it can be assumed that echo cancellation is supported on the voice path.
− If isFeatureSupported returns true for this feature, and the standard Android API android.media.audiofx.AcousticEchoCanceler.isAvailable() also returns true, then it can be assumed that echo cancellation is also available on the audio path.
11
Snapdragon Audio SDK
FEATURE_SURROUND_SOUND_RECORDING
public static final DeviceInfo.Features FEATURE_SURROUND_SOUND_RECORDINGSurround Sound Recording is designed to allow for 5.1 surround sound input using existing microphones on the device
12
OpenDSP program
13
Audio Modules – Common Interface
Common interface for audio modules that are defined to ease integration with the Elite Framework
Audio Postprocessor Interface (APPI) for audio processing algorithms
Common Audio Processor Interface (CAPI) for audio codec modules
Processing
Algorithm Wrapper with common interface (APPI)
Codec Decoder / Encoder
Wrapper with common interface (CAPI)
Note: The terms algorithm, module, and library are used interchangeably in this presentation.
14
Audio Modules – Static and Dynamic
Two types of modules, based on loading: − Static modules (*.lib)
− Linked to the aDSP image at compile time
− Symbol are resolved at compile time
− aDSP image must be recompiled when a module changes
− Dynamic modules (*.so) − Loaded at run-time into aDSP heap memory
− Symbols are resolved when the module is loaded into memory
− aDSP image does not need to be recompiled when a module changes
Each module is identified by unique global module ID
Audio static modules are added to the AMDB at boot time
Audio dynamic modules are added to the AMDB at run time
APPI mod-n (*.so)
CAPI mod-3 (*.so)
CAPI mod-2 (*.lib)
APPI mod-1 (*.lib)
15
Per Object PostProcessing (POPP)
Per Object PreProcessing (POPreP )
Processing chain per audio stream
Default POPP topologies: − Playback session
− Record session
DSP audio mixer
Downmix if needed
Resampler Audio Encoder Volume
DSP audio mixer
Volume Audio Decoder Resampler
POPP Topologies
16
Common Object PostProcessing (COPP)
Common Object PreProcessing (COPreP)
Processing chain per device
Default COPP topologies: − Playback session
− Record session
AFE Audio
pregain
AFE CodecRX Gain
DSP Audio Mixer
Audio Volume
DSP Audio Mixer
COPP Topologies
17
Custom Audio Topology – Static and Dynamic Two types of audio topologies: − Static topologies
− Created at compile time
− aDSP image must be recompiled if the topology changes
− Dynamic topology − Created at run time
− aDSP image does not need to be recompiled if the topology changes
− APIs are defined for adding custom dynamic topologies from the applications processor:
Custom topology is designed to use any module defined in the AMDB by using Module IDs
Customized topology with an OEM algorithm
ASM_CMD_ADD_TOPOLOGIES // for POPP and POPREP ADM_CMD_ADD_TOPOLOGIES_V5 // for COPP and COPREP
Audio Mixer Volume OEM
Algorithm Audio Decoder Resampler
18
Audio Dynamic Topology How are dynamic topologies added at run time?
APR
ADM
Topo DB
ASM_CMD_ADD_TOPOLOGIES T2 (-M1-M4-), T3 (-M1-M2)
topo_db_add_topologies()
T1
T2
T3
aDSP
AMDB
M1
M2
M3
M4
New audio sessions can be created using T2 and T3 topologies after adding them to the topology database
Client Processor
ASM
19
ISV Module1 ISV Module2
Application
HAL
RTAC Kernel Driver
APR Handler DSP
Payload
ASM_CMD_SET_PP_PARAMS/ AMD_CMD_SET_PP_PARAMS
RTAC Userspace module Send_rtac()
ACDB Loader
Acdb_store()
Acdb_send()
Acdb_get()
acdb file
Get/setParameter(key”,value)
Software Architecture – Block Diagram
20
Software Architecture – Android APIs Existing set parameter android API extended to have new key value pairs
Setparameter() with persistence − set_parameter(key=”ISV_PERSIST_CFG”, value=” DataBinary, *”);
− Data Binary – Entire calibration data binary that is stored in calibration database. Even if user changes single parameter through UI, the app is responsible for packaging the entire data binary for persistence
Setparameter() without persistence − set_parameter(key=”ISV_NOPERSIST_CFG”, value, “List_MID_PID, *”);
− List_MID_PID – List of dynamically modified calibration to be applied on ISV modules in ADSP. This could be an array of Module ID, Parameter ID and data, or could be only one set of ModuleId, Parameter ID and values. Here, the data is not persisted in audio calibration database. The app is responsible for packaging the modified modules and parameters.
21
Demo
22
developer.qualcomm.com
Augmented reality
Context awareness
Peer to peer
Wireless health
Graphics & gaming
Computer vision
Android applications
App development SDKs & tools
Marketing opportunities
Case Studies & Tutorials
News & Information
Hardware development platforms
Forums & technical support
Tools & resources to help developers build, integrate, and optimize
Qualcomm Developer Network
23
For more information on Qualcomm, visit us at: www.qualcomm.com & www.qualcomm.com/blog
Thank you
FOLLOW US ON:
© 2013-2014 Qualcomm Technologies, Inc. and/or its affiliated companies. All rights reserved. Qualcomm, Snapdragon, Hexagon and Fluence are trademarks of Qualcomm Incorporated, registered in the United States and other countries. Uplinq is a trademark of Qualcomm Incorporated. All trademarks of Qualcomm Incorporated are used with permission. Other product and brand names may be trademarks or registered trademarks of their respective owners. References in this presentation to “Qualcomm” may mean Qualcomm Incorporated, Qualcomm Technologies, Inc., and/or other subsidiaries or business units within the Qualcomm corporate structure, as applicable. Qualcomm Incorporated includes Qualcomm’s licensing business, QTL, and the vast majority of its patent portfolio. Qualcomm Technologies, Inc., a wholly-owned subsidiary of Qualcomm Incorporated, operates, along with its subsidiaries, substantially all of Qualcomm’s engineering, research and development functions, and substantially all of its product and services businesses, including its semiconductor business, QCT.