Camera2 API, SHIM, and HAL 3.2 in Android 5.1
-
Upload
cheng-hsien-chen -
Category
Software
-
view
298 -
download
4
Transcript of Camera2 API, SHIM, and HAL 3.2 in Android 5.1
![Page 1: Camera2 API, SHIM, and HAL 3.2 in Android 5.1](https://reader033.fdocuments.net/reader033/viewer/2022052209/587122251a28abe4448b54b3/html5/thumbnails/1.jpg)
OUTLINE
Android camera API history
What’s new of Camera API 2
HAL 3.2
Titan Camera API 2 situation
Solutions & Hacks
Conclusions
![Page 2: Camera2 API, SHIM, and HAL 3.2 in Android 5.1](https://reader033.fdocuments.net/reader033/viewer/2022052209/587122251a28abe4448b54b3/html5/thumbnails/2.jpg)
API Lv 21
Camera API 1Camera API 1(camera)Camera API 2(camera2)
API Lv ##
Camera API 2
HISTORY
![Page 3: Camera2 API, SHIM, and HAL 3.2 in Android 5.1](https://reader033.fdocuments.net/reader033/viewer/2022052209/587122251a28abe4448b54b3/html5/thumbnails/3.jpg)
ICS
HAL 1.0
JB 4.2 JB 4.3
HAL 1.0 - 3.1HAL 3.2
HAL 1.0HAL 2.0HAL 3.0
KK
HAL 1.0HAL 2.0HAL 3.0HAL 3.1
L
HISTORY
![Page 4: Camera2 API, SHIM, and HAL 3.2 in Android 5.1](https://reader033.fdocuments.net/reader033/viewer/2022052209/587122251a28abe4448b54b3/html5/thumbnails/4.jpg)
WHAT’S NEW OF CAMERA API 2?
![Page 5: Camera2 API, SHIM, and HAL 3.2 in Android 5.1](https://reader033.fdocuments.net/reader033/viewer/2022052209/587122251a28abe4448b54b3/html5/thumbnails/5.jpg)
camera & camera2 are both available to App Developer
BUT - only HAL 3.2 enables all features/properties of camera2
Shim layer
being used to as the Facade layer between new camera2 API and older HAL 1.0-3.1 by using camera API
WHAT’S NEW OF CAMERA API 2?
![Page 6: Camera2 API, SHIM, and HAL 3.2 in Android 5.1](https://reader033.fdocuments.net/reader033/viewer/2022052209/587122251a28abe4448b54b3/html5/thumbnails/6.jpg)
WHAT’S NEW OF CAMERA API 2?
![Page 7: Camera2 API, SHIM, and HAL 3.2 in Android 5.1](https://reader033.fdocuments.net/reader033/viewer/2022052209/587122251a28abe4448b54b3/html5/thumbnails/7.jpg)
WHAT’S NEW OF CAMERA API 2?
much more metadata, and per-frame metadata.
more direct controls on sensor and Post Processing
Synchronisation of setting and captures
Full rate, full-res captures in various formats
Support raw image capturing and saving
Camera API 1 is necessary for legacy applications
![Page 8: Camera2 API, SHIM, and HAL 3.2 in Android 5.1](https://reader033.fdocuments.net/reader033/viewer/2022052209/587122251a28abe4448b54b3/html5/thumbnails/8.jpg)
WHAT’S NEW OF CAMERA API 2?
![Page 9: Camera2 API, SHIM, and HAL 3.2 in Android 5.1](https://reader033.fdocuments.net/reader033/viewer/2022052209/587122251a28abe4448b54b3/html5/thumbnails/9.jpg)
WHAT’S NEW OF CAMERA API 2?
![Page 10: Camera2 API, SHIM, and HAL 3.2 in Android 5.1](https://reader033.fdocuments.net/reader033/viewer/2022052209/587122251a28abe4448b54b3/html5/thumbnails/10.jpg)
HAL 3.2
support vendor tags by camera module HAL
support partial metadata result
enables all features/properties of camera2
Not support stream configurations for still captures
Framework assumes high-res JPEG stream can be configured at all times without showing down preview framerate.
![Page 11: Camera2 API, SHIM, and HAL 3.2 in Android 5.1](https://reader033.fdocuments.net/reader033/viewer/2022052209/587122251a28abe4448b54b3/html5/thumbnails/11.jpg)
HAL 3.2
HAL 3.2
FULL mode
LIMITED mode
HAL 1.0 - 3.1
LEGACY Mode
![Page 12: Camera2 API, SHIM, and HAL 3.2 in Android 5.1](https://reader033.fdocuments.net/reader033/viewer/2022052209/587122251a28abe4448b54b3/html5/thumbnails/12.jpg)
HAL 3.2
![Page 13: Camera2 API, SHIM, and HAL 3.2 in Android 5.1](https://reader033.fdocuments.net/reader033/viewer/2022052209/587122251a28abe4448b54b3/html5/thumbnails/13.jpg)
HAL 3.2
![Page 14: Camera2 API, SHIM, and HAL 3.2 in Android 5.1](https://reader033.fdocuments.net/reader033/viewer/2022052209/587122251a28abe4448b54b3/html5/thumbnails/14.jpg)
HAL 3.2
![Page 15: Camera2 API, SHIM, and HAL 3.2 in Android 5.1](https://reader033.fdocuments.net/reader033/viewer/2022052209/587122251a28abe4448b54b3/html5/thumbnails/15.jpg)
Step 1Step 2Step 3
TITAN CAMERA API 2 SITUATION
![Page 16: Camera2 API, SHIM, and HAL 3.2 in Android 5.1](https://reader033.fdocuments.net/reader033/viewer/2022052209/587122251a28abe4448b54b3/html5/thumbnails/16.jpg)
Step 1Step 2Step 3
TITAN CAMERA API 2 SITUATION
![Page 17: Camera2 API, SHIM, and HAL 3.2 in Android 5.1](https://reader033.fdocuments.net/reader033/viewer/2022052209/587122251a28abe4448b54b3/html5/thumbnails/17.jpg)
TITAN CAMERA API 2 SITUATION
CaptureRequest
frame meta
CaptureRequest
frame meta
camera2 HAL 3
CaptureRequest
frame meta
CaptureRequest
frame meta
CaptureRequest
frame meta
CaptureRequest
frame meta
![Page 18: Camera2 API, SHIM, and HAL 3.2 in Android 5.1](https://reader033.fdocuments.net/reader033/viewer/2022052209/587122251a28abe4448b54b3/html5/thumbnails/18.jpg)
TITAN CAMERA API 2 SITUATION
CaptureRequest
frame meta
CaptureRequest
frame meta
camera2 HAL 1
CaptureRequest
frame meta
CaptureRequest
frame meta
SHIM
CaptureRequest
frame meta
CaptureRequest
frame meta
![Page 19: Camera2 API, SHIM, and HAL 3.2 in Android 5.1](https://reader033.fdocuments.net/reader033/viewer/2022052209/587122251a28abe4448b54b3/html5/thumbnails/19.jpg)
TITAN CAMERA API 2 SITUATION
Arcsoft involved to use camera2 only for its camera app.
QCT never support YUV sensor with HAL 3, and Titan’s camera solution was forced to use HAL 1.
Vertu decide to use HAL 1 + shim + camera 2 in Apr 2015
Vertu would like to have a Legacy+ device.
![Page 20: Camera2 API, SHIM, and HAL 3.2 in Android 5.1](https://reader033.fdocuments.net/reader033/viewer/2022052209/587122251a28abe4448b54b3/html5/thumbnails/20.jpg)
TITAN CAMERA API 2 SITUATION
Arcsoft involved to use camera2 only for its camera app.
QCT never support YUV sensor with HAL 3, and Titan’s camera solution was forced to use HAL 1.
Vertu decide to use HAL 1 + shim + camera 2 in Apr 2015
Vertu would like to have a Vertu specific Legacy+ mode for Titan
![Page 21: Camera2 API, SHIM, and HAL 3.2 in Android 5.1](https://reader033.fdocuments.net/reader033/viewer/2022052209/587122251a28abe4448b54b3/html5/thumbnails/21.jpg)
TITAN CAMERA API 2 SITUATION
LEGACY+ mode
slowmo/HFR support
Cont. shot/ZSL/BurstCap support (abandoned by Vertu at wk 41)
PICAUTO support
AS HDR support
YUV420 to NV12/21, not YV12 format
![Page 22: Camera2 API, SHIM, and HAL 3.2 in Android 5.1](https://reader033.fdocuments.net/reader033/viewer/2022052209/587122251a28abe4448b54b3/html5/thumbnails/22.jpg)
TITAN CAMERA API 2 SITUATION
mediaserver
android.hardware.Camera
android.hardware.camera2.*
android.hardware.camera2.legacy.*
CameraService(Native)
Camera Module & Device HAL
App
Camera2 Legacy Mode
SHIM Layer
![Page 23: Camera2 API, SHIM, and HAL 3.2 in Android 5.1](https://reader033.fdocuments.net/reader033/viewer/2022052209/587122251a28abe4448b54b3/html5/thumbnails/23.jpg)
TITAN CAMERA API 2 SITUATION
CameraDeviceUserShimCameraManager
LegacyCameraDevice
RequestThreadManager
GLThreadManager
CaptureCollector
![Page 24: Camera2 API, SHIM, and HAL 3.2 in Android 5.1](https://reader033.fdocuments.net/reader033/viewer/2022052209/587122251a28abe4448b54b3/html5/thumbnails/24.jpg)
TITAN CAMERA API 2 SITUATION
RequestThreadManager
GLThreadManager
LegacyMetadataMapper
SurfaceTextureRender
LegacyRequestMapper
LegacyResultMapper
LegacyXXXMapper
LegacyCameraDevice
![Page 25: Camera2 API, SHIM, and HAL 3.2 in Android 5.1](https://reader033.fdocuments.net/reader033/viewer/2022052209/587122251a28abe4448b54b3/html5/thumbnails/25.jpg)
SOLUTIONS & HACKS
LegacyMetadataMapper
be used for create camera static characteristic from HAL 1
Legacy+ mode
convert HFR support from HAL 1 to HIGH_SPEED_VIDEO property and export it to camera characteristic
export AS HDR, PICAUTO to camera characteristic
export scene mode FOOD/BACKLIT to camera characteristic
![Page 26: Camera2 API, SHIM, and HAL 3.2 in Android 5.1](https://reader033.fdocuments.net/reader033/viewer/2022052209/587122251a28abe4448b54b3/html5/thumbnails/26.jpg)
SOLUTIONS & HACKS
LegacyRequestMapper
be used for mapping camera2 request to parameters for HAL 1
Legacy+ mode
adjust fps and HFR mode from HIGH_SPEED_VIDEO setting
set AS HDR and PICAUTO flag
wrapper and convert VERTU specific scene mode to pass CTS
Cont. shot/ZSL/Burst shot request process
![Page 27: Camera2 API, SHIM, and HAL 3.2 in Android 5.1](https://reader033.fdocuments.net/reader033/viewer/2022052209/587122251a28abe4448b54b3/html5/thumbnails/27.jpg)
SOLUTIONS & HACKS
LegacyResultMapper
be used for mapping camera2 request to parameters for HAL 1
Legacy+ mode
based on HIGH_SPEED_VIDEO setting, convert correct camera2 result back.
based on AS HDR and PICAUTO flag, convert correct camera2 result back.
wrapper and convert VERTU specific scene mode to pass CTS
![Page 28: Camera2 API, SHIM, and HAL 3.2 in Android 5.1](https://reader033.fdocuments.net/reader033/viewer/2022052209/587122251a28abe4448b54b3/html5/thumbnails/28.jpg)
SOLUTIONS & HACKS
More and more?
camera2 settings are stored in native space and extensions are supported by vendor tags. (not available for Legacy mode)
We must revise the accessor to native space for extra supported features with Java space support
camera native framework need to be extended (LegacyXXXMapper)
HAL 1 must be greatly modified for supports of all extra features.
![Page 29: Camera2 API, SHIM, and HAL 3.2 in Android 5.1](https://reader033.fdocuments.net/reader033/viewer/2022052209/587122251a28abe4448b54b3/html5/thumbnails/29.jpg)
SOLUTIONS & HACKS
More and more?
Performance for shim is really not good
parts of shim are not completed, cause most of all stability problem
not only HAL 1, but underly QCOM camera system also need to be customised repetitively
![Page 30: Camera2 API, SHIM, and HAL 3.2 in Android 5.1](https://reader033.fdocuments.net/reader033/viewer/2022052209/587122251a28abe4448b54b3/html5/thumbnails/30.jpg)
CONCLUSIONS
Shim is just a start, what is the next?
Performance tuning, shim is really not good in performance
Complete the shim, parts of shim are not done by Google, which cause most of all stability problem
Go to lower layer. Not only HAL 1, but underly QCOM camera system also need to be customised repetitively
PICAUTO Integration
AS HDR Integration