Image Sensor Porting Guide

WCP/MS/SA1MC Lin

www.RD3721.com

Outline

IntroductionHW InterfaceSW InterfaceSCCBSensor&ISP timingAE/AWB algorithm and banding effectCode Review

www.RD3721.com

Introduction

Camera Task6219 ISP

isp_if.c.h,ae_awb.c.h,camera_3a_core.c

Driverimage_sensor.c.h

Tuningcamera_para.c

Experience80 & 20 theorem

20% time gives to the unimportant 80% task.80% time gives to the important 20% task.

AE AWB

6219 ISP

control tuning

www.RD3721.com

HW Interface

Signal Bus

VSYNC

HSYNC

SCLK

SDATA

XCLK

PCLK

8 or 10

DATA

PWDN

POWER

6219ISP

Sensorwww.RD3721.com

SW Interface

void (* sensor_init)(void)Initialize sensor and power on sensor module.Sensor default Configuration.The first executed API.

void (* sensor_power_off) (void)Clean sensor state and power off sensor module.www.RD3721.com

SW Interfacevoid (* get_sensor_id)

kal_uint8 *sensor_write_idkal_uint8 *sensor_read_id

To report sensor R/W ID, but don’t care it now.SCCB(I2C) R/W has been customized.www.RD3721.com

SW Interface

void (* get_sensor_size)kal_uint16 *sensor_widthkal_uint16 *sensor_height

To report image sensor resolution.IMAGE_SENSOR_SXGA_WIDTHIMAGE_SENSOR_SXGA_HEIGHTwww.RD3721.com

SW Interface

void (* get_sensor_period)kal_uint16 *pixel_numberkal_uint16 *line_number

To report the period of image sensor in a frame.VGA_PERIOD_PIXEL_NUMSVGA_PERIOD_LINE_NUMS

These are the parameters of frame rate, base-shutter and exposure control.

Base-shutter affects the banding effect.

www.RD3721.com

SW Interfacevoid (* sensor_capture_setting)void (* sensor_preview_setting)

image_sensor_exposure_window_struct *image_windowkal_uint16 grab_start_x;kal_uint16 grab_start_y;kal_uint16 exposure_window_width;kal_uint16 exposure_window_height;kal_uint16 image_target_width;kal_uint16 image_target_height;kal_uint16 exposure_pixel;kal_uint16 exposure_line;kal_uint8 digital_zoom_factor;

kal_uint8 image_mirrorNormal, H-Flip, V-Flip, HV-Flip.

www.RD3721.com

SW Interface

void (* write_sensor_reg)kal_uint32 addr;kal_uint32 para;

kal_uint32 (* read_sensor_reg)kal_uint32 addr;

SCCB R/W interface for CCT access.Sensor address/parameter bit width ?

Micron(9M011,9M111)8bits/16bits(9M111 has 3 page address i.e. 3*0xFF)

OV(OV9650,OV7660)8bits/8bits

www.RD3721.com

SW Interface

void (* set_sensor_eshutter)kal_uint16 eshutter;

To set sensor exposure time.Important API associated with AE algorithm, banding.Depends on sensor characteristic.www.RD3721.com

SW Interface

void (* set_camera_night_mode)kal_bool enable;

To turn ON/OFF sensor night mode.Double MAX_EXPOSURE_LINES.Increase sensor global gain.www.RD3721.com

SCCB(I2C)

Serial Camera Control Bus DescriptionStart bitEnd bitDon’t care bit

ACT : Low signal activate by sensor when writing.NACT : High signal activate by host when reading.

SCLK

SDATA

www.RD3721.com

SCCB(I2C)

ProtocolOV

Write(PARA to ADDR)

Read(PARA from ADDR)

OV SCCB is standard I2C.

S WID ADDR PARA E

S WID ADDR E S RID PARA E

A A A

A A NA

www.RD3721.com

SCCB(I2C)Micron 8bit

Write(PARA to ADDR)

Read(PARA from ADDR)

Micron 16bitWrite(PARA to ADDR)

Read(PARA from ADDR)

S WID PARA1 EA ADDR A S AA WID A 0xF1 PARA2 A

S WID A ADDR A S RID A PARA1 N

S WID A 0xF1 A S RID A NPARA2 E

S WID A ADDR A PARA1 A PARA2 A E

S WID A ADDR A S RID A PARA1 NPARA2A E

www.RD3721.com

SCCB(I2C)MT6219 HW SCCB

WriteSet Length 3Enable SCCBSet WIDSet ADDRSet PARA

ReadSet Length 2Enable SCCBSet WIDSet ADDREnable SCCBSet RIDSet “0”Get PARA from register.

www.RD3721.com

Sensor&ISP timing

Sync. Signal

Set VSYNC polarity LOWSet HSYNC polarity LOW.

VSYNC

HSYNCwww.RD3721.com

Sensor&ISP timingTiming adjustment

Set CMOS Sensor Polarity LOW.SET_TG_OUTPUT_CLK_DIVIDER(1);SET_CMOS_RISING_EDGE(0);SET_CMOS_FALLING_EDGE(1);SET_TG_PIXEL_CLK_DIVIDER(3);SET_CMOS_DATA_LATCH(2);

PCLK

XCLK

MCLK0 1 2 3

www.RD3721.com

Sensor&ISP timing

First Valid Pixel/Line

VSYNC

HSYNC

First Valid Line

PCLK

XCLK

First Valid Pixel

www.RD3721.com

Sensor&ISP timing

Bayer pattern

SET_BAYER_PATTERN(FIRST_GRAB_COLOR)#define FIRST_GRAB_COLOR XX

Gr00

R01

B10

Gb11www.RD3721.com

Sensor&ISP timing

Valid Frame

VSYNC Valid Frame

SENSOR_PERIOD_PIXELSHSYNC

HBLANKING

VBLANKING

www.RD3721.com

AE/AWB Algorithm

LISR(AE/AWB HW)To get AE/AWB information at each frame end.

weighted average luminance for AE.R, G, B accumulation for AWB.

HISR(AE/AWB FW)To set AE/AWB parameter.

AE/AWB process.To set AWB gain.To set sensor exposure time.To set ISP pre-gain.

www.RD3721.com

AE/AWB Algorithm

AE/AWB Window

AE “Window size”, “region” and “weighting” are configurable, and reports the luminance.AWB window reports the R, G, B accumulation.

w1 w2 w3

w4 w5 w6

w7 w8 w9www.RD3721.com

AE/AWB Algorithm

AE Process(FW)AE banding factor.

Base-shutter should be at lease 1/2f.Base-shutter=PIXEL_CLK/(2f*SENSOR_PERIOD_PIXELS)

VSYNC ValidFrame

SENSOR_PERIOD_PIXELSHSYNCwww.RD3721.com

AE/AWB Algorithm

AE Process(FW)AE computation.

Exposure time(shutter), pre-gain.Linear interpolation.

avg_lum = weighted average of 9 windows.

Exposure time

Luminance

Y1

Y2Yt

T2 T1Tt

Desired Luminance

www.RD3721.com

AE/AWB Algorithm

AWB Process(FW)5 light source

7500 (D75) – CLOUD6500 (D65) – DAYLIGHT5000 (D50) – INCANDESCENCE4400 (CWF) – FLUORESCENT2800 (A) – TUNGSTEN

AWB R, G, B gain computation.AWB R gain = AccG/AccRAWB G gain = 1AWB B gain = AccG/AccB( AccG > AccR && AccG > AccB )

www.RD3721.com

AE/AWB Algorithm

AE/AWB parameter setting timing(HISR)

Three of them should be set precisely, or AE vibration occurred.

ae_awb_cal_period=4

VSYNC

ae_setting_gain_delay_frame=2ae_setting_shut_delay_frame=1

F0 F1 F2 F3 F0

ae_setting_cal_delay_frame=0www.RD3721.com

Code Review

PreviewISP/Sensor CLK setting.Mode/Resolution configurationActive window configuration

CaptureAll you did in “Preview”Mode switching

VGA/SXGAzoom<2x and 2x<=zoom<=4x.

Set ShutterControl the exposure time precisely.Depends on sensor characteristic.

www.RD3721.com

Case Study I (1/2)

CMMCLKIt should be a smooth waveform without glitch.

Micron 9M011 for example. MCLK=PCLK

www.RD3721.com

Case Study I (2/2)Stripe(RED or BLUE Line) everywhere on preview window.

Root Cause : sensor raw data output offset.

SolutionFine tune CMMCLK driving current.Add capacitor to make waveform smooth.Not to modify data latch point of ISP.(offset occurs randomly.)

Gr R

B Gb

Bayer Pattern

First Pixel ID = B

www.RD3721.com

Case Study II

VSYNCWhy OV9650 not be suitable for 30fps ?

VSYNC

HSYNCISP IDEL IRQ

VBLANK

I2C set shutter & gain

ISP IDEL IRQ

www.RD3721.com

Case Study III

DROP FRAMEOV7660 DROP FRAME bug.

If shutter step changes over a limitation, one frame will be dropped automatically.

VSYNC

HSYNC

dropped unclearly

www.RD3721.com