SMIA 1.0 Part 3.1: Software and application...

SMIA 1.0 Part 3.1: Software and application

specification

Nokia & ST Confidential

SMIA 1.0 Part 3.1: Software and application specification

DISCLAIMER

The contents of this document are copyright © 2004 Nokia Corporation, ST Microelectronics NV and their licensors. All rights reserved. You may not copy, modify nor distribute this document without prior written consent by Nokia and ST. No license to any Nokia’s, ST’s or their licensor’s intellectual property rights are granted herein. YOU ACKNOWLEDGE THAT THIS SMIA SPECIFICATION IS PROVIDED "AS IS" AND NEITHER NOKIA, ST NOR THEIR LICENSORS MAKE ANY REPRESENTATIONS OR WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE OR THAT THIS SMIA SPECIFICATION OR ANY PRODUCT, SOFTWARE APPLICATION OR SERVICE IMPLEMENTING THIS SMIA SPECIFICATION WILL NOT INFRINGE ANY THIRD PARTY PATENTS, COPYRIGHTS, TRADEMARKS OR OTHER RIGHTS. THERE IS NO WARRANTY BY NOKIA, ST OR BY ANY OTHER PARTY THAT THE FUNCTIONS CONTAINED IN THIS SMIA SPECIFICATION WILL MEET YOUR REQUIREMENTS. LIMITATION OF LIABILITY. IN NO EVENT SHALL NOKIA, ST OR THEIR EMPLOYEES, LICENSORS OR AGENTS BE LIABLE FOR ANY LOST PROFITS OR COSTS OF PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES, PROPERTY DAMAGE, PERSONAL INJURY, LOSS OF PROFITS, INTERRUPTION OF BUSINESS OR FOR ANY SPECIAL, INDIRECT, INCIDENTAL, ECONOMIC, COVER, PUNITIVE, OR CONSEQUENTIAL DAMAGES, HOWEVER CAUSED AND WHETHER ARISING UNDER CONTRACT, TORT, NEGLIGENCE, OR OTHER THEORY OF LIABILITY ARISING OUT OF THIS SMIA SPECIFICATION, EVEN IF NOKIA, ST OR THEIR LICENSORS ARE ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. IN THE EVENT THAT ANY EXCLUSION CONTAINED HEREIN SHALL BE HELD TO BE INVALID FOR ANY REASON AND NOKIA, ST OR THEIR LICENSORS BECOMES LIABLE FOR LOSS OR DAMAGE THAT MAY LAWFULLY BE LIMITED, SUCH LIABILITY SHALL BE LIMITED TO U.S.$50. Specifications mentioned in this publication are subject to change without notice. This document supersedes and replaces all versions previously supplied.

Nokia & ST Confidential Page ii of 27


History Version Date Author Status Notes

1.0 30-June-2004 Nokia and ST Approved

Nokia & ST Confidential Page iii of 27


Table of contents

1. Introduction ...................................................................................................................8 2. Software block diagram................................................................................................9

2.1 Software-only system.................................................................................................................. 10 2.2 System with a separate camera processor ................................................................................. 11

3. Recommended host behaviour..................................................................................11 3.1 Behavioral Sequences ................................................................................................................ 12

3.1.1 Typical Still Capture Sequence ........................................................................................... 12 3.1.2 Typical Video Capture Sequence........................................................................................ 12

3.2 Behavioural Stages ..................................................................................................................... 13 3.2.1 Power Up............................................................................................................................. 13 3.2.2 Power Down ........................................................................................................................ 14 3.2.3 Basic Initialisation ................................................................................................................ 14 3.2.4 Initialise for Viewfinder......................................................................................................... 14 3.2.5 Viewfinder ............................................................................................................................ 15 3.2.6 Initialise for Capture............................................................................................................. 16 3.2.7 Capture ................................................................................................................................ 16 3.2.8 Initialise for Video Viewfinder .............................................................................................. 17 3.2.9 Video Viewfinder.................................................................................................................. 17 3.2.10 Initialise for Video Capture............................................................................................... 17 3.2.11 Video Capture .................................................................................................................. 17

3.3 CCI Driver.................................................................................................................................... 17 3.4 CCP2 Driver ................................................................................................................................ 17 3.5 Colour Spaces............................................................................................................................. 17

4. Personality files...........................................................................................................18 4.1 Introduction.................................................................................................................................. 18 4.2 Personality File Syntax................................................................................................................ 20 4.3 Setting Specifications .................................................................................................................. 22

4.3.1 SETTING_IDENTIFIER_STRING ....................................................................................... 22 4.3.2 register_override, register_reprogram_once, register_reprogram_every ........................... 22 4.3.3 vignetting_elimination .......................................................................................................... 23 4.3.4 colour_component_sensitivities .......................................................................................... 25

4.4 Personality File Example............................................................................................................. 26 5. References...................................................................................................................27

Nokia & ST Confidential Page iv of 27


List of tables Table 1: Acronyms...................................................................................................................................vi Table 2: ECR .......................................................................................................................................... vii

Nokia & ST Confidential Page v of 27


Acronyms Abbreviations and Definitions: SMIA Standard Mobile Imaging Architecture CCI Camera Control Interface CCP2 Compact Camera Port 2 AWB Automatic White Balance AE Automatic Exposure AF Automatic Focusing DSP Digital Signal Processing DSC Digital Still Camera LCD Liquid Chrystal Display RGB Red Green Blue; primary colors

RGB444 A subtype of RGB where each color component uses 4 bits = 16 levels; a.k.a. 16K colors

RGB565 Also known as 64K colors EMC Electromagnetic Compatibility PLL Phase-Locked Loop RAM Random Access Memory VGA Video Graphics Array; common image sensor resolution of 640x480 pixels YUV Color space where the components are luminance and chrominance MPEG Moving Picture Experts Group; a family of audiovisual compression formats I/O Input / Output DMA Direct Memory Access ASCII American Standard Code for Information Interchange

Table 1: Acronyms

Nokia & ST Confidential Page vi of 27


PREFACE Specification Supersedes Earlier Documents This document contains the draft SMIA software and applications specification. Following publication of the SMIA Standard, there may be future approved errata and/or approved changes to the standard prior to the issuance of another formal revision. Incorporation of Engineering Change Requests (ECRs) The following ECRs have been incorporated into this version of the specification:

ECR DESCRIPTION

Table 2: ECR

Nokia & ST Confidential Page vii of 27



of 27

1. Introduction This is the high level document from SMIA software applications working group. It attempts to explain basic concepts on how to use SMIA image sensors in a product.

When a system is using a SMIA sensor there are basically two choices for the hardware architecture.

Host processor

CCP2

Cameraprocessor(Multimediaaccelerator)

Host processor

SDRAM

CCI CCICCP2 CCP2

CCP2

CCI

Lens

sys

tem

Lens

sys

tem SMIA

camerahead

SMIAcamerahead

A software-only system may perform image processing (color interpolation, AWB, noise filtering, antivignetting etc) and sensor control (AE, AF etc) with the host processor. This allows a low-cost implementation.

If the host processor is not very powerful, has other tasks to perform or the sensor has a large number of pixels then a separate processing unit could be used. This allows a higher performance (e.g. shorter shot-to-shot time for stills and higher frame rate for video) solution. There are many names for this kind of chips – camera processor, hardware accelerator, camera DSP, multimedia processor etc.



of 27

2. Software block diagram Regardless of the operating system some common software blocks can be identified.

Applications

Some of the most common imaging applications are

• Still camera (DSC)

• Video camera (Camcorder)

• Videophone (Video teleconferencing)

• Gallery (Album)

Java imaging applications talk to the camera server through a Java MMAPI component. See /4/ for details.

Camera server

As there may be several applications trying to use a camera it may be beneficial to have only one piece of software controlling the image sensor. This helps avoid conflicting CCI messages. It also allows one captured image to be used as the source of conversion for different kind of applications. For example a camcorder might require RGB565 for displaying on a LCD but YUV422 for a video encoder.



of 27

2.1 Software-only system

Still cameraapplication

Camera server

Imagereconstruction

library

Exposure controllibrary

Camera driver

CCP2 receiver

Video cameraapplication

Videophoneapplication

Business cardscanner

application

Camera driver

CCP2 receiverI2C transceiver

CCI driver

SMIAimage

sensor 1

SMIAimage

sensor 2

CCI

Sensoridentification and

personality fileprocessing

Java MMAPI

Surveillancecamera

application

Purikuraapplication

Image conversionlibrary

Host processor

External hardware

Gallery application

Native applications Java applications

JPEG codec

Here the camera server is taking care of several tasks. It identifies the image sensors, loads any sensor specific settings from a personality file, runs exposure control algorithm, reconstructs a RGB image from raw bayer and converts the RGB image to whatever the applications have requested. To control the image sensors it will talk to them over CCI and to capture images it talks to the CCP2 receiver hardware drivers.

/3/ describes a SMIA camera driver interface in Symbian environment.



of 27

2.2 System with a separate camera processor

Still cameraapplication

Camera server

Video cameraapplication

Videophoneapplication

Business cardscanner

application

CCP2 transmitterI2C transceiver

CCI driver

SMIAimage

sensor 1

SMIAimage

sensor 2

CCI

Java MMAPI

Surveillancecamera

application

Purikuraapplication

Image conversionlibrary

Host processor

External hardware

Camera processor

Gallery application

Native applications Java applications

Camera processordriver

CCP2 receiver

Imagereconstruction

library

Exposurecontrol library

Sensoridentification

and personalityfile processing

JPEG codec

The camera processor takes care of identifying the image sensors, capturing images, controlling exposure levels, color interpolation and outputting whatever format the camera server has requested.

If a high bandwidth back channel from the host processor to the camera processor is available then it might also take care of conversions as requested by the camera server or a gallery type of application.

3. Recommended host behaviour Setting up a sensor from power off to streaming mode is described from a hardware point of view in /1/.

This chapter describes the recommended structure and type of code required within a typical SMIA camera host. It describes actions and sequences at a hardware driver level.



of 27

3.1 Behavioral Sequences Typical SMIA camera host behavior can be split into a few unique stages. These stages can then be assembled into a sequence to produce a desired end-user effect.

3.1.1 Typical Still Capture Sequence

Though this diagram details typical still capture operation, there could be variations. Typically viewfinder operation is with a smaller image size and is relatively quick to process and display, whilst captured images are larger, therefore more processor intensive and also have to be saved. It may be desirable to capture and display a viewfinder-sized image and display this to the user whilst the full size image is captured and saved in the background.

It should also be considered that powering the camera up and down uneccesarily may have EMC implications to the host, the user may simply be aborting to alter settings before re-entering the routine. It may therefore be desireable to have power down delayed using a timer.

Basic Initialisation

START

Initialise for Viewfinder

Abort?

Viewfinder

Power Up

Initialise for Capture

No

Capture

END

Yes

Power Down

3.1.2 Typical Video Capture Sequence



of 27

Basic Initialisation

START

Initialise for Video Viewfinder

Abort?

Video Viewfinder

Power Up

Initialise for Video Capture

No

Video Capture

END

Yes

Power Down

3.2 Behavioural Stages /1/ describes among other things a generic SMIA camera register map, electrical requirements and power-up timing details.

In some stages SMIA camera registers have to be accessed, this is accomplished over CCI bus using a low level I2C driver (section 3.3).

SMIA camera image data is passed from camera to host over CCP2 bus, see section 3.4 CCP2 Driver.

3.2.1 Power Up This stage is very much host hardware specific.

Basically, power supplies to SMIA camera (VANA: Analogue supply & VDIG: Digital supply) should be setup and switched on. Then clock supplied to SMIA camera (EXTCLK) should be setup and applied (these may of course be fixed and always on in the host).



of 27

It is recommended that XSHUTDOWN (a.k.a. Vctrl, PDN) is set high at end of this sequence so that the SMIA camera is in software standby mode on completion of this step.

CCI Driver (section 3.3) should be initialized during this stage.

3.2.2 Power Down This stage is very much host hardware specific.

Please refer to /1/ for the recommended power-down procedure.

3.2.3 Basic Initialisation This involves reading and writing the camera registers to get it into a basic state of readiness.

3.2.3.1 Identify the sensor Read and store SMIA version (0x0004), manufacturer_id (0x0003), revision_number (0x0002) and model_id (0x0000 & 0x0001) registers.

3.2.3.2 Process the personality file The personality file may contain improved register settings compared to the silicon defaults.

3.2.3.3 Set up the camera’s PLL CCI communication with the sensor is possible without using a PLL but streaming over CCP2 requires programming the sensor’s PLL to output suitable clock frequencies for the sensor and the CCP2 interface. Please refer to the ‘Clock setup registers’ chapter in /1/ for details.

3.2.4 Initialise for Viewfinder Typically the host applications will use more than one size of output image. Higher level software should know what size of capture image is required and should be able to pass the required Viewfinder size down to the driver, this will usually be the same aspect ratio as the required still image, but of a size to match the host display. E.g. Capture size is VGA (640 x 480), display is 200 x 200, so viewfinder optimum size may be chosen as 160 x 120.

Viewfinder is very processor intensive, more frames per second (less processing time per frame) gives smoother (more desirable) viewfinder. Smaller image gives less data across CCP2 bus (quicker), less data to save (less RAM required) and less data to process (less processor time). But largest size that can fit upon display is desirable for end-user experience. Saved processor time could be used to enhance image to further increase quality.

3.2.4.1 Optional Image Scaling H scaler and/or V scaler on the sensor silicon may be used to decrease the image size during viewfinder. If this is the case then scaling needs to be setup at this stage. Please refer to /1/ for full details of scaling. Register set ‘Image Scaling Parameter Limit Registers’ will need to be read to fetch limits. Register set ‘Image Scaling Registers’ should be written with values to get nearest to the size required.



of 27

3.2.4.2 Optional Image Data Compression Image data may be compressed at the sensor to decrease data size over the CCP2 bus, but this will involve more processor time to uncompress data during viewfinder. Please refer to ‘Image compression’ chapter in /1/ for full details of compression. If compression is required then it needs to be setup at this stage. Register set ‘Image Compression Parameter Limit Registers’ will need to be read to fetch limits. Register set ‘Image Compression Registers’ should be written with values to enable compression.

3.2.4.3 Image Timing setup Please refer to ‘Video timing’ chapter in /1/ for full details of video timing.

Basically this will set the streaming speed (frames per second). This also determines the maximum exposure time. Consideration should be taken that the speed is as quick as possible to give smoothest (best) viewfinder, but enough time is available to process each frame. There is a trade off between speed and frame quality. Register set ‘Video Timing Parameter Limits’ should be read to determine limits and then register set ‘Video Timing Registers’ should be written to setup for desired frame timing.

Host may also desire different speeds of still capture, maybe one speed for outdoors (small exposure time required, fastest frames per second), another for indoors (medium exposure required, medium frames per second), another for night mode (long exposure time required, slowest frames per second).

Alternatively, in still capture, exposure time could be variable as part of viewfinder exposure calculation.

3.2.4.4 Start Viewfinder CCP2 driver (section 3.4) should be set up to accept image data over CCP2 bus.

Finally, the sensor should be put into streaming mode by writing to mode_select register (0x0100).

3.2.5 Viewfinder Throughout viewfinder, the camera will be in streaming mode.

Please refer to ‘Data format’ chapter of /1/ for full details of frame data format.

Please note that the top and bottom lines of each frame contain contents of all camera registers at the time of frame creation. This data should be parsed and used to determine parameters for all image processing. If this is done correctly then the same code may be re-used for any SMIA camera (VGA may be upgraded to SVGA or Mega Pixel with no code changes).

From Frame register data, the following should be used during frame processing:-

Pixel_order (0x0006), pixel_depth (0x000C), x_output_size (0x034C), y_output_size (0x034E), compression_mode (0x0500), frame_acc_greenR_value (0x2100)



of 27

As each frame arrives, the followings steps have to be performed:

3.2.5.1 Exposure calculation Please refer to ‘Integration time and gain control’ chapter of /1/ for full details of exposure control.

Average pixel value can either be read from frame statistics (frame_acc_greenR) or image data may be traversed and an average calculated. Based on the average, exposure must be calculated and new values sent to camera. Register sets ‘Integration Time Registers’ and ‘Analogue Gain Registers’ should be written.

3.2.5.2 Image conversion Each frame contains still image data in SMIA raw bayer format. This must be converted to a format required by the host display (typically RGB444, RGB565, YUV422 etc).

Data conversion techniques are beyond the scope of this document, but a bayer 640 x 480 frame may be considered as being made up of 4 pixel blocks:-

RG GB Taking top byte of Red, either Green and Blue and simply shifting and oring together will produce a quite reasonable RGB444 or RGB565 320 x 240 image.

3.2.5.3 Image enhancement Various enhancement algorithms may be run to increase quality of images. Enhancement algorithm descriptions are beyond the scope of this document.

3.2.5.4 Image display Image has to be displayed to user. This is again host specific.

3.2.6 Initialise for Capture Capture usually comes directly after viewfinder.

It is recommended that the sensor is left in streaming mode during this stage, though CCP2 driver (section 3.4) should probably be configured to disable interrupts.

Typically captured images are larger than viewfinder images, therefore H scaler and V scaler will probably need to be disabled or changed (see 3.2.4.1). Exposure and gain should now be correct for viewfinder size image. These will need to be re-calculated and written for new capture size (see 3.2.5.1).

At the end of this stage, CCP2 driver interrupts should be re-enabled so that Capture may receive camera image data.

3.2.7 Capture This should perform in a similar fashion to viewfinder in so far as it should examine camera frames. It must process frame register data to ensure exposure and size changes have taken effect. As soon as a correct frame has



of 27

been received, camera may be set to idle mode and can optionally be powered down. Frame data must now be processed and saved.

3.2.8 Initialise for Video Viewfinder This stage will be very similar to 3.2.4 Initialise for Viewfinder and may even be accomplished using the same code (see 3.5 Colour Spaces for possible differences).

3.2.9 Video Viewfinder This stage will be very similar to 3.2.5 Viewfinder and may even be accomplished using the same code (see 3.5 Colour Spaces for possible differences).

3.2.10 Initialise for Video Capture Though this is very similar to Video Viewfinder and Viewfinder, viewfinders have no end product, so the frame rate may be variable and all inter frame time is used for video enhancement. Usually video has to be processed in similar fashion to viewfinder, but is restricted in speed and must be further processed and saved.

3.2.11 Video Capture Whilst viewfinder is processed for just display, video has to be captured in a recognised standard format. Viewfinders image size is usually tailored to display size, but video is usually tailored to standard video sizes. Image data format is usually tailored for an encoder, e.g. MPEG encoders usually prefer YUV422 data format still images as input.

3.3 CCI Driver Please refer to /1/ and /2/ for full details of CCI bus.

This is the low level interface which allows the host to read from and write to the SMIA camera registers to determine / control camera behaviour.

In some hosts this may be simple, built on top of a standard hardware CCI implementation, in others it may be more complex built on top of standard hardware I/O lines.

3.4 CCP2 Driver Please refer to /2/ for full details of CCP2 bus.

This is the low level interface, which allows the host to receive images from the SMIA camera.

The speed at which data is transmitted over this bus will require careful planning at software driver level. Interrupt latency, DMA (if used) prioritising and bus bottlenecks could all become problems at these speeds.

3.5 Colour Spaces Not all display devices are equal in the way they show colour. They may be biased toward red or green or blue or combinations. The way that colours are shown is known as a colour space.



of 27

Viewfinders have no output except the host display, so it is usual for viewfinder image processing to be biased to produce the best possible image for the display colour space.

Still images and Video can usually be transferred between devices. Therefore it is typical for these images to be processed for a normalized colour space known as the ‘standard colour space’.

This can lead to unwanted situations where the user presses the capture key and the colours noticeable change from viewfinder to end image, or from video viewfinder to video capture.

4. Personality files

4.1 Introduction Equalising performance

One of the driving ideas behind the concept of SMIA is to move towards image sensors that are commodity products rather than custom products. Part of this goal should be the possibility to treat one sensor in the same way as all others using a common usage model and receiving an understood level of performance. While the SMIA functional specification offers a common usage mode there will, in reality, still be differences in terms of performance. Some of these differences may be subtle (e.g. a slight difference in colour accuracy) others more extreme (e.g. different sensitivity range). In an attempt to address these differences the concept of ‘personality information’ was born. The idea being that additional information concerning particular sensors could be used by a host to make several different sensors perform in an even more similar manner that provided automatically by the SMIA standard interface. It would be convenient if some of this information was published by sensor manufacturers but by necessity some of it will have to be derived by sensor users (e.g. setup or ‘control limit’ information designed to make two particular sensors appear more similar).

Updated Sensor Information

A secondary role for personality information is as a method of informing sensor hosts of any details that allow higher performance to be obtained from a SMIA sensor. Typically this information would be published by a sensor manufacturer and would clearly be in addition to the mandatory information advertised by the SMIA sensor itself.

This class of personality information falls into two broad categories: information that should be used by the host instead of information advertised by the sensor (i.e. data that should override values read from sensor capability registers – such as the recommended colour matrix) and details of non-default sensor settings that can be used to enhance the performance of the sensor. Typically these non-default settings would be of manufacturer specific parameters. If they are SMIA parameters then it is likely that the sensor does not conform fully to the SMIA specification (in such a case the use of this personality information could be used to deal with non-conformity – NOTE: however, sensor manufacturers should not rely on use of this information as sensor hosts are not obliged to use it).



of 27

Extensions to the SMIA capability parameter set

A third possible role for personality information is the communication of new capability parameters that are not yet included in the mandatory set advertised by SMIA sensors (e.g. an agreed parameterised description of sensor ‘relative illumination’).

File syntax

The general syntax of a SMIA personality file is described in Chapter 4.2 and a list of supported settings and the syntaxes of the corresponding setting implementations are described in Chapter 4.3.



of 27

4.2 Personality File Syntax < SMIA personality file > = [SMIA_PERSONALITY_FILE_ID] < Sensor block 1 > . . . < Sensor block m >

, where

[SMIA_PERSONALITY_FILE_ID] = “smia_personality_file_id : ” + real_to_string(personality_file_syntax_version_number)\n “date : “ + date_str\n and < Sensor block x > = [SENSOR_ID] < Setting block 1 > . . . < Setting block n >

, where

[SENSOR_ID] = “manufacturer_id : ” + integer_to_string(manufacturer_id)\n “model_id : ” + integer_to_string(model_id)\n “revision_number : ” + integer_to_string(revision_number)\n and < Setting block x > = [SETTING_ID] < Setting implementation block 1 > . . . < Setting implementation block o >

, where

[SETTING_ID] = “setting : ” + SETTING_IDENTIFIER_STRING\n and < Setting implementation block x > = [SETTING_IMPLEMENTATION_ID] < setting_implementation_id –specific syntax >

, where

[SETTING_IMPLEMENTATION_ID] = “setting_implementation_id : ” + real_to_string(setting_implementation_id)\n and < setting_implementation_id –specific syntax> is specified explicitly in this document for each setting implementation.

\n indicates new line character in the syntax descriptions. The file should be stored in 8-bit ASCII format.



of 27

The integer values are supposed to be given in either 10base or hexadecimal format. If an integer is given in hexadecimal format, it should be indicated with prefix “0x”.

All the strings in the file (identifiers, parameter names…) are case insensitive.

String “//” is reserved as comment indicator; when “//” is encountered in a line then “//” and the rest of the line should be ignored. White spaces are allowed, and the host should ignore all of them except the line break character \n. Empty lines are allowed.

The host should ignore all sensor/setting/implementation blocks that are incomplete or do not follow the syntax as it is described in this document. However, an invalid block that is lower in hierarchy does not invalidate the block that is higher in hierarchy, e.g. one faulty setting implementation block does not invalidate the whole sensor block; only the faulty block is ignored.

See /1/ for reference about model_id, revision_number and manufacturer_id.



of 27

4.3 Setting Specifications

4.3.1 SETTING_IDENTIFIER_STRING

4.3.1.1 Definition SETTING_IDENTIFIER_STRING ∈ { “register_override”, “register_reprogram_once”, “register_reprogram_every”, “vignetting_elimination”, “colour_component_sensitivities” }

4.3.2 register_override, register_reprogram_once, register_reprogram_every

4.3.2.1 Syntax (version 1.0) address = value\n . . .

4.3.2.2 Description (version 1.0) The setting block “register_override” defines register values that should be used by the host instead of the values that would be read back from the sensor capability registers that correspond to these parameters.

The setting block “register_reprogram_once” defines register values that should be written to the sensor registers that correspond to these parameters once after the host first puts the system into ‘software standby’ mode (to enhance system performance).

The setting block “register_reprogram_every” defines register values that should be written to the sensor registers that correspond to these parameters in ‘software standby’ mode prior to each time host starts the sensor streaming.

Multiple address-value lines are possible for each of the described setting blocks. In general, both address and value are supposed to be given in either 10base or hexadecimal format. Where the parameter is a standard SMIA parameter (address that is not manufacturer specific) the standard SMIA parameter name can be used in the address field [1]. If a number is represented in hexadecimal format, it should be indicated with prefix “0x”.



of 27

4.3.3 vignetting_elimination

4.3.3.1 Syntax (version 1.0) a_coeff = x.x\n a1 = x.x\n a2 = x.x\n a3 = x.x\n r_coeff = x.x\n g_coeff = x.x\n b_coeff = x.x\n vign_center_x = x.x\n vign_center_y = x.x\n

4.3.3.2 Description (version 1.0) This setting block is used to describe the vignetting of a sensor. Circularly symmetrical vignetting is assumed. The raw Bayer input data needs to be in linear color space and the black level must be zero. The vignetting is modeled as a 6th degree polynomial that is parametrized with aCOEFF (a_coeff), a1 (a1), a2 (a2) and a3 (a3). The center of vignetting has the smallest correction gain (1.0) and the point that is farthest from the center has the biggest correction gain. The differences in vignetting between the colour components are parametrized with multipliers RCOEFF (r_coeff), GCOEFF (g_coeff) and BCOEFF (b_coeff). The center of vignetting is indicated with (xVIGNCENTER,yVIGNCENTER) ((vign_center_x,vign_center_y)), where (0.5,0.5) corresponds to the center of the visible image area and 0.0 and 1.0 correpond to the left/top and right/bottom borders of the visible image area, respectively.

The vignetting elimination that corresponds to the parameters is described by the following equation, for all colour components R, G and B:

{ } [ [ [ [),(),(),(

:,0|,0|,,yxGAINyxCyxC

yyNyxxNxBGRC

VEORIGINALCORRECTED

SIZESIZE

⋅=∈∈∀∧∈∈∀∧∈∀

, where

( )63

42

21 ),(),(),(0.1),( yxdayxdayxdaaCyxGAIN COEFFCOEFFVE ⋅+⋅+⋅⋅⋅+=

and

⎩⎨⎧

∈∈

NyNx

SIZE

SIZE

, where

22

222

))1(())1(())1(())1((),(

−⋅−+−⋅−−⋅−+−⋅−

=SIZEVIGNCENTERFSIZEVIGNCENTERF

SIZEVIGNCENTERSIZEVIGNCENTER

yyyxxxyyyxxxyxd

and



of 27

{ }[ ]

⎪⎩

⎪⎨

⎧

∈∈∈

∈∈

RCaRa

iRa

COEFF

COEFFCOEFF

i

0.1,0.0|3,2,1|

, where

[ ][ ]⎩

⎨⎧

∈∈∈∈

0.1,0.0|0.1,0.0|

VIGNCENTERVIGNCENTER

VIGNCENTERVIGNCENTER

yRyxRx

and where (xF,yF) is the corner point of the visible image area that is the farthest from the center point of vignetting (xVIGNCENTER*(xSIZE-1),yVIGNCENTER*(ySIZE-1)). (x,y) is the pixel coordinate, and xSIZE and ySIZE are the width and height of the visible image area in pixels, respectively.



of 27

4.3.4 colour_component_sensitivities

4.3.4.1 Syntax (version 1.0) gain_g1 = x.x\n gain_r = x.x\n gain_b = x.x\n gain_g2 = x.x\n

4.3.4.2 Description (version 1.0) This setting block is used to indicate the differences in colour component sensitivities in the sensor raw output. One of the gains must be 1.0, others ≥1.0 (gain∈R|gain∈[1.0,∞[).

When an image is captured given the constraints that are listed below, and given that the raw output pixel values are multiplied after capture with the above gains according to the colour component in question, then, on the average, the output values of adjacent pixels in the circular middle area of the output image that constitutes 20% of the total output image area should not differ more than 3%.

- The light source is a homogenous, smooth and flat D65 illuminant - The sensor is positioned perpendicularly towards the illuminant and only the

illuminant is visible in the image - No other light sources emit light to the sensor - The intensity level of the illuminant and the exposure time that is used in the

sensor (analogue and digital gains are +0dB) are such that the average level of the raw output pixel values lies in the middle of the dynamic range and no pixels are saturated.



of 27

4.4 Personality File Example // Personality file syntax example smia_personality_file_id : 1.0 // File syntax ver. 1.0 date : 2004-05-14 manufacturer_id : 1 // ST Microelectronics model_id : 590 // S590 revision_number : 1 setting : register_override setting_implementation_id : 1.0 matrix_element_RedInRed = 1.34 // 3/Nov/2003 matrix_element_GrnInRed = -0.23 setting : register_reprogram_once setting_implementation_id : 1.0 atest0 = 0xf2 0x4006 = 0x01 setting : register_reprogram_every setting_implementation_id : 1.0 pmod = 5 setting : vignetting_elimination setting_implementation_id : 1.0 a_coeff = 1.0 a1 = 1.0992 a2 = 0.6915 a3 = 0.0301 r_coeff = 1.2021 g_coeff = 1.0000 b_coeff = 1.0248 vign_center_x = 0.5 vign_center_y = 0.5 setting : colour_component_sensitivities setting_implementation_id : 1.0 gain_g1 = 1.333 gain_r = 1.0 gain_b = 1.6 gain_g2 = 1.28



of 27

5. References /1/ Nokia and ST:

SMIA 1.0 Part 1: Functional specification /2/ Nokia and ST:

SMIA 1.0 Part 2: CCP2 Specification /3/ Nokia and ST:

SMIA 1.0 Part 3.2: Symbian SMIA Camera device driver interface /4/ Java community:

JSR-000135 Mobile Media API, http://www.jcp.org/aboutJava/communityprocess/final/jsr135/

http://www.jcp.org/aboutJava/communityprocess/final/jsr135/

SMIA 1.0 Part 3.1: Software and application...

Documents

Transcript of SMIA 1.0 Part 3.1: Software and application...