TRUE BREW Test Procedures

General Testing True Brew® Test Procedures

Section Req Step Test requirement and procedures Notes, motivation, and expected results Developer notes

2.3.1 0 Entrance Criteria2.3.1 3 0 Test Name: MIF Settings N/A2.3.1 3 Requirement: Advanced privileges should not be used by the IUT

unless prior operator approval is documented within the Item Specification Document. Approval will be verified before testing continues.

N/A

2.3.1 3 1 Procedure: Open the MIF listed in the ARM directory of the submission package.

Result: MIF contents appear without error.

2.3.1 3 4 Procedure: Find the Advanced Privilege information within the MIF. Result: None of the Advanced Privilege levels are checked.

Note: These privileges are reserved. If Advanced Privilege levels are required by the IUT, Developer is required to get approval from Operator prior to submission and document this approval in the Item Specification Document. Operator's approval will be verified by Qualcomm/Test house before testing.

In Item Specification Template, Developer is advised to mention if Operator has approved us of Advanced Privilege levels.

2.3.1 3 0 Test Name: MIF Graphics N/A2.3.1 3 Requirement: MIF graphics (small, medium, large, and x-large) are the

correct size. It must be possible to visually determine that a graphic has been selected in Brew® Application Manager. The sizes are as follows:● Small: 16 (w) x 16 (h) pixels● Medium: 26 (w) x 26 (h) pixels● Large: 40 (w) x 40 (h) pixels● X-Large: 50 (w) x 50 (h) pixels

N/A

2.3.1 3 1 Procedure: Use the Brew MP Resource Manager (1.0.8.102 or above) to inspect the graphics associated with every applet defined in the IUT (within every MIF). The image

Result: Dimension of each graphic (image, icon, thumbnail) satisfies test requirements.

pp ( y ) gdimensions are displayed within the Brew MP Resource Manager.

2.3.1 3 4 Procedure: Use the Brew MP Resource manager ((1.0.8.102 or above) to view the number of colors in the bitmap.

Result: Color depth of images satisfies device requirements (DDS). If color depth exceeds that of the device, verify that the image is displayed on the device clearly and correctly.

2.3.1 8 0 Test Name: Startup/Exit IUT N/A2.3.1 8 Requirement: The IUT can be started from BAM. IUT terminates

execution in response to a Clear Key (when invoked from the root application menu), any means described in the Item Specification Document (e.g. Exit menu selection), or the End Key. The IUT can be started and operated after termination.

It is acceptable for an IUT to exit onto device native main screen in response to End Key and will not be failed for this behavior.

2.3.1 8 1 Procedure: Install the IUT on the device. Start IUT. From the Main Menu or other applicable screens, exit IUT using different means of exit specified in Item Specification Document (e.g. "Exit" menu options, clear key, End Key). Restart the IUT after each exit option.

Result: IUT starts up, operates, and exits to Brew Application Manager or native OEM screen. IUT can be restarted after each exit option.

2.3.1 5 0 Test Name: IUT Name and Version Number N/A

Qualcomm Confidential and Proprietary 1 May contain U.S. and international export controlled information



2.3.1 5 Requirement: All IUT names and version numbers must remain consistent throughout the IUT. The following areas are where the names and versions must match:● Item Specification Document● MIF Information● IUT Screens (About, Splash, Menu, Other)● Submission Information

The software version number must increase with each new submission of the IUT.

Motivation: Ensure that IUT is tested against the correct information provided throughout the IUT. This test case also ensures consistency in naming and versioning between the Item Specification Document and the information within the MIF (which will be displayed through BAM), in the submission information and within the IUT screens.

NOTE Discrepancies in IUT name consisting of only punctuation, whitespace, or capitalization differences will be noted as an issue. Example: IUT name per Item Specification Document MyApplication; IUT name at the Main Menu: my Application.

Any differences falling outside of these categories will cause the IUT to fail this test case. Example: IUT name per Item Specification Document; MyApplication; IUT name at the Main Menu: MyGame.

2.3.1 5 1 Procedure: View the Item Specification Document for the IUT version number and name.

Result: Item Specification Document contains IUT version number and name.

2.3.1 5 2 Procedure: View IUT version number listed in the MIF, all IUT screens on the device, and the submission information. Compare all version numbers found.

Result: IUT on device contains at least one version number that end user can read (either in IUT properties in the Brew Application Manager or on IUT screens). All version numbers found throughout the IUT are consistent.

2.3.1 5 3 Procedure: View IUT name listed in the MIF, all IUT screens on the device, and the submission information. Compare all titles found.

Result: All names found throughout the IUT are consistent

2.3.1 5 4 Procedure: Record item name Note: Record for archival and reference. Use "D" for Test Result. Name recorded match the one used during IUT Submission

2.3.1 5 5 Procedure: Record item version Note: Record for archival and reference. Use "D" for Test Result. Version Number recorded should match the one used during IUT Submission

2.3.1 5 6 Procedure: Record device model and PID Note: Record for archival and reference. Used "D" for Test Result2.3.1 5 7 Procedure: Record device S/W version Note: Record for archival and reference. Used "D" for Test Result2.3.1 10 0 Test Name: Basic Functionality N/A2.3.1 10 Requirement: The IUT meets its basic functionality as specified in the

Item Goal and Purpose section of Item Specification Document.




2.3.1 10 1 Procedure: Operate IUT. Check if Item meets its basic functionality as specified in the Item Goal and Purpose section of Item Specification Document. Use Application in its main menu/screens and all of its features.

Notes: For Ringtone/Wallpaper items, IUT downloads, saves, and plays/displays ring tones/wallpapers (as specified in the Item Specification Document). Downloaded tones/Wallpapers can be accessed/deleted through the native/application UI. For Camera items, IUT is able to take, and/or store, and/or send/play pictures/videos (as specified in the Item Specification Document).

For LBS items ( Tracking and Non-Tracking), IUT is able to retrieve position fixes and use these fixes as specified in the Item Specification Document.

For Networking items, IUT establishes, maintains, and tears down a data call. After exiting the application (through any means), the data call ends within the default linger timeout value of 30 secs.

Result: IUT meets its basic functionalities

2.3.2 0 Functinality Testing2.3.2 1 0 Test Name: Detailed Functionality N/A2.3.2 1 Requirement: All Functions on the IUT need to be specified in the Item Reproducibility of functionality test errors:

Specification Document. The Specification Document must be updated with every new item version and must contain information specific to the IUT and device that the IUT is being targeted for. Each device's functionality and behavior must be listed clearly and accurately so testing can accurately be done on device (i.e. touch screen vs. non-touch screen, single orientations vs. multiple orientations, etc.).

For errors with significant side effects (i.e. device power-cycle), errors that are reproducible in 30% or more attempts (over a minimum of 10 repetitions) will cause IUT failure.For other errors (errors impacting only the IUT), errors that are reproducible in 60% or more attempts (over a minimum of 10 repetitions) will cause IUT failure.

2.3.2 1 1 Procedure: Operate IUT. Explore all screens and functions. Note all instances of non-compliance with Item Specification Document. Note unexpected functionality outside scope of Item Specification Document.

Result: IUT operates according to Item Specification Document.

2.3.2 2 0 Test Name: User Interface N/A2.3.2 2 Requirement: The IUT primary functionality can be operated through

the user interface provided by the IUT. The user interface occupies the available display region.

The IUT should provide the user with dynamic visual feedback (e.g. rotating hour glass or moving percentage bar) or when there are delays greater than 3 seconds in duration.

N/A




2.3.2 2 1 Procedure: Operate IUT. Explore all screens and functions, paying particular attention to the presentation of text, graphics, etc.

The IUT provides the user with dynamic visual feedback(e.g. rotating hour glass or moving percentage bar) when there are delays greater than 3 seconds in duration.

For IUT and devices supporting multiple screen orientations, inspect the IUT UI under all supported screen orientations.

Result: All aspects of user interface render appropriately for all supported screen orientations. The IUT should make full use of the available display, and must not leave "empty" or undrawn areas in the display. The only exception to this requirement is when viewing video/movies, which may allow the user to view the video without stretching to completely fit both the width and height extents of the display.

Similarly, all aspects of the user interface must not be unexpectedly cropped.

Motivation: For any process/loading screen that lasts longer than 3 seconds, dynamic visual indication displays and informs the user that IUT is properly functioning (and not hung up or frozen).

2.3.2 2 3 Procedure: Navigate different paths in the IUT, exploring all menu options and screens. Press the CLR key on each IUT screen.

Result: The Clear key moves the user up (back) one level in the IUT menu hierarchy where appropriate.

Note: this test only applies to devices that have a physical Clear key, or a virtual (on-screen) Clear key.

Note: in some cases it is acceptable to bypass this requirement in order to provide a more user-friendly IUT behavior. For instance, a game may pause itself in response to the Clear key and display a pause menu containing a "Return to Main Menu" option. Pressing Clear again could resume the current game rather than returning to the main menu. If the IUT does not immediately return to the previous menu in response to the Clear key, it must display an option for the user to return to the previous screen (i.e. the main menu in this example).

In all cases, exceptions to the standard behavior must be documented in the behavior exceptions section of the Item Specification Document.

2.3.2 2 2 Procedure: Start the IUT. Rotate the device's screen (if supported). Observe IUT behavior in response to screen rotations.

Note: This testcase applies to devices with accelerometers, "sliders", and/or device's with an external and internal display.

Result: Verify that IUT behaves as indicated by Item Specification Document.

If IUT supports multiple screen orientations, verify that the transition between screen orientations is seamless from a user perspective: 1) IUT should be usable on the new screen orientation without any loss of functionality (unless documented in Item Specification Document). 2) No significant delay (greater than 1 sec) in redrawing/refreshing the IUT UI to account for the new screen orientation3) IUT properly adjusts UI to account for changes in display orientation (no undesirable clipping of UI, UI adjusts to properly render on the new orientation)




2.3.2 2 4 Procedure: The IUT functions properly with any peripheral equipment distributed with the device.

Motivation: some devices ship with a peripheral device that user may optionally connect to the device as an input (for example, an extended keypad add-on or a game controller). The IUT must gracefully support user input through these peripheral devices. This test case is executed only for peripheral devices shipped with the device; it is not executed using any third-party or after-market peripheral devices.

Result: IUT properly accepts input from the peripheral device. IUT functionality is not adversely impacted by disconnecting or reconnecting the peripheral device. Developer should also provide information or describe feature pertaining to peripherals. If peripheral device is not supported by design, the end user must be visually informed that the peripheral device is not supported.

Testing must be done for all peripherals distributed with the device even if there is more than one additional peripheral. This is not optional for peripherals that are packaged with the device.

2.3.2 3 0 Test Name: Supported Languages N/A2.3.2 3 Requirement: The IUT functions as specified in the Item Specification

Document for each language supported by the IUT.Motivation: ensures that IUT behaves consistently for all languages supported per Item Specification Document.

2.3.2 3 4 Procedure: If IUT allows selection of language through IUT menu, set IUT to each language as specified in Item Specification Document. Perform functionality testing.

Result: Item displays each selected language.

2.3.3 0 device/IUT Interaction2.3.3 1 0 Test Name: Suspend/Resume N/A2.3.3 1 Requirement: The IUT suspends, maintaining persistent IUT state (for

example, it doesn't lose data entered by the user), and it resumes at or near the point of suspension. Suspend can occur because of incoming voice calls, incoming

Note: For this test case, "test device" refers to the device used in testing the IUT, while "calling device" refers to another device used to originate calls to the test devices. Calls are ended by pressing the End key on the specified device.

SMS, and alerts. The IUT does not interfere with the ability to read the OEM Caller ID display.

2.3.3 1 1 Procedure: Call the test device from calling device while the IUT is on the main screen.

Result: IUT suspends. Caller ID display visible.

2.3.3 1 2 Procedure: Answer call on test device. Result: IUT remains suspended while voice call connects. Test device can hear calling device and vice versa.Note: It is not necessary to evaluate voice quality.

2.3.3 1 3 Procedure: End call from test device. Result: IUT resumes back to the point of suspension.2.3.3 1 4 Procedure: Repeat steps 1 through 3, this time ending call from

calling device.Result: IUT suspends, caller ID screen visible, and IUT then resumes back to main screen.

2.3.3 1 5 Procedure: Repeat steps 1 through 3, this time refusing call on test device (i.e. hit "Ignore" on caller ID screen if available, or simply ignore incoming call).

Result: IUT suspends, caller ID screen visible, and IUT then resumes back to main screen.

2.3.3 1 6 Procedure: Call the test device, timing incoming calls to occur during IUT startup and exit.

Result: IUT suspends, caller ID screen visible, and IUT then resumes back to or near point of suspension. No Significant data lost.

2.3.3 1 7 Procedure: If IUT is a networking item, repeat steps 1 through 3 while data connection is dormant. After resume, transfer data across data connection (e.g. download ringtone).

Motivation: An IUT with a dormant data connection may suspend when receiving an incoming voice call.

Result: IUT suspend, caller ID screen visible, and IUT then resumes back to main screen. Data transfer succeeds.




2.3.3 1 8 Procedure: If device supports other means of generating suspend events (e.g. side volume control, attachment/removal of power cord, camera key, insertion/removal of headphones, OEM Alarm, etc.), generate suspend events while on IUT main screen.

Result: IUT suspends, native message visible, and IUT then resumes back to main screen.

2.3.3 1 9 Procedure: Generate periodic incoming SMS suspend events(e.g. every 10 seconds). Perform functionality testing.

Result: IUT suspends, native SMS display visible, and IUT then resumes back to or near point of suspension. No significant data lost.

2.3.3 1 10 Procedure: Generate SMS events, timing the suspend events to occur during IUT startup and exit.

Result: IUT suspends, native SMS display visible, and IUT then resumes back to or near point of suspension. No significant data lost.

2.3.3 1 11 Procedure: Generate periodic incoming SMS suspend events (e.g. every 10 seconds). Repeat functionality testing, this time focusing on data call setup, activity (data transfer) and idle/dormant time periods. Applies only to network IUT.

Result: IUT suspends, native SMS display visible, and IUT then resumes back to or near point of suspension. No significant data is lost. Data connection may be lost on many devices. Some devices may not suspend during a data call, and if this happens the user must be notified with a visual or audio indicator that an SMS has been received. It is not a requirement that the IUT re-establish the data connection if it is lost (though certainly a user-friendly behavior).

2.3.3 3 0 Test Name: Device Form Factor/IUT Interaction N/A2.3.3 3 Requirement: IUT gracefully handles device form factor changes and

associated events (e.g., EVT_FLIP, EVT_KEYGUARD, EVT_SCR_ROTATE). IUT behavior matches expectations documented in Item Specification Document.

Note: If the IUT handles events differently on different screens, all expected behaviors must be listed.

Motivation: Item behavior requirements in response to form factor changes will differ across operators. Testing is performed to validate that the IUT behaves in accordance with the behavior documented in the Item Specification Document which is assumed to be in accordance with operator requirements.

2.3.3 3 1 Procedure: Start the IUT, close the device's clamshell/slider if present (attach peripheral device if applicable). Open/Close the clamshell/slider, observe IUT state (specifically, whether the IUT is still running or has terminated).

Repeat above step for any three different screens in the IUT.

Result: Verify that IUT behaves as indicated by Item Specification Document.

Note: If Item Specification Document lists that IUT implements Qualcomm Slider/Clamshell reference code, execute the Slider/Clamshell Qualcomm reference application and ensure that the behavior of the IUT is identical to that of the reference application.

2.3.3 4 0 Test Name: Extended Keypad Modifier Key Awareness N/A2.3.3 4 Requirement: IUT gracefully handles device keypad modifier keys

(both sticky and non-sticky). Applies only to devices supporting extended keypads with modifier keys, or devices that support peripheral device add-ons providing this functionality.

Motivation: for extended keypads (e.g., QWERTY keypad devices), the IUT must correctly interpret modifier keys—keys that change the key value of pressed keys, such as the Symbol or Shift keys. IUT must also properly handle "sticky" modifier keys, which have an effect on key codes even when not held down (e.g., Caps Lock, Num Lock).

If this functionality is provided through a peripheral device add-on, this test case will be executed only for peripheral devices shipped with the device, not any third-party or after-market add-ons.

Test case is not applicable to IUT that does not contain text entry or if the text entry is by other means like using a "scrolling" method (up and down arcade style and not through QWERTY Keyboard.




2.3.3 4 1 Procedure: Attach peripheral extended keypad add-on, if applicable. Navigate to a data entry screen (with an editable text field). Enter a minimum of 10 characters for each modifier key supported by the device.

Result: text is entered as expected. Modifier keys produce the expected text input: For instance, Shift + 'a' = 'A'. The supported modifier keys will be device-specific.

2.3.3 4 2 Procedure: Terminate the IUT. Activate a "sticky" modifier key. Start the IUT, and navigate to a data entry screen. Enter text.

Result: text is entered as expected, modified by the activated sticky key.

2.3.3 4 3 Procedure: Deactivate the sticky modifier key. Enter text. Result: text is entered as expected, no longer modified by the sticky key.

2.3.3 4 4 Procedure: Repeat steps 2-3 for all sticky modifier keys supported by the device.

Result: same expectations as above.

2.3.5 0 Brew User Interface2.3.6 0 Adversarial2.3.6 1 0 Test Name: Loss of Service N/A2.3.6 1 Requirement: The IUT gracefully handles no-service and loss-of-

service conditions. A no-service condition will result in connection failure when the IUT attempts to establish a connection. A loss-of-service condition will affect the IUT after a connection has been established. The requirement applies to applications using data services or voice services. NOTE: Procedures 9 - 13 are applicable only to Application Value Billing Items.

Any device that obstructs the device signal (e.g. a microwave oven) may be used in loss of service testing.

2.3.6 1 1 Procedure: Prepare IUT to initiate data call (do not start data call). Place device in Hoffman Box until service is lost. Inspect device to ensure no service, initiate data call, close and leave device in Hoffman box for 1 minute. Remove

Motivation: Ensure IUT handles loss-of-service during call setup.

Result: IUT detects and gracefully handles to loss-of-service (e.g. reports error to user).

device from Hoffman box and view IUT.

2.3.6 1 2 Procedure: Identify the error message reported by the application. Motivation: Ensure the user is notified of the loss of service error.

Result: A relevant error message must be displayed and no adverse effect to the IUT or device is observed.

2.3.6 1 3 Procedure: Initiate data call after device has re-acquired service. Motivation: Ensure IUT error handling for loss-of-service does not affect ability to successfully make data call once service restored.

Result: IUT succeeds in making data call and transferring data, functioning as it did during functionality testing.

2.3.6 1 4 Procedure: Initiate and establish data call. As IUT is transferring data across data connection, place device in Hoffman Box until device loses service. Remove device from Hoffman box.

Motivation: Ensure IUT gracefully handles loss-of-service during data transfer.

Result: IUT detects and handles loss-of-service (e.g. reports error to user).

2.3.6 1 5 Procedure: Initiate data call after device has re-acquired service. Motivation: Ensure IUT error handling for loss-of-service does not affect ability to successfully make data call once service restored.

Result: IUT succeeds in making data call, functioning as it did during functionality testing.




2.3.6 1 6 Procedure: Let device sit idle after transfer in Step 5 for linger time plus 20 seconds. Repeat step 1.

Motivation: The intent is to determine how an application handles loss of a connection that is idle or dormant after already successfully transferring data across it.

Result: IUT detects and handles loss-of-service (e.g. reports error to user).

2.3.6 1 7 Procedure: Repeat steps 1-6 for each IUT screen or function that transfers data across a data connection.

Note: The purpose is to evaluate different screens or functions provided by the IUT, not to evaluate different content types on a server that are available through the same IUT screen.

Result: See individual steps.2.3.6 1 8 Procedure: Repeat step 1 for IUT that supports sending of SMS

messages. Instead of making data call, initiate SMS. Other steps the same as step 1.

Motivation: Ensure that IUT handles inability to send an SMS message.

Result: IUT supporting sending of SMS messages detects and handles loss-of-service (e.g. reports error to user).

2.3.6 1 9 Procedure: Prepare IUT to submit an A-VB transaction (do not submit an A-VB transaction yet). Place device in Hoffman Box until service is lost. Inspect device to ensure no service, submit an A-VB transaction, close and leave the device in the Hoffman Box for 1 minute. Remove device from Hoffman box and view IUT. Identify the error message reported by IUT.

Motivation: Ensure the user is notified of the error when loss of service occurs before the transaction is submitted/posted.

Result: A relevant error message must be displayed and no adverse affect to the IUT or device is observed.

2.3.6 1 10 Procedure: Submit the transaction again after device has re-acquired service.

Motivation: Ensure IUT error handling for loss-of-service does not affect ability to successfully submit/post the transaction once service is restored.

Result: IUT succeeds in submitting the transaction and transferring transaction data.

2.3.6 1 11 Procedure: Submit the transaction again. As IUT is transferring data across data connection, place device in Hoffman Box until device loses service. Wait for 1 min and then remove device from Hoffman box.

Motivation: Ensure IUT handles loss-of-service during transaction transfer.

Result: IUT detects and gracefully handles loss-of-service (e.g. reports error to user) without adverse effect to the device. Verify that transaction can be concluded after the IUT is restarted via some means described in the Item Specification Document. Verify that transaction correlates to 'single transaction ID' on the A-VB Webtool.You may see more than one transaction get posted, but it should belongs to the same/unique transaction ID.




2.3.6 1 12 Procedure: Submit the transaction again. After the transaction succeeds and as IUT is downloading content using the data connection, place device in Hoffman Box until device loses service. Wait for 1 min and then remove device from Hoffman box.

Motivation: Ensure IUT handles loss-of-service during content download and is able to recover the content download.

Result: IUT detects and gracefully handles loss-of-service (e.g. reports error to user) without adverse impact to the device. Verify that content can be re-downloaded after the data connection is restored.

Note: For content that already exists on the device (e.g. content being unlocked after purchase), this test case can be skipped. Such behavior needs to be described in the Item Specification Document.

2.3.6 1 13 Procedure: Repeat steps 9-12 for each IUT screen or function that allows transactions to be submitted.

Note: For an A-VB IUT that contains: 1) too much content (e.g. wallpaper and ringtone apps) 2) or transaction pages that are too difficult to reach (e.g. gaming apps with too many levels), take the exploratory approach for verifying the transaction behavior.

2.3.6 3 0 Test Name: Continual Keypad Entry N/A2.3.6 3 Requirement: The IUT is robust in the face of a rapid succession of

random keypad inputs.N/A

2.3.6 3 1 Procedure: Start IUT. Once started, randomly press keys for 10 seconds. Omit End key and Clear key.

Motivation: Ensure IUT handles large buffer of random key-presses. Result: IUT handles all keypad input by either correctly processing it or ignoring it. No adverse affect on IUT or device.

2.3.6 3 2 Procedure: Repeat step 1 for each IUT screen and function, including point in IUT for which data calls have been established.

Result: IUT handles all keypad input by either correctly processing it or ignoring it. No adverse affect on IUT or device.

Note: If specific multiple key combinations are interpreted by the device as a key event other than one of the keys that was pressed, this is not an IUT issue. This is a limitation of the device keypad, and does not cause the IUT to fail this test case.

2.3.6 3 3 Procedure: Navigate to a data entry (editable text field). Randomly press keys for 10 second to create a large string.

Motivation: Ensure IUT handles or limits large strings.

Result: IUT either accepts large strings or limits size. No adverse impact on IUT or device.

2.3.6 3 4 Procedure: Repeat step 3 for each editable text box presented by IUT.

Motivation: Ensure IUT handles or limits large strings.

Result: IUT either accepts large strings or limits size. No adverse impact on IUT or device.

2.3.6 4 0 Test Name: Low Flash Memory N/A2.3.6 4 Requirement: The IUT must be stable (e.g. does not crash device and

gracefully terminates execution) when faced with low Embedded File System (EFS) conditions (e.g. in the face of file create and write failures when attempting to write to flash memory). NOTE: Procedures 6 - 7 are applicable only to Application Value Billing Items

N/A




2.3.6 4 1 Procedure: Remove the IUT and power cycle the device. Re-install the IUT. Do not start the IUT. Run the MaxFileCount utility to fill the device's file system using all configuration parameters set to "TBT default".

Motivation: Fills the Brew filesystem space of the device, creating condition in which file creation by IUT will fail.

2.3.6 4 2 Procedure: Start IUT. Navigate through IUT, focusing on screens and functions that are known or expected to result in the creation of files.

Motivation: Ensure that IUT does not crash nor become unstable when file creations fail. Result: IUT detects and gracefully handles file creation failure (e.g. notifies the user with an error message).

2.3.6 4 3 Procedure: Run the MaxFileCount application, removing one file. Motivation: IUT may create multiple files. Removing one file allows IUT to create one file, then fail. This may test additional file creation code segments of the IUT. Result: One less file on device.

2.3.6 4 4 Procedure: Start IUT. Navigate through IUT, focusing on screens and functions that are known to result in the creation of files.

Motivation: Ensure IUT handles file creation failure.

Result: IUT detects and gracefully handles to file creation failure (e.g. notifies the user with an descriptive error message).

2.3.6 4 5 Procedure: Repeat steps 3-4 until IUT operates normally (can create all files it needs). NOTE: Remember to remove files created by MaxFileCount application before proceeding to next test.

Result: IUT detects and gracefully handles file creation failures until it can create all the files it requires.

2.3.6 4 6 Procedure: For IUT that supports A-VB, make sure to focus on initiating a transaction in steps 1 - 5.

Motivation: Ensure that A-VB IUT does not crash nor become unstable when initiating A-VB transactions under low memory conditions.

Result: IUT detects and gracefully handles file creation failure during an A-VB transaction (e.g. notifies the user with a descriptive error message).

2.3.6 4 7 Procedure: For IUT that supports A-VB, make sure to focus on Motivation: Ensure that A-VB IUT does not crash nor become unstable content download after a successful transaction in steps 1 - 5.

when downloading content under low memory conditions.

Result: IUT detects and gracefully handles file creation failure during an A-VB content download (e.g. notifies the user with an error message). Verify that content can be retrieved after the space is freed up via some means described in the Item Specification Document.

2.3.6 6 0 Test Name: Default Event Handling Behavior N/A2.3.6 6 Requirement: The IUT does not return TRUE to the default case in the

event handler method.Motivation: Many items use a switch statement within the event handling method. It is critical that the default case in this switch statement return FALSE -- this is how the application informs Brew that it does not know how to process the incoming event. If an application returns TRUE to an event but does not process it properly (such as EVT_APP_START_WINDOW), the application will cause undesired behavior.

2.3.6 6 1 Procedure: From BAM, launch the EventHandleTool application. Note: The IUT may be in any state while this test is performed (e.g. terminated, running in the background, etc.).

Result: EventHandleTool opens and IUT is listed2.3.6 6 2 Procedure: Within the EventHandleTool application, select the IUT

(or each module associated with the IUT for multi-module submissions).

Result: EventHandleTool displays PASSED for each module selected.


Application-Value Billing True Brew® Test Procedures

Section Req Step Test requirement and procedures Notes, motivation, and expected results

2.3.4 0 Brew Features2.3.4 19 0 Test Name: Application Value Billing Items N/A2.3.4 19 Requirement: The IUT supporting Application-Value Billing (A-VB) should accurately

process A-VB transactions. Notes: Item specification document should clearly indicate where all the purchase/transaction entries are in the application along with their corresponding pricing (e.g., price ID). Item spec also needs to describe clearly when the transaction is actually submitted/posted to the ADS and when and how the content is made available to the user. The A-VB test guidelines also only apply to A-VB apps that retrieve content pricing from the ADS.

2.3.4 19 1 Procedure: Per the Item Specification Document, navigate to the content purchase page. Enter IUT information in the Webtool and select "Show or Update VB Pricing". Locate the price point in the webtool based on the Price ID for the purchase, which is described in the Item Specification Document. Compare the price info (e.g. dollar amount) displayed in the IUT with the price info in the webtool for the purchase price point.

Motivation: IUT is able to obtain and display correctly the pricing info from the ADS for the transaction.

Result: The price info displayed in IUT matches the price info in the webtool.

2.3.4 19 2 Procedure: In the webtool main page, select "Error Conditions" and then select "Clear Error Condition" to ensure no error is returned as the response to the IUT for the transaction. Go back to the webtool main page and select "Show VB Transactions" to monitor the incoming transactions from the IUT. Submit the transaction (i.e. purchase) and verify using the web tool that the transaction data are correctly received by the ADS.

Motivation: Ensure that IUT does not impact the transaction data resulting in incorrect pricing.

Result: IUT sends the transaction data to the ADS correctly.

2.3.4 19 3 Procedure: Verify, using the webtool, that the transaction is accepted by the ADS (i.e. successful transaction) and IUT is able to indicate a successful transaction t th

Motivation: IUT is able to complete a transaction successfully.

R lt T ti i t d b th ADS d IUT i di t th t thto the user. Result: Transaction is accepted by the ADS and IUT indicates that the transaction is successful.

2.3.4 19 4 Procedure: Once the transaction succeeds, ensure that the content is made available to the user properly.

Result: IUT successfully makes the content purchased available to the end user.

2.3.4 19 5 Procedure: Update the pricing (dollar value) for the same content in step 1 by going to the "Show or Update VB Pricing" in the web tool and changing the Price Value for the purchase price point identified by the Price ID. After the pricing has been successfully updated, navigate to the same IUT purchase page as in step 1 and observe the pricing displayed is correctly updated. Submit the transaction within the IUT. Select "Show VB Transactions" in the webtool and verify the transaction data received by the ADS.

Motivation: The latest pricing from the server must always be used by the IUT to charge for the transaction, which can be verified by the webtool after the transaction is submitted.

Result: Either one of the following must be satisfied for this test case to pass:1. IUT always displays and uses the up-to-date pricing from the ADS - i.e. the change of pricing will be reflected in IUT when viewing the pricing for the content.2. IUT may cache the pricing and the displayed pricing may not be up-to-date but when the transaction is taking place, IUT needs to successfully update the pricing and indicate the change of pricing to the end user. The transaction can either be cancelled or completed with the up-to-date pricing.




2.3.4 19 6 Procedure: Remove the price point for the same content in step 1 by going to "Show or Update VB Pricing" in the webtool, selecting "Delete this PriceID" for the price point identified by the Price ID and hitting "Submit". After the price point has been successfully removed, navigate to the same IUT purchase page as in step 1 and verify that the IUT successfully indicates that the content is no longer available for purchase.

Motivation: Price plan may change during the life time of the application in the field. Therefore, when certain price points are removed from the price plan, it means the content is no longer available for purchase. IUT needs to be able to successfully express that.

Result: IUT gracefully handles Price Point deletion scenarios.

2.3.4 19 7 Procedure: Repeat steps 1-6 for each IUT screen or function that allows transactions to be submitted per item spec.

Note: For an A-VB IUT that contains: 1) too much content (e.g. wallpaper and ringtone apps) 2) or transaction pages that are too difficult to reach (e.g. gaming apps with too many levels), take the exploratory approach for verifying the transaction behavior. For a good sampling at least 10 transactions should be checked.

2.3.6 0 Adversarial2.3.6 9 0 Test Name: User Interruptions during A-VB Transactions N/A2.3.6 9 Requirement: The IUT supporting A-VB needs to behave properly when the application

gets interrupted in the middle of transaction posting. In the webtool main page, select "Error Conditions" and then select "Clear Error Condition" to ensure no error is returned as the response to the IUT for the transaction.

Motivation: Whenever a "Submit" or any of its equivalent commands has been executed for a transaction, the transaction must be considered to be posted and hence committed (e.g. transaction billed to the user account) since there is no way of cancelling that transaction. If A-VB application is unable to receive and handle the transaction response from the ADS due to the interruption, it must provide a mechanism (e.g. reposting the same transaction) to allow the transaction to be concluded for the end user (e.g. a pop up message confirming the result of the transaction) if the transaction has been interrupted in the middle of its posting. Item spec needs to document such a mechanism.

2.3.6 9 1 Procedure: Start IUT and go to the purchase/transaction page. Submit the transaction in the IUT and immediately hit the CLR key.

Result: IUT returns to the previous page of the application and there is no adverse effect to the UI. Verify that transaction can be concluded via some means described in the item specfication document. Verify that transaction correlates to 'single transaction ID' on the A-VB Webtool.You may see more than one transaction get posted, but it should belong to the same/unique transaction ID.

2.3.6 9 2 Procedure: Start IUT and go to the purchase/transaction page. Submit the transaction in the IUT and immediately hit the END key.

Result: IUT exits and verify that transaction can be concluded after the IUT is restarted via some means described in the item specification document. Verify that transaction correlates to 'single transaction ID' on the A-VB Webtool.You may see more than one transaction get posted, but it should belong to the same/unique transaction ID.

2.3.6 9 3 Procedure: Start IUT and go to the purchase/transaction page. Submit the transaction in the IUT and immediately hit and hold the END key to power down the device.

Result: device powers down and verify that after the device powers back up and IUT is restarted transaction can be concluded via some means described in the item specification document. Verify that transaction correlates to 'single transaction ID' on the A-VB Webtool.You may see more than one transaction get posted, but it should belong to the same/unique transaction ID.




2.3.6 9 4 Procedure: Start IUT and go to the purchase/transaction page. Submit the transaction in the IUT and immediately pull out the battery.

Result: device powers down and verify that after the device powers back up and IUT is restarted transaction can be concluded via some means described in the item specification document. Verify that transaction correlates to 'single transaction ID' on the A-VB Webtool.You may see more than one transaction get posted, but it should belong to the same/unique transaction ID.

2.3.6 9 5 Procedure: Start IUT and go to the purchase/transaction page. Submit the transaction in the IUT and immediately call the device the IUT is running on from another device.

Result: IUT gets suspended properly by the voice call and verify that transaction can be concluded after the IUT is resumed via some means described in the item spec. Verify that transaction correlates to 'single transaction ID' on the A-VB Webtool.You may see more than one transaction get posted, but it should belong to the same/unique transaction ID.

2.3.6 9 6 Procedure: Repeat steps 1-5 for each IUT screen or function that allows transactions to be submitted as specified in the item specification document.


2.3.6 10 0 Test Name: User Interruptions during A-VB Content Retrieval N/A2.3.6 10 Requirement: The IUT supporting Application Value Billing (A-VB) needs to behave

properly when the application gets interrupted in the middle of content retrieval. In the webtool main page, select "Error Conditions" and then select "Clear Error Condition" to ensure no error is returned as the response to the IUT for the transaction.

Motivation: IUT needs to ensure that the interruptions during the content download does not prevent end users from receiving the content eventually. Note: These tests should only be applicable if there is a window of time for potential user interruptions during the content retrieval (e.g. content downloaded OTA). Item spec needs to describe how the content is made available after the transaction succeeds so that the applicability of the following tests can be determined.

2.3.6 10 1 Procedure: Start IUT and go to the purchase/transaction page. Complete the Result: IUT returns to the previous page of the application and there is no2.3.6 10 1 Procedure: Start IUT and go to the purchase/transaction page. Complete the transaction and hit the CLR key in the middle of the content retrieval.

Result: IUT returns to the previous page of the application and there is no adverse effect to the UI. Verify that content can be retrieved via some means described in the item spec.

2.3.6 10 2 Procedure: Start IUT and go to the purchase/transaction page. Complete the transaction and hit the END key in the middle of the content retrieval.

Result: IUT exits and verify that the content can be retrieved after the IUT is restarted via some means described in the item spec.

2.3.6 10 3 Procedure: Start IUT and go to the purchase/transaction page. Complete the transaction and in the middle of the content retrieval, hit and hold the END key to power down the device.

Result: device powers down and verify that after the device powers back up and IUT is restarted the content can be retrieved via some means described in the item spec.

2.3.6 10 4 Procedure: Start IUT and go to the purchase/transaction page. Complete the transaction and in the middle of the content retrieval, pull out the battery.

Result: Verify that after the device powers back up and IUT is restarted the content can be retrieved via some means described in the item spec.

2.3.6 10 5 Procedure: Start IUT and go to the purchase/transaction page. Complete the transaction and in the middle of the content retrieval, call the device the IUT is running on from another device.

Result: IUT gets suspended properly by the voice call and verify that content can be retrieved after the IUT is resumed via some means described in the item spec.



2.3.6 11 0 Test Name: A-VB Pricing Download Error handling N/A




2.3.6 11 Requirement: The IUT supporting Application Value Billing (A-VB) needs to handle pricing-download related failures properly.

Motivation: IUT is able to handle errors properly when attempting to download the content pricing.Note: Item spec must specify where inside the app content pricing is downloaded/updated from the ADS (e.g. upon app start-up, when displaying pricing page for the content, etc...). Take the exploratory approach if there are multiple places in the IUT where pricing-download can be triggered.

2.3.6 11 1 Procedure: In the webtool, select "Error Conditions" and then select "server Error" and hit "Submit". Start IUT and per item spec, go to the page where the content pricing is downloaded from the ADS and observe the IUT.

Motivation: Ensure that IUT gracefully handles error.

Result: IUT must display a descriptive error message without any adverse impact on IUT or the device.

2.3.6 11 2 Procedure: In the webtool, select "Error Conditions" and then select "SID not authorized " and hit "Submit". Start IUT and per item spec, go to the page where the content pricing is downloaded from the ADS and observe the IUT.



2.3.6 11 3 Procedure: In the webtool, select "Error Conditions" and then select "invalid itemid" and hit "Submit". Start IUT and per Item Spec Template, go to the page where the content pricing is downloaded from the ADS and observe the IUT.



2.3.6 11 4 Procedure: In the webtool, select "Error Conditions" and then select "pricing not available" and hit "Submit". Start IUT and per item spec, go to the page where the content pricing is downloaded from the ADS and observe the IUT.



2.3.6 12 0 Test Name: A-VB Transaction Error handling N/A2.3.6 12 Requirement: The IUT supporting Application Value Billing (A-VB) needs to handle

transaction related failures properly. Motivation: Due to billing concerns, IUT supporting A-VB must always provide proper error messages and handling for various transaction related failures. End user should always be informed of problems that occur during the transaction lest he or she be concerned about any unintended charges. No content should be made available when any these errors occurs.

2.3.6 12 1 Procedure: Navigate to the purchase/transaction page. In the webtool, select "Error Conditions" and then select "Content Fee" and select "Submit". Submit the transaction in the IUT and observe the transaction result in IUT.

Motivation: Ensure that no content is unlocked or made available when error occurs.

Result: IUT should display a descriptive error message and no adverse impact on IUT or the device. No Content should be made available.

2.3.6 12 2 Procedure: Navigate to the purchase/transaction page. In the webtool, select "Error Conditions" and then select "Invalid Price (amount)" and select "Submit". Submit the transaction in the IUT and observe the transaction result in IUT.






2.3.6 12 3 Procedure: Navigate to the purchase/transaction page. In the webtool, select "Error Conditions" and then select "Invalid Currency" and select "Submit". Submit the transaction in the IUT and observe the transaction result in IUT.



2.3.6 12 4 Procedure: Navigate to the purchase/transaction page. In the webtool, select "Error Conditions" and then select "Invalid Billing Desc" and select "Submit". Submit the transaction in the IUT and observe the transaction result in IUT.



2.3.6 12 5 Procedure: Navigate to the purchase/transaction page. In the webtool, select "Error Conditions" and then select "Invalid Billing Data" and select "Submit". Submit the transaction in the IUT and observe the transaction result in IUT.



2.3.6 12 6 Procedure: Navigate to the purchase/transaction page. In the webtool, select "Error Conditions" and then select "Server Error" and select "Submit". Submit the transaction in the IUT and observe the transaction result in IUT.



2.3.6 12 7 Procedure: Navigate to the purchase/transaction page. In the webtool, select "Error Conditions" and then select "SID Not Authorized" and select "Submit". Submit the transaction in the IUT and observe the transaction result in IUT.



2.3.6 12 8 Procedure: Navigate to the purchase/transaction page. In the webtool, select "Error Conditions" and then select "Insufficient Funds" and select "Submit".

Motivation: Ensure that no content is unlocked or made available when error occurs.Conditions and then select Insufficient Funds and select Submit .

Submit the transaction in the IUT and observe the transaction result in IUT.

error occurs.


2.3.6 12 9 Procedure: Navigate to the purchase/transaction page. In the webtool, select "Error Conditions" and then select "Price Changed" and select "Submit". Submit the transaction in the IUT and observe the transaction result in IUT.


Result: IUT must either display a descriptive error message or trigger an update for the latest pricing without any adverse impact on IUT or the device. No Content should be made available.

2.3.6 12 10 Procedure: Navigate to the purchase/transaction page. In the webtool, select "Error Conditions" and then select "Invalid ItemID" and select "Submit". Submit the transaction in the IUT and observe the transaction result in IUT.






2.3.6 12 11 Procedure: Navigate to the purchase/transaction page. In the webtool, select "Error Conditions" and then select "Late Response" and select "Submit" so that the server will delay the response to cause the IUT to time out. Submit the transaction in the IUT and observe the transaction result in IUT.

Motivation: Ensure that the IUT gracefully handles situations where there is no response from the server. IUT should attempt to resubmit the transaction.

Result: IUT must time out and display a descriptive error message without any adverse impact on IUT or the device. No Content should be made available. IUT should attempt to resubmit the transaction as described in item specification document.




Best Practices True Brew® Test Procedures

Practice number Requirement name Validation steps Notes, motivation, and expected results

1For Brew Applications and Extensions: The MODs and MIFs shall be signed. All signed files must have valid developer signatures. Files that are modified on the device must be unsigned or signed-modifiable.

N/A N/A

2MIF graphics (small medium and large) are the correct size. It must be possible to visually determine that a graphic has been selected in BAM. The sizes are as follows: small image 16 (w) x 16 (h) pixels; medium image 26 (w) x 26 (h) pixels; large image (for Brew 2.0 and beyond) 40 (w) x 40 (h) pixels and X-Large image 50 (w) x 50 (h) pixels.

1. Use the Brew MP Resource Manager (1.0.8.102 or above) to inspect the graphics associated with every applet defined in the IUT (within every MIF). The image dimensions are displayed within the Brew MP Resource Manager.2. Use the Brew MP Resource manager (1.0.8.102 or above) to view the number of colors in the bitmap.3. On device, select application using App Manager. Repeat this step for each application View supported by device.

1. Dimension of each graphic (image, icon, thumbnail) satisfy test requirements.2.Color depth of images satisfies device requirements (If color depth exceeds that of device, verify that image is displayed on the device. If this is not the case, ignore the error).3. Tester can visually verify that IUT has been selected (border around graphic).Motivation: device may allow user to configure view (icon, thumbnail, etc). Ensure by visible inspection that graphic for IUT is highlighted when selected in each view. Result: Tester can visually verify that IUT has been selected (border around graphic).

31. IUT gracefully handles device form factor changes and associated events (e g EVT FLIP

N/A Application behavior requirements in response to form factor changes will differ across operators

Developer Signature

MIF Graphics

Device Form Factor/IUT Interaction

changes and associated events (e.g., EVT_FLIP, EVT_KEYGUARD, EVT_SCR_ROTATE). IUT behavior matches expected behavior including supported Peripheral devices add ons that may trigger these events.2.For application not supporting a particular device orientation, the application should display a message explaining the appropriate device orientation to ensure the application functions properly. Example: "Application does not support landscape mode."

changes will differ across operators.

Application meets expected behavior as required for its functionality

4 Licensing - Unlimited Type




For an IUT supporting unlimited licensing options (PT_PURCHASE, expiration value BV_UNLIMITED), the application should function properly without decrementing or expiring.

DEVELOPER TEST NOTE: The Brew MIF Editor may be used to add PT_PURCHASE, LT_USES licensing information to the MIF for testing of this functionality prior to TBT submission, with unlimited expiration value. The Brew MIF Editor does not currently support developer testing of time-based licensing functionality.

Note: This test applies to applications using either time-based or usage-based licensing, as unlimited licensing is supported for both.Result: IUT resides on device with unlimited uses. IUT supporting unlimited licensing (per Item Specification Template) operate without depleting licensing or expiring.

5

Use ILicense to handle licensing of only an application. If another file augments the ILicense APIs, do not include actual license information in that file. The file can include information such as a timestamp, and the file can be part of the application folder. Explain your customer licensing scenario in detail on the Item Specification created to submit your item to TBT.

N/A N/A

6For Networking IUT, IUT establishes, maintains, and tears down a data call. After exiting the application (through any means), the data call ends within the linger timeout value (per device data sheet). This is applicable only to applications

N/A IUT ends data call within linger timeout

Licensing - Ilicense Implementation guidelines

Networking

sheet). This is applicable only to applications supporting this capability.

7The IUT correctly handles the use of the Send key. This key is not used except for predefined operations associated with external over-the-air communications (voice calls, data calls, or mobile originated SMS).

N/A Send key should be reserved only for over-the-air communications.

8

Send Key

Touch User Interface Controls




For any "touchable" user interface control elements (those that accept touch-screen input) that require user interaction (e.g., menu options, softkey labels, button labels, etc.), touchable regions must be a minimum of 6 mm by 6 mm and must be discrete touchable regions with no overlap. It is allowed to have Picture/Bitmap of Touch Area less than 6X6mm. But the bitmap should be visible/readable and should not be less than 75% of touch area and should be centrally aligned in all directions. Applies only to devices and applications that support this capability.

Motivation: for devices supporting navigation and user interaction through touch input, touchable interaction areas on the display must be large enough for users to easily touch-activate the desired region.

Motivation: for devices supporting navigation and user interaction through touch input, touchable interaction areas on the display must be large enough for users to easily touch-activate the desired region.

9Brew applications that record QCP audio using full fixed-rate encoding must incorporate the application-level workaround described in Brew App Note: Fixing Audio Recording Issues .

1. Place a voice call on the device while connected to a 4GV network. Maintain the voice call for 10 seconds. Terminate the voice call.2.Start the IUT. Navigate to a screen that allows recording of audio. Record a segment of audio and verify that the audio recording is successful. Repeat step 2 for every screen that supports recording of audio.

When certain chipsets are provisioned on a 4GV (EVRC-B) network, Brew applications recording QCP audio in full fixed rate will not be able to record audio from the time after a voice call has been placed to the next power-cycle of the device. An application-level workaround has been provided by Qualcomm, which permits the application to circumvent this limitation. All Brew applications making use of audio recording must implement this application workaround to ensure that the application functions gracefully on all network

Audio Recording on 4GV (EVRC-B) Networks

infrastructures.Result:-1. Voice call connects and is terminated successfully.2.Audio records successfully.3. No errors are displayed within the IUT. If possible, preview the recorded audio and ensure that the recording was successful. If the IUT processes the audio in any way (including transferring the recorded audio to a server for processing), ensure that the operation completes successfully.

10 Verification of Smartlink File Update AppNote Implementation




A Brew item implementing the workaround described in Brew App Note: SmartLink File Update for Multiple Application Family Support must handle the notifications in the background and deregister from the notifications after updating the SmartLink configuration file.

Preventing smartlink apps from randomly appearing in the foreground by ensuring the App is launched in the background while it is updating the smartlink file.


TRUE BREW Test Procedures

Documents

Transcript of TRUE BREW Test Procedures