8/12/2019 UTI Testing Criteria

1/27

This is the UTI Testing Criteria (UTC) for Android applications

Unified Testing Initiative:

Section 1. Testing notes

The tests should be performed on a device to which a factory reset has been applied prior to the installation ofthe application to be tested. If an application uses another application to perform a function (such as Facebookapplication for accessing Facebook details) it should be tested as if the application performed that functionitself.

It is not within the scope of these criteria to be able to test the performance of the application on devices withmultiple applications installed.

Section 2. Different types of apps and their associated tests

Simple App definition and tests

A Simple Android Application would be one which;

Does not send SMS/MMSDoes not write data to standard data files e.g. contacts, calendarDoes not write data to external services e.g. social networks.Can access but does not change the state of networking services e.g 3G/Wifi/BluetoothCan access external sites to retrieve informationCan access location informationCan read standard data filesCan read SMS/MMSCan access screen, sound, camera, key boardCan write its own data e.g. store pictures, create documents

A simple Android application will require the following tests to be carried out;

Test Title

1.1 OTA Install1.2 Long launch time3.1 Send/Receive Data3.4 Resource downloading5.2 Message Receive5.3 Incoming call6.1 Memory card operation

7.1 Readability7.3 Screen Repainting7.5 Key Layout ease of use7.8 Function progress7.10 Multiple display format handling7.11 Different screen sizes7.12 Multiple format input handling7.14 Spelling errors7.15 Technical text errors8.1 Language correct operation8.3 Language supported formats9.1 Suspend/resume from main menu

8/12/2019 UTI Testing Criteria

2/27

9.2 Suspend/resume while executing10.1 Application Mute option11.1 Help and About12.1 Functionality Sanity Check13.1 Scrolling in menus13.3 Pause15.1 Application Stability15.2 Application behavior after forced close

16.2 Data DeletionFramework app definition and testsThere are many applications where the same application framework is used repeatedly to create newapplications. This is especially prevalent for dictionaries, books and magazine applications but may occur inany application genre.

For these applications it is clearly excessive to fully test the new application as it is to a large degree anexisting application with new resource files. For these applications the appropriate criteria (simple or complex)should be used in the first instance and then the following tests are to be performed for subsequent derivedapplications.

Test Title1.1 OTA Install3.1 Send/Receive Data3.4 Resource downloading7.1 Readability11.1 Help and About12.1 Functionality Sanity Check15.1 Application Stability

Complex ApplicationsAny application which does not fall into the above categories i.e.Frameworkor Simplewill be deemed to beComplexand will be subject to testing against the full criteria.

Smoke TestWikipedia defines a Smoke Test as a first test made after assembly or repairs to a system, to provide someassurance that the system under test will not catastrophically fail.

The Smoke Test is a very basic set of tests that is suitable to confirm that a tested app runs (at least at a basiclevel) on a subsequent device. The Smoke Test can be used to confirm basic handset compatibility, but doesnot guaranteed full functionality.

Test cases to be carried out for a Smoke Test:

Test Title1.1 OTA Install

1.2 Long Launch Time7.1 Readability11.1 Help and about15.1 Application Stability15.2 Application behavior after forced close

Section 3:critical tests, warning tests and levels of warning

Critical TestsAs the name suggests, a Criticaltest must be passed. If an application fails the test then the application has anoverall fail.

8/12/2019 UTI Testing Criteria

3/27

Warning TestsFor a test which is considered Warning, we have allowed for four different results;pass, annoying, difficultandimpossible.

These warninglevels are described as follows;

Pass= the app has passed the test. There are no issuesAnnoying= a minor error has occurred with the app - e.g. one or two typos that would make the application notperfect but still very useableDifficult= a more serious issue has occurred with the app e.g. multiple typos making the application difficult touse but not impossibleImpossible= a very serious issue has occurred with the app - the errors are so bad as to make the applicationunusable.

Once all appropriate tests have been carried out, points should be attributed according to the following scale.

Warning levels:

Annoying = 1 points Difficult = 2 points

Impossible = 4 points

Critical levels = 5 points

For an application to pass, the errors should not add up to more than 3 points. 4 points or more is a failure.

Section 4:The Tests

8/12/2019 UTI Testing Criteria

4/27

8/12/2019 UTI Testing Criteria

5/27

8/12/2019 UTI Testing Criteria

6/27

8/12/2019 UTI Testing Criteria

7/27

8/12/2019 UTI Testing Criteria

8/27

8/12/2019 UTI Testing Criteria

9/27

8/12/2019 UTI Testing Criteria

10/27

8/12/2019 UTI Testing Criteria

11/27

8/12/2019 UTI Testing Criteria

12/27

8/12/2019 UTI Testing Criteria

13/27

8/12/2019 UTI Testing Criteria

14/27

8/12/2019 UTI Testing Criteria

15/27

8/12/2019 UTI Testing Criteria

16/27

8/12/2019 UTI Testing Criteria

17/27

8/12/2019 UTI Testing Criteria

18/27

8/12/2019 UTI Testing Criteria

19/27

8/12/2019 UTI Testing Criteria

20/27

8/12/2019 UTI Testing Criteria

21/27

8/12/2019 UTI Testing Criteria

22/27

8/12/2019 UTI Testing Criteria

23/27

8/12/2019 UTI Testing Criteria

24/27

8/12/2019 UTI Testing Criteria

25/27

8/12/2019 UTI Testing Criteria

26/27

8/12/2019 UTI Testing Criteria

27/27