Software Test Plan for Recommender Agent to Extract Usefulness of Product Reviews

8/14/2019 Software Test Plan for Recommender Agent to Extract Usefulness of Product Reviews

1/25


2/25

II

Table of Contents

Revision History .............................................................................................................................4

1. Introduction ..............................................................................................................................5

1.1 Identification ..................................................................................................................... 5

1.2 Purpose ............................................................................................................................. 5

1.3 Scope ................................................................................................................................ 5

1.4 Test Objectives .................................................................................................................. 6

1.5 System Overview .............................................................................................................. 6

1.6 Intended Audience and Reading Suggestions ................................................................... 7

1.7 Definitions and Acronyms ................................................................................................. 7

1.8 Supporting Materials ........................................................................................................ 8

2. Software Test Environment ....................................................................................................9

2.1 Hardware .......................................................................................................................... 9

2.2 Software ........................................................................................................................... 9

2.3 Testing Tools and Environment ........................................................................................ 9

3. Outline of Planned Test .........................................................................................................10

3.1 Outline of Test Inclusion ................................................................................................. 10

3.2 Outline of Other Candidated for Potential Inclusion ...................................................... 10

3.1 Outline of Test Exclusion ................................................................................................. 10

4. Test Planning Process ............................................................................................................11

4.1 Test Plan Development ................................................................................................... 11

4.2 Test Case Development .................................................................................................. 11

4.3 Test Execution ................................................................................................................. 17


3/25

III

5. Testing .....................................................................................................................................18

5.1 Functional Testing ........................................................................................................... 18

5.2 Data integrity Testing ...................................................................................................... 18

5.3 User Interface Testing ..................................................................................................... 19

5.4 Stress Testing .................................................................................................................. 19

5.5 Installation Testing .......................................................................................................... 20

5.6 Integration Testing .......................................................................................................... 21

6. Test Tools Reports .................................................................................................................22

7. Project Enhancement.............................................................................................................24

9. Supporting Information ........................................................................................................25

Appendix A: Glossary..................................................................................................................25


4/25


5/25

A RECOMMNEDER AGENT TO EXTRACT USEFULNESS OF PRODUCT FROM

REVIEWS USING ARTIFICIAL INTELLIGENCE5

1. Introduction:

This document is a procedural guide for designing test cases and test documentation. This software test

plan (STP) addresses and provide guidance for the testing activities performed for the NexGen-S

Recommender system. It is designed to capture and convey the overall structure and objectives of the

NexGen-S Recommender Test and Evaluation (T&E) activities.

1.1 Identification:

This software test plan provides detailed tests that are used to evaluate the stability and compliance of the

Next Generation Shopping Recommender System ("NexGen-S Recommender") Version 1.0.

1.2 Purpose:

The purpose of this Test Plan is to gather all of the information necessary to plan and control the test

effort for testing of NexGen-S system. The main purpose of this document is provides a basis for

planning, performing, managing, monitoring, and measuring the NexGen-S recommender system testing

activities. This document relies heavily on the Software Requirement Specification and Software Design

Document of the same product as a basis for the requirements. Specifically, this plan documents the

following:

Management of a testing strategy that addresses the evolution of the design, incremental delivery,testing efficiency, and testing coverage as well as the systems known areas of risk;

An overview of the testing process to include testing phases and processes for evaluating testadequacy;

Test facility, test equipment, and test support requirements; Measurement and reporting of test work products and test results; and The approach for developing acceptance criteria.

1.3 Scope:

This test plan describes the unit, subsystem integration, and system level tests that will be performed on

components of NexGen-S system. The NexGen-S recommender system code developed for this project

builds heavily on the Dot Net Framework 2. The software test plan outlined in this document will focus

on the functionality developed for the NexGen-S code.


6/25



Unit tests will be done through test driver program that perform boundary checking and basic black box

testing.

Subsystem tests will verify defined interfaces between the following packages:

POS Tagger Sentimental Analyzer Concept Identifier Recommender Agent GUI

The external interfaces to be tested:

Ecommerce Shop. Database interfaces that are supported (MSQL).

1.4 Test objective:

The goal of testing that is covered in this document will be achieved by completing the following

objectives:

Verifying that the system adheres to program specifications Developing and updating NexGen-S Test Cases Producing test reports from testing activities

1.5 System Overview:

Recommender systems attempt to predict items in which a user might be interested, given some

information about the users and items profiles. The proposed system act as a recommender to address

the problem of using consumer opinion about products, expressed online in free-form text, to generate

product recommendations.

The intention of the proposed system is to reduce the information load of the customers and resolve the

ambiguities involved in the existing systems. Our proposed involves following steps, for review

information to be useful for the recommendation process, we must translate it into a structured form and


7/25



communicate it to the process in a form suitable for generating recommendations. Weve developed and

employed ontology to translate opinions quality and content into a form the recommender process can

use. The text-mining process automatically maps the review comments into the ontologys information

structure.

A ranking mechanism computes a products rating using the information from the consumer reviews. It

prioritizes that information with respect to the consumers level of expertise in using the product under

consideration. The recommender system makes a recommendation based on the computed data.

1.6 Intended Audience and Reading Suggestions:

The Intended audience of this document is system testers who can develop test plan and software

developers who use this test plan as a metric to determine how well NexGen-S has been tested.

1.7 Definitions, Acronyms or Abbreviations:

Acceptance criteria:The criteria that a system or component must satisfy in order to be acceptedby a user, customer, or other authorized entity.

Acceptance testing: Formal testing conducted to determine whether a system satisfies itsacceptance criteria and enables the customer to determine whether to accept the system.

Functional testing: Testing that ignores the internal mechanism of a system or component andfocuses solely on the outputs generated in response to selected inputs and execution conditions.

Integration testing:Testing in which software components, hardware components, or both arecombined and tested to evaluate the interaction between them.

Integration testing:Testing in which software components, hardware components, or both arecombined and tested to evaluate the interaction between them.

Load testing:Testing that studies the behavior of the program when it is working at its limits. NexGen-S: Next Generation Shopping Recommender Agent Pass/Fail criteria:Decision rules used to determine whether a software item or software feature

passes or fails a test.

Performance testing:Testing conducted to evaluate the compliance of a system or componentwith specified performance requirements.

Stress testing: Testing conducted to evaluate a system or component at or beyond the limits of itsspecified requirements.

Structural testing: Testing that takes into account the internal mechanism of a system orcomponent. Types include branch testing, path testing, statement testing.


8/25



System testing: Testing conducted on a complete, integrated system to evaluate the systemscompliance with its specified requirements.

Test Case. Test Case is a commonly used term for a specific test. This is usually the smallest unitof testing. A Test Case will consist of information such as requirements testing, test steps,

verification steps, prerequisites, outputs, test environment, etc.

Test Procedure. A document, providing detailed instructions for the [manual] execution of oneor more test cases.

Test Tools: Computer programs used in the testing of a system, a component of the system, or itsdocumentation.

Unit Testing: Testing of individual software components. Validation: The process of evaluating software at the end of the software development process to

ensure compliance with software requirements. The techniques for validation are testing,

inspection and reviewing.

Verification and Validation (V&V):Verification and validation performed by an organizationthat is technically, managerially, and financially independent of the development organization.

1.8 References:

Software Requirement Specification of Recommender Agent: Extract Usefulness of Product fromReviews using AI Techniques.

Silvana Aciar, Debbie Zhang, Simeon Simoff, and John Debenham. Informed Recommender:Basing Recommendations on Consumer Product Reviews.

Zhu Zhang, University of Arizona. Weighing Stars: Aggregating Online Product Reviews forIntelligent E-commerce Applications.

Kushal Dave, Steve Lawrence, and David M. Pennock. Mining the Peanut Gallery: OpinionExtraction and Semantic Classification of Product Reviews.

IEEE Standard for Test Documentation. (ANSI/IEEE Standard 829-1983)


9/25



2. Software Test Environment:

The following have been identified as targets for testing:

2.1 Hardware

Following will be the Hardware resources:

Intel Pentium based PCs

2.2 Software

Following will be the Software resources:

Browser: IE (5.x, 6.x)/ Mozilla (4.7, 7.0)

Dot Net Framework 2.0

NUnit 2.4.8

Ncover

Visual Studio 2005 Professional Edition.

2.3 Testing Tool & Environments:

We are using NUnit (http://www.nunit.org) with Ncover as a testing framework.

2.3.1 Testing Tools:

Microsoft Stress Tool. NUnit with Ncover Code Coverage Tool. Test Matrix.

Also testing will be conducted in the following environments:

Priority Browser/Environment Version Comments

1 Internet Explorer 6.0

2 Mozilla 3.0.9


10/25


11/25



4. Test Planning Process

NexGen-S software testing will be performed by developing test cases. The test process will be divided

into three categories:

Test planning

Test case development

Test case execution

4.1Test Plan Development:

The Test Plan reveals the definition of high-level program specifications and sets up the approach for

testing. The Test Plan also defines and describes the test environment, test schedules and resources.

4.2 Test Case Development:

Test cases will be developed to support the use cases identified in the system requirement specification of

NexGen-S and based on the subsystems.

4.2.1 Test Case Identification:

Some of the major test cases that are identified based on functional specifications and use cases are listed

below:

Register as User Login to the system. Browse Product Catalog. View Product Recommendations. Add or Remove products in Catalog. Finalize Order.

Test cases:

Test cases identified based on use cases and sub systems are categorized based on each sub system of the

NexGen-S recommender system. Each test case will contain a set of discrete test steps beginning with a

Login step and ending with a Logout step. The functional test cases will be developed to include each

identified Functional Program Specification.


12/25



4.2.2 Web Shop:

Login Page:

Test Case: Check the Logging Process (For Valid User)

Test Case # 1

Pre Condition Open a browser window and type the url to access the login page.

Case Description Verify whether the system correctly allows the valid user to log in

and access the user information

Test Case Steps 1. Enter the User Id in the textbox2. Enter the password in the textbox3. Click the Login Button

Expected result It allows the user to navigate to the user area and allowed to access

the payment service of the system.

Observed Result User is navigated to the user area.

Test Status Pass

Remarks The user with valid password and user id is only allowed to accessthe system.

Test Case: Verify Admin Logging process

Test Case # 2

Pre Condition Open a browser window and type the url to access the login page.


and access the system information

Test Case Steps 1. Enter the User Id as Admin in the textbox2. Enter the admin password in the textbox3. Click the Login Button

Expected result It allows the admin to logging in to secure area of the system and

access or modify other user details

Observed Result Admin is navigated to the secure area of the system.

Test Status Pass

Remarks The other users are restricted to use secure area and super user is

only allowed to access the secure area.


13/25



User Registration Page:

Test Case: Verify the User Registration Process

Test Case # 3

Pre Condition Open a browser window and type the url to access the main page.


and access the system information

Test Case Steps 1. Enter the url of main page2. Click the New User link3. Enter all the details without blank4. Click the button submit

Expected result It displays the message registration success and stores all the details

of user in the system database.

Observed Result It shows the message Registration Success and navigate user to the

user area of the system

Test Status Pass

Remarks All the details of the user is entered to the system database without

fails

Site Navigation Menus:

Test Case: Check Site navigation

Test Case # 4


Case Description Verify whether the link address points valid URL

Test Case Steps 1. Click the navigation URLs or LinksExpected result It navigates the user to the correct path

Observed Result User is navigated to the desired location or path

Test Status Pass

Remarks Links are working properly


14/25



View Products:

Test Case: View product Description with Images

Test Case # 5


Case Description Verify whether the system properly displays the product description

with proper images.

Test Case Steps 1. Enter the Product name in the search button or Select theproduct link

2. Click the Search button or Product LinkExpected result It allows the user to navigate to the product description page and

images are properly loaded.

Observed Result User is navigated to the product description area.

Test Status Pass

Remarks Product with Recommendation and Description page is properly

loaded with images

Add/Delete Products to catalog:

Test Case: Add the user interested product to the shopping cart

Test Case # 6


Case Description Verify whether the system properly add the interested product to the

user shopping cart.

Test Case Steps 1. Enter the Product name in the search button or Select theproduct link

2. Select the specific product from category.3. Click add product to cart link to add the product.

Expected result It allows the user to add the interested product to the product

catalog.

Observed Result User interested product is added to the shopping cart.

Test Status Pass

Remarks Interested product is added to the user product shopping cart.


15/25



Test Case: Delete the unwanted/uninterested product from the shopping cart

Test Case # 7


Case Description Verify whether the system properly delete the uninterested productto the user card.

Test Case Steps 1. Click the Shopping Cart link.2. Select the product to be cancelled.3. Click cancel button to cancel the uninterested product.

Expected result It allows the user to delete the uninterested product in the shoppingcart.

Observed Result User uninterested product is deleted in the shopping cart.

Test Status Pass

Remarks Uninterested product is deleted from the user shopping cart.

Finalize order:

Test Case: Finalize the user orders

Test Case # 8

Pre Condition User must log inn otherwise user is redirected to log in page before

finalize the order.Case Description Verify whether the system properly allows the user to pay for the

products and finalize the order.

Test Case Steps 1. Click the Proceed to Check Out button.2. Redirect the user to check out page.3. Check whether the user profile is completed.4. If profile is completed, then get the payment option.5. Redirect to Thank u page.

Expected result It allows the user to finalize the order.

Observed Result User order is finalized and user is redirected to thank u page.

Test Status Pass

Remarks Finalize order page is properly loaded with the shopping details of

customer.


16/25



4.2.3 Part of Speech Tagger:

Identification POS of words

Test Case: Identify part of speech of each word in the sentence

Test Case # 9

Pre Condition Sentence should be given as input

Case Description Verify whether the system properly identify Pos tag for each word

inn the sentence.

Test Case Steps 1. Input sentence is tokenized as words.2. Each word should be given as input to Pos tagger.3. Display identified tags for each word.

Expected result It displays the identified pos tag for each word in the sentence.

Observed Result POS tag for each word is displayed properly.

Test Status Pass

Remarks POS tagger properly identifies the pos tag of each word.

4.2.4 Sentimental Analyzer:

Identification of sentiment for each sentence:

Test Case: Identify sentiment of each sentence based on POS tagger

Test Case # 10

Pre Condition Sentence with identified POS tags should be given as input.

Case Description Verify whether the system properly identify sentiment of each

sentence.

Test Case Steps 1. Sentence with POS tag should be given as input.2. Sentence is either categorized as positive or negative.

Expected result It displays the sentiment of each sentence.

Observed Result Sentence sentiment is displayed properly.

Test Status Pass

Remarks Sentimental analyzer properly identifies sentiment of each sentence.


17/25



4.3 Test Execution

The goal of test execution is to exercise test cases to ensure that explicit and implicit specifications or

requirements have been addressed.

4.3.1 Test Execution Sequence:

Following are the sequence of operations conducted after each software version or build is delivered for

NexGen-S testing:

Functional Requirements Testing. Unit Testing. System Testing. Integration Testing. User Interface Testing Production Verification Testing. Regression Testing.


18/25


19/25



5.3 User Interface Testing:

User Interface (UI) testing verifies a users interaction with the software. The goal of UI testing is to

ensure that the UI provides the user with the appropriate access and navigation through the functions of

the target-of-test.

Objective: Verify the user interface of the system like characteristics of objects

(menus, size, position, state, and focus) and access methods (tab keys,

mouse movements, accelerator keys).

Technique: Create or modify tests for each window to verify proper navigation and

object states for each application window and object.

Success Criteria: All window objects can be exercised, proper navigation through test

target, and test target acts as expected.

5.4 Stress Testing

Stress testing is a type of performance test implemented and executed to understand how a system fails

due to conditions at the boundary, or outside of, the expected tolerances. This typically involves low

resources or competition for resources.

Objective: Verify the system behavior under the stress conditions (multiple users

request a service, running a system with below recommended

requirements) to observe and identify the conditions under which the

system fails to continue functioning properly.

Technique: To test limited resources, tests should be run on a singlemachine, and RAM and persistent storage space on the server

should be reduced or limited.

Make queries against a locked database, or a database under aheavy load.


20/25



Success Criteria: NexGen-S inability to handle the review process when thereviews are more than 1000.

NexGen-S inability to access the database resources and providethe responses to the user queries in the correct time.

5.5 Installation Testing

Installation testing has two purposes. The first is to ensure that the software can be installed under

different conditions such as a new installation, an upgrade, and a complete or custom

installation under normal and abnormal conditions. Abnormal conditions include insufficient disk

space, lack of privilege to create directories, and so on. The second purpose is to verify that, once

installed, the software operates correctly. This usually means running a number of the tests that were

developed for Function Testing.

Technique Objective: Verify the system installation in the target environment under following

installation behavior and observe the system configuration state changes:

New installation: a new machine, never installed previously withNexGen-S system.

Update: a machine previously installed NexGen-S, same versionTechnique: Install to a new machine and verify basic functionality.

Install to a previously installed machine and verify basicfunctionality.

Install to a new machine without dot net framework. Install to a new machine with an incorrect version of dot net

framework.

Success Criteria: The technique supports the testing of the installation of the developed

product in one or more installation configurations.


21/25



5.6 Integration Testing:

Integration testing is a testing of combined parts of an application to determine if they function together

correctly. Usually integration testing is performed after unit and functional testing.

Technique Objective: Verify the functionality of the system after it is integrated from its

subsystems or sub modules.

Integration Strategy: Functional grouping.Technique: Unit test all the sub systems before integration.

Verify the integrated system result with the result achievedwith the help of stubs.

Elements to be

Integrated and

Integration Sequence:

The elements to be integrated and sequence of these elements is listed

below:

POS tagger. Sentimental analyzer. Concept Identifier. Recommender Agent. GUI Subsystem.

Hardware Integration

and Elementsrequired: Nil

Tools Used: Ncover testing framework.

Success Criteria: The system produces the expected result after the integration of the

system by final functional test.


22/25


23/25



Screen Snapshot # 3: Code Coverage

Screen Snapshot # 3: Module Summary

7. Project Enhancement:


24/25



Designing a system to extract the usefulness of product from reviews or comments (in free form text) is

not an easy task due to some legacy problems involved natural language processing and text mining

process. To the best of our knowledge, this is the best way to extract the usefulness of product from the

free form text. For example, the author based analysis is incorporated in to the system to increase the

efficiency of recommendation because a reviews usefulness depends on not only its authors intri nsic

character but also the author expertise in the product category. Several other challenges like

recommendation data generated from the system is purely depend on the dataset used to generate

recommendations and we couldnt classify some long, complicated sentences in to any category. Such

sentences should be broken in to several short sentences before classification. The future enhancement of

the project is depends up on long sentence classification and introducing the power of neural networks. In

future work, we intend to evaluate our approach with the intended customer groups in the real-world

applications.


25/25



8. Supporting Information

Appendix A: Glossary

Acceptance Test: Acceptance tests are done performed by a group of end-users or clients of thesystem to ensure the developed system meets the requirements specified.

Domain Testing: The essence of this type of testing is sampling. We reduce a massive set ofpossible tests to a small group by dividing (partitioning) the set into subsets (sub domains) and

picking one or two representatives from each subset.

Function Testing: Test each function / feature / variable in isolation. Scenario Testing: A scenario is a story that describes a hypothetical situation. In testing, you

check how the program copes with this hypothetical situation.

Specification Based testing: Check the program against every claim made in a referencedocument, such as a design specification, a requirements list, a user interface description, a

published model, or a user manual.

Test Case: A test case in software engineering is a set of conditions or variables under which atester will determine whether an application or software system meets specifications.

Test Plan: A test plan is a systematic approach to testing a system such as a machine or software.The plan typically contains a detailed understanding of what the eventual workflow will be

User Testing: User testing is done by users. Not by testers pretending to be users. Not bysecretaries or executives pretending to be testers pretending to be users. By users. People who

will make use of the finished product.

Software Test Plan for Recommender Agent to Extract Usefulness of Product Reviews

Documents

Transcript of Software Test Plan for Recommender Agent to Extract Usefulness of Product Reviews