Top100summit 跨平台自動化測試與工具徐翔智

揭示研发管理白金定律，分享那些激动人心的创新与变革，使得团队获得过多源动力与更大的推动力！

跨平台的自动化测试与工具

Tony Hsu徐祥智Sr Quality Manager

[email protected]

趨勢科技

http://top100summit.com/content.php?id=365



跨平台自动化测试与工具

案例背景实践

案例ROI分析

• UI Error handled • Multi-threads • Reboot scenario

失敗與放棄

•Automation Framework •Build Portal •L10n Automation •Tools

案例启示

未来想尝试

•Automation Framework •Win Browser UI Automation •iOS Mobile Automation •Android Mobile Automation

參考資料

Manual Automation

30.84 hr/man 1.1 hr/man


案例背景 http://top100summit.com/content.php?id=365

• Launch in 2012 March

• Windows, iPhone, iPad, Android Phone and Android Pad. Support Browser IE, FireFox and Chrome.

• 20+ team members development team size (QA/RD)

• Cross-site testing with localization team in JP/China & support team in every region.

• Multilingual User Interface (MUI) build.

10 + Years

87600 hours 2102400 Min In software development

About Me

• Sr Quality Manager / Sr Project Manager •Consumer product development group • TrendMicro



跨平台的自动化测试与工具

Automation Framework

Testing Report

Build Portal for Cross-platform

L10n Testing Automation

目標


实践 1.1 - Automation Framework – Testing Flow

B1001 B1002 B1003 B1004….

•Build monitoring •Build/Testing Deployment •Auto-Testing Execution •Sending Report

•QA Report


实践 1.2 – Automation Framework Key Component

•Daily Build Monitor

•Information for automation server

•Testing target machines

•Source location

•Server

•Revert the client VM

• package the Build and Testing script

• Deploy resource to client

• Trigger automation testing

• Collect testing result and send email

•Client

• Testing scripts Execution

• Generate testing report


实践 1.3 - Automation Framework –Infrastructure Blocks

Product Library

Testing Script

Add-on Module

Common Library

Python STAF Core Basis

Add-on

Application Testing

7


实践 1.4 - Automation Framework – Infrastructure

Windows Auto Testing

Android Auto Testing

Common lib

Test suite

Test Runner

Test Cases

iOS Auto Testing

Linux Auto Testing

TestResult.txt


Common lib

Plink Tool

iOS Auto Testing

Profile

UIAutomation

Instruments

iOSview001.js

iOSview001.js

实践 1.5 - iOS

STAF

9


实践 1.6 – Android

Common lib

Android project

STAF

Psexec.exe

Android Test project - Robotium

Instruments

TestView01.apk

Androidview001.java

I i

Python & ADB

androidview001_001.py

PC for Android

Android emulator

10

实践 2.1 – Automation Testing Report – Key Info

•Build Number •Testing Machine Name/IP •Testing Time •Platform •Results •Crashes •Testing Cases •Further Link for logs/screenshot


实践 2.2 – Automation Testing Report–Link to details

•Testing Steps of failure • Logs • Screenshot


实践 3.1 – Build Status Portal

Problem to Solve under cross regional team Environment.

1. How and Where to download Build?

2. What’s change in the build?

3. What’s the build configuration Status?


实践 3.2 – Build Status Portal

1. Build Number

2. Build Result

3. FTP download URL for Windows

4. SAMBA download URL for Mac OS

5. What's New in the build

6. Build Time

7. Configuration Highlight

8. Build Log


实践 4.1 – Localization Automation Testing

The problem to Solve for L10n UI verification

1. UI changes impact on other languages?

2. How to efficiently validate OEM customized UI changes?

3. How to validate UI truncation issue after language translation?


实践 4.2 – Localization Automation Testing – UI Truncation

1. Source code parser for • “SetText” with Android • “NSString” with iOS 2. Word counts of the parsing results 3. Compare the UI width 4. List of potential UI truncation ISsue


实践 4.3 – Localization Automation Testing – UI Screenshot Automation

•Key Technology •iOS UIAutomation •Robotium


失败经验 • UI Error handled

• Multi-threads testing execution

• Reboot scenario

未来尝试

• More Testing Cases.

• MetroApp Automation


效益分析

Before (Manual) After (Automation)

Testing Efficiency 30.84 hr/man 1.1 hr/man

Testing Coverage

Platform Win7_x32 W7_x32 , Vista_x32 , W7_x64 + Vista_x64 + WinXp and so on.

Browser IE, Firefox, Chrome = 3 IE, Firefox, Chrome with different versions

Language EN = 1 EN, JP, FR, TW, DE = 5


成功要素

1. Clearly Define the automation testing scope and staged-goal

2. Uses of Tools

3. Plan automation tasks as parts of QA testing cycle

4. Testing Report links to further info/logs to trace the issue

成功经验总结

1. Automation Framework (Build Monitoring, Testing Server and Client)

2. Integration with iOS/Android and Windows

哪些技术促使项目成功。

1. For automation framework, Python and Software Testing Automation Framework (STAF)

2. For Android UI Automation, Robotium

3. For Browser UI control, Selenium and JsonWireProtocol

4. For Testing Client environment, VM workstation, ESXi, Lab Manager


案例启示 -跨平台的自动化测试与工具

•Automation Framework •Consolidated reporting •VM workstation, ESXi, LabManager •Software Testing Automation Framework (STAF) •Python •Build Portal •Consolidated build status and download for cross platform build.

•L10n Automation •UI verification •Tools •Selenium UI control •Robotium •iOS UIAutomation •iOS/Android emulator


參考資料

•Automation Framework •http://www.python.org/download/ •http://sourceforge.net/projects/pywin32/ •http://staf.sourceforge.net/getcurrent.php •Windows Browser UI Automation •http://code.google.com/p/selenium/ •iOS Mobile Automation •Introduction to Automating UI Testing in iOS •iOS Automated Tests with UIAutomation •UI Automation JavaScript Reference •Android Mobile Automation •http://code.google.com/p/robotium/

http://www.python.org/download/

http://staf.sourceforge.net/getcurrent.php

http://sourceforge.net/projects/pywin32/


http://guerratopia.com/en/introduction-to-automating-ui-testing-in-ios








http://blog.manbolo.com/2012/04/08/ios-automated-tests-with-uiautomation


http://developer.apple.com/library/ios/



http://code.google.com/p/robotium/


www.top100summit.com


www.top100summit.com


跨平台自动化测试与工具

案例背景实践

案例ROI分析

• UI Error handled • Multi-threads • Reboot scenario

失敗與放棄

•Automation Framework •Build Portal •L10n Automation •Tools

案例启示

未来想尝试

•Automation Framework •Win Browser UI Automation •iOS Mobile Automation •Android Mobile Automation

參考資料

Manual Automation

30.84 hr/man 1.1 hr/man

Top100summit 跨平台自動化測試與工具徐翔智

Documents

Transcript of Top100summit 跨平台自動化測試與工具徐翔智

Top100summit 跨平台自動化測試與工具 徐翔智

Documents

Transcript of Top100summit 跨平台自動化測試與工具 徐翔智

Top100summit 跨平台自動化測試與工具徐翔智

Transcript of Top100summit 跨平台自動化測試與工具徐翔智