How we tested our code "Google way"

HOW WE TESTED OUR CODE“Google way”

2@twincengray #xpdays

Oleksiy Rezchykov

About me

Software Test Engineer

Last 7 years working with Java and Spring

XP/Agile/Lean practitioner

Lazy Pragmatic programmer


Where it all began


Test size concept

My test is using mocks – is it still a Unit test?

I want to test concurrency, my test is using thread sleeps – can I consider this as Functional test?

My test which is using

in-memory DB running for

about a second –

Is it an Integration test?


Test size concept

Small tests Medium tests Large tests Enormous tests

Time Goals (per method)

Execute in less than 100 ms

Execute in less than 1 sec

Execute as quickly as possible

Execute as quickly as possible

Time Limits Enforced

Kill small test targets after 1 minute

Kill medium test targets after5 minutes

Kill large test targets after 15 minutes

Kill enormous test targets after1 hour


Source:http://books.google.com.ua/books?id=vHlTOVTKHeUC&printsec=frontcover&source=gbs_ge_summary_r&cad=0#v=onepage&q=small%20tests&f=false

http://books.google.com.ua/books?id=vHlTOVTKHeUC&printsec=frontcover&source=gbs_ge_summary_r&cad=0#v=onepage&q=small tests&f=false

Test size concept

Resource Large Medium Small

Network Services (Opens a Socket)

Yes localhost only Mocked

Database Yes Yes Mocked

File System Access Yes Yes Mocked

Access to User-Facing Yes Discouraged Mocked

Systems Invoke Syscalls

Yes Discouraged No

Multiple Threads Yes Yes Discouraged

Sleep Statements Yes Yes No

System Properties Yes Yes No


Source:http://books.google.com.ua/books?id=vHlTOVTKHeUC&printsec=frontcover&source=gbs_ge_summary_r&cad=0#v=onepage&q=small%20tests&f=false

http://books.google.com.ua/books?id=vHlTOVTKHeUC&printsec=frontcover&source=gbs_ge_summary_r&cad=0#v=onepage&q=small tests&f=false

Test size concept

Small tests lead to code quality.

Medium and large tests lead to product quality.


Context

Playtika factsFounded in 2010Multiplatform social games developerHeadquarters in Tel-AvivOffices in US, Canada, Ukraine, Belarus, Romania and Argentina

Industry factsSocial Casino segment is growing faster than other Social Games

segmentsMobile platforms is the main focus for developers

More about Playtika and Social Casino genre:http://www.slideshare.net/eladkushnir/final-ccsf-elad-kushnir


http://www.slideshare.net/eladkushnir/final-ccsf-elad-kushnir

Context

My team is called Infra (Infrastructure)

We are providing infrastructure (logic, which is mostly not related to the game itself) for different games (Slotomania, Caesars Casino), platforms (Web, Mobile) and social networks (FB, Yahoo, e.t.c.)

SOA

Services Microservices clusters

Spring Framework, SQL, NoSQL,

e.t.c.


One day …


Starting point

“Unit” “Functional” “Integration”

Present for each service

Yes No No

Run on CI Yes No No

Coverage Depends on legacy code %*

Some feature acceptance cases

“Smoke” suite, tricky cases

Environment set-up needed

No Yes Yes

Duration Up to 35 sec** Up to 10 min n/a***


*- All new services was written using TDD so coverage was quite high** - In some services we had to test stored procedures*** - Those tests were heavily depending on environment configuration

Small tests

JUnit tests/Mockito tests

Data driven unit/functional tests (framework)

Short tests with Spring Context

Even stored procedures tests

using embedded MySQL


A little bit more about our services

Service is a regular Spring/Spring MVC application

Each service provides Library and Module (Plugin) for other services to communicate with it

REST JSON and Hessian calls are used for communication

Configuration Service is used to load/reload settings during start/runtime

Separation between feature toggling and user A/B testing

New services are developed using Spring Boot


“Functional” to Medium tests

So called “Functional” tests

Were not run on CI

Required pre set up environment (DB’s)

Service started in a standalone container

External “fake” Configuration Service is used

Duration - up to 5min


“Functional” to Medium tests

Our improvements

Generally we are trying not to test anything that does not belong to specific service

DB migration to prepare DB’s state anywhere anytime

Meta information about the environment needed

No external processes - embedded container and Configuration

No @DirtiesContext


Medium tests as we have them now

Concurrency tests

Cache tests

Service Configuration testing

Service API testing

Service module/library testing

Persistence layer testing with real life data

Time - less than 2min


DEMO


“Integration” to Large tests

So called “Integration tests”

Integration tests used Docker to store “production snapshot”

Was not run on CI

Not any computer could handle such a large image

Each small change required some efforts updating the image


“Integration” to Large tests

Improvements

Real Conf. Manager, Real persistence, Real services (except payments and stuff like that)

Evolution – Docker, Ansible, Gradle

OPS will do their stuff, we’ll do ours


Large tests as we have them now

Regression

End-2-End scenarios

Acceptance tests for new features


DEMO


Build pipeline

Small

• Small tests run

• Code analyzers

• Artifact publish

Medium

• Medium tests for Game X

• Medium tests for Game Y

• This build is triggered by Git push

Large

• Large tests run

• Manual trigger

Release

• Release artifact publish

• Snapshot increment


QA

Test plan creation (Large, medium tests), AC for the new features

Manual testing (playing games basically)

Automating their work


Production


Plans/Challenges

Test run infrastructure

Load tests for ALL pain points to be run on regular basis

Back-up compatibility tests (Modules/Libraries vs Services)

Make large tests run automatically

Test results specified in artifact metadata

Environment provisioning in real time using TC as trigger


Summary

Testing is needed on all application levels

Each level of testing has it’s own goal

It us useful to group tests according to the run time and not only the test context

Opensource your code in the organization

If your code could be used elsewhere compose a library and write a HOWTO Wiki page

Check Wiki before writing code

Do not reverse your testing pyramid

Improve, innovate, improvise


Thank [email protected]

[email protected]

@twincengray


We are hiring!

mailto:[email protected]

mailto:[email protected]

How we tested our code "Google way"

Software

Transcript of How we tested our code "Google way"