CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing &...
Transcript of CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing &...
![Page 1: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/1.jpg)
Unit Testing & Testability
CMPT 473Software Quality Assurance
Nick Sumnerwith material from the GoogleTest documentation
![Page 2: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/2.jpg)
Test Suite Design
● Objectives– Functional correctness– Nonfunctional attributes (performance, ...)
![Page 3: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/3.jpg)
Test Suite Design
● Objectives– Functional correctness– Nonfunctional attributes (performance, ...)
● Components – The Automated Testing Pyramid
Unit
API/Integration/Component/
System UI
![Page 4: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/4.jpg)
Test Suite Design
● Objectives– Functional correctness– Nonfunctional attributes (performance, ...)
● Components – The Automated Testing Pyramid
Unit
API/Integration/Component/
System UI
![Page 5: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/5.jpg)
Test Suite Design
● Objectives– Functional correctness– Nonfunctional attributes (performance, ...)
● Components – The Automated Testing Pyramid
Unit
API/Integration/Component/
System UI
![Page 6: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/6.jpg)
Test Suite Design
● Objectives– Functional correctness– Nonfunctional attributes (performance, ...)
● Components – The Automated Testing Pyramid
Unit
API/Integration/Component/
System UI
![Page 7: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/7.jpg)
Test Suite Design
● Objectives– Functional correctness– Nonfunctional attributes (performance, ...)
● Components – The Automated Testing Pyramid
Unit
API/Integration/Component/
System UI
Integrated
Isolated
![Page 8: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/8.jpg)
Test Suite Design
● Objectives– Functional correctness– Nonfunctional attributes (performance, ...)
● Components – The Automated Testing Pyramid
Unit
API/Integration/Component/
System UI
Integrated
Isolated
Slow
Fast
![Page 9: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/9.jpg)
Test Suite Design
● Objectives– Functional correctness– Nonfunctional attributes (performance, ...)
● Components – The Automated Testing Pyramid
Unit
API/Integration/Component/
System UI
Integrated
Isolated
Slow
Fast
![Page 10: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/10.jpg)
10
Levels of Testing
● Many different levels of testing can be considered:
– Unit Tests
– Integration Tests
– System Tests
– Acceptance Tests
– …
![Page 11: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/11.jpg)
11
Levels of Testing
● Many different levels of testing can be considered:
– Unit Tests
– Integration Tests
– System Tests
– Acceptance Tests
– …
● The simplest of these is Unit Testing
– Testing the smallest possible fragments of a program
![Page 12: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/12.jpg)
12
Unit Testing
● Try to ensure that the functionality of each component works in isolation
![Page 13: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/13.jpg)
13
Unit Testing
● Try to ensure that the functionality of each component works in isolation
– Unit Test a car:Wheels work. Steering wheel works....
![Page 14: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/14.jpg)
14
Unit Testing
● Try to ensure that the functionality of each component works in isolation
– Unit Test a car:Wheels work. Steering wheel works....
– Integration Test a car:Steering wheel turns the wheels....
![Page 15: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/15.jpg)
15
Unit Testing
● Try to ensure that the functionality of each component works in isolation
– Unit Test a car:Wheels work. Steering wheel works....
– Integration Test a car:Steering wheel turns the wheels....
– System Test a car: Driving down the highway with the air conditioning on works...
![Page 16: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/16.jpg)
16
Unit Testing
● Try to ensure that the functionality of each component works in isolation
– Unit Test a car:Wheels work. Steering wheel works....
– Integration Test a car:Steering wheel turns the wheels....
– System Test a car: Driving down the highway with the air conditioning on works....
● Not testing how well things are glued together.
![Page 17: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/17.jpg)
17
Unit Testing
● Try to ensure that the functionality of each component works in isolation
– Unit Test a car:Wheels work. Steering wheel works....
– Integration Test a car:Steering wheel turns the wheels....
– System Test a car: Driving down the highway with the air conditioning on works....
● Not testing how well things are glued together.
● In practice, there is a lot more debate on this than you might expect
![Page 18: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/18.jpg)
18
Unit Testing
● Try to ensure that the functionality of each component works in isolation
– Unit Test a car:Wheels work. Steering wheel works....
– Integration Test a car:Steering wheel turns the wheels....
– System Test a car: Driving down the highway with the air conditioning on works....
● Not testing how well things are glued together.
● In practice, there is a lot more debate on this than you might expect– Degrees of isolation
![Page 19: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/19.jpg)
19
Unit Testing
● Try to ensure that the functionality of each component works in isolation
– Unit Test a car:Wheels work. Steering wheel works....
– Integration Test a car:Steering wheel turns the wheels....
– System Test a car: Driving down the highway with the air conditioning on works....
● Not testing how well things are glued together.
● In practice, there is a lot more debate on this than you might expect– Degrees of isolation– Big & Small vs Unit & Integration
![Page 20: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/20.jpg)
20
Unit Testing
● Try to ensure that the functionality of each component works in isolation
– Unit Test a car:Wheels work. Steering wheel works....
– Integration Test a car:Steering wheel turns the wheels....
– System Test a car: Driving down the highway with the air conditioning on works....
● Not testing how well things are glued together.
● In practice, there is a lot more debate on this than you might expect– Degrees of isolation– Big & Small vs Unit & Integration– ...
The rapid feedback advantageof unit tests persists for refactoring,
but there are judgement calls.
![Page 21: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/21.jpg)
21
Unit Tests
● A dual view:
– They specify the expected behavior of individual components
![Page 22: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/22.jpg)
22
Unit Tests
● A dual view:
– They specify the expected behavior of individual components
– An executable specification
![Page 23: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/23.jpg)
23
Unit Tests
● A dual view:
– They specify the expected behavior of individual components
– An executable specification
● Can even be built first & used to guide development
– Usually called Test Driven Development
![Page 24: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/24.jpg)
24
Unit Tests
● A dual view:
– They specify the expected behavior of individual components
– An executable specification
● Can even be built first & used to guide development
– Usually called Test Driven Development
In practice, this has fallen out of favor(if it ever was in favor).
![Page 25: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/25.jpg)
25
Unit Tests
● Some guiding principles:
– Focus on one component in isolation
– Be simple to set up & run
– Be easy to understand
![Page 26: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/26.jpg)
26
Unit Tests
● Some guiding principles:
– Focus on one component in isolation
– Be simple to set up & run
– Be easy to understand
● Usually managed by some automating framework ....
![Page 27: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/27.jpg)
27
GoogleTest
● Increasingly used framework for C++
– Not dissimilar from JUnit, which you have already seen.
![Page 28: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/28.jpg)
28
GoogleTest
● Increasingly used framework for C++
– Not dissimilar from JUnit, which you have already seen.
● Test cases are written as functions:
TEST(TriangleTest, isEquilateral) { Triangle tri{2,2,2}; EXPECT_TRUE(tri.isEquilateral());}
![Page 29: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/29.jpg)
29
TEST(TriangleTest, isEquilateral) { Triangle tri{2,2,2}; EXPECT_TRUE(tri.isEquilateral());}
GoogleTest
● Increasingly used framework for C++
– Not dissimilar from JUnit, which you have already seen.
● Test cases are written as functions:
The TEST macro definesindividual test cases.
![Page 30: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/30.jpg)
30
TEST(TriangleTest, isEquilateral) { Triangle tri{2,2,2}; EXPECT_TRUE(tri.isEquilateral());}
GoogleTest
● Increasingly used framework for C++
– Not dissimilar from JUnit, which you have already seen.
● Test cases are written as functions:The first argument
names related tests.
![Page 31: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/31.jpg)
31
TEST(TriangleTest, isEquilateral) { Triangle tri{2,2,2}; EXPECT_TRUE(tri.isEquilateral());}
GoogleTest
● Increasingly used framework for C++
– Not dissimilar from JUnit, which you have already seen.
● Test cases are written as functions:The second argument
names individual test cases.
![Page 32: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/32.jpg)
32
TEST(TriangleTest, isEquilateral) { Triangle tri{2,2,2}; EXPECT_TRUE(tri.isEquilateral());}
GoogleTest
● Increasingly used framework for C++
– Not dissimilar from JUnit, which you have already seen.
● Test cases are written as functions:
EXPECT and ASSERT macrosprovide correctness oracles.
![Page 33: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/33.jpg)
33
GoogleTest
● Increasingly used framework for C++
– Not dissimilar from JUnit, which you have already seen.
● Test cases are written as functions:
ASSERT oracles terminate the program when they fail.EXPECT oracles allow the program to continue running.
TEST(TriangleTest, isEquilateral) { Triangle tri{2,2,2}; EXPECT_TRUE(tri.isEquilateral());}
![Page 34: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/34.jpg)
34
GoogleTest
● Increasingly used framework for C++
– Not dissimilar from JUnit, which you have already seen.
● Test cases are written as functions.
● TEST() cases are automatically registered with GoogleTest and are executed by the test driver.
![Page 35: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/35.jpg)
35
GoogleTest
● Increasingly used framework for C++
– Not dissimilar from JUnit, which you have already seen.
● Test cases are written as functions.
● TEST() cases are automatically registered with GoogleTest and are executed by the test driver.
● Some tests require common setUp & tearDown
– Group them into test fixtures
– A fresh fixture is created for each test
![Page 36: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/36.jpg)
36
GoogleTest
● Increasingly used framework for C++
– Not dissimilar from JUnit, which you have already seen.
● Test cases are written as functions.
● TEST() cases are automatically registered with GoogleTest and are executed by the test driver.
● Some tests require common setUp & tearDown
– Group them into test fixtures
– A fresh fixture is created for each test
– Fixtures enable using the same configuration for multiple tests
![Page 37: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/37.jpg)
37
GoogleTest - Fixtures
class StackTest : public ::testing::Test { protected: void SetUp() override { s1.push(1); s2.push(2); s2.push(3); }
void TearDown() override { }
Stack<int> s1; Stack<int> s2;};
Derive from the fixture base class
![Page 38: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/38.jpg)
38
class StackTest : public ::testing::Test { protected: void SetUp() override { s1.push(1); s2.push(2); s2.push(3); }
void TearDown() override { }
Stack<int> s1; Stack<int> s2;};
GoogleTest - Fixtures
SetUp() will be called beforeall tests using the fixture
![Page 39: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/39.jpg)
39
class StackTest : public ::testing::Test { protected: void SetUp() override { s1.push(1); s2.push(2); s2.push(3); }
void TearDown() override { }
Stack<int> s1; Stack<int> s2;};
GoogleTest - Fixtures
TearDown() will be called afterall tests using the fixture
![Page 40: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/40.jpg)
40
GoogleTest - Fixtures
Use the fixture in test cases defined with TEST_F:
TEST_F(StackTest, popOfOneIsEmpty) { s1.pop(); EXPECT_EQ(0, s1.size());}
![Page 41: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/41.jpg)
41
TEST_F(StackTest, popOfOneIsEmpty) { s1.pop(); EXPECT_EQ(0, s1.size());}
GoogleTest - Fixtures
Use the fixture in test cases defined with TEST_F:
![Page 42: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/42.jpg)
42
GoogleTest - Fixtures
Use the fixture in test cases defined with TEST_F:
Behaves like
TEST_F(StackTest, popOfOneIsEmpty) { s1.pop(); EXPECT_EQ(0, s1.size());}
{ StackTest t; t.SetUp(); t.popOfOneIsEmpty(); t.TearDown();}
![Page 43: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/43.jpg)
43
TEST_F(StackTest, popOfOneIsEmpty) { s1.pop(); EXPECT_EQ(0, s1.size());}
GoogleTest - Fixtures
Use the fixture in test cases defined with TEST_F:
A different expectation than before!
![Page 44: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/44.jpg)
44
TEST_F(StackTest, popOfOneIsEmpty) { s1.pop(); EXPECT_EQ(0, s1.size());}
GoogleTest - Fixtures
Use the fixture in test cases defined with TEST_F:
expectedvalue
![Page 45: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/45.jpg)
45
TEST_F(StackTest, popOfOneIsEmpty) { s1.pop(); EXPECT_EQ(0, s1.size());}
GoogleTest - Fixtures
Use the fixture in test cases defined with TEST_F:
expectedvalue
observedvalue
![Page 46: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/46.jpg)
46
GoogleTest
● Increasingly used framework for C++
– Not dissimilar from JUnit, which you have already seen.
● Test cases are written as functions.
● TEST() cases are automatically registered with GoogleTest and are executed by the test driver.
● Some tests require common setUp & tearDown
● Many different assertions and expectations availableASSERT_TRUE(condition);ASSERT_FALSE(condition);ASSERT_EQ(expected,actual);ASSERT_NE(val1,val2);ASSERT_LT(val1,val2);ASSERT_LE(val1,val2);ASSERT_GT(val1,val2);ASSERT_GE(val1,val2);
EXPECT_TRUE(condition);EXPECT_FALSE(condition);EXPECT_EQ(expected,actual);EXPECT_NE(val1,val2);EXPECT_LT(val1,val2);EXPECT_LE(val1,val2);EXPECT_GT(val1,val2);EXPECT_GE(val1,val2);
…
![Page 47: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/47.jpg)
47
GoogleTest
● Increasingly used framework for C++
– Not dissimilar from JUnit, which you have already seen.
● Test cases are written as functions.
● TEST() cases are automatically registered with GoogleTest and are executed by the test driver.
● Some tests require common setUp & tearDown
● Many different assertions and expectations available
● More information available online
– github.com/google/googletest/blob/master/googletest/docs/Primer.md
– github.com/google/googletest/blob/master/googletest/docs/AdvancedGuide.md
![Page 48: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/48.jpg)
Designing a Unit Test
● Common structure
![Page 49: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/49.jpg)
Designing a Unit Test
● Common structure
TEST_CASE("empty") { Environment env; ExprTree tree;
auto result = evaluate(tree, env);
CHECK(!result.has_value());}
![Page 50: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/50.jpg)
Designing a Unit Test
● Common structure
TEST_CASE("empty") { Environment env; ExprTree tree;
auto result = evaluate(tree, env);
CHECK(!result.has_value());}
This specific test usesanother framework
called Doctest
![Page 51: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/51.jpg)
Designing a Unit Test
● Common structure
TEST_CASE("empty") { Environment env; ExprTree tree;
auto result = evaluate(tree, env);
CHECK(!result.has_value());}
Set up a scenario
![Page 52: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/52.jpg)
Designing a Unit Test
● Common structure
TEST_CASE("empty") { Environment env; ExprTree tree;
auto result = evaluate(tree, env);
CHECK(!result.has_value());}
Run the scenario
![Page 53: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/53.jpg)
Designing a Unit Test
● Common structure
TEST_CASE("empty") { Environment env; ExprTree tree;
auto result = evaluate(tree, env);
CHECK(!result.has_value());} Check the outcome
![Page 54: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/54.jpg)
Designing a Unit Test
● Common structure
TEST_CASE("empty") { Environment env; ExprTree tree;
auto result = evaluate(tree, env);
CHECK(!result.has_value());}
This is sometimes known as AAA:ArrangeActAssert
![Page 55: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/55.jpg)
Designing a Unit Test
● Common structure
● Tests should run in isolation
struct Frob { Frob() : conn{getDB().connect()} { } DBConnection conn;};
![Page 56: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/56.jpg)
Designing a Unit Test
● Common structure
● Tests should run in isolation
struct Frob { Frob() : conn{getDB().connect()} { } DBConnection conn;};
TEST_CASE("bad test 1") { Frob frob; ...}
TEST_CASE("bad test 2") { Frob frob; ...}
![Page 57: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/57.jpg)
Designing a Unit Test
● Common structure
● Tests should run in isolation
struct Frob { Frob() : conn{getDB().connect()} { } DBConnection conn;};
TEST_CASE("bad test 1") { Frob frob; ...}
TEST_CASE("bad test 2") { Frob frob; ...}
![Page 58: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/58.jpg)
Designing a Unit Test
● Common structure
● Tests should run in isolation
struct Frob { Frob() : conn{getDB().connect()} { } DBConnection conn;};
TEST_CASE("bad test 1") { Frob frob; ...}
TEST_CASE("bad test 2") { Frob frob; ...}
The order of the test can affect the results!
![Page 59: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/59.jpg)
Designing a Unit Test
● Common structure
● Tests should run in isolation
struct Frob { Frob() : conn{getDB().connect()} { } DBConnection conn;};
TEST_CASE("bad test 1") { Frob frob; ...}
TEST_CASE("bad test 2") { Frob frob; ...}
The order of the test can affect the results!
A flaky DB can affect results!
![Page 60: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/60.jpg)
Designing a Unit Test
● Common structure
● Tests should run in isolation!
![Page 61: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/61.jpg)
Designing a Unit Test
● Common structure
● Tests should run in isolation
struct Frob { Frob(Connection& inConn) : conn{inConn} { } Connection& conn;};
![Page 62: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/62.jpg)
Designing a Unit Test
● Common structure
● Tests should run in isolation
struct Frob { Frob(Connection& inConn) : conn{inConn} { } Connection& conn;};
Dependency injection allowsthe user of a class tocontrol its behavior
![Page 63: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/63.jpg)
Designing a Unit Test
● Common structure
● Tests should run in isolation
struct Frob { Frob(Connection& inConn) : conn{inConn} { } Connection& conn;};
Connection
Dependency injection allowsthe user of a class tocontrol its behavior
![Page 64: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/64.jpg)
Designing a Unit Test
● Common structure
● Tests should run in isolation
struct Frob { Frob(Connection& inConn) : conn{inConn} { } Connection& conn;};
Connection
DBConnection
Dependency injection allowsthe user of a class tocontrol its behavior
![Page 65: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/65.jpg)
Designing a Unit Test
● Common structure
● Tests should run in isolation
struct Frob { Frob(Connection& inConn) : conn{inConn} { } Connection& conn;};
Connection
DBConnection FakeConnection
Dependency injection allowsthe user of a class tocontrol its behavior
![Page 66: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/66.jpg)
Designing a Unit Test
● Common structure
● Tests should run in isolation
struct Frob { Frob(Connection& inConn) : conn{inConn} { } Connection& conn;};
TEST_CASE("better test 1") { FakeDB db; FakeConnection conn = db.connect(); Frob frob{conn}; ...}
Connection
DBConnection FakeConnection
![Page 67: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/67.jpg)
Designing a Unit Test
● Common structure
● Tests should run in isolation
struct Frob { Frob(Connection& inConn) : conn{inConn} { } Connection& conn;};
TEST_CASE("better test 1") { FakeDB db; FakeConnection conn = db.connect(); Frob frob{conn}; ...}
Connection
DBConnection
FakeConnection
![Page 68: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/68.jpg)
68
Common Patterns (Ammonn & Offutt)
● Checking State
– Final State● Prepare initial state● Run test● Check final state
![Page 69: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/69.jpg)
69
Common Patterns (Ammonn & Offutt)
● Checking State
– Final State● Prepare initial state● Run test● Check final state
– Pre and Post conditions● Check initial state as well as final state
![Page 70: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/70.jpg)
70
Common Patterns (Ammonn & Offutt)
● Checking State
– Final State● Prepare initial state● Run test● Check final state
– Pre and Post conditions● Check initial state as well as final state
– Relative effects● Check final state relative to some initial state
![Page 71: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/71.jpg)
71
Common Patterns (Ammonn & Offutt)
● Checking State
– Final State● Prepare initial state● Run test● Check final state
– Pre and Post conditions● Check initial state as well as final state
– Relative effects● Check final state relative to some initial state
– Round trips● Check behavior on transform/inverse transform pairs
![Page 72: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/72.jpg)
72
Common Patterns (Ammonn & Offutt)
● Checking Interactions/Behavior
– Use mocks
![Page 73: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/73.jpg)
73
Common Patterns (Ammonn & Offutt)
● Checking Interactions/Behavior
– Use mocks● Testing 'fakes' that verify expected interactions● http://martinfowler.com/articles/mocksArentStubs.html● http://googletesting.blogspot.ca/2013/03/testing-on-toilet-testing-state-vs.html
![Page 74: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/74.jpg)
74
Common Patterns (Ammonn & Offutt)
● Checking Interactions/Behavior
– Use mocks● Testing 'fakes' that verify expected interactions● http://martinfowler.com/articles/mocksArentStubs.html● http://googletesting.blogspot.ca/2013/03/testing-on-toilet-testing-state-vs.html
TEST_CASE("better test 1") { FakeDB db; FakeConnection conn = db.connect(); Frob frob{conn}; ...}
The FakeConnection could checkthat DB interactions are correct.
![Page 75: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/75.jpg)
75
Common Patterns (Ammonn & Offutt)
● Checking Interactions/Behavior
– Use mocks● Testing 'fakes' that verify expected interactions● http://martinfowler.com/articles/mocksArentStubs.html● http://googletesting.blogspot.ca/2013/03/testing-on-toilet-testing-state-vs.html
TEST_CASE("better test 1") { FakeDB db; FakeConnection conn = db.connect(); Frob frob{conn}; ...}
The FakeConnection could checkthat DB interactions are correct.The FakeConnection could checkthat DB interactions are correct.
NOTE: Test doubles for isolation are good,but mocks should be used sparingly.
![Page 76: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/76.jpg)
76
Testability
● What makes testing hard?
– Not just difficult to get adequacy
– What makes it difficult to write tests?
![Page 77: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/77.jpg)
77
Testability
● What makes testing hard?
– Not just difficult to get adequacy
– What makes it difficult to write tests?
● Dependencies
– Connections between classes
![Page 78: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/78.jpg)
78
Testability
● What makes testing hard?
– Not just difficult to get adequacy
– What makes it difficult to write tests?
● Dependencies
– Connections between classes– Singletons
![Page 79: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/79.jpg)
79
Testability
● What makes testing hard?
– Not just difficult to get adequacy
– What makes it difficult to write tests?
● Dependencies
– Connections between classes– Singletons– Nondeterminism
![Page 80: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/80.jpg)
80
Testability
● What makes testing hard?
– Not just difficult to get adequacy
– What makes it difficult to write tests?
● Dependencies
– Connections between classes– Singletons– Nondeterminism– Static binding (mitigated by parametric polymorphism)
![Page 81: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/81.jpg)
81
Testability
● What makes testing hard?
– Not just difficult to get adequacy
– What makes it difficult to write tests?
● Dependencies
– Connections between classes– Singletons– Nondeterminism– Static binding– Mixing construction & application logic– ...
![Page 82: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/82.jpg)
82
Testability
● What makes testing hard?
– Not just difficult to get adequacy
– What makes it difficult to write tests?
● Dependencies
– Connections between classes– Singletons– Nondeterminism– Static binding– Mixing construction & application logic– ...
But solutions exist!You can design code to be testable!
![Page 83: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/83.jpg)
83
Testability (by example)
● Let's work together to improve some difficult to test code....
![Page 84: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/84.jpg)
84
Testability
● Keys things to notice:
– Mocks & stubs allow us to isolate components under test
![Page 85: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/85.jpg)
85
Testability
● Keys things to notice:
– Mocks & stubs allow us to isolate components under test
– Dependency Injection allows us to use mocks and stubs as necessary
![Page 86: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/86.jpg)
86
Testability
● Keys things to notice:
– Mocks & stubs allow us to isolate components under test
– Dependency Injection allows us to use mocks and stubs as necessary
– But doing this can lead to a lot more work and boilerplate code when written by hand
![Page 87: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/87.jpg)
87
Testability
● Keys things to notice:
– Mocks & stubs allow us to isolate components under test
– Dependency Injection allows us to use mocks and stubs as necessary
– But doing this can lead to a lot more work and boilerplate code when written by hand
Given dependency injection,what happens to the way we create objects?
![Page 88: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/88.jpg)
88
Testability
● Keys things to notice:
– Mocks & stubs allow us to isolate components under test
– Dependency Injection allows us to use mocks and stubs as necessary
– But doing this can lead to a lot more work and boilerplate code when written by hand
Given dependency injection,what happens to the way we create objects?
How might we mitigateboilerplate issues?
![Page 89: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/89.jpg)
89
Mocking Framework Example
● Frameworks exist that can automate the boilerplate behind:
![Page 90: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/90.jpg)
90
Mocking Framework Example
● Frameworks exist that can automate the boilerplate behind:
– Mocking
e.g. GoogleMock, Mockito, etc.
![Page 91: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/91.jpg)
91
Mocking Framework Example
● Frameworks exist that can automate the boilerplate behind:
– Mocking● e.g. GoogleMock, Mockito, etc.
– Dependency Injection
e.g. Google Guice, Pico Container, etc.
![Page 92: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/92.jpg)
92
Using GoogleMock
● Steps:
1) Derive a mock class from the class you wish to fake
![Page 93: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/93.jpg)
93
Using GoogleMock
● Steps:
1) Derive a mock class from the class you wish to fake
class Thing { public: virtual int foo(int x); virtual void bar(int y);};
![Page 94: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/94.jpg)
94
Using GoogleMock
● Steps:
1) Derive a mock class from the class you wish to fake
class Thing { public: virtual int foo(int x); virtual void bar(int y);};
class MockThing : public Thing { public: ... MOCK_METHOD(int, foo, (int x), (override)); MOCK_METHOD(void, bar, (int y), (override));};
![Page 95: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/95.jpg)
95
Using GoogleMock
● Steps:
1) Derive a mock class from the class you wish to fake
2) Replace virtual calls with uses of MOCK_METHOD().
class MockThing : public Thing { public: ... MOCK_METHOD(int, foo, (int x), (override)); MOCK_METHOD(void, bar, (int y), (override));};
class Thing { public: virtual int foo(int x); virtual void bar(int y);};
![Page 96: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/96.jpg)
96
Using GoogleMock
● Steps:
1) Derive a mock class from the class you wish to fake
2) Replace virtual calls with uses of MOCK_METHOD().
3) Use the mock class in your tests.
![Page 97: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/97.jpg)
97
Using GoogleMock
● Steps:
1) Derive a mock class from the class you wish to fake
2) Replace virtual calls with uses of MOCK_METHOD().
3) Use the mock class in your tests.
4) Specify expectations before use via EXPECT_CALL().● What arguments? How many times? In what order?
InSequence dummy;EXPECT_CALL(mockThing, foo(Ge(20))) .Times(2) .WillOnce(Return(100)) .WillOnce(Return(200));EXPECT_CALL(mockThing, bar(Lt(5)));
![Page 98: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/98.jpg)
98
Using GoogleMock
● Steps:
1) Derive a mock class from the class you wish to fake
2) Replace virtual calls with uses of MOCK_METHOD().
3) Use the mock class in your tests.
4) Specify expectations before use via EXPECT_CALL().● What arguments? How many times? In what order?
5) Expectations are automatically checked in the destructor of the mock.
![Page 99: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/99.jpg)
99
Using GoogleMock
● Precisely specifying mock behavior
InSequence dummy;EXPECT_CALL(mockThing, foo(Ge(20))) .Times(2) // Can be omitted here .WillOnce(Return(100)) .WillOnce(Return(200));EXPECT_CALL(mockThing, bar(Lt(5)));
![Page 100: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/100.jpg)
100
Using GoogleMock
● Precisely specifying mock behavior
InSequence dummy;EXPECT_CALL(mockThing, foo(Ge(20))) .Times(2) // Can be omitted here .WillOnce(Return(100)) .WillOnce(Return(200));EXPECT_CALL(mockThing, bar(Lt(5)));
![Page 101: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/101.jpg)
101
Using GoogleMock
● Precisely specifying mock behavior
InSequence dummy;EXPECT_CALL(mockThing, foo(Ge(20))) .Times(2) // Can be omitted here .WillOnce(Return(100)) .WillOnce(Return(200));EXPECT_CALL(mockThing, bar(Lt(5)));
![Page 102: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/102.jpg)
102
Using GoogleMock
● Precisely specifying mock behavior
InSequence dummy;EXPECT_CALL(mockThing, foo(Ge(20))) .Times(2) // Can be omitted here .WillOnce(Return(100)) .WillOnce(Return(200));EXPECT_CALL(mockThing, bar(Lt(5)));
![Page 103: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/103.jpg)
103
Using GoogleMock
● Precisely specifying mock behavior
InSequence dummy;EXPECT_CALL(mockThing, foo(Ge(20))) .Times(2) // Can be omitted here .WillOnce(Return(100)) .WillOnce(Return(200));EXPECT_CALL(mockThing, bar(Lt(5)));
![Page 104: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/104.jpg)
104
Using GoogleMock
● Precisely specifying mock behavior
InSequence dummy;EXPECT_CALL(mockThing, foo(Ge(20))) .Times(2) // Can be omitted here .WillOnce(Return(100)) .WillOnce(Return(200));EXPECT_CALL(mockThing, bar(Lt(5)));Complex behaviors can be checked
using these basic pieces.
![Page 105: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/105.jpg)
105
Using GoogleMock
● Note, GoogleMock can use the same process for creatingboth stubs and mocks as well as test fakes in the middle.
![Page 106: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/106.jpg)
106
Using GoogleMock
● Note, GoogleMock can use the same process for creatingboth stubs and mocks as well as test fakes in the middle.
● A mock will check that a function is called in the right ways.
![Page 107: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/107.jpg)
107
Using GoogleMock
● Note, GoogleMock can use the same process for creatingboth stubs and mocks as well as test fakes in the middle.
● A mock will check that a function is called in the right ways.
● A stub will prevent interaction with external resources and possibly return fake data.
![Page 108: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/108.jpg)
108
Using GoogleMock
● Note, GoogleMock can use the same process for creatingboth stubs and mocks as well as test fakes in the middle.
● A mock will check that a function is called in the right ways.
● A stub will prevent interaction with external resources and possibly return fake data.
What might this imply aboutwhere you use mocks vs
where you use stubs?
![Page 109: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/109.jpg)
109
Summary
● Unit testing provides a way to automate much of the testing process.
![Page 110: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/110.jpg)
110
Summary
● Unit testing provides a way to automate much of the testing process.
● Testing small components bootstraps confidence in the system on confidence in its constituents.
![Page 111: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/111.jpg)
111
Summary
● Unit testing provides a way to automate much of the testing process.
● Testing small components bootstraps confidence in the system on confidence in its constituents.
● Tests can verify state or behaviors.
![Page 112: CMPT 473 Software Quality Assurance Unit Testing & Testability · 2020. 3. 27. · Unit Testing & Testability CMPT 473 Software Quality Assurance Nick Sumner with material from the](https://reader035.fdocuments.net/reader035/viewer/2022062219/5fde168feeae9e4ade33e12a/html5/thumbnails/112.jpg)
112
Summary
● Unit testing provides a way to automate much of the testing process.
● Testing small components bootstraps confidence in the system on confidence in its constituents.
● Tests can verify state or behaviors.
● Software must be designed for testing (or designed by testing)