Javantura v3 - Mutation Testing for everyone – Nicolas Fränkel
-
Upload
hujak-hrvatska-udruga-java-korisnika-croatian-java-user-association -
Category
Technology
-
view
2.008 -
download
0
Transcript of Javantura v3 - Mutation Testing for everyone – Nicolas Fränkel
![Page 1: Javantura v3 - Mutation Testing for everyone – Nicolas Fränkel](https://reader036.fdocuments.net/reader036/viewer/2022062401/58f089381a28ab621b8b4623/html5/thumbnails/1.jpg)
IMPROVE YOUR TEST QUALITY WITH MUTATION TESTINGNICOLAS FRANKEL@nicolas_frankel
![Page 2: Javantura v3 - Mutation Testing for everyone – Nicolas Fränkel](https://reader036.fdocuments.net/reader036/viewer/2022062401/58f089381a28ab621b8b4623/html5/thumbnails/2.jpg)
@nicolas_frankel
ME, MYSELF AND IDeveloper & Architect•As Consultant
Teacher/trainerBook AuthorBlogger
![Page 3: Javantura v3 - Mutation Testing for everyone – Nicolas Fränkel](https://reader036.fdocuments.net/reader036/viewer/2022062401/58f089381a28ab621b8b4623/html5/thumbnails/3.jpg)
@nicolas_frankel #mutationtesting 3
SHAMELESS SELF-PROMOTION
![Page 4: Javantura v3 - Mutation Testing for everyone – Nicolas Fränkel](https://reader036.fdocuments.net/reader036/viewer/2022062401/58f089381a28ab621b8b4623/html5/thumbnails/4.jpg)
MANY KINDS OF TESTINGUnit TestingIntegration TestingEnd-to-end TestingPerformance TestingPenetration TestingExploratory Testingetc.
@nicolas_frankel #mutationtesting 4
![Page 5: Javantura v3 - Mutation Testing for everyone – Nicolas Fränkel](https://reader036.fdocuments.net/reader036/viewer/2022062401/58f089381a28ab621b8b4623/html5/thumbnails/5.jpg)
THEIR ONLY SINGLE GOALEnsure the Quality of the production code
@nicolas_frankel #mutationtesting 5
![Page 6: Javantura v3 - Mutation Testing for everyone – Nicolas Fränkel](https://reader036.fdocuments.net/reader036/viewer/2022062401/58f089381a28ab621b8b4623/html5/thumbnails/6.jpg)
THE PROBLEMHow to check the Quality of the testing code?
@nicolas_frankel #mutationtesting 6
![Page 7: Javantura v3 - Mutation Testing for everyone – Nicolas Fränkel](https://reader036.fdocuments.net/reader036/viewer/2022062401/58f089381a28ab621b8b4623/html5/thumbnails/7.jpg)
CODE COVERAGE“Code coverage is a measure used to describe the degree to which the source code of a program is tested”
--Wikipediahttp://en.wikipedia.org/wiki/
Code_coverage
@nicolas_frankel #mutationtesting 7
![Page 8: Javantura v3 - Mutation Testing for everyone – Nicolas Fränkel](https://reader036.fdocuments.net/reader036/viewer/2022062401/58f089381a28ab621b8b4623/html5/thumbnails/8.jpg)
MEASURING CODE COVERAGECheck whether a source code line is executed during a test•Or Branch Coverage
@nicolas_frankel #mutationtesting 8
![Page 9: Javantura v3 - Mutation Testing for everyone – Nicolas Fränkel](https://reader036.fdocuments.net/reader036/viewer/2022062401/58f089381a28ab621b8b4623/html5/thumbnails/9.jpg)
@nicolas_frankel #mutationtesting 9
COMPUTING CODE COVERAGECC: Code Coverage(in percent)Lexecuted: Number of executed lines of codeLtotal: Number of total lines of code
![Page 10: Javantura v3 - Mutation Testing for everyone – Nicolas Fränkel](https://reader036.fdocuments.net/reader036/viewer/2022062401/58f089381a28ab621b8b4623/html5/thumbnails/10.jpg)
JAVA TOOLS FOR CODE COVERAGEJaCoCoCloverCoberturaetc.
@nicolas_frankel #mutationtesting 10
![Page 11: Javantura v3 - Mutation Testing for everyone – Nicolas Fränkel](https://reader036.fdocuments.net/reader036/viewer/2022062401/58f089381a28ab621b8b4623/html5/thumbnails/11.jpg)
100% CODE COVERAGE?“Is 100% code coverage realistic? Of course it is. If you can write a line of code, you can write another that tests it.”
Robert Martin (Uncle Bob)https://twitter.com/unclebobmarti
n/status/55966620509667328
@nicolas_frankel #mutationtesting 11
![Page 12: Javantura v3 - Mutation Testing for everyone – Nicolas Fränkel](https://reader036.fdocuments.net/reader036/viewer/2022062401/58f089381a28ab621b8b4623/html5/thumbnails/12.jpg)
@nicolas_frankel #mutationtesting 12
ASSERT-LESS TESTING@Testpublic void add_should_add() {new Math().add(1, 1);
}
But, where is the assert?
As long as the Code Coverage is OK…
![Page 13: Javantura v3 - Mutation Testing for everyone – Nicolas Fränkel](https://reader036.fdocuments.net/reader036/viewer/2022062401/58f089381a28ab621b8b4623/html5/thumbnails/13.jpg)
CODE COVERAGE AS A MEASURE OF TEST QUALITYAny metric can be gamed!Code coverage is a metric…
⇒ Code coverage can be gamed• On purpose• Or by accident
@nicolas_frankel #mutationtesting 13
![Page 14: Javantura v3 - Mutation Testing for everyone – Nicolas Fränkel](https://reader036.fdocuments.net/reader036/viewer/2022062401/58f089381a28ab621b8b4623/html5/thumbnails/14.jpg)
CODE COVERAGE AS A MEASURE OF TEST QUALITYCode Coverage lulls you into a false sense of security…
@nicolas_frankel #mutationtesting 14
![Page 15: Javantura v3 - Mutation Testing for everyone – Nicolas Fränkel](https://reader036.fdocuments.net/reader036/viewer/2022062401/58f089381a28ab621b8b4623/html5/thumbnails/15.jpg)
THE PROBLEM STILL STANDSCode coverage cannot ensure test quality• Is there another way?
Mutation Testing to the rescue!
@nicolas_frankel #mutationtesting 15
![Page 16: Javantura v3 - Mutation Testing for everyone – Nicolas Fränkel](https://reader036.fdocuments.net/reader036/viewer/2022062401/58f089381a28ab621b8b4623/html5/thumbnails/16.jpg)
@nicolas_frankel #mutationtesting 16
THE CAST
William StrykerOriginal Source Code
Jason StrykerModified Source Code
a.k.a “The Mutant”
![Page 17: Javantura v3 - Mutation Testing for everyone – Nicolas Fränkel](https://reader036.fdocuments.net/reader036/viewer/2022062401/58f089381a28ab621b8b4623/html5/thumbnails/17.jpg)
@nicolas_frankel #mutationtesting 17
public class Math { public int add(int i1, int i2) { return i1 + i2; }}
public class Math { public int add(int i1, int i2) { return i1 - i2; }}
![Page 18: Javantura v3 - Mutation Testing for everyone – Nicolas Fränkel](https://reader036.fdocuments.net/reader036/viewer/2022062401/58f089381a28ab621b8b4623/html5/thumbnails/18.jpg)
@nicolas_frankel #mutationtesting 18
STANDARD TESTING
✔Execute Test
![Page 19: Javantura v3 - Mutation Testing for everyone – Nicolas Fränkel](https://reader036.fdocuments.net/reader036/viewer/2022062401/58f089381a28ab621b8b4623/html5/thumbnails/19.jpg)
@nicolas_frankel #mutationtesting 19
MUTATION TESTING
?Execute SAME Test
MUTATION
![Page 20: Javantura v3 - Mutation Testing for everyone – Nicolas Fränkel](https://reader036.fdocuments.net/reader036/viewer/2022062401/58f089381a28ab621b8b4623/html5/thumbnails/20.jpg)
@nicolas_frankel #mutationtesting 20
MUTATION TESTING
✗
✔Execute SAME Test
Execute SAME Test
Mutant Killed
Mutant Survived
![Page 21: Javantura v3 - Mutation Testing for everyone – Nicolas Fränkel](https://reader036.fdocuments.net/reader036/viewer/2022062401/58f089381a28ab621b8b4623/html5/thumbnails/21.jpg)
KILLED OR SURVIVING?Surviving means changing the source code did not change the test result• It’s bad!
Killed means changing the source code changed the test result• It’s good
@nicolas_frankel #mutationtesting 21
![Page 22: Javantura v3 - Mutation Testing for everyone – Nicolas Fränkel](https://reader036.fdocuments.net/reader036/viewer/2022062401/58f089381a28ab621b8b4623/html5/thumbnails/22.jpg)
@nicolas_frankel #mutationtesting 22
TEST THE CODEpublic class Math {public int add(int i1, int i2) { return i1 + i2;}
}
@Testpublic void add_should_add() {new Math().add(1, 1);
}
✔Execute Test
![Page 23: Javantura v3 - Mutation Testing for everyone – Nicolas Fränkel](https://reader036.fdocuments.net/reader036/viewer/2022062401/58f089381a28ab621b8b4623/html5/thumbnails/23.jpg)
@nicolas_frankel #mutationtesting 23
SURVIVING MUTANTpublic class Math {public int add(int i1, int i2) { return i1 - i2;}
}
@Testpublic void add_should_add() {new Math().add(1, 1);
}
✔Execute SAME Test
![Page 24: Javantura v3 - Mutation Testing for everyone – Nicolas Fränkel](https://reader036.fdocuments.net/reader036/viewer/2022062401/58f089381a28ab621b8b4623/html5/thumbnails/24.jpg)
@nicolas_frankel #mutationtesting 24
TEST THE CODEpublic class Math {public int add(int i1, int i2) { return i1 + i2;}
}
@Testpublic void add_should_add() {int sum = new Math().add(1, 1);Assert.assertEquals(sum, 2);
}
✔Execute Test
![Page 25: Javantura v3 - Mutation Testing for everyone – Nicolas Fränkel](https://reader036.fdocuments.net/reader036/viewer/2022062401/58f089381a28ab621b8b4623/html5/thumbnails/25.jpg)
@nicolas_frankel #mutationtesting 25
KILLED MUTANTpublic class Math {public int add(int i1, int i2) { return i1 - i2;}
}
@Testpublic void add_should_add() {int sum = new Math().add(1, 1);Assert.assertEquals(sum, 2);
}
✗Execute SAME Test
![Page 26: Javantura v3 - Mutation Testing for everyone – Nicolas Fränkel](https://reader036.fdocuments.net/reader036/viewer/2022062401/58f089381a28ab621b8b4623/html5/thumbnails/26.jpg)
MUTATION TESTING IN JAVAPIT is a tool for Mutation testingAvailable as• Command-line tool•Ant target•Maven plugin
@nicolas_frankel #mutationtesting 26
![Page 27: Javantura v3 - Mutation Testing for everyone – Nicolas Fränkel](https://reader036.fdocuments.net/reader036/viewer/2022062401/58f089381a28ab621b8b4623/html5/thumbnails/27.jpg)
MUTATORSMutators are patterns applied to source code to produce mutations
@nicolas_frankel #mutationtesting 27
![Page 28: Javantura v3 - Mutation Testing for everyone – Nicolas Fränkel](https://reader036.fdocuments.net/reader036/viewer/2022062401/58f089381a28ab621b8b4623/html5/thumbnails/28.jpg)
@nicolas_frankel #mutationtesting 28
PIT MUTATORS SAMPLEName Example source ResultConditionals Boundary > >=Negate Conditionals == !=Remove Conditionals foo == bar trueMath + -Increments foo++ foo--Invert Negatives -foo fooInline Constant static final FOO= 42 static final FOO = 43Return Values return true return falseVoid Method Call System.out.println("foo")Non Void Method Call long t = System.currentTimeMillis() long t = 0Constructor Call Date d = new Date() Date d = null;
![Page 29: Javantura v3 - Mutation Testing for everyone – Nicolas Fränkel](https://reader036.fdocuments.net/reader036/viewer/2022062401/58f089381a28ab621b8b4623/html5/thumbnails/29.jpg)
IMPORTANT MUTATORSConditionals Boundary• Probably a potential serious
bug smell if (foo > bar)
@nicolas_frankel #mutationtesting 29
![Page 30: Javantura v3 - Mutation Testing for everyone – Nicolas Fränkel](https://reader036.fdocuments.net/reader036/viewer/2022062401/58f089381a28ab621b8b4623/html5/thumbnails/30.jpg)
IMPORTANT MUTATORSVoid Method Call Assert.checkNotNull()connection.close()
@nicolas_frankel #mutationtesting 30
![Page 31: Javantura v3 - Mutation Testing for everyone – Nicolas Fränkel](https://reader036.fdocuments.net/reader036/viewer/2022062401/58f089381a28ab621b8b4623/html5/thumbnails/31.jpg)
REMEMBER It’s not because the IDE
generates code safely that it will never change• equals()• hashCode()
@nicolas_frankel #mutationtesting 31
![Page 32: Javantura v3 - Mutation Testing for everyone – Nicolas Fränkel](https://reader036.fdocuments.net/reader036/viewer/2022062401/58f089381a28ab621b8b4623/html5/thumbnails/32.jpg)
FALSE POSITIVESMutation Testing is not 100% bulletproofMight return false positivesBe cautious!
@nicolas_frankel #mutationtesting 32
![Page 33: Javantura v3 - Mutation Testing for everyone – Nicolas Fränkel](https://reader036.fdocuments.net/reader036/viewer/2022062401/58f089381a28ab621b8b4623/html5/thumbnails/33.jpg)
@nicolas_frankel #mutationtesting 33
ENOUGH TALK…
Time for DEMO
![Page 34: Javantura v3 - Mutation Testing for everyone – Nicolas Fränkel](https://reader036.fdocuments.net/reader036/viewer/2022062401/58f089381a28ab621b8b4623/html5/thumbnails/34.jpg)
DRAWBACKSSlowSluggishCrawlingSulkyLethargicetc.
@nicolas_frankel #mutationtesting 38
![Page 35: Javantura v3 - Mutation Testing for everyone – Nicolas Fränkel](https://reader036.fdocuments.net/reader036/viewer/2022062401/58f089381a28ab621b8b4623/html5/thumbnails/35.jpg)
METRICS (KIND OF)On joda-moneymvn clean test-compilemvn surefire:test• Total time: 2.181 s
mvn pit-test...• Total time: 48.634 s
@nicolas_frankel #mutationtesting 39
![Page 36: Javantura v3 - Mutation Testing for everyone – Nicolas Fränkel](https://reader036.fdocuments.net/reader036/viewer/2022062401/58f089381a28ab621b8b4623/html5/thumbnails/36.jpg)
WHY SO SLOW?Analyze test codeFor each class under test• For each mutator
• Create mutation• For each mutation
• Run test• Analyze result• Aggregate results
@nicolas_frankel #mutationtesting 40
![Page 37: Javantura v3 - Mutation Testing for everyone – Nicolas Fränkel](https://reader036.fdocuments.net/reader036/viewer/2022062401/58f089381a28ab621b8b4623/html5/thumbnails/37.jpg)
WORKAROUNDSThis is not acceptable in a normal test runBut there are workarounds
@nicolas_frankel #mutationtesting 41
![Page 38: Javantura v3 - Mutation Testing for everyone – Nicolas Fränkel](https://reader036.fdocuments.net/reader036/viewer/2022062401/58f089381a28ab621b8b4623/html5/thumbnails/38.jpg)
@nicolas_frankel #mutationtesting 42
SET MUTATORS<configuration> <mutators> <mutator> CONSTRUCTOR_CALLS </mutator> <mutator> NON_VOID_METHOD_CALLS </mutator> </mutators></configuration>
![Page 39: Javantura v3 - Mutation Testing for everyone – Nicolas Fränkel](https://reader036.fdocuments.net/reader036/viewer/2022062401/58f089381a28ab621b8b4623/html5/thumbnails/39.jpg)
@nicolas_frankel #mutationtesting 43
SET TARGET CLASSES<configuration> <targetClasses> <param>ch.frankel.pit*</param> </targetClasses></configuration>
![Page 40: Javantura v3 - Mutation Testing for everyone – Nicolas Fränkel](https://reader036.fdocuments.net/reader036/viewer/2022062401/58f089381a28ab621b8b4623/html5/thumbnails/40.jpg)
@nicolas_frankel #mutationtesting 44
SET TARGET TESTS<configuration> <targetTests> <param>ch.frankel.pit*</param> </targetTests></configuration>
![Page 41: Javantura v3 - Mutation Testing for everyone – Nicolas Fränkel](https://reader036.fdocuments.net/reader036/viewer/2022062401/58f089381a28ab621b8b4623/html5/thumbnails/41.jpg)
@nicolas_frankel #mutationtesting 45
DEPENDENCY DISTANCE
1 2
![Page 42: Javantura v3 - Mutation Testing for everyone – Nicolas Fränkel](https://reader036.fdocuments.net/reader036/viewer/2022062401/58f089381a28ab621b8b4623/html5/thumbnails/42.jpg)
@nicolas_frankel #mutationtesting 46
LIMIT DEPENDENCY DISTANCE<configuration> <maxDependencyDistance> 4 </maxDependencyDistance></configuration>
![Page 43: Javantura v3 - Mutation Testing for everyone – Nicolas Fränkel](https://reader036.fdocuments.net/reader036/viewer/2022062401/58f089381a28ab621b8b4623/html5/thumbnails/43.jpg)
@nicolas_frankel #mutationtesting 47
LIMIT NUMBER OF MUTATIONS<configuration> <maxMutationsPerClass> 10 </maxMutationsPerClass></configuration>
![Page 44: Javantura v3 - Mutation Testing for everyone – Nicolas Fränkel](https://reader036.fdocuments.net/reader036/viewer/2022062401/58f089381a28ab621b8b4623/html5/thumbnails/44.jpg)
USE MUTATIONFILTERFrom extension points
@nicolas_frankel #mutationtesting 48
![Page 45: Javantura v3 - Mutation Testing for everyone – Nicolas Fränkel](https://reader036.fdocuments.net/reader036/viewer/2022062401/58f089381a28ab621b8b4623/html5/thumbnails/45.jpg)
PIT EXTENSION POINTSMust be packaged in JARHave Implementation-Vendor and Implementation-Title in MANIFEST.MF that match PIT’sSet on the classpathUse Java’s Service Provider feature
@nicolas_frankel #mutationtesting 49
![Page 46: Javantura v3 - Mutation Testing for everyone – Nicolas Fränkel](https://reader036.fdocuments.net/reader036/viewer/2022062401/58f089381a28ab621b8b4623/html5/thumbnails/46.jpg)
SERVICE PROVIDERInversion of control• Since Java 1.3!
Text file located in META-INF/servicesInterface• Name of the file
Implementation class• Content of the file
@nicolas_frankel #mutationtesting 50
![Page 47: Javantura v3 - Mutation Testing for everyone – Nicolas Fränkel](https://reader036.fdocuments.net/reader036/viewer/2022062401/58f089381a28ab621b8b4623/html5/thumbnails/47.jpg)
@nicolas_frankel #mutationtesting 51
SAMPLE STRUCTURE
JARch.frankel.lts.Sample
![Page 48: Javantura v3 - Mutation Testing for everyone – Nicolas Fränkel](https://reader036.fdocuments.net/reader036/viewer/2022062401/58f089381a28ab621b8b4623/html5/thumbnails/48.jpg)
@nicolas_frankel #mutationtesting 52
SERVICE PROVIDER API
![Page 49: Javantura v3 - Mutation Testing for everyone – Nicolas Fränkel](https://reader036.fdocuments.net/reader036/viewer/2022062401/58f089381a28ab621b8b4623/html5/thumbnails/49.jpg)
@nicolas_frankel #mutationtesting 53
SERVICE PROVIDER SAMPLEServiceLoader<ISample> loaders = ServiceLoader.load(ISample.class);for (ISample sample: loaders) {
// Use the sample}
![Page 50: Javantura v3 - Mutation Testing for everyone – Nicolas Fränkel](https://reader036.fdocuments.net/reader036/viewer/2022062401/58f089381a28ab621b8b4623/html5/thumbnails/50.jpg)
OUTPUT FORMATSOut-of-the-box•HTML• XML• CSV
@nicolas_frankel #mutationtesting 54
![Page 51: Javantura v3 - Mutation Testing for everyone – Nicolas Fränkel](https://reader036.fdocuments.net/reader036/viewer/2022062401/58f089381a28ab621b8b4623/html5/thumbnails/51.jpg)
@nicolas_frankel #mutationtesting 55
OUTPUT FORMATS<configuration> <outputFormats> <outputFormat>XML</outputFormat> <outputFormat>HTML</outputFormat> </outputFormats></configuration>
![Page 52: Javantura v3 - Mutation Testing for everyone – Nicolas Fränkel](https://reader036.fdocuments.net/reader036/viewer/2022062401/58f089381a28ab621b8b4623/html5/thumbnails/52.jpg)
@nicolas_frankel #mutationtesting 56
![Page 53: Javantura v3 - Mutation Testing for everyone – Nicolas Fränkel](https://reader036.fdocuments.net/reader036/viewer/2022062401/58f089381a28ab621b8b4623/html5/thumbnails/53.jpg)
MUTATION FILTERRemove mutations from the list of available mutations for a class
@nicolas_frankel #mutationtesting 57
![Page 54: Javantura v3 - Mutation Testing for everyone – Nicolas Fränkel](https://reader036.fdocuments.net/reader036/viewer/2022062401/58f089381a28ab621b8b4623/html5/thumbnails/54.jpg)
@nicolas_frankel #mutationtesting 58
![Page 55: Javantura v3 - Mutation Testing for everyone – Nicolas Fränkel](https://reader036.fdocuments.net/reader036/viewer/2022062401/58f089381a28ab621b8b4623/html5/thumbnails/55.jpg)
@nicolas_frankel #mutationtesting 59
Time for DEMO
![Page 56: Javantura v3 - Mutation Testing for everyone – Nicolas Fränkel](https://reader036.fdocuments.net/reader036/viewer/2022062401/58f089381a28ab621b8b4623/html5/thumbnails/56.jpg)
@nicolas_frankel #mutationtesting 60
DON’T BIND TO TEST PHASE!<plugin> <groupId>org.pitest</groupId> <artifactId>pitest-maven</artifactId> <executions> <execution> <goals> <goal>mutationCoverage</goal> </goals> <phase>test</phase> </execution> </executions></plugin>
![Page 57: Javantura v3 - Mutation Testing for everyone – Nicolas Fränkel](https://reader036.fdocuments.net/reader036/viewer/2022062401/58f089381a28ab621b8b4623/html5/thumbnails/57.jpg)
@nicolas_frankel #mutationtesting 61
USE SCMMUTATIONCOVERAGEmvn \org.pitest:pitest-maven:scmMutationCoverage \-DtimestampedReports=false
maven-scm-
plugin
must be configured!
![Page 58: Javantura v3 - Mutation Testing for everyone – Nicolas Fränkel](https://reader036.fdocuments.net/reader036/viewer/2022062401/58f089381a28ab621b8b4623/html5/thumbnails/58.jpg)
@nicolas_frankel #mutationtesting 62
DO USE ON CONTINUOUS INTEGRATION SERVERSmvn \org.pitest:pitest-maven:mutationCoverage \-DtimestampedReports=false
![Page 59: Javantura v3 - Mutation Testing for everyone – Nicolas Fränkel](https://reader036.fdocuments.net/reader036/viewer/2022062401/58f089381a28ab621b8b4623/html5/thumbnails/59.jpg)
IS MUTATION TESTING THE SILVER BULLET?Sorry, no!It only• Checks the relevance of your
unit tests• Points out potential bugs
@nicolas_frankel #mutationtesting 63
![Page 60: Javantura v3 - Mutation Testing for everyone – Nicolas Fränkel](https://reader036.fdocuments.net/reader036/viewer/2022062401/58f089381a28ab621b8b4623/html5/thumbnails/60.jpg)
WHAT IT DOESN’T DOValidate the assembled application• Integration Testing
Check the performance• Performance Testing
Look out for display bugs• End-to-end testing
Etc.
@nicolas_frankel #mutationtesting 64
![Page 61: Javantura v3 - Mutation Testing for everyone – Nicolas Fränkel](https://reader036.fdocuments.net/reader036/viewer/2022062401/58f089381a28ab621b8b4623/html5/thumbnails/61.jpg)
TESTING IS ABOUT ROIDon’t test to achieve 100% coverageTest because it saves money in the long runPrioritize:• Business-critical code• Complex code
@nicolas_frankel #mutationtesting 65
![Page 62: Javantura v3 - Mutation Testing for everyone – Nicolas Fränkel](https://reader036.fdocuments.net/reader036/viewer/2022062401/58f089381a28ab621b8b4623/html5/thumbnails/62.jpg)
@nicolas_frankel
Q&A@nicolas_frankelhttp://blog.frankel.ch/https://leanpub.com/integrationtest/