Introduction to DevOps - KU Leuvenjoost.vennekens/DN/introduction_to_code... · Introduction to...

Introduction to DevOps

Maarten Vandersteegen

De Nayer, KU Leuven

November 6, 2019

0 – What? 2

Introduction to DevOps – Maarten Vandersteegen

0 – Demo project 3


0 – Demo project - hardware 4


0 – Demo project - software 5

https://github.com/maartenvds/rpi-ci-lamp


https://github.com/maartenvds/rpi-ci-lamp

0 – Content 6

1 Code versioning

2 Working in team

3 Test automation

4 Static and Dynamic code analysis

5 Continuous Integration


1 – Outline 7

1 Code versioning

2 Working in team

3 Test automation




1 – Centralized version control 8


1 – Centralized version control - examples 9


1 – Distributed version control 10


1 – Distributed version control - examples 11


1 – Source code hosting in the cloud 12

• git• free public repos• free private (students)

• git, mercurial• free private repos (5 user

limit)

• git• free private repos

• svn, git, mercurial• free public repos


1 – Demo project - version control 13


1 – Demo project - workflow 14

# deve lopment branchg i t p u l lg i t add my f i l e ; g i t commit. . .g i t add my f i l e ; g i t commitg i t push

# r e l e a s e to masterg i t checkout masterg i t p u l lg i t merge deve lopmentg i t pushg i t checkout deve lopment


2 – Outline 15

1 Code versioning

2 Working in team

3 Test automation




2 – Task management 16


2 – Task management - github 17


2 – Documentation 18

API documentation (.h files)1 /*2 * Parse HTTP response3 * in: HTTP response string from server4 * regex_passed: Regex that matches ’in’ when the build has passed5 * regex_running: Regex that matches ’in’ when the build is running6 * regex_failed: Regex that matches ’in’ when the build has failed7 * state: The resulting build status8 * return: 0 on success, -1 on failure9 */

10 int response_parser_get_result(const char *in, const char *regex_passed,11 const char *regex_running,12 const char *regex_failed,13 enum BuildState *state);1415

Use wiki for functional documentation:• tutorials/HOWTOs• use case examples


2 – Code review - github 19


2 – Atlassian toolbox 20


3 – Outline 21

1 Code versioning

2 Working in team

3 Test automation




3 – Layers of testing 22


3 – Flow of a unit test 24

• fixating the test conditions to a knownstate

• One or more actions that operate on theSUT (system under test)

• One or more conditions that need to betrue

• undo setup


3 – Basic unit test in C using cmocka 25

1 #include <stdarg.h>2 #include <stddef.h>3 #include <setjmp.h>4 #include <cmocka.h> /* yes, all these includes are needed */56 void test_example(void **state)7 {8 assert_true(1);9 }

1011 int main(void)12 {13 const struct CMUnitTest tests[] = {14 cmocka_unit_test(test_example),15 };16 return cmocka_run_group_tests(tests, NULL, NULL);17 }18

gcc -Wall test.c -o test -lcmocka && ./test[==========] Running 1 test(s).[ RUN ] test_example[ OK ] test_example[==========] 1 test(s) run.[ PASSED ] 1 test(s).


3 – Unit test example 26

response_parser.h1 int response_parser_get_result(const char *in, const char *regex_passed,2 const char *regex_running,3 const char *regex_failed,4 enum BuildState *state);5

test.c1 void test_response_parser_build_passed(void **state)2 {3 int res;4 enum BuildState build_state;5 char *response = "{\"key\":\"value\", \"state\":\"passed\"}";67 /* act */8 res = response_parser_get_result(response, "\"state\":\"passed\"",9 "\"state\":\"started|created\",

10 "\"state\":\"failed\"",11 &build_state);1213 /* assert */14 assert_int_equal(res, 0);15 assert_int_equal(build_state, BUILD_STATE_PASSED);16 }17


3 – Extended unit test example 27


3 – What is a mock? 29



lamp_control.c (SUT)1 void lamp_control_set_state(enum LampState *lamp_state, enum BuildState

build_state)2 {3 build_state_to_lamp_state(lamp_state, build_state);4 lamp_io_set_state(*lamp_state);5 }67

lamp_io.c (need to create a mock for this)1 void lamp_io_set_state(enum LampState lamp_state)2 {3 switch (lamp_state) {4 case LAMP_STATE_OFF :5 digitalWrite(LAMP_IO_RED_PIN, LOW);6 digitalWrite(LAMP_IO_GREEN_PIN, LOW);7 break;8 case LAMP_STATE_GREEN :9 digitalWrite(LAMP_IO_RED_PIN, LOW);

10 digitalWrite(LAMP_IO_GREEN_PIN, HIGH);11 break;12 case LAMP_STATE_RED :13 ...14 };15 }16



test.c1 /* mock */2 void lamp_io_set_state(enum LampState lamp_state)3 {4 check_expected(lamp_state);5 }67 /* test */8 void test_set_state_build_failed(void **state)9 {

10 enum LampState current_lamp_state;11 /* setup */12 current_lamp_state = LAMP_STATE_OFF;13 expect_value(lamp_io_set_state, lamp_state, LAMP_STATE_RED);14 /* act */15 lamp_control_set_state(&current_lamp_state, BUILD_STATE_FAILED);16 /* assert */17 assert_int_equal(current_lamp_state, LAMP_STATE_RED);18 }19

NOTE: cmocka implementation


3 – Integration/system testing 33


3 – Integration/system testing 34


3 – Acceptance testing 36


3 – Acceptance testing 37

+ real end-to-end− slow running tests− high maintanance


4 – Outline 38

1 Code versioning

2 Working in team

3 Test automation




4 – Static code analysis 39

+ Finding potential bugs/security issues+ Detecting code smells+ Guarantees 100% code coverage+ Validating coding standards


4 – Static code analysis aspects 40

• compiler warnings

• coding standards

• abstract interpretation

• cyclomatic complexity


4 – Static code analysis process 41


4 – Static code analysis tools 42

• free/paid• C/C++, C#,

Java, JS,Python, Ruby

• open source• C/C++,

Objective-C

• open source• C/C++, C#,

Java, JS, PHP,Python,...

• paid• C/C++, C#,

Java, JS, Go,Python,...

• paid• C/C++, Java,

.NET

• open source• C/C++


4 – Abstract interpretation - demo project 43


4 – Dynamic code analysis 44

+ Measuring test coverage+ Finding complex issues not detectable by static analysis+ Gives insight for improving efficiency (speed, memory)− No 100% coverage ensured− Tests run (deadly) slow


4 – Dynamic code analysis aspects 45

• Code coverage

• Issue analysis

• Profiling


4 – Dynamic code analysis process 46


4 – Code coverage tools - C/C++ 47

• gcov/lcov• open source• line/branch coverage

• llvm-cov• open source• line/region coverage

• paid• decision/condition coverage

• paid• all types of coverage


4 – Code coverage gcov/lcov - demo project 48

Makefile1 CFLAGS += --coverage2 LDFLAGS += --coverage3

Report generation1 lcov ...2 genhtml ...3


4 – Code coverage - demo project 49


4 – Issue analysis & profiling tools - C/C++ 50

• open source• Mem leak,

threading,heap/cacheprofiling

• Sanitizers (alsoin gcc!)

• open source• Mem leak,

addr issues,threading, noninit mem

• Insure++• paid• Mem,

threading, fileI/O,network,...


4 – Address sanitizer - demo project 51

Instrumentation flags1 CFLAGS += -fno-omit-frame-pointer -fsanitize=address2 LDFLAGS += -fno-omit-frame-pointer -fsanitize=address3

On runtime error1 =================================================================2 ==6305==ERROR: AddressSanitizer: stack-buffer-overflow on ...3 READ of size 18 at 0x7ffe1ac4ab51 thread T04 #0 0x7f3da492f1e8 ...5 #1 0x7f3da492fbcc in vfprintf ...6 #2 0x7f3da492fcf9 in fprintf ...7 #3 0x40262a in regex_report_and_cleanup /home/maarten/code/rpi-ci-lamp/

lib/response_parser.c:158 #4 0x4027d8 in perform_regex /home/maarten/code/rpi-ci-lamp/lib/

response_parser.c:269 ...

10


5 – Outline 52

1 Code versioning

2 Working in team

3 Test automation




5 – Getting it all together 53


5 – CI pipeline in practice 54


5 – CI tools 55

• free• NXP, Netflix,

Facebook,Ebay, NASA

• free/paid• Apple,

Stackoverflow,Ebay, Cochlear

• free/paid,hosted

• Facebook,Twitter

• free trial• ...

• free/paid,hosted

• EAVISE,...

• paid• Materialise


5 – CI pipeline demo project 56


5 – Travis CI - how does it work? 57


5 – Travis CI - how does it work? 58

.travis.yml1 dist: trusty2 language: c34 addons:5 apt:6 packages:7 - libcurl4-openssl-dev8 - libconfig-dev9 install:

10 - wget https://cmocka.org/files/1.1/cmocka-1.1.0.tar.xz11 - tar -xvf cmocka-1.1.0.tar.xz12 - ...1314 script:15 - make DEBUG=1 NOWIRINGPI=1 test16 - ./test/run_unit_tests.sh1718 after_success:19 - gcov -bclp lib/*.c bin/*.c20 - bash <(curl -s https://codecov.io/bash)21


5 – Build status notifications 59

• e-mail/rss feed on failure• build lamp• build result badges


5 – Full demo 60


Introduction to DevOps - KU Leuvenjoost.vennekens/DN/introduction_to_code... · Introduction to...

Documents

Transcript of Introduction to DevOps - KU Leuvenjoost.vennekens/DN/introduction_to_code... · Introduction to...