Writing Dumb TestsLuke Lee

References: http://bit.ly/dumber_tests_pytx2014

Luke Lee• Scientific Python/Desktop Visualization

• Recovering C developer

• @durden20

• lukelee.me

• codrspace.com/durden

References: http://bit.ly/dumber_tests_pytx2014

The Last Day of a Sprint

References: http://bit.ly/dumber_tests_pytx2014

The Last Day of a Sprint

References: http://bit.ly/dumber_tests_pytx2014

Testing Woes

References: http://bit.ly/dumber_tests_pytx2014

Smart Tests• Too many details

• Data

• Too much software

References: http://bit.ly/dumber_tests_pytx2014

Smart TestsToo Many Details

import collectionsentry = collections.namedtuple('entry', ['name', 'value', 'units'])def lines_to_tuples(filename): tuples = [] with open(filename, 'r') as file_obj: for line in file_obj: fields = line.split() tuples.append(entry(*fields))

return tuples

References: http://bit.ly/dumber_tests_pytx2014

Smart TestsToo Many Details

def test_lines_to_tuples_1(): # Test.txt contains: # Height 60 in # Length 30 in filename = 'test.txt'

test_tuples = lines_to_tuples(filename)

with open(filename, 'r') as file_obj: for line, test_tuple in zip(file_obj, test_tuples): fields = line.split() assert entry(*fields) == test_tuple

References: http://bit.ly/dumber_tests_pytx2014

Smart TestsToo Many Details

"We've fallen into a trap of testing that the code does what the code says it does, rather than testing functional behavior we care about."

-- Daniel Pope, "Every mock.patch is a little smell"

References: http://bit.ly/dumber_tests_pytx2014

Smart TestsData

Your data is clean, users' data is gold standard

References: http://bit.ly/dumber_tests_pytx2014

Random Data

References: http://bit.ly/dumber_tests_pytx2014

Random DataFails sometimes

References: http://bit.ly/dumber_tests_pytx2014

Smart TestsToo Much Software

• Too easy to change

• Too much cognitive overhead

References: http://bit.ly/dumber_tests_pytx2014

import unittest

class TestSequenceFunctions(unittest.TestCase): def setUp(self): self.seq = range(10)

def test_shuffle(self): # make sure the shuffled sequence does not lose any elements random.shuffle(self.seq) self.seq.sort() self.assertEqual(self.seq, range(10))

# should raise an exception for an immutable sequence self.assertRaises(TypeError, random.shuffle, (1,2,3))

def test_sample(self): with self.assertRaises(ValueError): random.sample(self.seq, 20) for element in random.sample(self.seq, 5): self.assertTrue(element in self.seq)

References: http://bit.ly/dumber_tests_pytx2014

import pytest

@pytest.fixture(scope="module", params=["merlinux.eu", "mail.python.org"]) def smtp(request): smtp = smtplib.SMTP(request.param)

def fin(): print ("finalizing %s" % smtp) smtp.close()

request.addfinalizer(fin) return smtp

References: http://bit.ly/dumber_tests_pytx2014

Bugs hide in complexity

References: http://bit.ly/dumber_tests_pytx2014

Saved by the Tests

References: http://bit.ly/dumber_tests_pytx2014

Smart enough to find bugsbut dumb enough to

DEBUGReferences: http://bit.ly/dumber_tests_pytx2014

Dumb Tests• Doctests

• Data over code

• Think goals, not implementation

• Be brave

References: http://bit.ly/dumber_tests_pytx2014

Dumb Testsimport doctest

• Limited means simple

References: http://bit.ly/dumber_tests_pytx2014

Dumb TestsData over code

• Commit examples with real data

• numpy.savetxt

• StringIO

References: http://bit.ly/dumber_tests_pytx2014

Dumb TestsData over code

• difflib• filecmp• binary data

References: http://bit.ly/dumber_tests_pytx2014

Dumb TestsGoals, not implementation

• Docs guide testing

• Postpone code coverage

References: http://bit.ly/dumber_tests_pytx2014

Dumb TestsGoals, not implementation

def test_lines_to_tuples_2(): # Test.txt contains: # Height 60 in # Length 30 in filename = 'test.txt'

correct_tuples = [] correct_tuples.append(entry(name='Height', value='60', units='in')) correct_tuples.append(entry(name='Length', value='30', units='in'))

test_tuples = lines_to_tuples(filename) assert correct_tuples == test_tuples

References: http://bit.ly/dumber_tests_pytx2014

Dumb TestsGoals, not implementation

Embrace the experience of other people, people not schooled in the details of your implementation, you can learn how to make your work easier to absorb, and therefore more effective at what it's designed to do.

-- Dave Winer, Early author of RSS software

References: http://bit.ly/dumber_tests_pytx2014

Be Brave!

References: http://bit.ly/dumber_tests_pytx2014

Dumb TestsBe Brave!

import pytest @pytest.mark.parametrize("input,expected", [ ("3+5", 8), ("2+4", 6), ("6*9", 42), ]) def test_eval(input, expected): assert eval(input) == expected

References: http://bit.ly/dumber_tests_pytx2014

Use asserts

References: http://bit.ly/dumber_tests_pytx2014

Conclusion1. Uses real world data

2. Focuses on goals not implementation details

3. Lowers cognitive oveheard with brave, simple code

References: http://bit.ly/dumber_tests_pytx2014

Write Boring Tests

References: http://bit.ly/dumber_tests_pytx2014

@durden20www.lukelee.mecodrspace.com/durdenReferences: http://bit.ly/dumber_tests_pytx2014