Writing dumb tests

Writing Dumb TestsLuke Lee

References: http://bit.ly/dumber_tests_pytx2014

Luke Lee• Scientific Python/Desktop Visualization

• Recovering C developer

• @durden20

• lukelee.me

• codrspace.com/durden


http://twitter.com/durden20

http://www.lukelee.me

http://codrspace.com/durden

The Last Day of a Sprint


Testing Woes


Smart Tests• Too many details

• Data

• Too much software


Smart TestsToo Many Details

import collectionsentry = collections.namedtuple('entry', ['name', 'value', 'units'])def lines_to_tuples(filename): tuples = [] with open(filename, 'r') as file_obj: for line in file_obj: fields = line.split() tuples.append(entry(*fields))

return tuples



def test_lines_to_tuples_1(): # Test.txt contains: # Height 60 in # Length 30 in filename = 'test.txt'

test_tuples = lines_to_tuples(filename)

with open(filename, 'r') as file_obj: for line, test_tuple in zip(file_obj, test_tuples): fields = line.split() assert entry(*fields) == test_tuple



"We've fallen into a trap of testing that the code does what the code says it does, rather than testing functional behavior we care about."

-- Daniel Pope, "Every mock.patch is a little smell"


Smart TestsData

Your data is clean, users' data is gold standard


Random Data


Random DataFails sometimes


Smart TestsToo Much Software

• Too easy to change

• Too much cognitive overhead


import unittest

class TestSequenceFunctions(unittest.TestCase): def setUp(self): self.seq = range(10)

def test_shuffle(self): # make sure the shuffled sequence does not lose any elements random.shuffle(self.seq) self.seq.sort() self.assertEqual(self.seq, range(10))

# should raise an exception for an immutable sequence self.assertRaises(TypeError, random.shuffle, (1,2,3))

def test_sample(self): with self.assertRaises(ValueError): random.sample(self.seq, 20) for element in random.sample(self.seq, 5): self.assertTrue(element in self.seq)


import pytest

@pytest.fixture(scope="module", params=["merlinux.eu", "mail.python.org"]) def smtp(request): smtp = smtplib.SMTP(request.param)

def fin(): print ("finalizing %s" % smtp) smtp.close()

request.addfinalizer(fin) return smtp


Bugs hide in complexity


Saved by the Tests


Smart enough to find bugsbut dumb enough to

DEBUGReferences: http://bit.ly/dumber_tests_pytx2014

Dumb Tests• Doctests

• Data over code

• Think goals, not implementation

• Be brave


Dumb Testsimport doctest

• Limited means simple


Dumb TestsData over code

• Commit examples with real data

• numpy.savetxt

• StringIO


Dumb TestsData over code

• difflib• filecmp• binary data


Dumb TestsGoals, not implementation

• Docs guide testing

• Postpone code coverage



def test_lines_to_tuples_2(): # Test.txt contains: # Height 60 in # Length 30 in filename = 'test.txt'

correct_tuples = [] correct_tuples.append(entry(name='Height', value='60', units='in')) correct_tuples.append(entry(name='Length', value='30', units='in'))

test_tuples = lines_to_tuples(filename) assert correct_tuples == test_tuples



Embrace the experience of other people, people not schooled in the details of your implementation, you can learn how to make your work easier to absorb, and therefore more effective at what it's designed to do.

-- Dave Winer, Early author of RSS software


Be Brave!


Dumb TestsBe Brave!

import pytest @pytest.mark.parametrize("input,expected", [ ("3+5", 8), ("2+4", 6), ("6*9", 42), ]) def test_eval(input, expected): assert eval(input) == expected


Use asserts


Conclusion1. Uses real world data

2. Focuses on goals not implementation details

3. Lowers cognitive oveheard with brave, simple code


Write Boring Tests


@durden20www.lukelee.mecodrspace.com/durdenReferences: http://bit.ly/dumber_tests_pytx2014

Writing dumb tests

Technology

Transcript of Writing dumb tests