cells: - markdown: | # Advanced topics in test driven development metadata: slideshow: slide_type: slide - markdown: | ## Introduction - Already seen the basics - Learn some advanced topics metadata: slideshow: slide_type: slide - markdown: | ## The hypothesis package - http://hypothesis.readthedocs.io - `pip install hypothesis` - General idea earlier: - Make test data. - Perform operations - Assert something after operation - Hypothesis automates this! - Describe range of scenarios - Computer explores these and tests - With hypothesis: - Generate random data using specification - Perform operations - assert something about result metadata: slideshow: slide_type: slide - markdown: | ### Example metadata: slideshow: slide_type: slide - code: | from hypothesis import given from hypothesis import strategies as st from gcd import gcd @given(st.integers(min_value=0), st.integers(min_value=0)) def test_gcd(a, b): result = gcd(a, b) # Now what? # assert a%result == 0 id: 0 metadata: collapsed: true slideshow: slide_type: fragment - markdown: | ### Example: adding a specific case metadata: slideshow: slide_type: slide - code: | @given(st.integers(min_value=0), st.integers(min_value=0)) @example(a=44, b=19) def test_gcd(a, b): result = gcd(a, b) # Now what? # assert a%result == 0 id: 1 metadata: collapsed: true slideshow: slide_type: fragment - markdown: | ### More details - `given` generates inputs - `strategies`: provides a strategy for inputs - Different stratiegies - `integers` - `floats` - `text` - `booleans` - `tuples` - `lists` - ... - See: http://hypothesis.readthedocs.io/en/latest/data.html metadata: slideshow: slide_type: slide - markdown: | ### Example exercise - Write a simple run-length encoding function called `encode` - Write another called `decode` to produce the same input from the output of `encode` metadata: slideshow: slide_type: slide - code: | def encode(text): return [] def decode(lst): return '' id: 2 metadata: collapsed: true slideshow: slide_type: fragment - markdown: | ### The test - code: | from hypothesis import given from hypothesis import strategies as st @given(st.text()) def test_decode_inverts_encode(s): assert decode(encode(s)) == s id: 3 metadata: collapsed: true - markdown: | ### Summary - Much easier to test - hypothesis does the hard work - Can do a lot more! - Read the docs for more - For some detailed articles: http://hypothesis.works/articles/intro/ - Here in particular is one interesting article: http://hypothesis.works/articles/calculating-the-mean/ ---- ## Unittest module - Basic idea and style is from JUnit - Some consider this old style ### How to use unittest - Subclass `unittest.TestCase` - Create test methods ### A simple example - Let us test gcd.py with unittest - code: | # gcd.py def gcd(a, b): if b == 0: return a return gcd(b, a%b) id: 4 metadata: collapsed: true - markdown: | ### Writing the test - code: | # test_gcd.py from gcd import gcd import unittest class TestGCD(unittest.TestCase): def test_gcd_works_for_positive_integers(self): self.assertEqual(gcd(48, 64), 16) self.assertEqual(gcd(44, 19), 1) if __name__ == '__main__': unittest.main() id: 5 metadata: collapsed: true - markdown: | ### Running it - Just run `python test_gcd.py` - Also works with `nosetests` and `pytest` ### Notes - Note the name of the method. - Note the use of `self.assertEqual` - Also available: `assertNotEqual, assertTrue, assertFalse, assertIs, assertIsNot` - `assertIsNone, assertIn, assertIsInstance, assertRaises` - `assertAlmostEqual, assertListEqual, assertSequenceEqual ` ... - https://docs.python.org/2/library/unittest.html ### Fixtures - What if you want to do something common before all tests? - Typically called a **fixture** - Use the `setUp` and `tearDown` methods for method-level fixtures ### Silly fixture example - code: | # test_gcd.py import gcd import unittest class TestGCD(unittest.TestCase): def setUp(self): print("setUp") def tearDown(self): print("tearDown") def test_gcd_works_for_positive_integers(self): self.assertEqual(gcd(48, 64), 16) self.assertEqual(gcd(44, 19), 1) if __name__ == '__main__': unittest.main() id: 6 metadata: collapsed: true - markdown: | ### Exercise - Fix bug with negative numbers in gcd.py. - Use TDD. ### Using hypothesis with unittest - code: | # test_gcd.py from hypothesis import given from hypothesis import strategies as st import gcd import unittest class TestGCD(unittest.TestCase): @given(a=st.integers(min_value=0), b=st.integers(min_value=0)) def test_gcd_works_for_positive_integers(self, a, b): result = gcd(a, b) assert a%result == 0 assert b%result == 0 assert result <= a and result <= b if __name__ == '__main__': unittest.main() id: 7 metadata: collapsed: true - markdown: | ### Some notes on style - Use descriptive function names - Intent matters - Segregate the test code into the following - code: | - Given: what is the context of the test? - When: what action is taken to actually test the problem - Then: what do we actually ensure. id: 8 metadata: collapsed: true - markdown: | ### More on intent driven programming - "Programs must be written for people to read, and only incidentally for machines to execute.” Harold Abelson - The code should make the intent clear. For example: - code: | if self.temperature > 600 and self.pressure > 10e5: message = 'hello you have a problem here!' message += 'current temp is %s'%(self.temperature) print(message) self.valve.close() self.raise_warning() id: 9 metadata: collapsed: true - markdown: | is totally unclear as to the intent. Instead refactor as follows: - code: | if self.reactor_is_critical(): self.shutdown_with_warning() id: 10 metadata: collapsed: true - markdown: | ### A more involved testing example - Motivational problem: > Find all the git repositories inside a given directory recursively. > Make this a command line tool supporting command line use. - Write tests for the code - Some rules: 0. The test should be runnable by anyone (even by a computer), almost anywhere. 1. Don't write anything in the current directory (use a temporary directory). 2. Cleanup any files you create while testing. 3. Make sure tests do not affect global state too much. ### Solution 1. Create some test data. 2. Test! 3. Cleanup the test data ### Class-level fixtures - Use `setupClass` and `tearDownClass` classmethods for class level fixtures. ### Module-level fixtures - `setup_module`, `teardown_module` - Can be used for a module-level fixture - http://nose.readthedocs.io/en/latest/writing_tests.html ## Coverage - Assess the amount of code that is covered by testing - http://coverage.readthedocs.io/ - `pip install coverage` - Integrates with nosetests/pytest ### Typical coverage usage - code: | $ coverage run -m nose.core my_package $ coverage report -m $ coverage html id: 11 metadata: collapsed: true - markdown: | ## mock - Mocking for advanced testing. - Example: reading some twitter data - Example: function to post an update to facebook or twitter - Example: email user when simulation crashes - Can you test it? How? ### Using mock: the big picture - Do you really want to post something on facebook? - Or do you want to know if the right method was called with the right arguments? - Idea: "mock" the objects that do something and test them - Quoting from the Python docs: > It allows you to replace parts of your system under test with mock objects > and make assertions about how they have been used. ### Installation - Built-in on Python >= 3.3 - code: | - `from unittest import mock` id: 12 metadata: collapsed: true - markdown: | - else `pip install mock` - code: | - `import mock` id: 13 metadata: collapsed: true - markdown: | ### Simple examples Say we have a class: - code: | class ProductionClass(object): def method(self, *args): # This does something we do not want to actually run in the test # ... pass id: 14 metadata: collapsed: true - markdown: | To mock the `ProductionClass.method` do this: - code: | from unittest.mock import MagicMock thing = ProductionClass() thing.method = MagicMock(return_value=3) thing.method(3, 4, 5, key='value') thing.method.assert_called_with(3, 4, 5, key='value') id: 15 metadata: collapsed: true - markdown: | ### More practical use case - Mocking a module or system call - Mocking an object or method - Remember that after testing you want to restore original state - Use `mock.patch` ### An example - Write code to remove generated files from LaTeX compilation, i.e. remove the *.aux, *.log, *.pdf etc. Here is a simple attempt: - code: | # clean_tex.py import os def cleanup(tex_file_pth): base = os.path.splitext(tex_file_pth)[0] for ext in ('.aux', '.log'): f = base + ext if os.path.exists(f): os.remove(f) id: 16 metadata: collapsed: true - markdown: | ### Testing this with mock - code: | import mock @mock.patch('clean_tex.os.remove') def test_cleanup_removes_extra_files(mock_remove): cleanup('foo.tex') expected = [mock.call('foo.' + x) for x in ('aux', 'log')] mock_remove.assert_has_calls(expected) id: 17 metadata: collapsed: true - markdown: | - Note the mocked argument that is passed. - Note that we did not mock `os.remove` - Mock where the object is looked up ### Doing more - code: | import mock @mock.patch('clean_tex.os.path') @mock.patch('clean_tex.os.remove') def test_cleanup_does_not_fail_when_files_dont_exist(mock_remove, mock_path): # Setup the mock_path to return False mock_path.exists.return_value = False cleanup('foo.tex') mock_remove.assert_not_called() id: 18 metadata: collapsed: true - markdown: | - Note the order of the passed arguments - Note the name of the method ### Patching instance methods Use `mock.patch.object` to patch an instance method - code: | @mock.patch.object(ProductionClass, 'method') def test_method(mock_method): obj = ProductionClass() obj.method(1) mock_method.assert_called_once_with(1) id: 19 metadata: collapsed: true - markdown: | Mock works as a context manager: - code: | with mock.patch.object(ProductionClass, 'method') as mock_method: obj = ProductionClass() obj.method(1) mock_method.assert_called_once_with(1) id: 20 metadata: collapsed: true - markdown: | ### More articles on mock - See more here https://docs.python.org/3/library/unittest.mock.html - https://www.toptal.com/python/an-introduction-to-mocking-in-python ## Pytest Offers many useful and convenient features that are useful ## Odds and ends ### Linters - `pyflakes` - `flake8` ### IPython goodies - Use `%run` - Use `%pdb` - `%debug` ### Debugging - Debug with `%run` - pdb.set_trace() - IPython set trace: - code: | from IPython.core.debugger import Tracer; Tracer()() id: 21 metadata: collapsed: true - markdown: | - See here: http://www.scipy-lectures.org/advanced/debugging/ # The lines below here may be deleted if you do not need them. # --------------------------------------------------------------------------- metadata: celltoolbar: Slideshow kernelspec: display_name: Python 2 language: python name: python2 language_info: codemirror_mode: name: ipython version: 2 file_extension: .py mimetype: text/x-python name: python nbconvert_exporter: python pygments_lexer: ipython2 version: 2.7.11 nbformat: 4 nbformat_minor: 0 # --------------------------------------------------------------------------- data: [{execution_count: null, outputs: []}, {execution_count: null, outputs: []}, {execution_count: null, outputs: []}, {execution_count: null, outputs: []}, {execution_count: null, outputs: []}, {execution_count: null, outputs: []}, {execution_count: null, outputs: []}, {execution_count: null, outputs: []}, {execution_count: null, outputs: []}, {execution_count: null, outputs: []}, {execution_count: null, outputs: []}, {execution_count: null, outputs: []}, {execution_count: null, outputs: []}, {execution_count: null, outputs: []}, {execution_count: null, outputs: []}, {execution_count: null, outputs: []}, {execution_count: null, outputs: []}, {execution_count: null, outputs: []}, {execution_count: null, outputs: []}, {execution_count: null, outputs: []}, {execution_count: null, outputs: []}, {execution_count: null, outputs: []}]