{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "#  Advanced topics in test driven development"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Introduction\n",
    "\n",
    "\n",
    "- Already seen the basics\n",
    "- Learn some advanced topics"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## The hypothesis package\n",
    "\n",
    "- http://hypothesis.readthedocs.io\n",
    "\n",
    "- `pip install hypothesis`\n",
    "\n",
    "- General idea earlier:\n",
    "  - Make test data.\n",
    "  - Perform operations\n",
    "  - Assert something after operation\n",
    "\n",
    "- Hypothesis automates this!\n",
    "  - Describe range of scenarios\n",
    "  - Computer explores these and tests\n",
    "\n",
    "- With hypothesis:\n",
    "  - Generate random data using specification\n",
    "  - Perform operations\n",
    "  - assert something about result\n",
    "  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Example"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "\n",
    "from hypothesis import given\n",
    "from hypothesis import strategies as st\n",
    "\n",
    "from gcd import gcd\n",
    "\n",
    "@given(st.integers(min_value=0), st.integers(min_value=0))\n",
    "def test_gcd(a, b):\n",
    "    result = gcd(a, b)\n",
    "    # Now what?\n",
    "    # assert a%result == 0\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Example: adding a specific case"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "@given(st.integers(min_value=0), st.integers(min_value=0))\n",
    "\n",
    "@example(a=44, b=19)\n",
    "\n",
    "def test_gcd(a, b):\n",
    "    result = gcd(a, b)\n",
    "    # Now what?\n",
    "    # assert a%result == 0\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### More details\n",
    "\n",
    "- `given` generates inputs\n",
    "- `strategies`: provides a strategy for inputs\n",
    "- Different stratiegies\n",
    "    - `integers`\n",
    "    - `floats`\n",
    "    - `text`\n",
    "    - `booleans`\n",
    "    - `tuples`\n",
    "    - `lists`\n",
    "    - ...\n",
    "\n",
    "- See: http://hypothesis.readthedocs.io/en/latest/data.html"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Example exercise\n",
    "\n",
    "- Write a simple run-length encoding function called `encode`\n",
    "- Write another called `decode` to produce the same input from the output of\n",
    "  `encode`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "def encode(text):\n",
    "    return []\n",
    "\n",
    "def decode(lst):\n",
    "    return ''\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### The test"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from hypothesis import given\n",
    "from hypothesis import strategies as st\n",
    "\n",
    "@given(st.text())\n",
    "def test_decode_inverts_encode(s):\n",
    "    assert decode(encode(s)) == s"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Summary\n",
    "\n",
    "- Much easier to test\n",
    "- hypothesis does the hard work\n",
    "- Can do a lot more!\n",
    "- Read the docs for more\n",
    "- For some detailed articles:\n",
    "  http://hypothesis.works/articles/intro/\n",
    "- Here in particular is one interesting article:\n",
    "  http://hypothesis.works/articles/calculating-the-mean/\n",
    "\n",
    "----\n",
    "\n",
    "## Unittest module\n",
    "\n",
    "- Basic idea and style is from JUnit\n",
    "- Some consider this old style\n",
    "\n",
    "\n",
    "### How to use unittest\n",
    "\n",
    "- Subclass `unittest.TestCase`\n",
    "- Create test methods\n",
    "\n",
    "### A simple example\n",
    "\n",
    "- Let us test gcd.py with unittest"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# gcd.py\n",
    "def gcd(a, b):\n",
    "    if b == 0:\n",
    "        return a\n",
    "    return gcd(b, a%b)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Writing the test"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# test_gcd.py\n",
    "from gcd import gcd\n",
    "import unittest\n",
    "\n",
    "class TestGCD(unittest.TestCase):\n",
    "    def test_gcd_works_for_positive_integers(self):\n",
    "        self.assertEqual(gcd(48, 64), 16)\n",
    "        self.assertEqual(gcd(44, 19), 1)\n",
    "\n",
    "if __name__ == '__main__':\n",
    "    unittest.main()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Running it\n",
    "\n",
    "- Just run `python test_gcd.py`\n",
    "- Also works with `nosetests` and `pytest`\n",
    "\n",
    "\n",
    "### Notes\n",
    "\n",
    "- Note the name of the method.\n",
    "- Note the use of `self.assertEqual`\n",
    "- Also available: `assertNotEqual, assertTrue, assertFalse, assertIs, assertIsNot`\n",
    "- `assertIsNone, assertIn, assertIsInstance, assertRaises`\n",
    "- `assertAlmostEqual, assertListEqual, assertSequenceEqual ` ...\n",
    "\n",
    "- https://docs.python.org/2/library/unittest.html\n",
    "\n",
    "\n",
    "### Fixtures\n",
    "\n",
    "- What if you want to do something common before all tests?\n",
    "- Typically called a **fixture**\n",
    "\n",
    "- Use the `setUp` and `tearDown` methods for method-level fixtures\n",
    "\n",
    "### Silly fixture example"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# test_gcd.py\n",
    "import gcd\n",
    "import unittest\n",
    "\n",
    "class TestGCD(unittest.TestCase):\n",
    "    def setUp(self):\n",
    "        print(\"setUp\")\n",
    "    def tearDown(self):\n",
    "        print(\"tearDown\")\n",
    "    def test_gcd_works_for_positive_integers(self):\n",
    "        self.assertEqual(gcd(48, 64), 16)\n",
    "        self.assertEqual(gcd(44, 19), 1)\n",
    "\n",
    "if __name__ == '__main__':\n",
    "    unittest.main()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise\n",
    "\n",
    "- Fix bug with negative numbers in gcd.py.\n",
    "- Use TDD.\n",
    "\n",
    "\n",
    "### Using hypothesis with unittest"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# test_gcd.py\n",
    "from hypothesis import given\n",
    "from hypothesis import strategies as st\n",
    "\n",
    "import gcd\n",
    "import unittest\n",
    "\n",
    "class TestGCD(unittest.TestCase):\n",
    "    @given(a=st.integers(min_value=0), b=st.integers(min_value=0))\n",
    "    def test_gcd_works_for_positive_integers(self, a, b):\n",
    "        result = gcd(a, b)\n",
    "        assert a%result == 0\n",
    "        assert b%result == 0\n",
    "        assert result <= a and result <= b\n",
    "\n",
    "if __name__ == '__main__':\n",
    "    unittest.main()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Some notes on style\n",
    "\n",
    "- Use descriptive function names\n",
    "- Intent matters\n",
    "\n",
    "- Segregate the test code into the following"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "- Given: what is the context of the test?\n",
    "- When: what action is taken to actually test the problem\n",
    "- Then: what do we actually ensure."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### More on intent driven programming\n",
    "\n",
    "- \"Programs must be written for people to read, and only incidentally for\n",
    "  machines to execute.”  Harold Abelson\n",
    "\n",
    "- The code should make the intent clear.\n",
    "\n",
    "For example:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "if self.temperature > 600 and self.pressure > 10e5:\n",
    "    message = 'hello you have a problem here!'\n",
    "    message += 'current temp is %s'%(self.temperature)\n",
    "    print(message)\n",
    "    self.valve.close()\n",
    "    self.raise_warning()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "is totally unclear as to the intent.  Instead refactor as follows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "if self.reactor_is_critical():\n",
    "    self.shutdown_with_warning()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### A more involved testing example\n",
    "\n",
    "- Motivational problem:\n",
    "\n",
    "> Find all the git repositories inside a given directory recursively.\n",
    "> Make this a command line tool supporting command line use.\n",
    "\n",
    "- Write tests for the code\n",
    "- Some rules:\n",
    "\n",
    "   0. The test should be runnable by anyone (even by a computer), almost anywhere.\n",
    "   1. Don't write anything in the current directory (use a temporary directory).\n",
    "   2. Cleanup any files you create while testing.\n",
    "   3. Make sure tests do not affect global state too much.\n",
    "\n",
    "\n",
    "### Solution\n",
    "\n",
    "1. Create some test data.\n",
    "2. Test!\n",
    "3. Cleanup the test data\n",
    "\n",
    "\n",
    "### Class-level fixtures\n",
    "\n",
    "- Use `setupClass` and `tearDownClass` classmethods for class level fixtures.\n",
    "\n",
    "\n",
    "### Module-level fixtures\n",
    "\n",
    "- `setup_module`, `teardown_module`\n",
    "- Can be used for a module-level fixture\n",
    "\n",
    "- http://nose.readthedocs.io/en/latest/writing_tests.html\n",
    "\n",
    "\n",
    "## Coverage\n",
    "\n",
    "- Assess the amount of code that is covered by testing\n",
    "- http://coverage.readthedocs.io/\n",
    "- `pip install coverage`\n",
    "- Integrates with nosetests/pytest\n",
    "\n",
    "### Typical coverage usage"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "$ coverage run -m nose.core my_package\n",
    "$ coverage report -m\n",
    "$ coverage html"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## mock\n",
    "\n",
    "- Mocking for advanced testing.\n",
    "\n",
    "- Example: reading some twitter data\n",
    "- Example: function to post an update to facebook or twitter\n",
    "- Example: email user when simulation crashes\n",
    "\n",
    "- Can you test it?  How?\n",
    "\n",
    "### Using mock: the big picture\n",
    "\n",
    "- Do you really want to post something on facebook?\n",
    "- Or do you want to know if the right method was called with the right arguments?\n",
    "\n",
    "- Idea: \"mock\" the objects that do something and test them\n",
    "\n",
    "- Quoting from the Python docs:\n",
    "\n",
    "> It allows you to replace parts of your system under test with mock objects\n",
    "> and make assertions about how they have been used.\n",
    "\n",
    "### Installation\n",
    "\n",
    "- Built-in on Python >= 3.3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "- `from unittest import mock`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- else `pip install mock`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "- `import mock`\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Simple examples\n",
    "\n",
    "Say we have a class:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "class ProductionClass(object):\n",
    "    def method(self, *args):\n",
    "        # This does something we do not want to actually run in the test\n",
    "        # ...\n",
    "        pass"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To mock the `ProductionClass.method` do this:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from unittest.mock import MagicMock\n",
    "thing = ProductionClass()\n",
    "thing.method = MagicMock(return_value=3)\n",
    "thing.method(3, 4, 5, key='value')\n",
    "thing.method.assert_called_with(3, 4, 5, key='value')\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### More practical use case\n",
    "\n",
    "- Mocking a module or system call\n",
    "- Mocking an object or method\n",
    "- Remember that after testing you want to restore original state\n",
    "- Use `mock.patch`\n",
    "\n",
    "### An example\n",
    "\n",
    "- Write code to remove generated files from LaTeX compilation, i.e. remove the\n",
    "  *.aux, *.log, *.pdf etc.\n",
    "\n",
    "Here is a simple attempt:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# clean_tex.py\n",
    "import os\n",
    "\n",
    "def cleanup(tex_file_pth):\n",
    "    base = os.path.splitext(tex_file_pth)[0]\n",
    "    for ext in ('.aux', '.log'):\n",
    "        f = base + ext\n",
    "        if os.path.exists(f):\n",
    "            os.remove(f)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Testing this with mock"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import mock\n",
    "\n",
    "@mock.patch('clean_tex.os.remove')\n",
    "def test_cleanup_removes_extra_files(mock_remove):\n",
    "    cleanup('foo.tex')\n",
    "\n",
    "    expected = [mock.call('foo.' + x) for x in ('aux', 'log')]\n",
    "\n",
    "    mock_remove.assert_has_calls(expected)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- Note the mocked argument that is passed.\n",
    "- Note that we did not mock `os.remove`\n",
    "- Mock where the object is looked up\n",
    "\n",
    "### Doing more"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import mock\n",
    "\n",
    "@mock.patch('clean_tex.os.path')\n",
    "@mock.patch('clean_tex.os.remove')\n",
    "def test_cleanup_does_not_fail_when_files_dont_exist(mock_remove, mock_path):\n",
    "    # Setup the mock_path to return False\n",
    "    mock_path.exists.return_value = False\n",
    "\n",
    "    cleanup('foo.tex')\n",
    "\n",
    "    mock_remove.assert_not_called()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- Note the order of the passed arguments\n",
    "- Note the name of the method\n",
    "\n",
    "### Patching instance methods\n",
    "\n",
    "Use `mock.patch.object` to patch an instance method"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "@mock.patch.object(ProductionClass, 'method')\n",
    "def test_method(mock_method):\n",
    "    obj = ProductionClass()\n",
    "    obj.method(1)\n",
    "\n",
    "    mock_method.assert_called_once_with(1)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Mock works as a context manager:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "with mock.patch.object(ProductionClass, 'method') as mock_method:\n",
    "    obj = ProductionClass()\n",
    "    obj.method(1)\n",
    "\n",
    "mock_method.assert_called_once_with(1)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### More articles on mock\n",
    "\n",
    "- See more here https://docs.python.org/3/library/unittest.mock.html\n",
    "- https://www.toptal.com/python/an-introduction-to-mocking-in-python\n",
    "\n",
    "\n",
    "## Pytest\n",
    "\n",
    "Offers many useful and convenient features that are useful\n",
    "\n",
    "\n",
    "## Odds and ends\n",
    "\n",
    "### Linters\n",
    "\n",
    "- `pyflakes`\n",
    "- `flake8`\n",
    "\n",
    "### IPython goodies\n",
    "\n",
    "- Use `%run`\n",
    "- Use `%pdb`\n",
    "- `%debug`\n",
    "\n",
    "### Debugging\n",
    "\n",
    "- Debug with `%run`\n",
    "- pdb.set_trace()\n",
    "- IPython set trace:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from IPython.core.debugger import Tracer; Tracer()()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- See here: http://www.scipy-lectures.org/advanced/debugging/"
   ]
  }
 ],
 "metadata": {
  "celltoolbar": "Slideshow",
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}