{
  "cells": [
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "%matplotlib inline"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "\n\n# Writing your own test case\n\n\nIn this tutorial, you will learn how to write a test case for your contributed\ncode, and make sure the test passes.\n\nOutline\n=======\n\n* `Prerequisites`_\n* `Scenario`_\n* `Implementation`_\n\n<div class=\"alert alert-info\"><h4>Note</h4><p>This tutorial is intended for people interested in contributing code\n    `contributing code <dev-contributing>` to pulse2percept.</p></div>\n\nPrerequisites\n=============\n\nThis tutorial assumes the following:\n\n* pulse2percept is already installed on your machine.\n  If you are having trouble with installation, see\n  `Installation <install>`.\n\n* You have read `Contributing to pulse2percept <dev-contributing>`\n  and have followed the steps to fork and clone a development version of\n  pulse2percept to your local computer.\n\nScenario\n========\n\nThe goal of this tutorial is to add a new function to the\n:py:mod:`~pulse2percept.stimuli.pulse_trains` subpackage that returns the\nlargest element in a :py:class:`~pulse2percept.stimuli.TimeSeries` object.\n\nThe function, which we will call ``largest_timeseries_element``, should:\n\n* accept a :py:class:`~pulse2percept.stimuli.TimeSeries` object, or throw a\n  ``TypeError`` if a different object is passed\n* throw a warning if the largest value is 0\n* return the largest value in the\n  :py:class:`~pulse2percept.stimuli.TimeSeries`' ``data`` container\n\n<div class=\"alert alert-info\"><h4>Note</h4><p>If this were a real problem, there would already be an issue for it in\n    pulse2percept's `issue tracker`_.\n\n    In that case, you would open the specific issue on GitHub and assign\n    yourself to it. You can also post a comment to let the community know that\n    you are going to submit a pull request on this issue.</p></div>\n\n\nImplementation\n==============\n\nIn this tutorial, we will follow what is known as `test-driven development`_\n(TDD). TDD turns the common work flow (write code, then test it interactively)\non its head with the goal of producing better code faster. Instead, we will\nfollow the following recipe:\n\n* Write the test function, ``test_largest_timeseries_element`` **first**.\n* Then write a ``largest_timeseries_element`` function that should pass those\n  tests.\n* If the function produces any wrong answers, fix it and re-run the test\n  function.\n\n<div class=\"alert alert-info\"><h4>Note</h4><p>In Python, function and variable names are generally lowercase, with words\n    separated by underscores. Class names, on the other hand, typically use\n    CapWords convention.\n\n    See `PEP8`_ for a full Python style guide.</p></div>\n\n\nCreating a new branch\n---------------------\n\nFollowing the general guidelines outlined in\n`Contributing to pulse2percept <dev-contributing>`, we\nneed to perform all our work on a new branch.\n\nFirst, we need to make sure we are working off the latest code:\n\n.. code-block: bash\n\n    git checkout master\n    git pull upstream master\n\nThen we will create a new branch (aptly named \"largest-timeseries-element\"\nor similar):\n\n.. code-block: bash\n\n    git checkout -b largest-timeseries-element\n\nWriting the test function\n-------------------------\n\nBecause our code is related to :py:class:`~pulse2percept.stimuli.TimeSeries`,\nwhich lives in the :py:mod:`pulse2percept.stimuli.pulse_trains` subpackage, our\nnew function should go in the same subpackage.\n\nThe corresponding test file is\n\"pulse2percept/stimuli/tests/test_pulse_trains.py\".\n\nIn this file, we will create a new test function.\nFor consistency, it is important that our function be named\n\"test\\_<name of function to test>\", where \"<name of function to test>\" is\nidentical to the function added to the\n:py:mod:`~pulse2percept.stimuli.pulse_trains` subpackage.\nFor example:\n\n* ``def test_TimeSeries`` for testing the\n  :py:class:`~pulse2percept.stimuli.TimeSeries` object (note that this function\n  already exists).\n* ``def test_TimeSeries_resample`` for testing the\n  :py:meth:`~pulse2percept.stimuli.TimeSeries.resample` method of the\n  :py:class:`~pulse2percept.stimuli.TimeSeries` object.\n* ``def test_newfunc`` for a new function called ``newfunc``.\n\nOur test function should therefore be called\n``test_largest_timeseries_element``.\n\n.. important::\n\n    For `pytest`_ to run your test function, its name must start with \"test\\_\".\n\nWithin our function, we have access to a number of `numpy-testing`_ routines\nthat can compare desired to actual output, such as:\n\n* ``assert_equal(actual, desired)`` returns an ``AssertionError`` if two\n  objects are not equal.\n* ``assert_almost_equal(actual, desired, decimal=7)`` returns an\n  ``AssertionError`` if two items are not equal up to desired precision\n  (good for testing doubles).\n* ``assert_raises(exception_class)`` fails unless an ``Exception`` of class\n  ``exception_class`` is thrown.\n\nTypically, we want to make sure the function works for a few simple cases.\nOur first draft for a test function might thus look like this:\n\n.. code-block:: python\n\n    import numpy as np\n    import numpy.testing as npt\n    from pulse2percept.stimuli import (largest_timeseries_element, TimeSeries,\n                                       PulseTrain)\n\n\n    def test_largest_timeseries_element():\n        # Create a simple TimeSeries object:\n        ts = TimeSeries(1, np.array([0, 1.5, 2]))\n        # Use almost_equal because we are comparing doubles:\n        npt.assert_almost_equal(largest_timeseries_element(ts), 2.0)\n\n\nWe can now run the entire test suite from the pulse2percept root directory:\n\n.. code-block:: bash\n\n    make tests\n\nAlternatively, we can run a single test file by specifying its path:\n\n.. code-block:: bash\n\n    pytest pulse2percept/stimuli/tests/test_pulse_trains.py\n\nEven better yet, we can run just a single test from a single file:\n\n.. code-block:: bash\n\n    pytest pulse2percept/stimuli/tests/test_pulse_train.py::test_largest_timeseries_element\n\nWhat we expect to see is an ``ImportError``, because we have not actually\nwritten the ``largest_timeseries_element`` function yet! So let's get going.\n\n\nWriting the actual function\n---------------------------\n\nThe next step is to add the actual function to\n\"pulse2percept/stimuli/pulse_trains.py\":\n\n.. code-block:: python\n\n    import numpy as np\n\n\n    def largest_timeseries_element(ts):\n        \"\"\"Return the largest element of a TimeSeries object\n\n        Parameters\n        ----------\n        ts:\n            TimeSeries\n            A TimeSeries object\n\n        Returns\n        -------\n        max:\n            double\n            The largest value in the TimeSeries data\n        \"\"\"\n        return np.max(ts.data)\n\nNote how we make use of `docstring`_ notation here to document the functions\npurpose, input arguments, and return values.\n\nNow we can run the test suite again, and find... still an ``ImportError``.\nWhat's going on here?\n\n\nAdding the function to the subpackage __init__\n----------------------------------------------\n\nTo make sure the function gets imported, we have to edit the subpackage's\n\"__init__.py\" file. It might look something like this:\n\n.. code-block:: python\n\n    \"\"\"Stimuli\n\n    This module provides a number of stimuli.\n    \"\"\"\n\n    from .base import Stimulus\n    from .pulse_trains import (TimeSeries, MonophasicPulse, BiphasicPulse,\n                               PulseTrain)\n\n    __all__ = [\n        'BiphasicPulse',\n        'MonophasicPulse',\n        'PulseTrain',\n        'Stimulus',\n        'TimeSeries'\n    ]\n\n<div class=\"alert alert-info\"><h4>Note</h4><p>One of the purposes of this file is to enumerate all the functions and\n    objects that should be imported in this subpackage.\n\n    The ``__all__`` variable lists all functions and objects to be imported\n    when somebody types ``from pulse2percept import *``.</p></div>\n\nTo make sure our new function gets imported, we need to modify the file as\nfollows:\n\n.. code-block:: python\n\n    \"\"\"Stimuli\n\n    This module provides a number of stimuli.\n    \"\"\"\n\n    from .base import Stimulus\n    from .pulse_trains import (TimeSeries, MonophasicPulse, BiphasicPulse,\n                               PulseTrain, largest_timeseries_element)\n\n    __all__ = [\n        'BiphasicPulse',\n        'largest_timeseries_element'\n        'MonophasicPulse',\n        'PulseTrain',\n        'Stimulus',\n        'TimeSeries'\n    ]\n\n<div class=\"alert alert-info\"><h4>Note</h4><p>In agreement with `PEP8`_, our function name is lowercase and uses\n    underscores to separate words, whereas the other variables listed in this\n    file are all class names (and thus should use CapWords convention).</p></div>\n\nNow we are able to run the test suite without ``ImportError``!\n\n<div class=\"alert alert-info\"><h4>Note</h4><p>You might have to run ``make`` first to install the code changes.\n    ``make tests`` does that automatically.</p></div>\n\nUpdating the test function\n--------------------------\n\nNow the test passes, but have not yet implemented all the functionality\noutlined under `Scenario`_. Specifically, we need to throw a ``TypeError`` if\nthe input argument is not of type\n:py:class:`~pulse2percept.stimuli.TimeSeries`, and throw a warning if the\nlargest value is 0.\n\nWe should therefore update ``test_largest_timeseries_element`` as follows:\n\n.. code-block:: python\n\n    import pytest\n    import numpy as np\n    import numpy.testing as npt\n    from pulse2percept.stimuli import (largest_timeseries_element, TimeSeries,\n                                       PulseTrain)\n    from pulse2percept.utils.testing import assert_warns_msg\n\n    def test_largest_timeseries_element():\n        # Create a simple TimeSeries object:\n        ts = TimeSeries(1, np.array([0, 1.5, 2]))\n        # Use almost_equal because we are comparing doubles:\n        npt.assert_almost_equal(largest_timeseries_element(ts), 2.0)\n\n        # Make sure an error is thrown here:\n        with pytest.raises(TypeError):\n            largest_timeseries_element(3.0)\n        with pytest.raises(TypeError):\n            largest_timeseries_element([0, 1.5, 2])\n\n        # Make sure a warning is thrown here:\n        ts = TimeSeries(1, np.array([0, 0, 0]))\n        assert_warns_msg(UserWarning, largest_timeseries_element, [ts],\n                         \"is zero\")\n\nThe :py:func:`~pulse2percept.utils.assert_warns_msg` takes as input the\n`warning category`_ to expect (``UserWarning``), the function to test\n(``largest_timeseries_element``), a list of objects to pass (``[ts]``),\nand a warning message (or substring thereof) to expect (``\"is zero\"``).\n\nOther things to add:\n\n* You might want to check whether your function works on different NumPy\n  arrays: ``np.array([1])``? 2-D? 3-D? etc.\n* You might want to check whether the warning is raised when the maximum\n  values is *really close but not exactly* 0.\n* You might want to check if your function works for subclasses of\n  :py:class:`~pulse2percept.stimuli.TimeSeries`, such as\n  :py:class:`~pulse2percept.stimuli.MonophasicPulse` and\n  :py:class:`~pulse2percept.stimuli.PulseTrain`.\n\n\nUpdating the actual function\n----------------------------\n\nTo make this test pass, we have to make a few changes to our function.\n\nFor one, we can check a variable's type with ``isinstance``.\n\nFor another, we can produce a warning with the ``logging`` package.\n\n.. code-block:: python\n\n    import logging\n    import numpy as np\n\n    def largest_timeseries_element(ts):\n        \"\"\"Return the largest element of a TimeSeries object\n\n        Parameters\n        ----------\n        ts:\n            TimeSeries\n            A TimeSeries object\n\n        Returns\n        -------\n        max:\n            double\n            The largest value in the TimeSeries data\n        \"\"\"\n        if not isinstance(ts, TimeSeries):\n            raise TypeError(\"Input argument 'ts' is not a TimeSeries object.\")\n        max_val = np.max(ts.data)\n        if np.isclose(max_val, 0):\n            # Use `isclose` because we are comparing doubles:\n            logging.getLogger(__name__).warn(\"Max val is zero.\")\n        return max_val\n\nA few things to observe here:\n\n* We raise ``TypeError``, which is the same type of error that we test\n  against in our test function. We could have also chosen a different\n  `exception class`_.\n\n* We don't need to import :py:class:`~pulse2percept.stimuli.TimeSeries` here,\n  because it is defined in the same file.\n\n* We use `isinstance`_ to check if the input argument is an instance or\n  subclass of :py:class:`~pulse2percept.stimuli.TimeSeries`. Note that a\n  :py:class:`~pulse2percept.stimuli.PulseTrain` would also return True,\n  because it is a subclass of :py:class:`~pulse2percept.stimuli.TimeSeries`.\n\n* We use `np.isclose`_ to check whether two values are equal within a\n  tolerance. We could also pass an absolute/relative tolerance level.\n\n* We use ``logging.getLogger(__name__)`` to access the `logger`_ that was\n  already created by pulse2percept for this very file (\"__name__\").\n\n<div class=\"alert alert-info\"><h4>Note</h4><p>You would import :py:class:`~pulse2percept.stimuli.Stimulus` by writing\n    ``from .base import Stimulus``, because it lives in the same directory\n    (\".\") in the file \"base.py\".\n    Similarly, you could import :py:class:`~pulse2percept.implants.ArgusII` via\n    ``from ..implants import ArgusII``.</p></div>\n\n\nVerifying the result\n--------------------\n\nThe final test is to make sure that ``make tests`` marks all tests with\n\"PASSED\". It is important to run all tests, because sometimes our code change\n(e.g., a bug fix) inadvertently breaks other pieces of the software.\nRunning the full test suite makes sure this doesn't happen.\n\nOnce all tests pass, you are ready to submit your pull request\n(see `Contributing to pulse2percept <dev-contributing>`).\n\nGood luck!\n"
      ]
    }
  ],
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.7.3"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}