Change language

Testing in Python: unittest and pytest. Tutorial for beginners

It is no secret that developers create programmes that sooner or later become very large-scale (if you look at the number of lines of code). And with this comes a great responsibility for quality.

Now I will tell you how unittest and pytest will help you find errors in programs and eliminate them in future.

So, testing

Everyone who wrote the first programs (be it the classic "hello, world" or calculator) always ran tests to check their work.

The very fact of running it is the very first, invisible touch of testing technology in your life. Consider it as the process of finding bugs on a slightly more complex program.

For example, you need to enter three numbers (a, b, c) and find the roots of a square equation. To solve it, we write the code:

from math import sqrt

def square_eq_solver(a, b, c):
   result = []
   discriminant = b * b - 4 * a * c

   if discriminant == 0:
       result.append(-b / (2 * a))
   else:
       result.append((-b + sqrt(discriminant)) / (2 * a))
       result.append((-b - sqrt(discriminant)) / (2 * a))

   return result

def show_result(data):
   if len(data) > 0:
       for index, value in enumerate(data):
           print(f'Solution number {index+1} equals {value:.02f}')
   else:
       print('No solution s')

def main():
   a, b, c = map(int, input('Please enter 3 values separated by spaces ').split())
   result = square_eq_solver(a, b, c)
   show_result(result)

if __name__ == '__main__':
   main()

I should say right away: I view any task, however brief it may be, from the perspective of "someday it will grow and become very large". So I always try to divide the program into different subprograms (input/processing/output).

You may have already noticed a bug in the code. But in some cases it may be so deeply hidden that you cannot easily detect it. In this case, the only way to find it out is to test the code. How to do it?

  • Knowing the algorithm for finding the roots of an equation, we determine the sets of input data which will be given to the program;
  • knowing the input data, you can manually calculate the answer the program should give;
  • run the program and give it the input data;
  • get its answer and compare it with the one it should get. If they are the same, go to the next set of data, if not, report an error.

For example, the following tests can be matched for a given task:

  • 10x2 = 0 - the only root of x=0 2x2 + 5x - 3 = 0 - this equation has two roots (x1 = 0.5, x2 = 3)
  • 10x**2 + 2 = 0 - this equation has no roots.

Tests picked up, what next? That's right, let's run them:

Test number 1
> python.exe example.py
Please enter three numbers separated by a space: 10 0 0
The root number 0 is 0.00

Test number 2:
> python.exe example.py
Please enter three numbers followed by a space: 2 5 -3
Root number 1 is 0.50
Root number 2 is -3.00

Test number 3:
> python.exe example.py
Please enter three numbers separated by a space: 10 0 2
Traceback (most recent call last):
  File "C:PyProjectstprogerexample.py", line 32, in <module>
    main()
  File "C:PyProjectstprogerexample.py", line 27, in main
    result = square_eq_solver(a, b, c)
  File "C:PyProjectstprogerexample.py", line 11, in square_eq_solver
    result.append((-b + sqrt(discriminant)) / (2 * a))
ValueError: math domain error

Oops… There was an error in the third test. Exactly the one you might have noticed in the source code of the program - the case with zero discriminant was not processed. As a result, you can tweak the function code so that this variant is processed correctly:

def square_eq_solver(a, b, c):
   result = []
   discriminant = b * b - 4 * a * c

   if discriminant == 0:
       result.append(-b / (2 * a))
   elif discriminant > 0: # <--- changed the condition, now
                           # if the discriminant is zero
                           # will not be evaluated
       result.append((-b + sqrt(discriminant)) / (2 * a))
       result.append((-b - sqrt(discriminant)) / (2 * a))

   return result

Run all the tests again and they will work fine.

But be aware that it will take a few minutes to re-test the program and re-test all three variants of the input values. If there are a lot of such variants, it would be very expensive to call them manually. This is where automated testing comes in.

An automated testing program is run on the basis of pre-prepared input/output data and the program that will call it. It is essentially a program that tests other programs. And within the Python language ecosystem, there are several packages that allow you to automate the testing process.

Unittest and pytest: writing tests

The two most popular libraries are unittest and pytest. Let's try each one to evaluate the syntax objectively.

We will start with unittest, because it is the place where many people start to learn the world of testing. The reason for this is simple: the library is integrated by default into the standard Python language library.

Code format

In terms of format it closely resembles the JUnit library used in Java for writing tests:

  • tests must be written in a class;
  • the class must be inherited from the base class unittest.TestCase;
  • names of all functions, which are tests, must begin with the keyword test;
  • inside functions there must be calls of comparison operators (assertX) - they will check our obtained values for compliance with the declared ones.

An example of using unittest for our task

import unittest

class SquareEqSolverTestCase(unittest.TestCase):
   def test_no_root(self):
       res = square_eq_solver(10, 0, 2)
       self.assertEqual(len(res), 0)

   def test_single_root(self):
       res = square_eq_solver(10, 0, 0)
       self.assertEqual(len(res), 1)
       self.assertEqual(res, [0])

   def test_multiple_root(self):
       res = square_eq_solver(2, 5, -3)
       self.assertEqual(len(res), 2)
       self.assertEqual(res, [0.5, -3])

This code is run with the following command

python.exe -m unittest example.py

And as a result, the following will be displayed on the screen:

> python.exe -m unittest example.py
...
------------------------------------------------------------------
Ran 3 tests in 0.001s

OK

If an error is found in any of the tests, unittest will not hesitate to report it:

> python.exe -m unittest example.py
F..
==================================================================
FAIL: test_multiple_root (hello.SquareEqSolverTestCase)
------------------------------------------------------------------
Traceback (most recent call last):
  File "C:PyProjectstprogerexample.py", line 101, in test_multiple_root
    self.assertEqual(len(res), 3)
AssertionError: 2 != 3
------------------------------------------------------------------
Ran 3 tests in 0.001s

FAILED (failures=1)

unittest: arguments for

  • Is part of the standard Python language library: no need to install anything extra;
  • Flexible structure and conditions for running tests. For each test, you can assign tags, according to which either one or another group of tests will be run;
  • Fast generation of test reports, both in plaintext and XML format.

unittest: arguments against

  • You would have to write a rather large amount of code (as compared to other libraries) to carry out testing;
  • Because the developers were inspired by the JUnit library format, the names of the main functions are written in camelCase style (e.g. setUp and assertEqual);
  • The python language should use the snake_case name format (e.g. set_up and assert_equal), as recommended by pep8.

Pytest

Probably the most popular open source framework presented here.

Pytest allows unit testing (testing of individual program components), functional testing (testing of the ability of code to meet business requirements), API testing (application programming interface) and more.

Code format

Writing tests is much simpler here than in unittest. You simply write several functions that satisfy the following conditions:

The function name must begin with the keyword test;
Within the function, a boolean expression must be tested using the assert operator.

An example of using pytest for our task

def test_no_root():
res = square_eq_solver(10, 0, 2)
assert len(res) == 0

def test_single_root():
res = square_eq_solver(10, 0, 0)
assert len(res) == 1
assert res == [0]

def test_no_root():
res = square_eq_solver(10, 0, 2)
assert len(res) == 0

def test_single_root():
res = square_eq_solver(10, 0, 0)
assert len(res) == 1
assert res == [0]

def test_multiple_root():
res = square_eq_solver(2, 5, -3)
assert len(res) == 3
assert res == [0.5, -3]

This code is run with the following command

pytest.exe example.py

And as a result, the following will be displayed on the screen:

> pytest.exe example.py
======================= test session starts ======================
platform win32 -- Python 3.9.6, pytest-7.1.2, pluggy-1.0.0
rootdir: C:PyProjectstproger
collected 3 items

example.py ...                                              [100%]

======================== 3 passed in 0.03s =======================

In the event of an error, the output will be slightly larger:

> pytest.exe example.py
======================= test session starts ======================
platform win32 -- Python 3.9.6, pytest-7.1.2, pluggy-1.0.0
rootdir: C:PyProjectstproger
collected 3 items

example.py ..F                                              [100%]

============================ FAILURES ============================
_______________________ test_multiple_root _______________________

    def test_multiple_root():
        res = square_eq_solver(2, 5, -3)
>       assert len(res) == 3
E       assert 2 == 3
E        +  where 2 = len([0.5, -3.0])

example.py:116: AssertionError
===================== short test summary info ====================
FAILED example.py::test_multiple_root - assert 2 == 3

=================== 1 failed, 2 passed in 0.10s ==================

Pytest: arguments for

  • Allows you to write compact (compared to unittest) test suites;
  • Outputs much more information about errors when they occur;
  • Allows to run tests written for other testing systems;
  • Has a system of plugins (and hundreds of them) that extend the framework's capabilities. Examples of such plugins: pytest-cov, pytest-django, pytest-bdd;
  • Allows tests to be run in parallel (using the pytest-xdist plugin).

Pytest: arguments against

  • pytest is not part of the standard Python language library. Therefore, you will have to install it separately using the pip install pytest command;
  • there is no compatibility with other frameworks. So if you write code for pytest, you will not be able to run it with the built-in unittest.

So what's better?

If you need basic unit testing and are familiar with frameworks like xUnit, then unittest is fine.
If you're looking for a framework that allows you to create concise and elegant tests that implement complex logic checks, pytest is the way to go.

Post Scriptum

The topic of quality control is very broad. And even the code I've written is very easy to pick on. At the very least, there's no check that the input data must necessarily be integers. If you type in any other number or even a string, the program will make an error.

By the way, I intentionally left one more error in this program (this time a logical one) related to finding the root.

Shop

Learn programming in R: courses

$

Best Python online courses for 2022

$

Best laptop for Fortnite

$

Best laptop for Excel

$

Best laptop for Solidworks

$

Best laptop for Roblox

$

Best computer for crypto mining

$

Best laptop for Sims 4

$

Latest questions

NUMPYNUMPY

Common xlabel/ylabel for matplotlib subplots

12 answers

NUMPYNUMPY

How to specify multiple return types using type-hints

12 answers

NUMPYNUMPY

Why do I get "Pickle - EOFError: Ran out of input" reading an empty file?

12 answers

NUMPYNUMPY

Flake8: Ignore specific warning for entire file

12 answers

NUMPYNUMPY

glob exclude pattern

12 answers

NUMPYNUMPY

How to avoid HTTP error 429 (Too Many Requests) python

12 answers

NUMPYNUMPY

Python CSV error: line contains NULL byte

12 answers

NUMPYNUMPY

csv.Error: iterator should return strings, not bytes

12 answers

News


Wiki

Python | How to copy data from one Excel sheet to another

Common xlabel/ylabel for matplotlib subplots

Check if one list is a subset of another in Python

sin

How to specify multiple return types using type-hints

exp

Printing words vertically in Python

exp

Python Extract words from a given string

Cyclic redundancy check in Python

Finding mean, median, mode in Python without libraries

cos

Python add suffix / add prefix to strings in a list

Why do I get "Pickle - EOFError: Ran out of input" reading an empty file?

Python - Move item to the end of the list

Python - Print list vertically