Testing

Python Testing with pytest: Fixtures and Coverage

Reuven M. Lerner — Mon, 14 Jan 2019 12:30:00 +0000

Improve your Python testing even more.

In my last two articles, I introduced pytest, a library for testing Python code (see "Testing Your Code with Python's pytest" Part I and Part II). pytest has become quite popular, in no small part because it's so easy to write tests and integrate those tests into your software development process. I've become a big fan, mostly because after years of saying I should get better about testing my software, pytest finally has made it possible.

So in this article, I review two features of pytest that I haven't had a chance to cover yet: fixtures and code coverage, which will (I hope) convince you that pytest is worth exploring and incorporating into your work.

Fixtures

When you're writing tests, you're rarely going to write just one or two. Rather, you're going to write an entire "test suite", with each test aiming to check a different path through your code. In many cases, this means you'll have a few tests with similar characteristics, something that pytest handles with "parametrized tests".

But in other cases, things are a bit more complex. You'll want to have some objects available to all of your tests. Those objects might contain data you want to share across tests, or they might involve the network or filesystem. These are often known as "fixtures" in the testing world, and they take a variety of different forms.

In pytest, you define fixtures using a combination of the pytest.fixture decorator, along with a function definition. For example, say you have a file that returns a list of lines from a file, in which each line is reversed:


def reverse_lines(f):
   return [one_line.rstrip()[::-1] + '\n'
           for one_line in f]

Note that in order to avoid the newline character from being placed at the start of the line, you remove it from the string before reversing and then add a '\n' in each returned string. Also note that although it probably would be a good idea to use a generator expression rather than a list comprehension, I'm trying to keep things relatively simple here.

If you're going to test this function, you'll need to pass it a file-like object. In my last article, I showed how you could use a StringIO object for such a thing, and that remains the case. But rather than defining global variables in your test file, you can create a fixture that'll provide your test with the appropriate object at the right time.

Here's how that looks in pytest:

Go to Full Article

Unit Testing in the Linux Kernel

Zack Brown — Thu, 03 Jan 2019 13:00:00 +0000

by Zack Brown

Brendan Higgins recently proposed adding unit tests to the Linux kernel, supplementing other development infrastructure such as perf, autotest and kselftest. The whole issue of testing is very dear to kernel developers' hearts, because Linux sits at the core of the system and often has a very strong stability/security requirement. Hosts of automated tests regularly churn through kernel source code, reporting any oddities to the mailing list.

Unit tests, Brendan said, specialize in testing standalone code snippets. It was not necessary to run a whole kernel, or even to compile the kernel source tree, in order to perform unit tests. The code to be tested could be completely extracted from the tree and tested independently. Among other benefits, this meant that dozens of unit tests could be performed in less than a second, he explained.

Giving credit where credit was due, Brendan identified JUnit, Python's unittest.mock and Googletest/Googlemock for C++ as the inspirations for this new KUnit testing idea.

Brendan also pointed out that since all code being unit-tested is standalone and has no dependencies, this meant the tests also were deterministic. Unlike on a running Linux system, where any number of pieces of the running system might be responsible for a given problem, unit tests would identify problem code with repeatable certainty.

Daniel Vetter replied extremely enthusiastically to Brendan's work. In particular, he said, "Having proper and standardized infrastructure for kernel unit tests sounds terrific. In other words: I want." He added that he and some others already had been working on a much more specialized set of unit tests for the Direct Rendering Manager (DRM) driver. Brendan's approach, he said, would be much more convenient than his own more localized efforts.

Dan Williams was also very excited about Brendan's work, and he said he had been doing a half-way job of unit tests on the libnvdimm (non-volatile device) project code. He felt Brendan's work was much more general-purpose, and he wanted to convert his own tests to use KUnit.

Go to Full Article

Testing Your Code with Python's pytest

Reuven M. Lerner — Tue, 27 Nov 2018 12:30:00 +0000

by Reuven M. Lerner

Don't test your code? pytest removes any excuses.

Software developers don't just write software; they also use software. So, they're the first to recognize, and understand, that software is complex and inevitably contains bugs.

But, just because bugs are inevitable doesn't mean that developers can or should try to prevent them. And, thus, during the past few decades, there's been rapid growth in software testing. Testing is no longer seen as an optional or "nice to have" part of software development; it's considered an absolute must—part of the software development process. In many cases, the people in the Python courses I teach at various companies aren't developers per se, but instead testers—people with the full-time job of writing tests to ensure that the company's software is robust.

I must admit that even though I've been writing software for a long time, I have rarely been as good about testing as I'd like to be. Sure, when I'm working on a large, complex app, I'll write tests, but it always seemed to be a bit of a burden. I know that it's good for me, will save tons of time in the future, will make the software more robust and maintenance easier, but really, if I just want to get my program out the door, why test? And besides, the various test frameworks I've used through the years never struck me as very impressive or easy to use.

So for the past few years, I've been in a bit of a holding pattern. I want to test more, but testing is annoying, so I don't test, which makes it seem like even more of a burden, because it's not part of my regular process.

All of this has changed for me recently, thanks to my discovery (long after other people, I admit) of the pytest library for Python. pytest turns out to be easy to use, easy to work with and easy to integrate into my work. Part of the reason for this is that pytest abandons the Python idea of "there's only one way to do it", giving developers a great degree of flexibility and freedom in choosing how to write tests.

So in this article, I provide an introduction to pytest, showing how to start integrating it into your development process today. I plan to expand on this in my next article and describe some more advanced pytest features that you might need to use.

Basic pytest

The idea behind pytest is that if you want to test a function, you'll write a separate function to test it. Actually, you'll probably want to write more than one test function, but that's in addition.

For example, let's assume you have the following function that sums numbers:

Go to Full Article

It's about the User: Applying Usability in Open-Source Software

Jim Hall — Tue, 18 Feb 2014 19:34:49 +0000

by Jim Hall

Open-source software developers have created an array of amazing programs that provide a great working environment with rich functionality. At work and at home, I routinely run Linux on my desktop, using Firefox and LibreOffice for most of my daily tasks. I prefer to run open-source software tools, and I think most Linux Journal readers do too. But as comfortable as the open-source software ecosystem can be, we've all shared or heard the same comments about some of our favorite Linux programs:

"___ is a great program, once you figure out how to use it."
"You can do a lot in ___, after you get past the awkward menus."
"You'll like using ___, if you can learn the user interface."

That's the problem. No matter how powerful the program, that functionality is lost if people have to figure out how to use the program in order to unlock its secrets. Typical users with average knowledge should be able to operate a general-purpose program. If a program is hard to use, that suggests the problem is with the program, not with the user.

Usability and Open-Source Software

"Usability" refers to how easily users can learn and start using software, or any similar "information product". Usability is separate from the functionality of the program, and so usability testing is different from unit testing. Instead, usability testing allows us to uncover issues that prevent users from using our programs.

Most open-source software programs are written by developers for other developers. Although some large open-source programs, such as GNOME and Drupal, have undergone usability testing, most projects lack the resources or interest to pursue a usability evaluation. As a result, open-source software programs often are utilitarian, focused on the functionality and features, with little attention paid to how people will use it. Applying usability practices tends to be antithetical to how open-source software is created. Open-source developers prefer functionality over appearance. Although some projects may have a maintainer who dictates a particular design aesthetic, many more do not. In an interview for this article, open-source advocate Eric Raymond commented to me that most programmers view menus and icons "like the frosting on a cake after you've baked it", which is an apt metaphor. Open-source software developers tend to prefer assembling the ingredients and baking the cake, not applying frosting to make it look nice.

So how can open-source developers easily apply usability to their own programs? There are many ways to implement usability practices. Alice Preston described 11 different techniques to evaluate usability in the STC Usability SIG newsletter. These methods run the gamut from interviews and focus groups to heuristic reviews and formal usability tests:

Go to Full Article