Mypy

Python's Mypy: Callables and Generators

Reuven M. Lerner — Mon, 29 Jul 2019 11:00:00 +0000

Learn how Mypy's type checking works with functions and generators.

In my last two articles I've described some of the ways Mypy, a type checker for Python, can help identify potential problems with your code. [See "Introducing Mypy, an Experimental Optional Static Type Checker for Python" and "Python's Mypy—Advanced Usage".] For people (like me) who have enjoyed dynamic languages for a long time, Mypy might seem like a step backward. But given the many mission-critical projects being written in Python, often by large teams with limited communication and Python experience, some kind of type checking is an increasingly necessary evil.

It's important to remember that Python, the language, isn't changing, and it isn't becoming statically typed. Mypy is a separate program, running outside Python, typically as part of a continuous integration (CI) system or invoked as part of a Git commit hook. The idea is that Mypy runs before you put your code into production, identifying where the data doesn't match the annotations you've made to your variables and function parameters.

I'm going to focus on a few of Mypy's advanced features here. You might not encounter them very often, but even if you don't, it'll give you a better picture of the complexities associated with type checking, and how deeply the Mypy team is thinking about their work, and what tests need to be done. It'll also help you understand more about the ways people do type checking, and how to balance the beauty, flexibility and expressiveness of dynamic typing with the strictness and fewer errors of static typing.

Callable Types

When I tell participants in my Python classes that everything in Python is an object, they nod their heads, clearly thinking, "I've heard this before about other languages." But then I show them that functions and classes are both objects, and they realize that Python's notion of "everything" is a bit more expansive than theirs. (And yes, Python's definition of "everything" isn't as wide as Smalltalk's.)

When you define a function, you're creating a new object, one of type "function":


>>> def foo():
...     return "I'm foo!"

>>> type(foo)

Similarly, when you create a new class, you're adding a new object type to Python:


>>> class Foo():
...     pass

>>> type(Foo)

It's a pretty common paradigm in Python to write a function that, when it runs, defines and runs an inner function. This is also known as a "closure", and it has a few different uses. For example, you can write:

Go to Full Article

Python's Mypy--Advanced Usage

Reuven M. Lerner — Mon, 24 Jun 2019 11:30:00 +0000

by Reuven M. Lerner

Mypy can check more than simple Python types.

In my last article, I introduced Mypy, a package that enforces type checking in Python programs. Python itself is, and always will remain, a dynamically typed language. However, Python 3 supports "annotations", a feature that allows you to attach an object to variables, function parameters and function return values. These annotations are ignored by Python itself, but they can be used by external tools.

Mypy is one such tool, and it's an increasingly popular one. The idea is that you run Mypy on your code before running it. Mypy looks at your code and makes sure that your annotations correspond with actual usage. In that sense, it's far stricter than Python itself, but that's the whole point.

In my last article, I covered some basic uses for Mypy. Here, I want to expand upon those basics and show how Mypy really digs deeply into type definitions, allowing you to describe your code in a way that lets you be more confident of its stability.

Type Inference

Consider the following code:


x: int = 5
x = 'abc'
print(x)

This first defines the variable x, giving it a type annotation of int. It also assigns it to the integer 5. On the next line, it assigns x the string abc. And on the third line, it prints the value of x.

The Python language itself has no problems with the above code. But if you run mypy against it, you'll get an error message:


mytest.py:5: error: Incompatible types in assignment
   (expression has type "str", variable has type "int")

As the message says, the code declared the variable to have type int, but then assigned a string to it. Mypy can figure this out because, despite what many people believe, Python is a strongly typed language. That is, every object has one clearly defined type. Mypy notices this and then warns that the code is assigning values that are contrary to what the declarations said.

In the above code, you can see that I declared x to be of type int at definition time, but then assigned it to a string, and then I got an error. What if I don't add the annotation at all? That is, what if I run the following code via Mypy:

Go to Full Article

Introducing Mypy, an Experimental Optional Static Type Checker for Python

Reuven M. Lerner — Mon, 13 May 2019 11:30:00 +0000

by Reuven M. Lerner

Tighten up your code and identify errors before they occur with mypy.

I've been using dynamic languages—Perl, Ruby and Python—for many years. I love the flexibility and expressiveness that such languages provide. For example, I can define a function that sums numbers:


def mysum(numbers):
    total = 0
   for one_number in numbers:
       total += one_number
   return total

The above function will work on any iterable that returns numbers. So I can run the above on a list, tuple or set of numbers. I can even run it on a dictionary whose keys are all numbers. Pretty great, right?

Yes, but for my students who are used to static, compiled languages, this is a very hard thing to get used to. After all, how can you make sure that no one passes you a string, or a number of strings? What if you get a list in which some, but not all, of the elements are numeric?

For a number of years, I used to dismiss such worries. After all, dynamic languages have been around for a long time, and they have done a good job. And really, if people are having these sorts of type mismatch errors, then maybe they should be paying closer attention. Plus, if you have enough testing, you'll probably be fine.

But as Python (and other dynamic languages) have been making inroads into large companies, I've become increasingly convinced that there's something to be said for type checking. In particular, the fact that many newcomers to Python are working on large projects, in which many parts need to interoperate, has made it clear to me that some sort of type checking can be useful.

How can you balance these needs? That is, how can you enjoy Python as a dynamically typed language, while simultaneously getting some added sense of static-typing stability?

One of the most popular answers is a system known as mypy, which takes advantage of Python 3's type annotations for its own purposes. Using mypy means that you can write and run Python in the normal way, gradually adding static type checking over time and checking it outside your program's execution.

In this article, I start exploring mypy and how you can use it to check for problems in your programs. I've been impressed by mypy, and I believe you're likely to see it deployed in a growing number of places, in no small part because it's optional, and thus allows developers to use it to whatever degree they deem necessary, tightening things up over time, as well.

Dynamic and Strong Typing

In Python, users enjoy not only dynamic typing, but also strong typing. "Dynamic" means that variables don't have types, but that values do. So you can say:

Go to Full Article