UP | HOME

Personal Python anti-patterns

1 Preface

Here's a collection of things I'm finding to be better to avoid while writing in Python. It's partly humorous and exaggerated, and of course most of Python programmers won't agree with it.

Obviously, Python itself should be avoided, if there are better alternatives, but here are those things.

2 Anti-patterns

2.1 Usual anti-patterns

Those are Python anti-patterns, OO anti-patterns, and general programming anti-patterns, which are widely known and could easily be googled; things like global variables.

2.2 FP

A language is said to support a style of programming if it provides facilities that make it convenient (reasonably easy, safe, and efficient) to use that style. A language does not support a technique if it takes exceptional effort or skill to write such programs; it merely enables the technique to be used.

– Bjarne Stroustrup, The C++ Programming Language

It's often being said that Python supports FP, what tricked me into it in the first place, but the truth is that it doesn't. There's simply nothing that makes it better for FP than virtually any other modern high-level language. The first problem one might notice is mutation, which breaks pretty much any possibility of sane FP, and then there goes a cumbersome syntax for lambdas, non-FP-friendly library, a lack of TCO, etc.

2.3 OOP

OO without static typing, and without strict interfaces, but with possibility to hack things easily (which is generally a good thing, but.), could easily lead into a design rabbit-hole, or simply to bugs. Though OO by itself is pretty error-prone. Hence, KISS, and only use it minimally.

2.4 Laziness

That is, things like generators and coroutines. It's just overcomplicated in Python, so, unless it really simplifies something else, or is necessary (e.g., amount of data or computations is really big; though it's also probably not a good idea to use Python for such tasks), it's better to avoid, and to force computations/values if a library function returns lazy ones (what they tend to do when it doesn't [always] make much sense: e.g., asyncio.create_subprocess_shell). Here's a function to force coroutines:

def force(x):
    loop = asyncio.get_event_loop()
    return loop.run_until_complete(x)

2.5 Exceptions

Gradually these objectives have been sacrificed in favor of power, supposedly achieved by a plethora of features and notational conventions, many of them unnecessary and some of them, like exception handling, even dangerous. We relive the history of the design of the motor car. Gadgets and glitter prevail over fundamental concerns of safety and economy.

[…]

In this last resort, I appeal to you, representatives of the programming profession in the United States, and citizens concerned with the welfare and safety of your own country and of mankind: Do not allow this language in its present state to be used in applications where reliability is critical, i.e., nuclear power stations, cruise missiles, early warning systems, anti-ballistic missile defense systems.

– C.A.R. Hoare, The Emperor’s Old Clothes

The primary duty of an exception handler is to get the error out of the lap of the programmer and into the surprised face of the user. Provided you keep this cardinal rule in mind, you can’t go far wrong.

– Verity Stob

It's not only about Python, but about languages with exceptions in general. Exceptions introduce a lot of complexity by breaking the control flow (just as goto's and premature returns), and proper handling (of every different situation) isn't quite practical: one would end up wrapping tens (or even hundreds) of LOC around every ORM call, for instance. Hence, contrary to "Catch What You Can Handle", I tend to wrap such calls into a cycle (for reset+retry), and try-catch with catch-all (catching specific exceptions when it makes sense as well, of course) – so that's just 7 lines of copypasted wrapper code on every such call. There's also no practical way to move that into a function, to write a macro, or to wrap the whole ORM. It's incredibly ugly, yet apparently inevitable. But here's what could be done to minimize its harm a little:

  • Avoid using additional libraries.
  • Avoid raising new exceptions, unless they certainly make sense.
  • Avoid logging.exception, when handling expected/known exceptions: huge traces make logs much harder to read.
  • Don't let exceptions to walk through your program, catch them as early as possible. Often it's required to implement proper handling anyway.

2.6 Concurrency

I'm not inclined to avoid concurrency in general, and it's actually not the worst in Python, but things are pretty awkward nevertheless. Its thread-based parallelism uses pthreads (and hence native threads), but doesn't run threads simultaneously, because of the global interpreter lock. Furthermore, somehow major libraries still manage to be not quite threads-friendly.

2.7 Using libraries

Python libraries tend to be abandoned and bitrotten, and those that work at all – buggy. I imagine that it's how it feels when one visits a festival area the next morning after that festival: there still are inviting plates that promise fun, decorations are there, too, though part of them is broken, but all the previously celebrating people are, as one might guess, sitting at home and having a hangover, and there's only rotting food, a lot of garbage, maybe some rats and dogs, and a few hobos around. But when one is about to conclude that he or she is late, strange things attract their attention: a crazy hobo gathers some rotten food, and places it back on a table; some folks are bringing a new inviting plate; other group of people consumes fermented fruits. And that's when one realizes that it's not the case that he or she is late; that's just what happens here every day, and not a single festival, but this continued weirdness by itself has generated all that rotting garbage. Then one installs a stand they intended to install, because it's even more pointless to bring it back, unintentionally adding a bit to the whole thing, and leaves the area. Madness thrives.

A bit more seriously, there's plenty of abandoned libraries in any language, yet one usually doesn't have to go through them, looking for a usable one.

Of course, there are some exceptions, and big and nice projects, but when something like "I could also implement a nice feature X, will only need a library Y, which surely should be around" comes to mind, it's often easier replace it with "I'd better spend my time doing something else" at once if using Python.

2.8 Writing programs

Python "programs" are called "scripts" even in its setuptools, and it probably becomes less awful if one uses it as a scripting language: when executing a little script manually, or generating a web page, things like exceptions don't matter as much as when it comes to, say, daemons: in the first case, it's fine if it'd crash with an exception (or will use a catch-all in a single place, without retries).

Indeed, if comparing Python to a shell, such as bash, rather than to a general-purpose language, it is even pretty nice. Well, not for shell scripting tasks, but for general-purpose language ones. Though it's rather like saying that a poor hammer is good for dealing with nails, comparing to a screwdriver.

2.9 Hope

It's easier to go into it without any hope to write nice programs, or to stay completely sane: neither of those is realistic while working with broken things, using broken tools. The trick is to pick kinds of software misbehaviour and insanity wisely.

3 Patterns

3.1 Move fast and break things

There are some patterns, too! Or, rather, approaches. "Worse is better" seems to be the slogan that drives such languages and their libraries, and problems tend to arise when trying to use a tool together with an approach which it is not supposed to be used with. So, it helps to extend "Worse it better" to your programs. Though it's a pain to maintain programs developed with such an approach.

3.2 Enforce or check types manually

It might be useful to enforce types (e.g., int, str, bytes) even where it doesn't seem to make sense. Transposed a list of regular integers with numpy.transpose, tried to insert those into a database using sqlalchemy+pymysql? Chances are that there will be a huge trace with a type error in the end, claiming that it doesn't like a bytes-like object in place of a string, mentioning regexps and other sqlalchemy internals, not numpy. After some time wasted on debug and checks aimed to find erroneous strings and bytes, one would find that transpose converted values into numpy.int64, and then pymysql didn't like it. There's no even single problem: each mentioned part is broken here, they just happen to work sometimes.

So, probably the only way to have at least some vague understanding of what happens, is to enforce (or check, perhaps?) types in a program, between library calls. Even if it doesn't make sense: who would expect transpose to change types at all?

Libraries, as this example shows, won't check them for us, and would fail much later, throwing a misleading error message.