What's up Python? Lazy imports, django gets a task queue...

September, 2025. And a bit of october.

Summary

• Python may finally get lazy imports

• Django will definitely get built-in task queues

• And a few small stuff.

PEP 810: lazy imports

I’m a week late for writing the monthly Python recap, but then something interesting happened: PEP 810 dropped a few days ago. Technically October’s news, but it’s too good to pass on, so let’s dive into that.

When you import a module in Python, its entire code is executed. This includes its own contained imports, executing other files, and so on. Because of this, importing a single name from a package can cascade so that almost all of it runs.

E.G, importing a single function from numpy executes 129 modules!

❯ python -c “import sys; old = len(sys.modules); print(old); from numpy import sum; new = len(sys.modules);print(new) ; print(new - old);” 

36
165
129

This is an issue in two cases:

• One-shot CLI tools. Sometimes the import time dominates the whole execution time. Even running --help could be costly! Mercurial famously suffered from this and had special functions to deal with it.

• Very big code bases that pay a huge starting cost because they have so many imports to resolve before being able to do anything. FAANG monorepos are the first victims.

For these reasons, we witnessed year after year new projects using some form of mitigation, like putting imports in function calls, setting lazy module attribute getters, or even creating new features. Cinder, the Facebook Python fork, did the latter.

Lazy imports are the logical solution to this: delay the load of the file, so that it’s executed not when you write the import statement, but later on, when you access the module content for the first time. When you actually use the module.

Lazy imports have been proposed before, but a consensus couldn’t be found. This time, they could land in Python 3.15 as the reception has been very positive. It is indeed an elegantly and meticulously crafted PEP, that comes with:

• An optional lazy keyword to mark an import as potentially lazy.

• A new interpreter mode for lazy imports that can be either default, disabled or enabled.

• A Python flag -x lazy_import=”<mode>” to set this mode.

• A PYTHON_LAZY_IMPORTS env var to set this mode.

• A Python API to set this mode with sys.set_lazy_imports()

• A __lazy_modules__ magic variable you can use to list modules and mark them as potentially lazy. This requires no new syntax, unlike the lazy keyword.

• A __lazy_import__() function that is the lazy equivalent of __import__().

• A sys.set_lazy_imports_filter() callback to programmatically exclude imports from being lazy.

If an import is marked this way:

lazy import json

Or this way:

lazy from json import dumps

Then it is marked as “potentially lazy”. If the interpreter's lazy import mode is default, then the marked import will indeed be lazy. If the mode is disabled, it won’t be. If the mode is enabled, all imports, marked or not, will be lazy.

This seems like a strange design until you realize that:

• The default is to follow what the keyword tells you.

• A global OFF switch is very interesting anywhere you need predictable performances. Because lazy imports could trigger a hot path anywhere. Disabling them make sure you pay the cost up front, like for a web server, where you want it at the start of the process, not at the first HTTP request handling.

• A global ON switch is very useful for testing.

• __lazy_modules__ allows to opt in early, but make it compatible with old Python versions that don’t yet have the new syntax: they just get ordinary imports.

• set_lazy_imports_filter() is a safety net for all the things the spec designers didn’t think about.

The proposal has a very narrow scope as well:

• lazy only works on imports. It is not a general lazy language feature. In fact, it mostly binds a proxy object to the module holding variable until an attribute is accessed.

• Any introspection of the import state will trigger reification (meaning imports will be executed). You look at it, it resolves, period.

• They don’t try to solve circular import.

• It’s not recursive. lazy only affects the import it marks, not the ones in the lazily imported modules.

The specs authors have done a good job at thinking about a lot of edge cases, without overdoing it. Given they are touching a very complex and sensitive part of Python, an immensely popular language among a super diverse crowd, it’s an incredible balancing job they did.

And there are many such details:

• __future__ can’t be lazy.

• sys.modules don’t contain the module name until first access, so that in checks work as expected.

• import *, imports in try/except (or context managers) and in functions cannot be marked as lazy.

• ImportError tracebacks on lazy-loaded modules will contain both the error at the access site and the import site.

• In the context of multi-threading, exactly one thread performs the import and atomically rebinds the importing module’s global to the resolved object, making it thread safe.

Of course, lazy imports come with a “here be dragons” warning, since:

• Imports might trigger in a different order than listed in the file.

• Side effects now occur at access time (that’s the point).

• If you look at the module object, it now contains a proxy.

• This will play interestingly if you lazily import something in one parent thread and use it first in a child thread, as the side effects occur there.

So if this gets implemented, don’t jump on it, even if it looks like an easy win. Use it only if the need arises; it is definitely an advanced feature.

And so much respect to the 7 people that came up with this, even if it’s not. It’s a beautiful example of how one should approach a problem, solve it, then explain it to the world.

Django finally gets background tasks

Any big Django website ends up having some kind of task queue to run blocking or long-running processes. 10 years ago, Celery was popular, but then more lightweight solutions like rq and huey took hold.

However, Django is typically batteries included, and it was about time it would come with its own solution out of the box, which finally was specified in DEP 14, thanks to Jake Howard, a part of the excellent Wagtail team.

It is expected to be available for version 6.0, coming out at the end of this year.

So what does it look like?

Well, let’s say you need to encode a video, and it takes a long, long time, so you don’t want to block your requests while you do.

First, you configure your task backend, which defines where the tasks and results are going to be stored. Appart from dummy and instant that are made for testing, the only option is currently storing stuff in the DB:

TASKS = {
    “default”: {
        “BACKEND”: “django_tasks.backends.database.DatabaseBackend”
    }
}
INSTALLED_APPS = [
    # ...
    “django_tasks”,
    “django_tasks.backends.database”,
] # don’t forget to ./manage.py migrate

You can define a task:

from imaginary_video_module import encode_video
from django.tasks import task

@task()
def encode_video_task(video_file_path):
    return encode_video(video_file_path)

You can then start your worker process:

./manage.py db_worker

This is the process, separate from your web process, that will handle the tasks.

And then, anywhere in your endpoints:

result = encode_video_task.enqueue(
    video_file_path=”./path/to/video.mp4”
)

This call will not block the endpoint, and instead, send a request to the task queue, which will deal with the video in a separate process. This way, your users get a response immediately, while the video is encoded in the background.

result is a special object that contains an id (so you can retrieve it elsewhere with default_task_backend.get_result()), a status (so you can know if the task is ready, pending, failed, etc), a return_value (it raises ValueError if the task status is not ResultStatus.SUCCEEDED) and errors, a list of objects containing information about potentially raised exceptions during your task.

It’s not a very fancy system (in fact, I managed to contribute a couple of lines to the project because the code was very accessible). For now, there is no possible custom backend, no alternative to the db backend, traceback information is just strings, there is no priority, and in fact, no timeout handling, no retry, no cancellation, not even observability!

But it’s also a no-brainer to setup, and should be sufficient for projects that just want to offload a few things without having to build a babel tower just yet.

So this first step is, as usual, to test the water. It’s not going to replace your multi-tenant terraformed airflow to Kafka deployment any time soon, but it will improve.

Small stuff

• six, the 2 to 3 compat library, is still in the top 20 of most downloaded libs from Pypi.

• PEP 757 wants to normalize how Python imports and exports integers.

• Mypy 1.18.1 is out, tooting a big bump in performance.

• Still more attacks on Pypi.

• uv now allows installing invalid wheels (it’s opt-in). Sounds like pyx is hitting corporate land.

• nanodjango, a microframework-like API for Django, has a new website. Good time to let you know this tool exists.