For modern web applications, running asynchronous tasks in the background is often a must. Whether you need to parallelize something not-so-time-critical (say, thumbnail generation) or access that miraculous-but-really-slow machine learning API in the background, there is a plethora of other use cases that require the developer to isolate time-consuming operations from Django’s default synchronous request-response cycle. This is done to spare the user having to wait for a response while being left to stare at an unresponsive browser window.
Of course you could go ahead and develop your own asynchronous task queue by means of Python’s good ol’ threading module. But why reinvent the wheel? On the other hand, if asynchronous programming is an area you would like to learn more about, the DIY approach might be the way to go. In case you are going for a ready-made solution, keep on reading.
Because executing code in the background is so important these days, various people have come up with various solutions to the problem. For Django alone, at least 3 well-supported packages are available via PyPI: Celery, Channels and Django Background Tasks.
Each of the mentioned packages is a great choice for implementing background tasks. In this post, I will focus on Django Background Tasks (DBT) as in my experience it’s the easiest to set up. This is mostly due to its simple design which is, by default, database-backed. Thanks to this property you are not required to install an external message broker such as Redis.
In order to set up DBT, you have to install the package by means of pip, add the background_task app to your INSTALLED_APPS and then migrate your database to install the required tables:
# In your shell
pip install django-background-tasks
# In settings.py
INSTALLED_APPS = (
(…),
'background_task',
)
# In your shell again
python manage.py makemigrations background_task
python manage.py migrate
Next, it is your turn to decorate a function with that sweet @background decorator included with DBT. The convention is to define these background functions in myapp/tasks.py but of course you could define them virtually anywhere in your codebase. Let’s stick with the convention, though, in order not to irritate your co-coders and risk exclusion from the next company event.
# In myapp/tasks.py
from background_task import background
@background
def do_xyz_in_the_background(**kwargs):
…
Whenever you call this function, DBT will transparently create an instance of DBT’s Task model and append it to its database-backed task queue. If you needed to, you could still call the decorated function synchronously:
do_xyz_in_the_background.now(some_kwarg=123)
One more important thing: For the scheduled background tasks to actually be executed, you need to run the python manage.py process_tasks command in parallel with your Django server. process_tasks periodically checks the database for new Tasks and, if necessary, launches them asynchronously.