How to add Celery, Celerybeat and Redis to a Django website.

09 Apr 2022 / Oscar F
django, celerybeat and redis logo and a celery

Install

Redis-server

sudo apt install redis-server

Python packages

In your python environment

pip install redis celery[redis] django-celery-beat

Celery

add to settings.py

INSTALLED_APPS += [
    'django_celery_beat',
]

and

CELERY_BROKER_URL = 'redis://localhost:6379'
CELERY_RESULT_BACKEND = 'redis://localhost:6379'

to /project/webbapp/__init__.py add

from .celery import app as celery_app

__all__ = ("celery_app",)

add a celery.py to /project/webbapp/ with

import os

from celery import Celery

os.environ.setdefault("DJANGO_SETTINGS_MODULE", "webbapp.settings")

app = Celery("webbapp")
app.config_from_object("django.conf:settings", namespace="CELERY")
app.autodiscover_tasks()

Development

redis-server
redis-cli ping
celery -A webapp worker -l info
celery -A webapp beat -l info -S django

Production

sudo nano /etc/redis/redis.conf

change "supervised no" to "supervised systemd"

sudo systemctl enable redis-server.service
sudo systemctl daemon-reload
sudo apt install supervisor

In /etc/supervisor/conf.d add two new files

celery.conf

[program:celery]
directory=/project/webapp
command=/project/webbapp/env/bin/celery -A webapp worker --loglevel=INFO
autostart=true
autorestart=true
startsecs=10
stopwaitsecs = 600

celerybeat.conf

[program:celerybeat]
directory=/project/webapp
command=/project/webbapp/env/bin/celery -A webbapp beat -S django --loglevel=INFO
autostart=true
autorestart=true
startsecs=10

run the two commands

sudo supervisorctl start all
sudo supervisorctl status all

References:

https://docs.celeryq.dev/en/stable/getting-started/introduction.html#installation

https://docs.celeryq.dev/en/stable/getting-started/backends-and-brokers/redis.html

https://docs.celeryq.dev/en/stable/django/first-steps-with-django.html

https://docs.celeryq.dev/en/stable/userguide/daemonizing.html

https://github.com/celery/celery/blob/master/extra/supervisord/celeryd.conf

https://github.com/celery/celery/blob/master/extra/supervisord/celerybeat.conf