File: introduction.txt

package info (click to toggle)
python-django-celery-beat 2.4.0-1
  • links: PTS, VCS
  • area: main
  • in suites: bookworm
  • size: 996 kB
  • sloc: python: 2,694; makefile: 326; sh: 22
file content (258 lines) | stat: -rw-r--r-- 8,585 bytes parent folder | download
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
:Version: 2.4.0
:Web: http://django-celery-beat.readthedocs.io/
:Download: http://pypi.python.org/pypi/django-celery-beat
:Source: http://github.com/celery/django-celery-beat
:Keywords: django, celery, beat, periodic task, cron, scheduling

About
=====

This extension enables you to store the periodic task schedule in the
database.

The periodic tasks can be managed from the Django Admin interface, where you
can create, edit and delete periodic tasks and how often they should run.

Using the Extension
===================

Usage and installation instructions for this extension are available
from the :ref:`Celery documentation <beat-custom-schedulers>`.

Important Warning about Time Zones
==================================

.. warning::

    If you change the Django :setting:`TIME_ZONE` setting your periodic task schedule
    will still be based on the old timezone.

    To fix that you would have to reset the "last run time" for each periodic
    task:

       >>> from django_celery_beat.models import PeriodicTask, PeriodicTasks
       >>> PeriodicTask.objects.update(last_run_at=None)
       >>> PeriodicTasks.changed()

    Note that this will reset the state as if the periodic tasks have never run
    before.

Models
======

- :class:`django_celery_beat.models.PeriodicTask`

  This model defines a single periodic task to be run.

  It must be associated with a schedule, which defines how often the task should
  run.

- :class:`django_celery_beat.models.IntervalSchedule`

  A schedule that runs at a specific interval (e.g. every 5 seconds).

- :class:`django_celery_beat.models.CrontabSchedule`

  A schedule with fields like entries in cron:
  ``minute hour day-of-week day_of_month month_of_year``.

- :class:`django_celery_beat.models.PeriodicTasks`

  This model is only used as an index to keep track of when the schedule has
  changed.

Whenever you update a :class:`~django_celery_beat.models.PeriodicTask`, a counter in this table is also
incremented, which tells the ``celery beat`` service to reload the schedule
from the database.

If you update periodic tasks in bulk, you will need to update the counter
manually:

   >>> from django_celery_beat.models import PeriodicTasks
   >>> PeriodicTasks.changed()

Example creating interval-based periodic task
---------------------------------------------

To create a periodic task executing at an interval you must first
create the interval object::

   >>> from django_celery_beat.models import PeriodicTask, IntervalSchedule

   # executes every 10 seconds.
   >>> schedule, created = IntervalSchedule.objects.get_or_create(
   ...     every=10,
   ...     period=IntervalSchedule.SECONDS,
   ... )

That's all the fields you need: a period type and the frequency.

You can choose between a specific set of periods:


- :data:`IntervalSchedule.DAYS <django_celery_beat.models.IntervalSchedule.DAYS>`
- :data:`IntervalSchedule.HOURS <django_celery_beat.models.IntervalSchedule.HOURS>`
- :data:`IntervalSchedule.MINUTES <django_celery_beat.models.IntervalSchedule.MINUTES>`
- :data:`IntervalSchedule.SECONDS <django_celery_beat.models.IntervalSchedule.SECONDS>`
- :data:`IntervalSchedule.MICROSECONDS <django_celery_beat.models.IntervalSchedule.MICROSECONDS>`

.. note::

    If you have multiple periodic tasks executing every 10 seconds,
    then they should all point to the same schedule object.

There's also a "choices tuple" available should you need to present this
to the user:

   >>> IntervalSchedule.PERIOD_CHOICES


Now that we have defined the schedule object, we can create the periodic task
entry:

   >>> PeriodicTask.objects.create(
   ...     interval=schedule,                  # we created this above.
   ...     name='Importing contacts',          # simply describes this periodic task.
   ...     task='proj.tasks.import_contacts',  # name of task.
   ... )


Note that this is a very basic example, you can also specify the arguments
and keyword arguments used to execute the task, the ``queue`` to send it
to [#f1]_, and set an expiry time.

Here's an example specifying the arguments, note how JSON serialization is
required:

   >>> import json
   >>> from datetime import datetime, timedelta

   >>> PeriodicTask.objects.create(
   ...     interval=schedule,                  # we created this above.
   ...     name='Importing contacts',          # simply describes this periodic task.
   ...     task='proj.tasks.import_contacts',  # name of task.
   ...     args=json.dumps(['arg1', 'arg2']),
   ...     kwargs=json.dumps({
   ...        'be_careful': True,
   ...     }),
   ...     expires=datetime.utcnow() + timedelta(seconds=30)
   ... )


.. [#f1] you can also use low-level AMQP routing using the ``exchange`` and
   ``routing_key`` fields.

Example creating crontab-based periodic task
--------------------------------------------

A crontab schedule has the fields: ``minute``, ``hour``, ``day_of_week``,
``day_of_month`` and ``month_of_year``, so if you want the equivalent
of a ``30 * * * *`` (execute at 30 minutes past the hour every hour) crontab
entry you specify:

   >>> from django_celery_beat.models import CrontabSchedule, PeriodicTask
   >>> schedule, _ = CrontabSchedule.objects.get_or_create(
   ...     minute='30',
   ...     hour='*',
   ...     day_of_week='*',
   ...     day_of_month='*',
   ...     month_of_year='*',
   ... )


Then to create a periodic task using this schedule, use the same approach as
the interval-based periodic task earlier in this document, but instead
of ``interval=schedule``, specify ``crontab=schedule``:

   >>> PeriodicTask.objects.create(
   ...     crontab=schedule,
   ...     name='Importing contacts',
   ...     task='proj.tasks.import_contacts',
   ... )

Temporarily disable a periodic task
-----------------------------------

You can use the ``enabled`` flag to temporarily disable a periodic task:

   >>> periodic_task.enabled = False
   >>> periodic_task.save()


Example running periodic tasks
------------------------------

The periodic tasks still need 'workers' to execute them.
So make sure the default **Celery** package is installed.
(If not installed, please follow the installation instructions
here: :github_project:`celery/celery`)

Both the worker and beat services need to be running at the same time.

1. Start a Celery worker service (specify your Django project name):

   .. code-block:: sh

      $ celery -A [project-name] worker --loglevel=info


2. As a separate process, start the beat service (specify the Django scheduler):

   .. code-block:: sh

      $ celery -A [project-name] beat -l info --scheduler django_celery_beat.schedulers:DatabaseScheduler


  **OR** you can use the -S (scheduler flag), for more options see ``celery beat --help``):

  .. code-block:: sh

     $ celery -A [project-name] beat -l info -S django

  **OR** you can set the scheduler through Django's settings:

  .. code-block:: sh

     CELERYBEAT_SCHEDULER = 'django_celery_beat.schedulers:DatabaseScheduler'



Also, as an alternative, you can run the two steps above (worker and beat services)
with only one command (recommended for **development environment only**):

.. code-block:: sh

   $ celery -A [project-name] worker --beat --scheduler django --loglevel=info


3. Now you can add and manage your periodic tasks from the Django Admin interface.



Working with django-celery-results
-----------------------------------

Now you can store :attr:`PeriodicTask.name <django_celery_beat.models.PeriodicTask.name>`
to django-celery-results (``TaskResult.periodic_task_name``).

Suppose we have two periodic tasks, their schedules are different, but the tasks are the same.

+-----------+------------------+------+---------------+
|   name    |       task       | args |   schedule    |
+===========+==================+======+===============+
| schedule1 | some.celery.task | (1,) | every hour    |
| schedule2 | some.celery.task | (2,) | every 2 hours |
+-----------+------------------+------+---------------+

Now you can distinguish the source of the task from the results by the ``periodic_task_name`` field.

+--------+------------------+--------------------+
|   id   |    task_name     | periodic_task_name |
+========+==================+====================+
| uuid1  | some.celery.task | schedule1          |
| uuid2  | some.celery.task | schedule1          |
| uuid3  | some.celery.task | schedule2          |
| uuid4  | some.celery.task | schedule2          |
+--------+------------------+--------------------+

(more technical details here: :github_pr:`477`, :github_pr:`261`)