Celery 初步

Celery 是一个“自带电池”的的任务队列。它易于使用,所以你可以无视其所解决问题的复杂程度而轻松入门。它遵照最佳实践设计,所以你的产品可以扩展,或与其他语言集成,并且它自带了在生产环境中运行这样一个系统所需的工具和支持。

此教程中,你会了解使用 Celery 的最基础部分。包括:

  • 选择和安装消息传输方式(中间人)。
  • 安装 Celery 并创建第一个任务。
  • 运行 worker 并调用任务.
  • 追踪任务在不同状态间的迁移,并检视返回值。

Celery 起初可能令人却步——但不要担心——此教程会快速带你入门。此教程刻意简化,所以不会让你在高级特性上困扰。在你完成了此教程的学习后, 阅读文档其余部分是个明智的选择,例如展示 Celery 能力的 next-steps 教程。

选择中间人

Celery 需要一个发送和接收消息的解决方案,其通常以独立服务形式出现, 称为 消息中间人

可行的选择包括:

RabbitMQ

RabbitMQ 功能完备、稳定、耐用,并且安装简便,是生产环境的绝佳选择。 配合 Celery 使用 RabbitMQ 的详情见:

broker-rabbitmq

如果你使用 Ubuntu 或 Debian,可以执行这条命令来安装 RabbitMQ:

  1. $ sudo apt-get install rabbitmq-server

命令执行完成后,中间人就已经运行在后台,准备好传输消息: Starting rabbitmq-server: SUCCESS.

如果你不使用 Ubuntu 或 Debian 也无须担心,你可以访问这个网站来寻找同样简单的其他平台上(包括 Microsoft Windows)的安装指南:

http://www.rabbitmq.com/download.html

Redis

Redis 也是功能完备的,但更易受突然中断或断电带来数据丢失的影响。使用 Redis 的详细信息见:

broker-redis

Using a database

Using a database as a message queue is not recommended, but can be sufficient for very small installations. Your options include:

  • broker-sqlalchemy
  • broker-django

If you’re already using a Django database for example, using it as your message broker can be convenient while developing even if you use a more robust system in production.

Other brokers

In addition to the above, there are other experimental transport implementations to choose from, including Amazon SQS <broker-sqs>, broker-mongodb and IronMQ <broker-ironmq>.

See broker-overview for a full list.

Installing Celery

Celery is on the Python Package Index (PyPI), so it can be installed with standard Python tools like pip or easy_install:

  1. $ pip install celery

Application

The first thing you need is a Celery instance, which is called the celery application or just “app” for short. Since this instance is used as the entry-point for everything you want to do in Celery, like creating tasks and managing workers, it must be possible for other modules to import it.

In this tutorial you will keep everything contained in a single module, but for larger projects you want to create a dedicated module <project-layout>.

Let’s create the file tasks.py:

  1. from celery import Celery
  2. app = Celery('tasks', broker='amqp://guest@localhost//')
  3. @app.task
  4. def add(x, y):
  5. return x + y

The first argument to ~celery.app.Celery is the name of the current module, this is needed so that names can be automatically generated, the second argument is the broker keyword argument which specifies t