参考来源:
https://www.cnblogs.com/alex3714/p/6351797.html
http://www.cnblogs.com/dachenzi/p/8082730.html
https://www.cnblogs.com/alex3714/p/6351797.html
本文主要是做记录,便于自己往后用到的时候能找到相关的资料
第1步在:pycham 中 创建项目名称 proj(创建python 包)
第2步在:建立相关的python项目文件
编写:celery.py文件内容
"""
@Create_Time: 2018/1/9 17:33
@File: celery.py
@文件功能描述:
"""
from __future__ import absolute_import, unicode_literals
from celery import Celery
broker = 'redis://:123456@localhost:6379/0'
backend = 'redis://:123456@localhost:6379/1'
app = Celery('proj',
broker=broker,
backend=backend,
include=['proj.tasks'])
# Optional configuration, see the application user guide.
app.conf.update(
result_expires=3600,
)
if __name__ == '__main__':
app.start()
编写:tasks.py文件内容
#!/usr/bin/evn python
# coding=utf-8
from .celery import app # 导入应用
@app.task
def add(x, y):
return x + y
@app.task
def minus(x, y):
return x - y
查看相关的项目目录情况:
# yum -y install tree
[root@localhost proj]# tree
.
├── celery.py
├── __init__.py
└── tasks.py
0 directories, 3 files
第3步在:后台启动多进程celery work
上一篇笔记中使用的方式是前台启动的方式,若终端结束,则相关的进程也会结束,并且有相关的日志输出,实际工作中是一般不允许的!
PS: 后台启动worker
$ celery multi start w1 -A proj -l info
celery multi v4.0.0 (latentcall)
Starting nodes...
w1.halcyon.local: OK
补充命令信息:
启动:
$ celery multi start w1 -A proj -l info
重启:
$ celery multi restart w1 -A proj -l info
立即停止
$ celery multi stop w1 -A proj -l info
延迟停止,确保了所有正在执行的任务是完成之前退出:
$ celery multi stopwait w1 -A proj -l info
首先进入到当前的项目的同级目录下:
# 需在 proj的同一级目录(在同级目录下执行:就是在项目名称同一个目录中!!!)执行celery 后台启动 celery worker进程
celery multi start work_1 -A proj
或
celery multi start work_1 -A /data/app/proj
# work_1 为woker的名称,可以用来进行对该进程进行管理
后台运行成功
导入任务模块进行使用:
>>> import appcelery
>>> tasks.add.delay(3,676)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'tasks' is not defined
>>> from appcelery import tasks
>>> tasks.add.delay(3,676)
<AsyncResult: b9d200ed-8b87-49c3-8ff2-744b58a4cafa>
>>>
查看对应的redis数据信息:
image.png
执行3次:>>> tasks.add.delay(3,676)
image.png
运行错误:
image.png
image.png
在同级目录下执行:就是在项目名称同一个目录中!!!
image.png
原因:配置文件的redis的写错!
image.png
补充:
如果你手动的清除了相关的信息 需要重新再启动一下celery
就是需要重新再次执行以下:
# celery -A proj2 worker -l info
然后测试导入的话是需要使用:
[root@localhost app]# python
Python 3.5.2 (default, Jan 8 2018, 09:14:33)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-16)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import proj2.tasks
>>> proj2.tasks.add.delay(222,344)
<AsyncResult: 6ac6626e-7fa5-40b7-b37f-064b6d448cc1>
>>> proj2.tasks.add.delay(222,3443453)
<AsyncResult: 81ef3703-422a-490f-8122-0ab7d8a631ad>
>>>
# 为了导入方便,这里在appcelery同级目录下 执行python解释器
[root@namenode celerymodule]# ls
appcelery __pycache__ task.py
[root@namenode celerymodule]# python3
Python 3.6.4 (default, Dec 25 2017, 17:45:03)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-16)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from appcelery import tasks # 相对导入
>>> tasks.add.delay(1,2) # 执行任务
<AsyncResult: bc90bfa6-7972-4679-b897-4f5a98c70ae0>
>>>
补充:celery的其他参数或命令
[root@localhost proj]# celery
usage: celery <command> [options]
Show help screen and exit.
positional arguments:
args
optional arguments:
-h, --help show this help message and exit
--version show program's version number and exit
Global Options:
-A APP, --app APP
-b BROKER, --broker BROKER
--loader LOADER
--config CONFIG
--workdir WORKDIR
--no-color, -C
--quiet, -q
---- -- - - ---- Commands- -------------- --- ------------
+ Main:
| celery worker
| celery events
| celery beat
| celery shell
| celery multi
| celery amqp
+ Remote Control:
| celery status
| celery inspect --help
| celery inspect active
| celery inspect active_queues
| celery inspect clock
| celery inspect conf [include_defaults=False]
| celery inspect memdump [n_samples=10]
| celery inspect memsample
| celery inspect objgraph [object_type=Request] [num=200 [max_depth=10]]
| celery inspect ping
| celery inspect query_task [id1 [id2 [... [idN]]]]
| celery inspect registered [attr1 [attr2 [... [attrN]]]]
| celery inspect report
| celery inspect reserved
| celery inspect revoked
| celery inspect scheduled
| celery inspect stats
| celery control --help
| celery control add_consumer <queue> [exchange [type [routing_key]]]
| celery control autoscale [max [min]]
| celery control cancel_consumer <queue>
| celery control disable_events
| celery control election
| celery control enable_events
| celery control heartbeat
| celery control pool_grow [N=1]
| celery control pool_restart
| celery control pool_shrink [N=1]
| celery control rate_limit <task_name> <rate_limit (e.g., 5/s | 5/m | 5/h)>
| celery control revoke [id1 [id2 [... [idN]]]]
| celery control shutdown
| celery control terminate <signal> [id1 [id2 [... [idN]]]]
| celery control time_limit <task_name> <soft_secs> [hard_secs]
+ Utils:
| celery purge
| celery list
| celery call
| celery result
| celery migrate
| celery graph
| celery upgrade
+ Debugging:
| celery report
| celery logtool
---- -- - - --------- -- - -------------- --- ------------
Type 'celery <command> --help' for help using a specific command.
# 多进程相关
celery multi stop WOERNAME # 停止worker进程,有的时候这样无法停止进程,就需要加上-A 项目名,才可以删掉
celery multi restart WORKNAME # 重启worker进程
# 查看进程数
celery status -A celery # 查看该项目运行的进程数
一些错误的解决:
示例在windows下运行的时候:
执行:
D:\python_learn\celeryTest>celery -A tasks worker --loglever=info
因为proj才是模块的名称!
image.png
如果是:D:\python_learn\celeryTest>celery -A tasks worker --loglever=info
对应的模块应该是:
# coding:utf-8
from celery import Celery
app = Celery('tasks', broker='amqp://guest@localhost//')
@app.task
def add(x, y):
return x + y
windos下执行:
D:\python_learn\celeryTest>celery -A proj worker --loglever=info
usage: celery worker [options]
celery: error: unrecognized arguments: --loglever=info
错误原因估计最新版本不支持!是不支持windos!
卸载重新安装最新版本:
D:\python_learn\celeryTest>pip uninstall celery
Uninstalling celery-4.1.0:
重新安装最新版本:
pip install celery==3.1.18