CentOS 7系统下的项目中如何使用celery

参考来源:
https://www.cnblogs.com/alex3714/p/6351797.html
http://www.cnblogs.com/dachenzi/p/8082730.html
https://www.cnblogs.com/alex3714/p/6351797.html
本文主要是做记录,便于自己往后用到的时候能找到相关的资料

第1步在:pycham 中 创建项目名称 proj(创建python 包)

第2步在:建立相关的python项目文件

编写:celery.py文件内容
"""
@Create_Time: 2018/1/9 17:33
@File: celery.py
@文件功能描述:
"""
from __future__ import absolute_import, unicode_literals
from celery import Celery

broker = 'redis://:123456@localhost:6379/0'

backend = 'redis://:123456@localhost:6379/1'

app = Celery('proj',
             broker=broker,
             backend=backend,
             include=['proj.tasks'])

# Optional configuration, see the application user guide.
app.conf.update(
    result_expires=3600,
)

if __name__ == '__main__':
    app.start()
编写:tasks.py文件内容
#!/usr/bin/evn python
# coding=utf-8

from .celery import app  # 导入应用


@app.task
def add(x, y):
    return x + y


@app.task
def minus(x, y):
    return x - y

查看相关的项目目录情况:

# yum -y install tree
[root@localhost proj]# tree 
.
├── celery.py
├── __init__.py
└── tasks.py

0 directories, 3 files

第3步在:后台启动多进程celery work

上一篇笔记中使用的方式是前台启动的方式,若终端结束,则相关的进程也会结束,并且有相关的日志输出,实际工作中是一般不允许的!
PS: 后台启动worker
$ celery multi start w1 -A proj -l info
celery multi v4.0.0 (latentcall)

Starting nodes...
w1.halcyon.local: OK

补充命令信息:
启动:
$ celery multi start w1 -A proj -l info
重启:
$ celery  multi restart w1 -A proj -l info
立即停止
$ celery multi stop w1 -A proj -l info
延迟停止,确保了所有正在执行的任务是完成之前退出:
$ celery multi stopwait w1 -A proj -l info

首先进入到当前的项目的同级目录下:

# 需在 proj的同一级目录(在同级目录下执行:就是在项目名称同一个目录中!!!)执行celery 后台启动 celery worker进程
 
celery multi start work_1 -A proj
或
celery multi start work_1 -A /data/app/proj
 
# work_1 为woker的名称,可以用来进行对该进程进行管理
后台运行成功

导入任务模块进行使用:

>>> import appcelery
>>> tasks.add.delay(3,676)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'tasks' is not defined
>>> from appcelery import tasks
>>> tasks.add.delay(3,676)     
<AsyncResult: b9d200ed-8b87-49c3-8ff2-744b58a4cafa>
>>> 

查看对应的redis数据信息:

image.png

执行3次:>>> tasks.add.delay(3,676)
image.png

运行错误:


image.png
image.png

在同级目录下执行:就是在项目名称同一个目录中!!!

image.png

原因:配置文件的redis的写错!

image.png

补充:
如果你手动的清除了相关的信息 需要重新再启动一下celery
就是需要重新再次执行以下:

# celery -A proj2 worker -l info

然后测试导入的话是需要使用:

[root@localhost app]# python
Python 3.5.2 (default, Jan  8 2018, 09:14:33) 
[GCC 4.8.5 20150623 (Red Hat 4.8.5-16)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import proj2.tasks
>>> proj2.tasks.add.delay(222,344)
<AsyncResult: 6ac6626e-7fa5-40b7-b37f-064b6d448cc1>
>>> proj2.tasks.add.delay(222,3443453)
<AsyncResult: 81ef3703-422a-490f-8122-0ab7d8a631ad>
>>> 

# 为了导入方便,这里在appcelery同级目录下 执行python解释器
[root@namenode celerymodule]# ls
appcelery  __pycache__  task.py 
 
[root@namenode celerymodule]# python3
Python 3.6.4 (default, Dec 25 2017, 17:45:03)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-16)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from appcelery import tasks        # 相对导入
>>> tasks.add.delay(1,2)        # 执行任务
<AsyncResult: bc90bfa6-7972-4679-b897-4f5a98c70ae0>
>>>   

补充:celery的其他参数或命令

[root@localhost proj]# celery
usage: celery <command> [options] 

Show help screen and exit.

positional arguments:
  args

optional arguments:
  -h, --help            show this help message and exit
  --version             show program's version number and exit

Global Options:
  -A APP, --app APP
  -b BROKER, --broker BROKER
  --loader LOADER
  --config CONFIG
  --workdir WORKDIR
  --no-color, -C
  --quiet, -q

---- -- - - ---- Commands- -------------- --- ------------

+ Main: 
|    celery worker
|    celery events
|    celery beat
|    celery shell
|    celery multi
|    celery amqp

+ Remote Control: 
|    celery status
 
|    celery inspect --help
|    celery inspect active 
|    celery inspect active_queues 
|    celery inspect clock 
|    celery inspect conf [include_defaults=False]
|    celery inspect memdump [n_samples=10]
|    celery inspect memsample 
|    celery inspect objgraph [object_type=Request] [num=200 [max_depth=10]]
|    celery inspect ping 
|    celery inspect query_task [id1 [id2 [... [idN]]]]
|    celery inspect registered [attr1 [attr2 [... [attrN]]]]
|    celery inspect report 
|    celery inspect reserved 
|    celery inspect revoked 
|    celery inspect scheduled 
|    celery inspect stats 
 
|    celery control --help
|    celery control add_consumer <queue> [exchange [type [routing_key]]]
|    celery control autoscale [max [min]]
|    celery control cancel_consumer <queue>
|    celery control disable_events 
|    celery control election 
|    celery control enable_events 
|    celery control heartbeat 
|    celery control pool_grow [N=1]
|    celery control pool_restart 
|    celery control pool_shrink [N=1]
|    celery control rate_limit <task_name> <rate_limit (e.g., 5/s | 5/m | 5/h)>
|    celery control revoke [id1 [id2 [... [idN]]]]
|    celery control shutdown 
|    celery control terminate <signal> [id1 [id2 [... [idN]]]]
|    celery control time_limit <task_name> <soft_secs> [hard_secs]

+ Utils: 
|    celery purge
|    celery list
|    celery call
|    celery result
|    celery migrate
|    celery graph
|    celery upgrade

+ Debugging: 
|    celery report
|    celery logtool
---- -- - - --------- -- - -------------- --- ------------

Type 'celery <command> --help' for help using a specific command.
# 多进程相关
celery multi stop WOERNAME      # 停止worker进程,有的时候这样无法停止进程,就需要加上-A 项目名,才可以删掉
celery multi restart WORKNAME        # 重启worker进程
 
# 查看进程数
celery status -A celery       # 查看该项目运行的进程数

一些错误的解决:
示例在windows下运行的时候:
执行:
D:\python_learn\celeryTest>celery -A tasks worker --loglever=info

因为proj才是模块的名称!

image.png
image.png

如果是:D:\python_learn\celeryTest>celery -A tasks worker --loglever=info
对应的模块应该是:

# coding:utf-8

from celery import Celery

app = Celery('tasks', broker='amqp://guest@localhost//')

@app.task
def add(x, y):
    return x + y

windos下执行:

D:\python_learn\celeryTest>celery -A proj worker --loglever=info
usage: celery worker [options]
celery: error: unrecognized arguments: --loglever=info

错误原因估计最新版本不支持!是不支持windos!

卸载重新安装最新版本:

D:\python_learn\celeryTest>pip uninstall celery
Uninstalling celery-4.1.0:
image.png

重新安装最新版本:

pip install celery==3.1.18
最后编辑于
©著作权归作者所有,转载或内容合作请联系作者
平台声明:文章内容(如有图片或视频亦包括在内)由作者上传并发布,文章内容仅代表作者本人观点,简书系信息发布平台,仅提供信息存储服务。