如果我们想让celery worker运行在后台而不是终端上,在后台以守护进程的方式运行,我们可以使用supervisor进行管理。
celery 的安装pip install celery
celery的安装很简单,直接通过pip就能够安装,并且本身没有任何的配置,如果需要命令行运行,直接以下命令:
celery -A proj(个人的项目,一般为main文件的相对路径) worker -l info
注意:如果你是win10系统,使用上面命令,可能报错:
Celery ValueError: not enough values to unpack (expected 3, got 0)
请查看这篇文章解决问题
使用通用的 init-script
首先获取 Celery 的 GitHub 仓库extra/generic-init.d/
文件夹下的文件。
启动 worker
将上面 GitHub 仓库文件夹下的celeryd
文件复制到本地的/etc/init.d/celeryd
文件中并赋予可执行的权限:sudo chmod +x /etc/init.d/celeryd
,然后在/etc/default/
文件夹下创建celeryd
配置文件:
#Names of nodes to start #most people will only start one node: CELERYD_NODES="worker1" #but you can also start multiple and configure settings #for each in CELERYD_OPTS #CELERYD_NODES="worker1 worker2 worker3" #alternatively, you can specify the number of nodes to start: #CELERYD_NODES=10 #Absolute or relative path to the 'celery' command: CELERY_BIN="/usr/local/bin/celery" #CELERY_BIN="/virtualenvs/def/bin/celery" #App instance to use #comment out this line if you don't use an app CELERY_APP="proj" #or fully qualified: #CELERY_APP="proj.tasks:app" #Where to chdir at start. CELERYD_CHDIR="/opt/Myproject/" #Extra command-line arguments to the worker CELERYD_OPTS="--time-limit=300 --concurrency=8" #Configure node-specific settings by appending node name to arguments: #CELERYD_OPTS="--time-limit=300 -c 8 -c:worker2 4 -c:worker3 2 -Ofair:worker1" #Set logging level to DEBUG #CELERYD_LOG_LEVEL="DEBUG" #%n will be replaced with the first part of the nodename. CELERYD_LOG_FILE="/var/log/celery/%n%I.log" CELERYD_PID_FILE="/var/run/celery/%n.pid" #Workers should run as an unprivileged user. #You need to create this user manually (or you can choose #a user/group combination that already exists (e.g., nobody). CELERYD_USER="celery" CELERYD_GROUP="celery" #If enabled pid and log directories will be created if missing, #and owned by the userid/group configured. CELERY_CREATE_DIRS=1
这个配置文件中,你可能需要更改CELERY_BIN
,CELERY_APP
,CELERYD_CHDIR
,CELERYD_USER
,CELERYD_GROUP
。
根据你自身项目配置好上述配置后就可以通过/etc/init.d/celeryd {start|stop|restart|graceful|kill|dryrun|create-paths}
这些命令来控制 Celery worker 进程。
启动 beat
如果你的项目中还有周期任务,那么还得把 Celery beat 启动起来。
把 GitHub 仓库中的celerybeat
文件复制到/etc/init.d/celerybeat
文件中,然后赋予可执行权限:sudo chmod +x /etc/init.d/celerybeat
。
然后同样得有一个/etc/default/celerybeat
配置文件。不过,如果没有什么特殊配置,可以直接使用/etc/default/celeryd
配置文件,不必创建/etc/default/celerybeat
配置文件。
如果你依然要创建的,那么这个文件可能是长这样的:
#Absolute or relative path to the 'celery' command: CELERY_BIN="/usr/local/bin/celery" #CELERY_BIN="/virtualenvs/def/bin/celery" #App instance to use #comment out this line if you don't use an app CELERY_APP="proj" #or fully qualified: #CELERY_APP="proj.tasks:app" #Where to chdir at start. CELERYBEAT_CHDIR="/opt/Myproject/" #Extra arguments to celerybeat CELERYBEAT_OPTS="--schedule=/var/run/celery/celerybeat-schedule"
完成配置后,同样通过/etc/init.d/celerybeat {start|stop|restart|create-paths|status}
这些命令控制 Celery beat 进程。