supervisor部分

安装启动

  1. yum install supervisor.noarch
  2. // supervisor.noarch 0:3.4.0-1.el7 
  4. systemctl start supervisord.service
  5. systemctl enable supervisord.service

修改配置

  1. 配置文件
  2. /etc/supervisord.conf
  3. 修改
  4. [inet_http_server]         ; inet (TCP) server disabled by default
  5. port=127.0.0.1:9001        ; (ip_address:port specifier, *:port for all iface)
  6. username=user              ; (default is no username (open server))
  7. password=123
  9. childlogdir=/var/log/supervisor
  12. minfds=65533  ; 这是允许打开的最大文件数，默认1024，改大一点

同时，在 /etc/security/limits.conf 末尾添加：

  1. * soft nofile 65533
  2. * hard nofile 65533

这设计系统默认的

项目日志

路径：/var/log/supervisor

  1. douyin_00-stdout---supervisor-XDONdL.log
  2. douyin_02-stdout---supervisor-zON3KH.log
  3. douyin_01-stdout---supervisor-KEC9eu.log
  4. douyin_03-stdout---supervisor-4vH7Jc.log
  6. soubao_00-stdout---supervisor-cBim5q.log
  7. soubao_02-stdout---supervisor-fVAWIw.log
  8. soubao_01-stdout---supervisor-2GF_p0.log
  9. soubao_03-stdout---supervisor-uSGl2S.log

项目部署

中国搜索

  1. /root/projects/ChinasoService
  3. 搜索网页
  4. /root/.virtualenvs/ChinasoService-LkkRWD5_/bin/python -u   main_api.py
  5. 搜索视频
  6. /root/.virtualenvs/ChinasoService-LkkRWD5_/bin/python -u   main_api.py -m video
  8. selenium搜索网页
  9. main_selenium.py -m web
  11. selenium搜索视频
  12. main_selenium.py -m video

路径：/etc/supervisord.d/cnso.ini

  1. [program:cnso]
  2. directory=/root/projects/ChinasoService
  3. command=/root/.virtualenvs/ChinasoService-LkkRWD5_/bin/python -u  main_api.py  -p 98%(process_num)02d
  4. process_name=%(program_name)s_%(process_num)02d
  5. numprocs=6
  6. autorestart=true
  7. autostart=true
  8. stopasgroup=true
  9. killasgroup=true
  10. redirect_stderr=true
  11. user=root
  12. stdout_logfile_maxbytes=50MB
  13. stdout_logfile_backups=5
  14. stdout_capture_maxbytes=1MB
  15. stdout_events_enabled=false
  16. loglevel=warn

搜狗

  1. 搜狗视频
  2. /root/projects/SougouService
  3. /root/.virtualenvs/SougouService-R81tdI-0/bin/python -u  run_video.py
  5. 搜狗微信公众号搜索
  6. xvfb-run --auto-servernum --server-args="-screen 0 1280x760x24"   /root/.virtualenvs/SougouService-R81tdI-0/bin/python -u  run_weixin.py
  1. sougou.ini
  2. [program:sougou_video]
  3. directory=/root/projects/SougouService
  4. command=/root/.virtualenvs/SougouService-R81tdI-0/bin/python -u  run_video.py
  5. process_name=%(program_name)s_%(process_num)02d
  6. numprocs=2
  7. autorestart=true
  8. autostart=true
  9. stopasgroup=true
  10. killasgroup=true
  11. redirect_stderr=true
  12. user=root
  13. stdout_logfile_maxbytes=50MB
  14. stdout_logfile_backups=5
  15. stdout_capture_maxbytes=1MB
  16. stdout_events_enabled=false
  17. loglevel=warn
  20. sougou_weixin.ini
  21. [program:sougou_weixin]
  22. directory=/root/projects/SougouService
  23. command=xvfb-run --auto-servernum --server-args="-screen 0 1280x760x24"   /root/.virtualenvs/SougouService-R81tdI-0/bin/python -u  run_weixin.py
  24. process_name=%(program_name)s_%(process_num)02d
  25. numprocs=2
  26. autorestart=true
  27. autostart=true
  28. stopasgroup=true
  29. killasgroup=true
  30. redirect_stderr=true
  31. user=root
  32. stdout_logfile_maxbytes=50MB
  33. stdout_logfile_backups=5
  34. stdout_capture_maxbytes=1MB
  35. stdout_events_enabled=false
  36. loglevel=warn

搜报

  1. /root/.virtualenvs/SoubaoService-5L9_HNSo/bin/python   main_api.py
  2. /root/projects/SoubaoService

路径：/etc/supervisord.d/soubao.ini

  1. [program:soubao]
  2. directory=/root/projects/SoubaoService
  3. command=/root/.virtualenvs/SoubaoService-5L9_HNSo/bin/python -u  main_api.py  -p 98%(process_num)02d
  4. process_name=%(program_name)s_%(process_num)02d
  5. numprocs=4
  6. autorestart=true
  7. autostart=true
  8. stopasgroup=true
  9. killasgroup=true
  10. redirect_stderr=true
  11. user=root
  12. stdout_logfile_maxbytes=50MB
  13. stdout_logfile_backups=5
  14. stdout_capture_maxbytes=1MB
  15. stdout_events_enabled=false
  16. loglevel=warn

抖音

  1. /root/.virtualenvs/DouyinService-IWX9AG-R/bin/python  run.py
  2. /root/projects/DouyinService

路径：/etc/supervisord.d/douyin.ini

  1. [program:douyin]
  2. directory=/root/projects/DouyinService
  3. command=/root/.virtualenvs/DouyinService-IWX9AG-R/bin/python -u  run.py -p 98%(process_num)02d
  4. process_name=%(program_name)s_%(process_num)02d
  5. numprocs=4
  6. autorestart=true
  7. autostart=true
  8. stopasgroup=true
  9. killasgroup=true
  10. redirect_stderr=true
  11. user=root
  12. stdout_logfile_maxbytes=50MB
  13. stdout_logfile_backups=5
  14. stdout_capture_maxbytes=1MB
  15. stdout_events_enabled=false
  16. loglevel=warn

自动检测重启（已放弃，转为系统crontab部署）

  1. /root/projects/self_check
  2. /root/.virtualenvs/self_check-1yFL0V5E/bin/python
  4. python run_cron.py  # 定时任务 
  5. python run_srv.py  # 服务

路径：/etc/supervisord.d/self_check.ini

  1. [program:self_check]
  2. directory=/root/projects/self_check
  3. command=/root/.virtualenvs/self_check-1yFL0V5E/bin/python -u  run.py -p 98%(process_num)02d
  4. process_name=%(program_name)s_%(process_num)02d
  5. numprocs=1
  6. autorestart=true
  7. autostart=true
  8. stopasgroup=true
  9. killasgroup=true
  10. redirect_stderr=true
  11. user=root
  12. stdout_logfile_maxbytes=50MB
  13. stdout_logfile_backups=5
  14. stdout_capture_maxbytes=1MB
  15. stdout_events_enabled=false
  16. loglevel=warn

贴吧

  1. /root/.virtualenvs/TiebaService-a1o7RGKc/bin/python
  3. /root/projects/TiebaService
  5. python main_selenium_tie.py  # 帖子的搜索
  6. python main_selenium_ba.py  # 指定吧的搜索
  9. python kws_name.py  # 根据关键字重新获取吧名

定时任务

  1.  /root/.virtualenvs/TiebaService-a1o7RGKc/bin/python
  3. 4 0 * * * flock -xn /tmp/tieba_kw.lock -c 'cd /root/projects/TiebaService && /root/.virtualenvs/TiebaService-a1o7RGKc/bin/python -u kws_name.py >> "logs/kws_name-$(date +"\%Y-\%m-\%d").log" 2>&1 '

路径：/etc/supervisord.d/tie.ini

  1. [program:tie]
  2. directory=/root/projects/TiebaService
  3. command=/root/.virtualenvs/TiebaService-a1o7RGKc/bin/python -u main_selenium_tie.py -p 98%(process_num)02d
  4. process_name=%(program_name)s_%(process_num)02d
  5. numprocs=4
  6. autorestart=true
  7. autostart=true
  8. stopasgroup=true
  9. killasgroup=true
  10. redirect_stderr=true
  11. user=root
  12. stdout_logfile_maxbytes=50MB
  13. stdout_logfile_backups=5
  14. stdout_capture_maxbytes=1MB
  15. stdout_events_enabled=false
  16. loglevel=warn

路径：/etc/supervisord.d/ba.ini

  1. [program:ba]
  2. directory=/root/projects/TiebaService
  3. command=/root/.virtualenvs/TiebaService-a1o7RGKc/bin/python -u main_selenium_ba.py -p 98%(process_num)02d
  4. process_name=%(program_name)s_%(process_num)02d
  5. numprocs=4
  6. autorestart=true
  7. autostart=true
  8. stopasgroup=true
  9. killasgroup=true
  10. redirect_stderr=true
  11. user=root
  12. stdout_logfile_maxbytes=50MB
  13. stdout_logfile_backups=5
  14. stdout_capture_maxbytes=1MB
  15. stdout_events_enabled=false
  16. loglevel=warn

B站

  1. /root/.virtualenvs/BiliService-zAb0pCC0/bin/python
  3. /root/projects/BiliService

路径：/etc/supervisord.d/bi.ini

  1. [program:bi]
  2. directory=/root/projects/BiliService
  3. command=/root/.virtualenvs/BiliService-zAb0pCC0/bin/python -u main_selenium.py -p 98%(process_num)02d
  4. process_name=%(program_name)s_%(process_num)02d
  5. numprocs=4
  6. autorestart=true
  7. autostart=true
  8. stopasgroup=true
  9. killasgroup=true
  10. redirect_stderr=true
  11. user=root
  12. stdout_logfile_maxbytes=50MB
  13. stdout_logfile_backups=5
  14. stdout_capture_maxbytes=1MB
  15. stdout_events_enabled=false
  16. loglevel=warn