查看进程

ps aux | grep python 或者 ps -ef | grep python
可以通过查看 /proc/PID/ 目录的文件信息来得到这个进程的一些信息(Linux中一切皆文件,进程信息也在文件中)
cd /proc/12293(12293是进程的id号)
sudo ls -ahl
该进程的详细信息显示如下:
image.png

杀死进程

kill PID或者kill -9 PID(强制杀死进程)
kill -l查看kill命令的所有信号。
image.png
kill -9 PIDkill -SIGKILL PID命令等价,强制杀死一个进程。
信号SIGHUP通常程序用这个信号进行优雅重载配置文件,重新启动并且不影响正在运行的服务。比如
kill -1 uwsgi 优雅重启uwsgi 进程,对服务器没有影响
kill -1 NGINX_PID 优雅重启nginx进程,对服务器没有影响
CTRL-C发送 SIGINT 信号给前台进程组中的所有进程,常用于终止正在运行的程序。
CTRL-Z发送 SIGTSTP 信号给前台进程组中的所有进程,常用于挂起一个进程。

查看进程当前状态

sudo strace -p PID

端口分析

sudo netstat -nltp可以查看服务器中监控了哪些端口,netstat选项如下。-a或—all 显示所有连接中的Socket,默认不显示 LISTEN 相关的。
-c或—continuous 持续列出网络状态,不断自动刷新输出。
-l或—listening 显示监听中的服务器的Socket。
-n或—numeric 直接使用IP地址,而不是展示域名。
-p或—programs 显示正在使用Socket的程序进程PID和名称。
-t或—tcp 显示TCP传输协议的连接。
-u或—udp 显示UDP传输协议的连接。

进程管理软件supervisor

安装

1 pip安装:pip install supervisor
2 源码安装:下载源文件压缩包.tar.gz,解压缩,进入setup.py目录,运行命令python setup.py install

配置

cd /home/pi/supervisor/
echo_supervisord_conf > supervisord.conf
以上命令将默认配置重定向到/home/pi/supervisor/supervisord.conf

配置文件说明

  1. ; Sample supervisor config file.
  2. ;
  3. ; For more information on the config file, please see:
  4. ; http://supervisord.org/configuration.html
  5. ;
  6. ; Notes:
  7. ; - Shell expansion ("~" or "$HOME") is not supported. Environment
  8. ; variables can be expanded using this syntax: "%(ENV_HOME)s".
  9. ; - Quotes around values are not supported, except in the case of
  10. ; the environment= options as shown below.
  11. ; - Comments must have a leading space: "a=b ;comment" not "a=b;comment".
  12. ; - Command will be truncated if it looks like a config file comment, e.g.
  13. ; "command=bash -c 'foo ; bar'" will truncate to "command=bash -c 'foo ".
  14. [unix_http_server]
  15. file=/var/run/supervisor.sock ; the path to the socket file
  16. ;chmod=0700 ; socket file mode (default 0700)
  17. ;chown=nobody:nogroup ; socket file uid:gid owner
  18. ;username=user ; default is no username (open server)
  19. ;password=123 ; default is no password (open server)
  20. [inet_http_server] ; inet (TCP) server disabled by default(网页端访问配置)
  21. port=127.0.0.1:9001 ; ip_address:port specifier, *:port for all iface
  22. username=user ; default is no username (open server)
  23. password=123 ; default is no password (open server)
  24. [supervisord]
  25. logfile=/var/run/supervisord.log ; main log file; default $CWD/supervisord.log
  26. logfile_maxbytes=50MB ; max main logfile bytes b4 rotation; default 50MB
  27. logfile_backups=10 ; # of main logfile backups; 0 means none, default 10
  28. loglevel=info ; log level; default info; others: debug,warn,trace
  29. pidfile=/var/run/supervisord.pid ; supervisord pidfile; default supervisord.pid
  30. nodaemon=false ; start in foreground if true; default false
  31. minfds=1024 ; min. avail startup file descriptors; default 1024
  32. minprocs=200 ; min. avail process descriptors;default 200
  33. ;umask=022 ; process file creation umask; default 022
  34. ;user=supervisord ; setuid to this UNIX account at startup; recommended if root
  35. ;identifier=supervisor ; supervisord identifier, default is 'supervisor'
  36. ;directory=/tmp ; default is not to cd during start
  37. ;nocleanup=true ; don't clean up tempfiles at start; default false
  38. ;childlogdir=/tmp ; 'AUTO' child log dir, default $TEMP
  39. ;environment=KEY="value" ; key value pairs to add to environment
  40. ;strip_ansi=false ; strip ansi escape codes in logs; def. false
  41. ; The rpcinterface:supervisor section must remain in the config file for
  42. ; RPC (supervisorctl/web interface) to work. Additional interfaces may be
  43. ; added by defining them in separate [rpcinterface:x] sections.
  44. [rpcinterface:supervisor]
  45. supervisor.rpcinterface_factory = supervisor.rpcinterface:make_main_rpcinterface
  46. ; The supervisorctl section configures how supervisorctl will connect to
  47. ; supervisord. configure it match the settings in either the unix_http_server
  48. ; or inet_http_server section.
  49. [supervisorctl]
  50. serverurl=unix:///var/run/supervisor.sock ; use a unix:// URL for a unix socket
  51. serverurl=http://127.0.0.1:9001 ; use an http:// url to specify an inet socket
  52. username=user ; should be same as in [*_http_server] if set
  53. password=123 ; should be same as in [*_http_server] if set
  54. ;prompt=mysupervisor ; cmd line prompt (default "supervisor")
  55. ;history_file=~/.sc_history ; use readline history if available
  56. ; The sample program section below shows all possible program subsection values.
  57. ; Create one or more 'real' program: sections to be able to control them under
  58. ; supervisor.
  59. ;[program:theprogramname]
  60. ;command=/bin/cat ; the program (relative uses PATH, can take args)
  61. ;process_name=%(program_name)s ; process_name expr (default %(program_name)s)
  62. ;numprocs=1 ; number of processes copies to start (def 1)
  63. ;directory=/tmp ; directory to cwd to before exec (def no cwd)
  64. ;umask=022 ; umask for process (default None)
  65. ;priority=999 ; the relative start priority (default 999)
  66. ;autostart=true ; start at supervisord start (default: true)
  67. ;startsecs=1 ; # of secs prog must stay up to be running (def. 1)
  68. ;startretries=3 ; max # of serial start failures when starting (default 3)
  69. ;autorestart=unexpected ; when to restart if exited after running (def: unexpected)
  70. ;exitcodes=0 ; 'expected' exit codes used with autorestart (default 0)
  71. ;stopsignal=QUIT ; signal used to kill process (default TERM)
  72. ;stopwaitsecs=10 ; max num secs to wait b4 SIGKILL (default 10)
  73. ;stopasgroup=false ; send stop signal to the UNIX process group (default false)
  74. ;killasgroup=false ; SIGKILL the UNIX process group (def false)
  75. ;user=chrism ; setuid to this UNIX account to run the program
  76. ;redirect_stderr=true ; redirect proc stderr to stdout (default false)
  77. ;stdout_logfile=/a/path ; stdout log path, NONE for none; default AUTO
  78. ;stdout_logfile_maxbytes=1MB ; max # logfile bytes b4 rotation (default 50MB)
  79. ;stdout_logfile_backups=10 ; # of stdout logfile backups (0 means none, default 10)
  80. ;stdout_capture_maxbytes=1MB ; number of bytes in 'capturemode' (default 0)
  81. ;stdout_events_enabled=false ; emit events on stdout writes (default false)
  82. ;stdout_syslog=false ; send stdout to syslog with process name (default false)
  83. ;stderr_logfile=/a/path ; stderr log path, NONE for none; default AUTO
  84. ;stderr_logfile_maxbytes=1MB ; max # logfile bytes b4 rotation (default 50MB)
  85. ;stderr_logfile_backups=10 ; # of stderr logfile backups (0 means none, default 10)
  86. ;stderr_capture_maxbytes=1MB ; number of bytes in 'capturemode' (default 0)
  87. ;stderr_events_enabled=false ; emit events on stderr writes (default false)
  88. ;stderr_syslog=false ; send stderr to syslog with process name (default false)
  89. ;environment=A="1",B="2" ; process environment additions (def no adds)
  90. ;serverurl=AUTO ; override serverurl computation (childutils)
  91. ; The sample eventlistener section below shows all possible eventlistener
  92. ; subsection values. Create one or more 'real' eventlistener: sections to be
  93. ; able to handle event notifications sent by supervisord.
  94. ;[eventlistener:theeventlistenername]
  95. ;command=/bin/eventlistener ; the program (relative uses PATH, can take args)
  96. ;process_name=%(program_name)s ; process_name expr (default %(program_name)s)
  97. ;numprocs=1 ; number of processes copies to start (def 1)
  98. ;events=EVENT ; event notif. types to subscribe to (req'd)
  99. ;buffer_size=10 ; event buffer queue size (default 10)
  100. ;directory=/tmp ; directory to cwd to before exec (def no cwd)
  101. ;umask=022 ; umask for process (default None)
  102. ;priority=-1 ; the relative start priority (default -1)
  103. ;autostart=true ; start at supervisord start (default: true)
  104. ;startsecs=1 ; # of secs prog must stay up to be running (def. 1)
  105. ;startretries=3 ; max # of serial start failures when starting (default 3)
  106. ;autorestart=unexpected ; autorestart if exited after running (def: unexpected)
  107. ;exitcodes=0 ; 'expected' exit codes used with autorestart (default 0)
  108. ;stopsignal=QUIT ; signal used to kill process (default TERM)
  109. ;stopwaitsecs=10 ; max num secs to wait b4 SIGKILL (default 10)
  110. ;stopasgroup=false ; send stop signal to the UNIX process group (default false)
  111. ;killasgroup=false ; SIGKILL the UNIX process group (def false)
  112. ;user=chrism ; setuid to this UNIX account to run the program
  113. ;redirect_stderr=false ; redirect_stderr=true is not allowed for eventlisteners
  114. ;stdout_logfile=/a/path ; stdout log path, NONE for none; default AUTO
  115. ;stdout_logfile_maxbytes=1MB ; max # logfile bytes b4 rotation (default 50MB)
  116. ;stdout_logfile_backups=10 ; # of stdout logfile backups (0 means none, default 10)
  117. ;stdout_events_enabled=false ; emit events on stdout writes (default false)
  118. ;stdout_syslog=false ; send stdout to syslog with process name (default false)
  119. ;stderr_logfile=/a/path ; stderr log path, NONE for none; default AUTO
  120. ;stderr_logfile_maxbytes=1MB ; max # logfile bytes b4 rotation (default 50MB)
  121. ;stderr_logfile_backups=10 ; # of stderr logfile backups (0 means none, default 10)
  122. ;stderr_events_enabled=false ; emit events on stderr writes (default false)
  123. ;stderr_syslog=false ; send stderr to syslog with process name (default false)
  124. ;environment=A="1",B="2" ; process environment additions
  125. ;serverurl=AUTO ; override serverurl computation (childutils)
  126. ; The sample group section below shows all possible group values. Create one
  127. ; or more 'real' group: sections to create "heterogeneous" process groups.
  128. ;[group:thegroupname]
  129. ;programs=progname1,progname2 ; each refers to 'x' in [program:x] definitions
  130. ;priority=999 ; the relative start priority (default 999)
  131. ; The [include] section can just contain the "files" setting. This
  132. ; setting can list multiple files (separated by whitespace or
  133. ; newlines). It can also contain wildcards. The filenames are
  134. ; interpreted as relative to this file. Included files *cannot*
  135. ; include files themselves.
  136. [include]
  137. files = /home/pi/supervisor/config.d/*.ini

[program:theprogramname]进程配置可以直接在supervisord.conf文件中书写,但是不建议,应该把程序配置单独写到.ini文件中,然后放到/home/pi/supervisor/config.d/目录中,目录文件关系如下所示。
image.png
各程序配置文件如下:
uwsgi网络服务器:

[program:uwsgi]
directory=/home/pi/projects/django/monitor/  ;服务文件所在的目录,如果是.py文件,引入自定义的.py模块,指定其文件目录就非常重要
command=/usr/local/bin/uwsgi --emperor /etc/uwsgi/vassals ;启动程序的命令,也就是在控制台输入的命令
autostart=true               ;在supervisord启动的时候也自动启动
startsecs=5                  ;启动 5 秒后没有异常退出,就当作已经正常启动了
autorestart=true             ;程序异常退出后自动重启
startretries=3               ;启动失败自动重试次数
redirect_stderr=true         ;把stderr重定向到stdout,默认false
stdout_logfile_maxbytes=1MB  ;stdout日志文件大小
stdout_logfile_backups=10    ;stdout日志文件备份数
stdout_logfile=/home/pi/projects/django/monitor/supervisor_uwsgi.log  ;stdout日志文件,需要注意当指定目录不存在时无法正常启动,所以需要手动创建目录(supervisord 会自动创建日志文件)
loglevel=info                ;日志级别

python脚本:serverRun.py和savedb.py

[program:serverRun]
directory=/home/pi/projects/raspberrypi/ ;如果在severRun.py代码中引入了模块Operations.py,这就需要指定py文件的目录/home/pi/projects/raspberrypi/ 
command=python /home/pi/projects/raspberrypi/serverRun.py
autostart=true
startsecs=5
autorestart=true
startretries=3
redirect_stderr=true
stdout_logfile_maxbytes=1MB
stdout_logfile_backups=10
stdout_logfile=/home/pi/projects/raspberrypi/supervisor_serverRun.log
loglevel=info
[program:savedb]
directory=/home/pi/projects/raspberrypi/
command=python /home/pi/projects/raspberrypi/savedb.py
autostart=true
startsecs=5
autorestart=true
startretries=3
redirect_stderr=true
stdout_logfile_maxbytes=1MB
stdout_logfile_backups=10
stdout_logfile=/home/pi/projects/raspberrypi/supervisor_savedb.log
loglevel=info

启动

进入supervisord.conf配置文件目录,输入如下命令启动:supervisord -c supervisord.conf

使用supervisorctl监控

运行命令:supervisorctl 或者 supervisorctl -c supervisord.conf
进入supervisorctl的shell界面,在shell界面可以使用如下命令:

help                 #查看所有命令
help reread          #查看reread命令帮助文档
status               # 查看程序状态
stop procName        #关闭procName程序
start procName       #启动procName程序
restart procName     #重启procName程序
reread               #读取有更新(增加)的配置文件,不会启动新添加的程序
update               #重启配置文件修改过的程序

也可以直接在linux终端使用supervisorctl命令监控进程:

supervisorctl status
supervisorctl stop procName
supervisorctl start procName
supervisorctl restart procName
supervisorctl reread
supervisorctl update

在web端监控

supervisord.conf配置文件有如下内容,配置了网页端访问的端口和账户密码。

[inet_http_server]         ; inet (TCP) server disabled by default(网页端访问配置)
port=127.0.0.1:9001        ; ip_address:port specifier, *:port for all iface
username=user              ; default is no username (open server)
password=123               ; default is no password (open server)

打开浏览器,输入127.0.0.1:9001,打开监控界面如下图所示:

image.png

管理进程组

待开发

开机自启动

配置systemctl服务

创建supervisor.service文件:sudo vim /lib/systemd/system/supervisor.service

[Unit]
Description=supervisor
After=network.target

[Service]
Type=forking
ExecStart=/usr/local/bin/supervisord -c /home/pi/supervisor/supervisord.conf
ExecStop=/usr/local/bin/supervisorctl shutdown
ExecReload=/usr/local/bin/supervisorctl reload
KillMode=process
Restart=on-failure
RestartSec=42s

[Install]
WantedBy=multi-user.target

设置开机启动:
systemctl enable supervisor.service
systemctl daemon-reload
修改文件权限:
chmod 766 supervisor.service

配置service类型服务

sudo vim /etc/rc.d/init.d/supervisor

#!/bin/bash
#
# supervisord  This scripts turns supervisord on
#
# Author:    Mike McGrath <mmcgrath@redhat.com> (based off yumupdatesd)
#
# chkconfig:  - 95 04
#
# description: supervisor is a process control utility. It has a web based
#        xmlrpc interface as well as a few other nifty features.
# processname: supervisord
# config: /home/pi/supervisor/supervisord.conf
# pidfile: /var/run/supervisord.pid
#

# source function library
. /etc/rc.d/init.d/functions

RETVAL=0

start() {
  echo -n $"Starting supervisord: "
  daemon "supervisord -c /home/pi/supervisor/supervisord.conf "
  RETVAL=$?
  echo
  [ $RETVAL -eq 0 ] && touch /var/lock/subsys/supervisord
}

stop() {
  echo -n $"Stopping supervisord: "
  killproc supervisord
  echo
  [ $RETVAL -eq 0 ] && rm -f /var/lock/subsys/supervisord
}

restart() {
  stop
  start
}

case "$1" in
 start)
  start
  ;;
 stop) 
  stop
  ;;
 restart|force-reload|reload)
  restart
  ;;
 condrestart)
  [ -f /var/lock/subsys/supervisord ] && restart
  ;;
 status)
  status supervisord
  RETVAL=$?
  ;;
 *)
  echo $"Usage: $0 {start|stop|status|restart|reload|force-reload|condrestart}"
  exit 1
esac

exit $RETVAL

修改文件权限,设置开机自启动
chmod 755 /etc/rc.d/init.d/supervisor
chkconfig supervisor on
其它Linux发行版开机启动脚本:https://github.com/Supervisor/initscripts

rc.local脚本

Linux 在启动的时候会执行 /etc/rc.local 里面的脚本,所以只要在rc.local文件添加执行命令就可以。

sudo /usr/local/bin/supervisord -c /home/pi/supervisor/supervisord.conf

以上内容需要添加在 exit 0命令前。而且由于在执行 rc.local 脚本时,PATH 环境变量未全部初始化,因此命令需要使用绝对路径。

注意事项

1、supervisor的日志文件.log只能是英文编码,不能打印中文。所以程序里有打印中文日志的代码运行会出错。
2、supervisord.pid 以及 supervisor.sock 是放在 /tmp 目录下,但是 /tmp 目录是存放临时文件,里面的文件是会被 Linux 系统删除的,所以建议把.pid或者.sock文件放在/var/run/目录下。
3、supervisord管理的进程必须由supervisord来启动,supervisord还要求管理的程序是非daemon程序,supervisord会帮你把它转成daemon程序,因此如果用supervisord来管理nginx的话,必须在nginx的配置文件里添加一行设置daemon off让nginx以非daemon方式启动。

参考文档

http://supervisord.org/index.html
https://www.jianshu.com/p/bf2b3f4dec73
https://www.cnblogs.com/xingzc/p/6612270.html
https://serverfault.com/questions/96499/how-to-automatically-start-supervisord-on-linux-ubuntu
https://www.jb51.net/article/142427.htm

其他进程管理软件

Monitlaunchd, daemontools, and runit.