1. Access Log

1.1. http_log_module

1.1.1. log_format

  • Directive

    1. Syntax: log_format name [escape=default|json|none] string ... ;
    2. Default: log_format combined "..." ;
    3. Context: http

  • Instroduction

Specify log_format. Reference variables：Nginx Alphabetical index of variables .

1.1.2. access_log

  • Directive

    1. Syntax: access_log path [format [buffer=size] [gzip[=level]] [flush=time] [if=condition]];
    2.               access_log off;
    3. Default: access_log logs/access.log combined ;
    4. Context: http,server,location,if in location;limit_except

  • Instroduction

Sets the path, format, and configuration for a buffered log write. Several logs can be specified on the same level. Logging to syslog can be configured by specifying the “syslog:” prefix in the first parameter. The special value off cancels all access_log directives on the current level.
Paremeters:

  • path: specify access.log path.
  • format: reference name from log_format directive
  • buffer: sets buffer size befor write to disk.
  • gzip: compress log content in buffer,then write to disk.Default level is 1.
  • flush: sets time flush buffer to disk
  • if: if value is empty or equal to 0, nginx will ignore to record log.

    1.2. Access Log Example

    1.2.1. Nginx Configuration

    1. log_format access '$time_local|$http_x_real_ip|$remote_addr|$http_x_forwarded_for|$upstream_addr|'
    2.              '$request_method|$server_protocol|$host|$request_uri|$http_referer|$http_user_agent|'
    3.              '$proxy_host|$status' ;
    4. access_log  /opt/logs/nginx/access.log  access ;

    1.2.2. Test

    [root@centos-81 ~]# tail -f /opt/logs/nginx/access.log
    1. 16/Feb/2019:22:26:16 +0800|-|192.168.1.81|-|192.168.1.50:8081|GET|HTTP/1.1|www.heyang.info|/|-|curl/7.29.0|test_proxy|200
    2. 16/Feb/2019:22:26:16 +0800|-|192.168.1.81|-|192.168.1.50:8081|GET|HTTP/1.1|www.heyang.info|/|-|curl/7.29.0|test_proxy|200
    3. 16/Feb/2019:22:26:16 +0800|-|192.168.1.81|-|192.168.1.50:8080|GET|HTTP/1.1|www.heyang.info|/|-|curl/7.29.0|test_proxy|200

    1.3. Change Time Format

    Sometime we need change time format “16/Feb/2019:22:26:16 +0800” to “yyyy-mm-dd HH:MM:SS”.The second time format is better for log statistics and analysis.

    1.3.1. time_iso8601

    The variable $time_iso8601 is built-in variable.But the format is not well.

  • Nginx configuraion

    1. log_format access '$time_iso8601|$http_x_real_ip|$remote_addr|$http_x_forwarded_for|$upstream_addr|'
    2.                 '$request_method|$server_protocol|$host|$request_uri|$http_referer|$http_user_agent|'
    3.                 '$proxy_host|$status' ;
    4. access_log  /opt/logs/nginx/access.log  access ;

    [root@centos-81 ~]# tail -1 /opt/logs/nginx/access.log

  • Test

    1. 2019-07-07T11:44:38+08:00|-|192.168.1.1|-|ngx_backend|GET|HTTP/1.1|www.heyingsheng.com|/|-|Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.100 Safari/537.36|-|404

    1.3.2. Modify source code

  • Change ngx_http_log_module.c

[root@centos-81 nginx-1.14.2]# vim src/http/modules/ngx_http_log_module.c

  1. // old
  2. { ngx_string("time_local"), sizeof("28/Sep/1970:12:00:00 +0600") - 1,
  3.                       ngx_http_log_time },
  4. // new
  5. { ngx_string("time_local"), sizeof("1970-09-28 12:00:00 +0600") - 1,
  6.                       ngx_http_log_time },
  8. //old
  9. {
  10.     return ngx_cpymem(buf, ngx_cached_http_log_time.data,
  11.                       ngx_cached_http_log_time.len);
  12. }
  13. //new
  14. {
  15.     return ngx_cpymem(buf, ngx_cached_err_log_time.data,
  16.                       ngx_cached_err_log_time.len);
  17. }
  • Change ngx_times.c

[root@centos-81 nginx-1.14.2]# vim src/core/ngx_times.c

  1. //old
  2. (void) ngx_sprintf(p1, "%4d/%02d/%02d %02d:%02d:%02d",
  3.                        tm.ngx_tm_year, tm.ngx_tm_mon,
  4.                        tm.ngx_tm_mday, tm.ngx_tm_hour,
  5.                        tm.ngx_tm_min, tm.ngx_tm_sec);
  6. //new
  7. (void) ngx_sprintf(p1, "%4d-%02d-%02d %02d:%02d:%02d",
  8.                        tm.ngx_tm_year, tm.ngx_tm_mon,
  9.                        tm.ngx_tm_mday, tm.ngx_tm_hour,
  10.                        tm.ngx_tm_min, tm.ngx_tm_sec);

  • Nginx configuraion

    1. log_format access '$time_local|$http_x_real_ip|$remote_addr|$http_x_forwarded_for|$upstream_addr|'
    2.                 '$request_method|$server_protocol|$host|$request_uri|$http_referer|$http_user_agent|'
    3.                 '$proxy_host|$status' ;
    4. access_log  /opt/logs/nginx/access.log  access ;

  • Test

[root@centos-81 ~]# tail -3 /opt/logs/nginx/access.log

  1. 2019-07-07 12:23:33|-|192.168.1.1|-|58.213.154.120:80|GET|HTTP/1.1|www.heyang.com|/gtzz/images/flexslider.js|https://www.heyang.com/gtzz/gy/tyljls/|Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.100 Safari/537.36|jsmlr|200
  2. 2019-07-07 12:23:33|-|192.168.1.1|-|58.213.154.120:80|GET|HTTP/1.1|www.heyang.com|/gtzz/images/scroll.js|https://www.heyang.com/gtzz/gy/tyljls/|Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.100 Safari/537.36|jsmlr|200
  3. 2019-07-07 12:23:33|-|192.168.1.1|-|-|GET|HTTP/1.1|www.heyang.com|/favicon.ico|https://www.heyang.com/|Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.100 Safari/537.36|-|404

1.3.3. Define format by lua

Reference: https://blog.csdn.net/ystyaoshengting/article/details/84951087

2. Error Log

2.1. error_log

  • Directive

    1. Syntax: error_log file [level] ;
    2. Defautl: error_log logs/error.log error ;
    3. Context: main,http,mail,stream,server,location

  • Instroduction

Configures logging. Several logs can be specified on the same level (1.5.2). If on the main configuration level writing a log to a file is not explicitly defined, the default file will be used.
The first parameter defines a file that will store the log. The special value stderr selects the standard error file. Logging to syslog can be configured by specifying the “syslog:” refix. Logging to a cyclic memory buffer can be configured by specifying the “memory:” prefix and buffer size, and is generally used for debugging (1.7.11).
The second parameter determines the level of logging, and can be one of the following:debug, info, notice, warn, error, crit, alert, or emerg.
Log levels above are listed in the order of increasing severity. Setting a certain log level will cause all messages of the specified and more severe log levels to be logged. For example, the default level error will cause error, crit, alert, and emerg messages to be logged. If this parameter is omitted then error is used.

3. Log Rotate

  • Shell Script: ```bash

    !/usr/bin/env bash

    Author: Heyingsheng

    Date: 2019-01-04

    export PATH=”/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin” LOG_DIR=”/opt/logs/nginx/“ PID=”/opt/logs/nginx/nginx.pid”

cd $LOG_DIR [ -e $(date +%Y) ] || mkdir $(date +%Y) mv access.log $(date +%Y)/access-$(date +%F).log mv error.log $(date +%Y)/error-$(date +%F).log

kill -s USR1 $(cat $PID)

find ./ -maxdepth 2 -type f -name “error-.log” -mtime +30 -exec rm -f {} \; find ./ -maxdepth 2 -type f -name “access-.log” -mtime +60 -exec rm -f {} \;

[ -e $(date -d “-1 years” +%Y) ] && FILE_COUNT=$(find ./$(date -d “-1 years” +%Y) -type f | wc -l) && [ $FILE_COUNT -eq 0 ] && rm -fr ./$(date -d “-1 years” +%Y) ```