05-缓存服务器

1. 缓存类型
2. http_proxy_module
3. ngx_cache_purge
4. Example
- 4.1. Basic Example

1. 缓存类型

1.1. 服务端缓存

缓存服务常用 Memcached,Redis 等Key-Value方式
未命名图片.png

1.2. 代理缓存

由Nginx服务器本身做缓存，针对未过期的数据，直接从缓存中读取，减少与后端服务器的交互。类似于CDN加速静态资源访问速度。
未命名图片.png

1.3. 客户端缓存

在Web架构中，主要由浏览器提供缓存机制。服务器应用会缓存数据库数据……
未命名图片.png

2. http_proxy_module

2.1. proxy_cache

Directives

Syntax: proxy_cache zone|off ;
Default: proxy_cache off ;
Context: http,server,location

Instroduction

Defines a shared memory zone for caching.The zone can be used in several places.Parameter value can contain vairables since 1.7.9.
The value off means that the level disables caching inherited from the previous configuration level.

2.2. proxy_cache_path

Directives

Syntax: proxy_cache_path path [levels=levels] [use_temp_path=on|off] keys_zone=name:size [inactive=time] [max_size=size] [manager_files=number] [manager_sleep=time] [manager_threshold=time] [loader_files=number] [loader_sleep=time] [loader_threshold=time] ;
Default: Close
Context: http

Instroduction

Cache data are stored in files,the file name in a cache is a result of applying the MD5 function to the cache key.

levels

Defines hierarchy levels of cache(from 1 to 3),each level accepts value 1 or 2. If level sets levels=1:2,then file name looks like: $cache_path/c/29/b7f54b2df7773722d382f4809d65029c.

use_temp_path

A cached response is first written to a temporary file,then the file is renamed.If sets to the value on or omitted,the directory is seted by the proxy_temp_path.If the value is set to off,temporary files will be put directly in the cache directory.

keys_zone

A actived keys and information of response data will stored in share memory.The parameter defines zone name and size,1m share memory can store 8000 keys.

inactive

Cached data that are not accessed during the time specified by the inactive get removed for the cache.Default value is 10min.

max_size

If cache size large than max size value,cache manager process will removes the least recently used data.

manager_files

Defines maximum files can be remove in one iteration (1.11.5).

manager_threshold

Defines maximum time of one interation,default 200 milliseconds (1.11.5).

manager_sleep

Defines sleep time of tow interations,default 50 milliseconds (1.11.5).

loader_files

After nginx start,the cache loader process will loads information about previously cached data stored from disk done in interations.load_files parameter specified maximum files information in one interation,default 100.

loader_sleep

Defines sleep time between tow interations,default 50 milliseconds.

loader_threshold

Defines maximum time of one interation,default 200 milliseconds.

2.3. proxy_cache_key

Directives

Syntax: proxy_cache_key string ;
Default: proxy_cache_key $scheme$proxy_host$request_uri ;
Context: http,server,location

Instroduction

Defines a key for response cache.

2.4. proxy_cache_bypass

Directives

Syntax: proxy_cache_bypass string ... ;
Default: Close
Context: http,server,location

Instroduction

Defines conditions under which the response will not be taken from a cache.If at least one value of the string parameters is not empty or not equal to 0 ,then the response will not taken from cache.The directive can be used along with proxy_no_cache. The directive effiect is same as _proxy_no_cache. _For example :

proxy_cache_bypass $cookie_nocache $arg_nocache $arg_comment $http_pragma $http_authorization ;
# $cookie_nocache: cookie name of nocache
# $arg_nocache: argument name of nocache
# $arg_comment: argument name of comment
# $http_pragma: http request filed named pragma
# $http_authorization: http request filed named authorization

2.5. proxy_no_cache

Directives

Syntax: proxy_no_cache string ... ;
Default: Close
Context: http,server,location

Instroduction

Defines conditions under which the response will not be taken from a cache.If at least one value of the string parameters is not empty or not equal to 0 ,then the response will not taken from cache.The directive can be used along with proxy_cache_bypass. The directive effiect is same as _proxy_cache_bypass. _For example :

proxy_no_cache $cookie_nocache $arg_nocache $arg_comment $http_pragma $http_authorization ;

2.6. proxy_cache_min_uses

Directives

Syntax: proxy_cache_min_uses number ;
Default: proxy_cache_min_uses 1 ;
Context: http,server,location

Instroduction

Defines the number of requests that after client requst the response will be cached .It very useful to reduce cached files size and increase hite rate.

2.7. proxy_cache_valid

Directives

Syntax: proxy_cache_valid [code|any] time ;
Default: Close
Context: http,server,location

Instroduction

Sets caching time for different response codes.For example:

proxy_cache_valid 200 302 301 12h ;
proxy_cache_valid 404 1m ;
proxy_cache_valid any 2m ;

any: specified any response
: not specify any or response code means only 200,301,302 response are cached.
2.8. proxy_cache_use_stale

Directives

Syntax: proxy_cache_use_stale error | timeout | invalid_header | updating | http_500 | http_502 | http_503 | http_504 | http_403 | http_404 | http_429 | off ...;
Default: proxy_cache_use_stale off ;
Context: http,server,location

Instroduction

Determines in which cases a stale cached response can be used during communication with the proxied server.If proxied server rarely update resource and proxied server is return error,the directive is very usefull.

error

nginx can’t select proxied server to send request.an error occurred while establishing a connection with the server, passing a request to it, or reading the response header

timeout

a timeout has occurred while establishing a connection with the server, passing a request to it, or reading the response header

invalid_header

a server returned an empty or invalid response.

updating

the cache is updating by other request.

http_(500|503|504|403|404|429)

resopnse code.

2.9. proxy_ignore_headers

Directives

Syntax: proxy_ignore_headers field ... ;
Default: Close
Context: http,server,location

Instroduction

Parameter of caching can alse be set directly in response header.This has higher priority than proxy_cache :

X-Accel-Expires

sets caching time of a response in seconds.
The zero value disables caching for a response. If the value starts with the @ prefix, it sets an absolute time in seconds since Epoch, up to which the response may be cached.If the header does not include the “X-Accel-Expires” field, parameters of caching may be set in the header fields “Expires” or “Cache-Control”.

Set-Cookie

If the header includes the “Set-Cookie” field, such a response will not be cached.

Vary

If the header includes the “Vary” field with the special value “*”, such a response will not be cached (1.7.7). If the header includes the “Vary” field with another value, such a response will be cached taking into account the corresponding request header fields (1.7.7).
The drective can ignore fields of response,such as “Set-Cookie”.

3. ngx_cache_purge

3.1. Install

[root@centos-81 ~]# cd /opt/apps/nginx-1.14.2 && make clean
[root@centos-81 nginx-1.14.2]# mkdir modules
[root@centos-81 nginx-1.14.2]# wget -O modules/ngx_cache_purge.zip https://github.com/FRiCKLE/ngx_cache_purge/archive/master.zip
[root@centos-81 nginx-1.14.2]# unzip modules/ngx_cache_purge.zip -d modules/

[root@centos-81 nginx-1.14.2]# /opt/apps/nginx/sbin/nginx -V

......
configure arguments: --prefix=/opt/apps/nginx --user=nginx --group=nginx --with-compat --with-file-aio --with-threads --with-http_addition_module --with-http_auth_request_module --with-http_dav_module --with-http_flv_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_mp4_module --with-http_random_index_module --with-http_realip_module --with-http_secure_link_module --with-http_slice_module --with-http_ssl_module --with-http_stub_status_module --with-http_sub_module --with-http_v2_module --with-mail --with-mail_ssl_module --with-stream --with-stream_realip_module --with-stream_ssl_module --with-stream_ssl_preread_module --with-cc-opt='-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -fPIC' --with-ld-opt='-Wl,-z,relro -Wl,-z,now -pie'

[root@centos-81 nginx-1.14.2]# ./configure —prefix=/opt/apps/nginx —user=nginx —group=nginx —with-compat —with-file-aio —with-threads —with-http_addition_module —with-http_auth_request_module —with-http_dav_module —with-http_flv_module —with-http_gunzip_module —with-http_gzip_static_module —with-http_mp4_module —with-http_random_index_module —with-http_realip_module —with-http_secure_link_module —with-http_slice_module —with-http_ssl_module —with-http_stub_status_module —with-http_sub_module —with-http_v2_module —with-mail —with-mail_ssl_module —with-stream—with-stream_realip_module —with-stream_ssl_module —with-stream_ssl_preread_module —with-cc-opt=’-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong —param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -fPIC’ —with-ld-opt=’-Wl,-z,relro -Wl,-z,now -pie’ —add-module=modules/ngx_cache_purge-master # add module to nginx.
[root@centos-81 nginx-1.14.2]# make -j 4

[root@centos-81 nginx-1.14.2]# mv /opt/apps/nginx/sbin/nginx /opt/apps/nginx/sbin/nginx.old
[root@centos-81 nginx-1.14.2]# mv objs/nginx /opt/apps/nginx/sbin/

3.2. proxy_cache_purge

Directives

Syntax: proxy_cache_purge zone string;
Default: Close
Context: location

Instroduction

Delete caching from disk and memory by manual.The zone sets by proxy_cache_path ; string sets by proxy_cache_key .

3.3. fastcgi_cache_purge

Directives

Syntax: fastcgi_cache_purge zone string;
Default: Close
Context: location

Instroduction

Delete fastcgi cache from disk and memory by manual.

3.4. scgi_cache_purge

Directives

Syntax: scgi_cache_purge zone string;
Default: Close
Context: location

Instroduction

Delete scgi cache from disk and memory by manual.

3.5. uwsgi_cache_purge

Directives

Syntax: uwsgi_cache_purge zone string;
Default: Close
Context: location

Instroduction

Delete uwsgi cache from disk and memory by manual.

4. Example

4.1. Basic Example

4.1.1. Nginx configuration

[root@centos-81 conf.d]# cat /opt/apps/nginx/conf/nginx.conf

user  nginx;
worker_processes  2;
error_log  /opt/logs/nginx/error.log  notice;
pid        /opt/logs/nginx/nginx.pid;
events {
    worker_connections  1024;
}
http {
    include       mime.types ; 
    include       conf.d/heyang.conf ;
    default_type  application/octet-stream ;
    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';
    access_log  /opt/logs/nginx/access.log  main;
    sendfile        on;
    keepalive_timeout  65;
    ## Gzip;disabled on ie 6
    gzip  on;
    gzip_min_length 1024 ;
    gzip_types text/css text/plain application/javascript application/xml image/gif image/jpeg image/png;
    gzip_disable msie6 ; 
    gunzip on ;
    ## Client Cache.
    map $sent_http_content_type $expires {
        default off;
        ~image/ 2d ;
        ~vedio/ 7d ;
        text/plain 24h;
        text/css   24h;
        text/html  24h;
        application/javascript 24h;
        application/pdf 30d;
        application/zip 30d;
    }
    expires $expires ;
    ## proxy server settings
    proxy_max_temp_file_size 4096m ;
    proxy_connect_timeout 20s ;
    proxy_read_timeout 20s ;
    proxy_send_timeout 10s ;
    proxy_temp_path /opt/cache/jsmlr_temp 1 2 ;
    ## cache settings 
    proxy_cache jsmlr_cache ;
    proxy_cache_path /opt/cache/jsmlr_cache levels=1:2 use_temp_path=on keys_zone=jsmlr_cache:10m inactive=20m max_size=1024m ;
    proxy_cache_key $scheme$host$request_uri ;
    proxy_cache_bypass $cookie_trswcmv6username $http_authorization ;
    proxy_ignore_headers Set-Cookie ; ## warning!Usually can't ignore Set-Cookie field.
    proxy_cache_valid 200 302 301 12h ;
    proxy_cache_valid 404 403 1m ;
    proxy_cache_use_stale error timeout updating http_500 http_503 http_504 ;
}

[root@centos-81 conf.d]# cat /opt/apps/nginx/conf/conf.d/heyang.conf

server {
    listen       80;
    server_name  *.heyang.com;
    root   /opt/website ;
    location / {
        index  index.html index.htm;
    }
    location ~ ^/gt(wzqxxgk|app|zz)/ {
        if ($request_uri ~ .*\.js$) {
            expires off ;
        }
        proxy_set_header Host $proxy_host ;
        proxy_set_header X-Real-IP $remote_addr ;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for ;
        proxy_pass http://www.jsmlr.gov.cn ;
    }
    location ~ ^/test/ {
        rewrite ^/test/(.*)$  /$1 permanent ;
    }
    location /yzgy/ {
        if ($request_uri ~ .*\.js$) {
            expires off ;
        }
        proxy_set_header Host $proxy_host ;
        proxy_set_header X-Real-IP $remote_addr ;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for ;
        proxy_pass http://www.jsmlr.gov.cn ;
    }
    ## Clean cache by URL.
    location ~ /purge(/.*) {
        allow 127.0.0.1 ;
        allow 192.168.1.0/24 ;
        deny all ;
        proxy_cache_purge jsmlr_cache $scheme$host$1$is_args$args ;
    }
    error_page   500 502 503 504  /50x.html;
}

4.1.2. Browser Test

4.1.3. Cache Purge Test

[root@centos-81 cache]# find ./ -type f | wc -l ## no cache
0
[root@centos-81 ~]# curl -v “http://192.168.1.81/gtapp/nrglIndex.action?type=2&messageID=2c9082b567f78e560168265127f0026d“ >/dev/null

......
< Server: nginx/1.14.2
< Date: Sun, 27 Jan 2019 03:14:02 GMT
< Content-Type: text/html;charset=utf-8
< Content-Length: 6020
< Connection: keep-alive
< Content-Language: zh-CN
< Access-Control-Allow-Origin: http://www.jsmlr.gov.cn:80/gtxx/
< Set-Cookie: JSESSIONID=C687FF44B0AFC0B30FBE50B4C7573758; Path=/gtapp

[root@centos-81 cache]# find ./ -type f | wc -l
1

[root@centos-81 ~]# curl “http://192.168.1.81/purge/gtapp/nrglIndex.action?type=2&messageID=2c9082b567f78e560168265127f0026d“ ## Clean the cache

<html>
<head><title>Successful purge</title></head>
<body bgcolor="white">
<center><h1>Successful purge</h1>
<br>Key : http192.168.1.81/gtapp/nrglIndex.action?type=2&messageID=2c9082b567f78e560168265127f0026d
<br>Path: /opt/cache/jsmlr_cache/9/d7/dea75c8055254935ddb6c85ffb71ed79
</center>
<hr><center>nginx/1.14.2</center>
</body>
</html>

[root@centos-81 cache]# find ./ -type f | wc -l
0

1. 缓存类型

1.1. 服务端缓存

1.2. 代理缓存

1.3. 客户端缓存

2. http_proxy_module

2.1. proxy_cache

2.2. proxy_cache_path

2.3. proxy_cache_key

2.4. proxy_cache_bypass

2.5. proxy_no_cache

2.6. proxy_cache_min_uses

2.7. proxy_cache_valid

2.8. proxy_cache_use_stale

2.9. proxy_ignore_headers

3. ngx_cache_purge

3.1. Install

3.2. proxy_cache_purge

3.3. fastcgi_cache_purge

3.4. scgi_cache_purge

3.5. uwsgi_cache_purge

4. Example

4.1. Basic Example

4.1.1. Nginx configuration

4.1.2. Browser Test

4.1.3. Cache Purge Test