Checking kubernetes pod CPU and memory

https://kubernetes.io/docs/tasks/debug-application-cluster/resource-usage-monitoring/

pod

kubectl top

https://gitee.com/ploynomail/python/blob/master/kubernetes/README.md#documentation-for-api-endpoints
gitee 国内镜像 api 接口

github 国外 api 接口
https://github.com/kubernetes-client/python/blob/master/kubernetes/README.md

kubectl top pod -n <namespace> --no-headers

kubectl exec <podname> -n <namespace> -- cat /sys/fs/cgroup/memory/memory.usage_in_bytes

注意以下Deployment配置清单和kubectl top指令查看pod资源使用率中，都有cpu和内存的两个数量单位（m和Mi），这里把这两个单位解释做个记录：

  • cpu单位m：代表 “千分之一核心”，譬如50m的含义是指50/1000核心，即5%
  • 内存单位Mi：1Mi = 1024乘1024，而平时使用的单为M是1M = 1000乘1000
  2. [root@k8s-master ~]# echo $((`kubectl exec le-bc5200ee-fcc1-4168-b447-fc28b5aaf9bb-bvrpt -n zj31 -- cat  /sys/fs/cgroup/memory/memory.usage_in_bytes`/1000/1000))
  3. 633
  4. [root@k8s-master ~]# kubectl top pod le-bc5200ee-fcc1-4168-b447-fc28b5aaf9bb-bvrpt  -n zj31
  5. NAME                                            CPU(cores)   MEMORY(bytes)   
  6. le-bc5200ee-fcc1-4168-b447-fc28b5aaf9bb-bvrpt   5m           600Mi    
  7. [root@k8s-master ~]# kubectl top pod -n zj31 --no-headers
  8. le-bc5200ee-fcc1-4168-b447-fc28b5aaf9bb-bvrpt   6m    601Mi 
  10. [root@k8s-master ~]# kubectl get all -n zj31
  11. NAME                                                READY   STATUS    RESTARTS   AGE
  12. pod/le-bc5200ee-fcc1-4168-b447-fc28b5aaf9bb-bvrpt   1/1     Running   0          2d2h
  14. NAME                                              TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)   AGE
  15. service/le-bc5200ee-fcc1-4168-b447-fc28b5aaf9bb   ClusterIP   10.1.62.2    <none>        80/TCP    2d2h
  17. NAME                                                COMPLETIONS   DURATION   AGE
  18. job.batch/le-bc5200ee-fcc1-4168-b447-fc28b5aaf9bb   0/1           2d2h       2d2h

api

https://stackoverflow.com/questions/58403810/how-to-get-pod-cpu-and-memory-usage-from-metrics-server

  2. from kubernetes import client
  4. api_client = client.ApiClient()
  5. ret_metrics = api_client.call_api(
  6.             '/apis/metrics.k8s.io/v1beta1/namespaces/' + 'default' + '/pods', 'GET',
  7.             auth_settings=['BearerToken'], response_type='json', _preload_content=False)
  8. response = ret_metrics[0].data.decode('utf-8')
  9. print('RESP', json.loads(response))

https://stackoverflow.com/questions/57238390/getting-metrics-via-metrics-server-kubernetes

I tried this - using metrics-server
kubectl get --raw /apis/metrics.k8s.io/v1beta1/namespace/<namespace>/pods/<podname>

The above command returns the cpu and memory usage of a specified pod at that time.
Sample output :slight_smile:

  1. {“kind”:“PodMetrics”,“apiVersion”:“metrics.k8s.io/v1beta1”,
  3. “metadata”:{“name”:“podname”,“namespace”:“default”,“selfLink”:"/apis/metrics.k8s.io/v1beta1/namespaces/default/pods/podname",
  5. “creationTimestamp”:“2019-06-26T03:41:25Z”},“timestamp”:“2019-06-26T03:41:00Z”,“window”:“1m0s”,
  7. “containers”:[{“name”:“cassandra”,“usage”:{“cpu”:“4m”,“memory”:“816388Ki”}}]}

I tried the same with curl command also.
curl http://127.0.0.1:8001/apis/metrics.k8s.io/v1beta1/namespace/<namespace-name>/pods/<pod-name> -k
This also returns the same..

  1. [root@k8s-master ~]# kubectl top pod le-bc5200ee-fcc1-4168-b447-fc28b5aaf9bb-bvrpt -n zj31
  2. NAME                                            CPU(cores)   MEMORY(bytes)   
  3. le-bc5200ee-fcc1-4168-b447-fc28b5aaf9bb-bvrpt   5m           602Mi
  1. >>> from kubernetes import client
  2. >>> api_client = client.ApiClient()
  3. >>> 
  4. >>> namesapce = 'zj31'
  5. >>> url = '/apis/metrics.k8s.io/v1beta1/namespaces/' + namesapce + '/pods'
  6. >>> ret_metrics = api_client.call_api(url, 'GET', auth_settings=['BearerToken'], response_type='json', _preload_content=False)
  7. >>> response = ret_metrics[0].data.decode('utf-8')
  8. >>> res = json.loads(response)
  9. >>> for i in res['items']:
  10. ...     for j in i['containers']:
  11. ...         print('## memory: ',j['usage']['memory'])
  12. ... 
  13. ## memory:  617224Ki
  16. >>> namesapce = 'zj31'
  17. >>> podname = 'le-bc5200ee-fcc1-4168-b447-fc28b5aaf9bb-bvrpt'
  18. >>> url_pod = '/apis/metrics.k8s.io/v1beta1/namespaces/' + namesapce + '/pods/' + podname
  19. >>> ret_metric = api_client.call_api(url_pod, 'GET', auth_settings=['BearerToken'], response_type='json', _preload_content=False)
  20. >>> response = ret_metric[0].data.decode('utf-8')
  21. >>> res = json.loads(response)
  22. >>> for j in res['containers']:
  23. ...     print('## memory: ',j['usage']['memory'])
  24. ... 
  25. ## memory:  617224Ki

node

[https://dzone.com/articles/kubernetes-resource-usage-how-do-you-manage-and-mo]

To see the how much of its quota each node is using we can use this command, with example output for a 3 node cluster:

kubectl top node
kubectl get nodes --no-headers | awk '{print $1}' | xargs -I {} sh -c 'echo {}; kubectl describe node {} | grep Allocated -A 5 | grep -ve Event -ve Allocated -ve percent -ve -- ; echo'

  1. kubectl get nodes --no-headers | awk '{print $1}' | xargs -I {} sh -c 'echo {}; kubectl describe node {} | grep Allocated -A 5 | grep -ve Event -ve Allocated -ve percent -ve -- ; echo'
  3. gke-rel3170-default-pool-3459fe6a-n03g
  4.   CPU Requests CPU Limits Memory Requests Memory Limits
  5.   358m (38%) 138m (14%) 516896Ki (19%) 609056Ki (22%)
  7. gke-rel3170-default-pool-3459fe6a-t3b3
  8.   CPU Requests CPU Limits Memory Requests Memory Limits
  9.   460m (48%) 0 (0%) 310Mi (11%) 470Mi (17%)
  11. gke-rel3170-default-pool-3459fe6a-vczz
  12.   CPU Requests CPU Limits Memory Requests Memory Limits
  13.   570m (60%) 110m (11%) 430Mi (16%) 790Mi (29%)
  16. [root@k8s-master ~]# kubectl top node
  17. NAME         CPU(cores)   CPU%   MEMORY(bytes)   MEMORY%   
  18. k8s-master   406m         0%     5678Mi          2%        
  19. k8s-node1    2643m        4%     41689Mi         10%       
  20. k8s-node2    261m         0%     26782Mi         5%