docker容器C和其他主机B有网络连接,但是在容器宿主机A用netstat看不到。

    现象:
    B的服务日志可以看到来自A的请求
    B上执行netstat看到有来自A的连接
    A上执行netstat看不到来自B的连接
    B是k8s节点
    在上执行tcpdump dst A,可以看到发给A的网络包

    说明网络连接是存在的,但netstat无法看到,是B上某个容器产生的
    执行如下命令找出相关容器:
    for cid in docker ps -q;do pid=docker inspect -f "{{.State.Pid}}" $cid;echo $cid,$pid;nsenter -t $pid -n netstat -anp|grep ${hosta-ip};done

    docker inspect可以看到容器对应的进程号
    如果容器没有相关命令,可以使用nsenter进入该进程执行netstat,从而查看到对应的网络连接和相关进程
    nsenter -t $pid -n netstat

    1. NSENTER(1) User Commands NSENTER(1)
    2. NAME
    3. nsenter - run program with namespaces of other processes
    4. SYNOPSIS
    5. nsenter [options] [program [arguments]]
    6. DESCRIPTION
    7. Enters the namespaces of one or more other processes and then executes the specified program. Enterable namespaces are:
    8. mount namespace
    9. Mounting and unmounting filesystems will not affect the rest of the system (CLONE_NEWNS flag), except for filesystems which are explicitly
    10. marked as shared (with mount --make-shared; see /proc/self/mountinfo for the shared flag).
    11. UTS namespace
    12. Setting hostname or domainname will not affect the rest of the system. (CLONE_NEWUTS flag)
    13. IPC namespace
    14. The process will have an independent namespace for System V message queues, semaphore sets and shared memory segments. (CLONE_NEWIPC flag)
    15. network namespace
    16. The process will have independent IPv4 and IPv6 stacks, IP routing tables, firewall rules, the /proc/net and /sys/class/net directory trees,
    17. sockets, etc. (CLONE_NEWNET flag)
    18. PID namespace
    19. Children will have a set of PID to process mappings separate from the nsenter process (CLONE_NEWPID flag). nsenter will fork by default if
    20. changing the PID namespace, so that the new program and its children share the same PID namespace and are visible to each other. If --no-fork
    21. is used, the new program will be exec'ed without forking.
    22. user namespace
    23. The process will have a distinct set of UIDs, GIDs and capabilities. (CLONE_NEWUSER flag)
    24. See clone(2) for the exact semantics of the flags.
    25. If program is not given, then ``${SHELL}'' is run (default: /bin/sh).
    26. OPTIONS
    27. -t, --target pid
    28. Specify a target process to get contexts from. The paths to the contexts specified by pid are:
    29. /proc/pid/ns/mnt the mount namespace
    30. /proc/pid/ns/uts the UTS namespace
    31. /proc/pid/ns/ipc the IPC namespace
    32. /proc/pid/ns/net the network namespace
    33. /proc/pid/ns/pid the PID namespace
    34. /proc/pid/ns/user the user namespace
    35. /proc/pid/root the root directory
    36. /proc/pid/cwd the working directory respectively
    37. -m, --mount[=file]
    38. Enter the mount namespace. If no file is specified, enter the mount namespace of the target process. If file is specified, enter the mount
    39. namespace specified by file.
    40. -u, --uts[=file]
    41. Enter the UTS namespace. If no file is specified, enter the UTS namespace of the target process. If file is specified, enter the UTS names‐
    42. pace specified by file.
    43. -i, --ipc[=file]
    44. Enter the IPC namespace. If no file is specified, enter the IPC namespace of the target process. If file is specified, enter the IPC names‐
    45. pace specified by file.
    46. -n, --net[=file]
    47. Enter the network namespace. If no file is specified, enter the network namespace of the target process. If file is specified, enter the
    48. network namespace specified by file.
    49. -p, --pid[=file]
    50. Enter the PID namespace. If no file is specified, enter the PID namespace of the target process. If file is specified, enter the PID names‐
    51. pace specified by file.
    52. -U, --user[=file]
    53. Enter the user namespace. If no file is specified, enter the user namespace of the target process. If file is specified, enter the user
    54. namespace specified by file. See also the --setuid and --setgid options.
    55. -G, --setgid gid
    56. Set the group ID which will be used in the entered namespace and drop supplementary groups. nsenter(1) always sets GID for user namespaces,
    57. the default is 0.
    58. -S, --setuid uid
    59. Set the user ID which will be used in the entered namespace. nsenter(1) always sets UID for user namespaces, the default is 0.
    60. --preserve-credentials
    61. Don't modify UID and GID when enter user namespace. The default is to drops supplementary groups and sets GID and UID to 0.
    62. -r, --root[=directory]
    63. Set the root directory. If no directory is specified, set the root directory to the root directory of the target process. If directory is
    64. specified, set the root directory to the specified directory.
    65. -w, --wd[=directory]
    66. Set the working directory. If no directory is specified, set the working directory to the working directory of the target process. If direc
    67. tory is specified, set the working directory to the specified directory.
    68. -F, --no-fork
    69. Do not fork before exec'ing the specified program. By default, when entering a PID namespace, nsenter calls fork before calling exec so that
    70. any children will also be in the newly entered PID namespace.
    71. -Z, --follow-context
    72. Set the SELinux security context used for executing a new process according to already running process specified by --target PID. (The util-
    73. linux has to be compiled with SELinux support otherwise the option is unavailable.)
    74. -V, --version
    75. Display version information and exit.
    76. -h, --help
    77. Display help text and exit.
    78. SEE ALSO
    79. setns(2), clone(2)