1. secnetperf使用

1.1 help text

  1. 【help】
  2. ./secnetperf -?
  4. secnetperf usage:
  6. Server: secnetperf [options]
  8.   -bind:<addr>                A local IP address to bind to.
  10. Client: secnetperf -TestName:<Throughput|RPS|HPS> [options]
  13. 【tput】
  14. ./secnetperf -test:tput
  15. Must specify '-target' argument!
  17. Throughput Client options:
  19.   -target:<####>               The target server to connect to.
  20.   -bind:<addr>                 A local IP address to bind to.
  21.   -port:<####>                 The UDP port of the server. (def:4433)
  22.   -ip:<0/4/6>                  A hint for the resolving the hostname to an IP address. (def:0)
  23.   -encrypt:<0/1>               Enables/disables encryption. (def:1)
  24.   -sendbuf:<0/1>               Whether to use send buffering. (def:0)
  25.   -pacing:<0/1>                Whether to use pacing. (def:1)
  26.   -timed:<0/1>                 Indicates the upload/download arg time (ms). (def:0)
  27.   -upload:<####>               The length of data (or time with -timed:1 arg) to send. (def:0)
  28.   -download:<####>             The length of data (or time with -timed:1 arg) to request/receive. (def:0)
  29.   -iosize:<####>               The size of each send request queued. (def:65536)
  30.   -tcp:<0/1>                   Indicates TCP/TLS should be used instead of QUIC. (def:0)
  31.   -stats:<0/1>                 Indicates connection stats should be printed at the end of the run. (def:0)
  34. 【rps】
  35. ./secnetperf -test:rps
  36. Must specify '-target' argument!
  38. RPS Client options:
  40.   -target:<####>              The target server to connect to.
  41.   -runtime:<####>             The total runtime (in ms). (def:10000)
  42.   -port:<####>                The UDP port of the server. (def:4433)
  43.   -ip:<0/4/6>                  A hint for the resolving the hostname to an IP address. (def:0)
  44.   -conns:<####>               The number of connections to use. (def:1000)
  45.   -requests:<####>            The number of requests to send at a time. (def:2*conns)
  46.   -request:<####>             The length of request payloads. (def:0)
  47.   -response:<####>            The length of request payloads. (def:0)
  48.   -threads:<####>             The number of threads to use. Defaults and capped to number of cores
  49.   -affinitize:<0/1>           Affinitizes threads to a core. (def:0)

1.2 server

  1. # 启动服务器
  2. ./secnetperf

1.3 client

  1. # 发送1G字节流报文，每次1300字节
  2. ./secnetperf -test:tput -target:10.0.32.139 -upload:1000000000 -iosize:1300 -stats:1
  3. Started!
  4. Result: 999848200 bytes @ 883695 kbps (9051.517 ms).
  5. App Main returning status 0

2. secnetperf调用栈

2.1 server

总计开启线程 cpu*6 + 1

2.1.1 data path

  1. 开启cpu个数线程
  2. CxPlatDataPathInitialize
  4. #0  QuicMainStart (argc=0, argc@entry=1, argv=0x308afc8, argv@entry=0x308afc0, StopEvent=StopEvent@entry=0x7fffffffde70, SelfSignedCredConfig=SelfSignedCredConfig@entry=0x308afe0) at /root/source/msquic/src/perf/lib/SecNetPerfMain.cpp:131
  5. #1  0x0000000000404649 in QuicUserMain (argc=argc@entry=1, argv=argv@entry=0x308afc0, KeyboardWait=KeyboardWait@entry=false, SelfSignedCredConfig=SelfSignedCredConfig@entry=0x308afe0, FileName=FileName@entry=0x0) at /root/source/msquic/src/perf/bin/appmain.cpp:116
  6. #2  0x0000000000404ace in main (argc=1, argv=0x7fffffffe038) at /root/source/msquic/src/perf/bin/appmain.cpp:448

2.1.2 open

  1. 开启cpu个数线程
  2. MsQuic = new(std::nothrow) MsQuicApi
  4. #0  CxPlatThreadCreate -> CxPlatDataPathWorkerThread (epoll_wait)
  5. #1  CxPlatProcessorContextInitialize (Datapath=Datapath@entry=0x605560, Index=Index@entry=0, ProcContext=ProcContext@entry=0x605600) at /root/source/msquic/src/platform/datapath_epoll.c:608
  6. #2  0x00007ffff7961b57 in CxPlatDataPathInitialize (ClientRecvContextLength=ClientRecvContextLength@entry=48, UdpCallbacks=UdpCallbacks@entry=0x7fffffffddd0, TcpCallbacks=TcpCallbacks@entry=0x0, NewDataPath=NewDataPath@entry=0x7ffff7dc97e0 <MsQuicLib+320>) at /root/source/msquic/src/platform/datapath_epoll.c:701
  7. #3  0x00007ffff7925c9a in MsQuicLibraryInitialize () at /root/source/msquic/src/core/library.c:393
  8. #4  0x00007ffff7926271 in MsQuicAddRef () at /root/source/msquic/src/core/library.c:598
  9. #5  0x00007ffff79262ca in MsQuicOpen (QuicApi=QuicApi@entry=0x603080 <MsQuic>) at /root/source/msquic/src/core/library.c:1325
  10. #6  0x00007ffff79264b9 in MsQuicOpenVersion (Version=Version@entry=1, QuicApi=QuicApi@entry=0x603080 <MsQuic>) at /root/source/msquic/src/core/library.c:1405

2.1.3 new server

  1. 开启cpu*3个数线程 = data path(cpu个数) + quic worker(cpu个数) + tcp worker(cpu个数)
  2. TestToRun = new(std::nothrow) PerfServer(SelfSignedCredConfig)
  4. TcpEngine::TcpEngine
  5. 1. CxPlatDataPathInitialize
  6. 2. MsQuicRegistrationOpen (构造函数中调用)
  7. 3. TcpWorker::Initialize
  10. #0  0x00007ffff66bedb5 in clone () from /lib64/libc.so.6
  11. #1  0x00007ffff698e00f in create_thread () from /lib64/libpthread.so.0
  12. #2  0x00007ffff698f9c6 in pthread_create@@GLIBC_2.2.5 () from /lib64/libpthread.so.0
  13. #3  0x00007ffff795cd9c in CxPlatThreadCreate (Config=0x7fffffffdc30, Thread=0x30f3190) at /root/source/msquic/src/platform/platform_posix.c:583
  14. #4  0x00007ffff792d9e5 in QuicWorkerInitialize (Owner=Owner@entry=0x30f28f0, ThreadFlags=ThreadFlags@entry=0, IdealProcessor=IdealProcessor@entry=1, Worker=Worker@entry=0x30f3100) at /root/source/msquic/src/core/worker.c:83
  15. #5  0x00007ffff792f041 in QuicWorkerPoolInitialize (Owner=Owner@entry=0x30f28f0, ThreadFlags=0, WorkerCount=4, NewWorkerPool=0x30f2928) at /root/source/msquic/src/core/worker.c:783
  16. #6  0x00007ffff79255c6 in MsQuicRegistrationOpen (Config=0x7fffffffdda0, NewRegistration=0x308c848) at /root/source/msquic/src/core/registration.c:120
  17. #7  0x000000000040686c in MsQuicRegistration::MsQuicRegistration (AutoCleanUp=true, Profile=QUIC_EXECUTION_PROFILE_LOW_LATENCY, AppName=0x5b74e8 "secnetperf-server", this=0x308c840) at /root/source/msquic/src/inc/msquic.hpp:316
  18. #8  PerfServer::PerfServer (CredConfig=0x308afe0, this=0x308c830) at /root/source/msquic/src/perf/lib/PerfServer.h:23
  19. #9  QuicMainStart (argc=0, argc@entry=1, argv=0x308afc8, argv@entry=0x308afc0, StopEvent=StopEvent@entry=0x7fffffffde70, SelfSignedCredConfig=SelfSignedCredConfig@entry=0x308afe0) at /root/source/msquic/src/perf/lib/SecNetPerfMain.cpp:176
  20. #10 0x0000000000404649 in QuicUserMain (argc=argc@entry=1, argv=argv@entry=0x308afc0, KeyboardWait=KeyboardWait@entry=false, SelfSignedCredConfig=SelfSignedCredConfig@entry=0x308afe0, FileName=FileName@entry=0x0) at /root/source/msquic/src/perf/bin/appmain.cpp:116
  21. #11 0x0000000000404ace in main (argc=1, argv=0x7fffffffe038) at /root/source/msquic/src/perf/bin/appmain.cpp:448

2.1.4 listener

  1. 开启cpu个数线程 = quic worker(cpu个数)
  3. #0  CxPlatThreadCreate -> QuicWorkerThread
  4. #1  QuicWorkerInitialize (Owner=Owner@entry=0x62af20, ThreadFlags=ThreadFlags@entry=0, IdealProcessor=IdealProcessor@entry=2, Worker=Worker@entry=0x6edb70) at /root/source/msquic/src/core/worker.c:83
  5. #2  0x00007ffff7932041 in QuicWorkerPoolInitialize (Owner=Owner@entry=0x62af20, ThreadFlags=0, WorkerCount=4, NewWorkerPool=0x62af58) at /root/source/msquic/src/core/worker.c:783
  6. #3  0x00007ffff79285c6 in MsQuicRegistrationOpen (Config=Config@entry=0x7fffffffdd70, NewRegistration=NewRegistration@entry=0x7ffff7dc9808 <MsQuicLib+360>) at /root/source/msquic/src/core/registration.c:120
  7. #4  0x00007ffff7925152 in QuicLibraryOnListenerRegistered (Listener=Listener@entry=0x6eb380) at /root/source/msquic/src/core/library.c:1778  (StatelessRegistration)
  8. #5  0x00007ffff7926b2c in MsQuicListenerStart (Handle=0x6eb380, AlpnBuffers=0x602d90 <Alpn>, AlpnBufferCount=1, LocalAddress=0x7fffffffde30) at /root/source/msquic/src/core/listener.c:256

2.2 client

总计开启线程 cpu*4 + 1

2.2.1 open

  1. 开启cpu个数线程
  2. MsQuic = new(std::nothrow) MsQuicApi
  4. #0  CxPlatThreadCreate -> CxPlatDataPathWorkerThread (epoll_wait)
  5. #1  CxPlatProcessorContextInitialize (Datapath=Datapath@entry=0x605560, Index=Index@entry=0, ProcContext=ProcContext@entry=0x605600) at /root/source/msquic/src/platform/datapath_epoll.c:608
  6. #2  0x00007ffff7961b57 in CxPlatDataPathInitialize (ClientRecvContextLength=ClientRecvContextLength@entry=48, UdpCallbacks=UdpCallbacks@entry=0x7fffffffddd0, TcpCallbacks=TcpCallbacks@entry=0x0, NewDataPath=NewDataPath@entry=0x7ffff7dc97e0 <MsQuicLib+320>) at /root/source/msquic/src/platform/datapath_epoll.c:701
  7. #3  0x00007ffff7925c9a in MsQuicLibraryInitialize () at /root/source/msquic/src/core/library.c:393
  8. #4  0x00007ffff7926271 in MsQuicAddRef () at /root/source/msquic/src/core/library.c:598
  9. #5  0x00007ffff79262ca in MsQuicOpen (QuicApi=QuicApi@entry=0x603080 <MsQuic>) at /root/source/msquic/src/core/library.c:1325
  10. #6  0x00007ffff79264b9 in MsQuicOpenVersion (Version=Version@entry=1, QuicApi=QuicApi@entry=0x603080 <MsQuic>) at /root/source/msquic/src/core/library.c:1405

2.2.2 new client

  1. 开启cpu*3个数线程 = data path(cpu个数) + quic worker(cpu个数) + tcp worker(cpu个数)
  2. TestToRun = new(std::nothrow) ThroughputClient;
  4. TcpEngine::TcpEngine
  5. 1. CxPlatDataPathInitialize
  6. 2. MsQuicRegistrationOpen (构造函数中调用)
  7. 3. TcpWorker::Initialize
  9. 同【2.1.3 new server】调用栈一样