结果对比:Tournaments

假设我们要分析改进对性能的影响。 首先，我们需要进行benchmark。例如：

  1. esrally --track=pmc --revision=latest --user-tag="intention:baseline_github_1234"

上面我们根据Elasticsearch的最新源代码版本运行benchmark。我们可以使用命令行参数--user-tag提供一个键值对，以记录比赛的指标。

新的测试如下：

  1. esrally --track=pmc --revision=current --user-tag="intention:reduce_alloc_1234"

在进行了两次比赛之后，我们想知道性能的对比。借助Rally，我们可以轻松分析两个测试的差异。首先，我们需要通过esrally list races来找到两个比赛:

  1. dm@io:~ $ esrally list races
  3.     ____        ____
  4.    / __ \____ _/ / /_  __
  5.   / /_/ / __ `/ / / / / /
  6.  / _, _/ /_/ / / / /_/ /
  7. /_/ |_|\__,_/_/_/\__, /
  8.                 /____/
  9. Recent races:
  11. Race ID                               Race Timestamp    Track    Track Parameters    Challenge            Car       User Tags
  12. ------------------------------------  ----------------  -------  ------------------  ------------------   --------  ------------------------------
  13. beb154e4-0a05-4f45-ad9f-e34f9a9e51f7  20160518T122341Z  pmc                          append-no-conflicts  defaults  intention:reduce_alloc_1234
  14. 0bfd4542-3821-4c79-81a2-0858636068ce  20160518T112057Z  pmc                          append-no-conflicts  defaults  intention:baseline_github_1234
  15. 0cfb3576-3025-4c17-b672-d6c9e811b93e  20160518T101957Z  pmc                          append-no-conflicts  defaults

我们可以看到用户标签可以帮助我们识别两次测试。我们要比较两个最近的比赛，并在下一步中必须提供两个比赛ID：

  1. dm@io:~ $ esrally compare --baseline=0bfd4542-3821-4c79-81a2-0858636068ce --contender=beb154e4-0a05-4f45-ad9f-e34f9a9e51f7
  3.     ____        ____
  4.    / __ \____ _/ / /_  __
  5.   / /_/ / __ `/ / / / / /
  6.  / _, _/ /_/ / / / /_/ /
  7. /_/ |_|\__,_/_/_/\__, /
  8.                 /____/
  10. Comparing baseline
  11.   Race ID: 0bfd4542-3821-4c79-81a2-0858636068ce
  12.   Race timestamp: 2016-05-18 11:20:57
  13.   Challenge: append-no-conflicts
  14.   Car: defaults
  16. with contender
  17.   Race ID: beb154e4-0a05-4f45-ad9f-e34f9a9e51f7
  18.   Race timestamp: 2016-05-18 12:23:41
  19.   Challenge: append-no-conflicts
  20.   Car: defaults
  22. ------------------------------------------------------
  23.     _______             __   _____
  24.    / ____(_)___  ____ _/ /  / ___/_________  ________
  25.   / /_  / / __ \/ __ `/ /   \__ \/ ___/ __ \/ ___/ _ \
  26.  / __/ / / / / / /_/ / /   ___/ / /__/ /_/ / /  /  __/
  27. /_/   /_/_/ /_/\__,_/_/   /____/\___/\____/_/   \___/
  28. ------------------------------------------------------
  29.                                                   Metric    Baseline    Contender               Diff
  30. --------------------------------------------------------  ----------  -----------  -----------------
  31.                         Min Indexing Throughput [docs/s]       19501        19118  -383.00000
  32.                      Median Indexing Throughput [docs/s]       20232      19927.5  -304.45833
  33.                         Max Indexing Throughput [docs/s]       21172        20849  -323.00000
  34.                                Total indexing time [min]     55.7989       56.335    +0.53603
  35.                                   Total merge time [min]     12.9766      13.3115    +0.33495
  36.                                 Total refresh time [min]     5.20067      5.20097    +0.00030
  37.                                   Total flush time [min]   0.0648667    0.0681833    +0.00332
  38.                          Total merge throttle time [min]    0.796417     0.879267    +0.08285
  39.                Query latency term (50.0 percentile) [ms]     2.10049      2.15421    +0.05372
  40.                Query latency term (90.0 percentile) [ms]     2.77537      2.84168    +0.06630
  41.               Query latency term (100.0 percentile) [ms]     4.52081      5.15368    +0.63287
  42.         Query latency country_agg (50.0 percentile) [ms]     112.049      110.385    -1.66392
  43.         Query latency country_agg (90.0 percentile) [ms]     128.426      124.005    -4.42138
  44.        Query latency country_agg (100.0 percentile) [ms]     155.989      133.797   -22.19185
  45.              Query latency scroll (50.0 percentile) [ms]     16.1226      14.4974    -1.62519
  46.              Query latency scroll (90.0 percentile) [ms]     17.2383      15.4079    -1.83043
  47.             Query latency scroll (100.0 percentile) [ms]     18.8419      18.4241    -0.41784
  48.  Query latency country_agg_cached (50.0 percentile) [ms]     1.70223      1.64502    -0.05721
  49.  Query latency country_agg_cached (90.0 percentile) [ms]     2.34819      2.04318    -0.30500
  50. Query latency country_agg_cached (100.0 percentile) [ms]     3.42547      2.86814    -0.55732
  51.             Query latency default (50.0 percentile) [ms]     5.89058      5.83409    -0.05648
  52.             Query latency default (90.0 percentile) [ms]     6.71282      6.64662    -0.06620
  53.            Query latency default (100.0 percentile) [ms]     7.65307       7.3701    -0.28297
  54.              Query latency phrase (50.0 percentile) [ms]     1.82687      1.83193    +0.00506
  55.              Query latency phrase (90.0 percentile) [ms]     2.63714      2.46286    -0.17428
  56.             Query latency phrase (100.0 percentile) [ms]     5.39892      4.22367    -1.17525
  57.                             Median CPU usage (index) [%]     668.025       679.15   +11.12499
  58.                             Median CPU usage (stats) [%]      143.75        162.4   +18.64999
  59.                            Median CPU usage (search) [%]       223.1        229.2    +6.10000
  60.                                   Total Young Gen GC [s]      39.447       40.456    +1.00900
  61.                                     Total Old Gen GC [s]       7.108        7.703    +0.59500
  62.                                          Index size [GB]     3.25475      3.25098    -0.00377
  63.                                       Total written [GB]     17.8434      18.3143    +0.47083
  64.                              Heap used for segments [MB]     21.7504      21.5901    -0.16037
  65.                            Heap used for doc values [MB]     0.16436      0.13905    -0.02531
  66.                                 Heap used for terms [MB]     20.0293      19.9159    -0.11345
  67.                                 Heap used for norms [MB]    0.105469    0.0935669    -0.01190
  68.                                Heap used for points [MB]    0.773487     0.772155    -0.00133
  69.                                Heap used for points [MB]    0.677795     0.669426    -0.00837
  70.                                            Segment count         136          121   -15.00000
  71.                      Indices Stats(90.0 percentile) [ms]     3.16053      3.21023    +0.04969
  72.                      Indices Stats(99.0 percentile) [ms]     5.29526      3.94132    -1.35393
  73.                     Indices Stats(100.0 percentile) [ms]     5.64971      7.02374    +1.37403
  74.                        Nodes Stats(90.0 percentile) [ms]     3.19611      3.15251    -0.04360
  75.                        Nodes Stats(99.0 percentile) [ms]     4.44111      4.87003    +0.42892
  76.                       Nodes Stats(100.0 percentile) [ms]     5.22527      5.66977    +0.44450