测试HDFS写性能

使用命令：

  1. hadoop jar /opt/module/hadoop-2.7.2/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.7.2-tests.jar\
  2.  TestDFSIO\
  3.  -write\
  4.  -nrFiles 10\
  5.  -fileSize 128MB

输出结果：

  1. 20/08/01 21:03:08 INFO fs.TestDFSIO: ----- TestDFSIO ----- : write
  2. 20/08/01 21:03:08 INFO fs.TestDFSIO:            Date & time: Sat Aug 01 21:03:08 HKT 2020
  3. 20/08/01 21:03:08 INFO fs.TestDFSIO:        Number of files: 10
  4. 20/08/01 21:03:08 INFO fs.TestDFSIO: Total MBytes processed: 1280.0
  5. 20/08/01 21:03:08 INFO fs.TestDFSIO:      Throughput mb/sec: 31.757846421039574
  6. 20/08/01 21:03:08 INFO fs.TestDFSIO: Average IO rate mb/sec: 72.23583984375
  7. 20/08/01 21:03:08 INFO fs.TestDFSIO:  IO rate std deviation: 49.9093861009565
  8. 20/08/01 21:03:08 INFO fs.TestDFSIO:     Test exec time sec: 49.157
  9. 20/08/01 21:03:08 INFO fs.TestDFSIO:

可以看到，我的写入吞吐量是 31MB/s

测试HDFS读性能

使用命令：

  1. hadoop jar /opt/module/hadoop-2.7.2/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.7.2-tests.jar\
  2.  TestDFSIO\
  3.  -read\
  4.  -nrFiles 10\
  5.  -fileSize 128MB

输出结果：

  1. 20/08/01 21:07:53 INFO fs.TestDFSIO: ----- TestDFSIO ----- : read
  2. 20/08/01 21:07:53 INFO fs.TestDFSIO:            Date & time: Sat Aug 01 21:07:53 HKT 2020
  3. 20/08/01 21:07:53 INFO fs.TestDFSIO:        Number of files: 10
  4. 20/08/01 21:07:53 INFO fs.TestDFSIO: Total MBytes processed: 1152.0
  5. 20/08/01 21:07:53 INFO fs.TestDFSIO:      Throughput mb/sec: 42.38098741814436
  6. 20/08/01 21:07:53 INFO fs.TestDFSIO: Average IO rate mb/sec: 84.15180969238281
  7. 20/08/01 21:07:53 INFO fs.TestDFSIO:  IO rate std deviation: 76.78312138865212
  8. 20/08/01 21:07:53 INFO fs.TestDFSIO:     Test exec time sec: 48.78
  9. 20/08/01 21:07:53 INFO fs.TestDFSIO: