0x01 DIST
Checker
ThrottledAsyncChecker: Scheduling a check for [DISK]/var/hadoop/data/
0x02 启动metrics system
load hadoop-metrics2.properties
Scheduled Metric snapshot period at 10 second(s).
0x03 初始化 Block Scanner
Initialized block scanner with targetBytesPerSec 1048576
0x04 启动 Streaming Server 50010端口
0x05 设置自动平衡
Balancing bandwidth is 10485760 bytes/s
Number threads for balancing is 50
0x06 启动 HTTP Server 50075端口
0x07 Starting JVM pause monitor
0x08 LinkedBlockingQueue queueCapacity: 1000
0x09 启动 IPC Server 50020端口
向 Block pool注册
如果NN没启动。一组10次,每隔1秒向NN RPC Server(默认9000端口) 建立连接,期望注册。失败会再来10次,不停重复。
如果NN启动。handshakeBlock pool
Block pool
2019-09-23 12:09:34,670 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: DatanodeCommand action : DNA_REGISTER from tail1/192.168.124.13:9000 with active state
Blockpool handshake with NN
向Block pool 注册。
successfully registered with NN
上报Block
sent block report
Got finalize command for block pool BP-2050056729-192.168.124.13-1568174426431
启动完成
向NN注册
2019-12-05 10:20:34,230 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: DatanodeCommand action : DNA_REGISTER from tail1/192.168.124.13:9000 with active state
2019-12-05 10:20:34,274 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Block pool BP-2004589352-192.168.124.13-1572572416418 (Datanode Uuid a8358c71-d0b9-4610-b00a-2de4b1c1f8a6) service to tail1/192.168.124.13:9000 beginning handshake with NN
2019-12-05 10:20:34,327 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Block pool Block pool BP-2004589352-192.168.124.13-1572572416418 (Datanode Uuid a8358c71-d0b9-4610-b00a-2de4b1c1f8a6) service to tail1/192.168.124.13:9000 successfully registered with NN
2019-12-05 10:20:34,359 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Successfully sent block report 0x5d107e374f32179c, containing 1 storage report(s), of which we sent 1. The reports had 30 total blocks and used 1 RPC(s). This took 2 msec to generate and 24 msecs for RPC and NN processing. Got back one command: FinalizeCommand/5.
2019-12-05 10:20:34,360 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Got finalize command for block pool BP-2004589352-192.168.124.13-1572572416418
2019-12-05 10:51:29,141 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Successfully sent block report 0x5d107e374f32179d, containing 1 storage report(s), of which we sent 1. The reports had 30 total blocks and used 1 RPC(s). This took 1 msec to generate and 6 msecs for RPC and NN processing. Got back one command: FinalizeCommand/5.
2019-12-05 10:51:29,142 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Got finalize command for block pool BP-2004589352-192.168.124.13-1572572416418
2019-12-05 12:49:51,907 INFO org.apache.hadoop.util.JvmPauseMonitor: Detected pause in JVM or host machine (eg GC): pause of approximately 1635ms
No GCs detected
2019-12-05 12:50:09,995 INFO org.apache.hadoop.util.JvmPauseMonitor: Detected pause in JVM or host machine (eg GC): pause of approximately 1486ms
No GCs detected
2019-12-05 17:01:55,159 INFO org.apache.hadoop.hdfs.server.datanode.DirectoryScanner: BlockPool BP-2004589352-192.168.124.13-1572572416418 Total blocks: 30, missing metadata files:0, missing block files:0, missing blocks in memory:0, mismatched blocks:0
2019-12-05 18:01:32,718 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Successfully sent block report 0x5d107e374f32179e, containing 1 storage report(s), of which we sent 1. The reports had 30 total blocks and used 1 RPC(s). This took 3 msec to generate and 66 msecs for RPC and NN processing. Got back one command: FinalizeCommand/5.
2019-12-05 18:01:32,720 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Got finalize command for block pool BP-2004589352-192.168.124.13-1572572416418