由于客户做虚拟机迁移导致fsimage文件被损坏

    1. 2022-06-27 22:42:19,261 INFO org.apache.hadoop.hdfs.qjournal.client.QuorumJournalManager: Recovery prepare phase complete. Responses:
    2. 192.168.28.235:8485: segmentState { startTxId: 10067784 endTxId: 10068201 isInProgress: true } lastWriterEpoch: 5 lastCommittedTxId: 10068195
    3. 192.168.28.234:8485: lastWriterEpoch: 5 lastCommittedTxId: 10068204
    4. 2022-06-27 22:42:19,262 INFO org.apache.hadoop.hdfs.qjournal.client.QuorumJournalManager: Using longest log: 192.168.28.235:8485=segmentState {
    5. startTxId: 10067784
    6. endTxId: 10068201
    7. isInProgress: true
    8. }
    9. lastWriterEpoch: 5
    10. lastCommittedTxId: 10068195
    12. 2022-06-27 22:42:19,262 FATAL org.apache.hadoop.hdfs.server.namenode.FSEditLog: Error: recoverUnfinalizedSegments failed for required journal (JournalAndStream(mgr=QJM to [192.168.28.222:8485, 192.168.28.234:8485, 192.168.28.235:8485], stream=null))
    13. java.lang.AssertionError: Decided to synchronize log to startTxId: 10067784
    14. endTxId: 10068201
    15. isInProgress: true
    16. but logger 192.168.28.234:8485 had seen txid 10068204 committed
    17. at org.apache.hadoop.hdfs.qjournal.client.QuorumJournalManager.recoverUnclosedSegment(QuorumJournalManager.java:334)
    18. at org.apache.hadoop.hdfs.qjournal.client.QuorumJournalManager.recoverUnfinalizedSegments(QuorumJournalManager.java:457)
    19. at org.apache.hadoop.hdfs.server.namenode.JournalSet$8.apply(JournalSet.java:624)
    20. at org.apache.hadoop.hdfs.server.namenode.JournalSet.mapJournalsAndReportErrors(JournalSet.java:393)
    21. at org.apache.hadoop.hdfs.server.namenode.JournalSet.recoverUnfinalizedSegments(JournalSet.java:621)
    22. at org.apache.hadoop.hdfs.server.namenode.FSEditLog.recoverUnclosedStreams(FSEditLog.java:1521)
    23. at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startActiveServices(FSNamesystem.java:1180)
    24. at org.apache.hadoop.hdfs.server.namenode.NameNode$NameNodeHAContext.startActiveServices(NameNode.java:1919)
    25. at org.apache.hadoop.hdfs.server.namenode.ha.ActiveState.enterState(ActiveState.java:61)
    26. at org.apache.hadoop.hdfs.server.namenode.ha.HAState.setStateInternal(HAState.java:64)
    27. at org.apache.hadoop.hdfs.server.namenode.ha.StandbyState.setState(StandbyState.java:49)
    28. at org.apache.hadoop.hdfs.server.namenode.NameNode.transitionToActive(NameNode.java:1777)
    29. at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.transitionToActive(NameNodeRpcServer.java:1649)
    30. at org.apache.hadoop.ha.protocolPB.HAServiceProtocolServerSideTranslatorPB.transitionToActive(HAServiceProtocolServerSideTranslatorPB.java:107)
    31. at org.apache.hadoop.ha.proto.HAServiceProtocolProtos$HAServiceProtocolService$2.callBlockingMethod(HAServiceProtocolProtos.java:4460)
    32. at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:503)
    33. at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:989)
    34. at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:871)
    35. at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:817)
    36. at java.security.AccessController.doPrivileged(Native Method)
    37. at javax.security.auth.Subject.doAs(Subject.java:422)
    38. at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1893)
    39. at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2606)
    40. 2022-06-27 22:42:19,264 INFO org.apache.hadoop.util.ExitUtil: Exiting with status 1: Error: recoverUnfinalizedSegments failed for required journal (JournalAndStream(mgr=QJM to [192.168.28.222:8485, 192.168.28.234:8485, 192.168.28.235:8485], stream=null))
    41. 2022-06-27 22:42:19,268 INFO org.apache.hadoop.hdfs.server.namenode.NameNode: SHUTDOWN_MSG:
    42. /************************************************************
    43. SHUTDOWN_MSG: Shutting down NameNode at br-apm-006/192.168.28.235
    44. ************************************************************/

    可以通过hadoop namenode -recover命令进行修复

    1. #修复
    2. hadoop namenode -recover