组成:

Crc
MagicByte
Attributes
Key
Value

需要注意的是,在Kafka的文档以及源码中,消息(Message)并不包括它的offset。Kafka的log是由一条一条的记录构成的,Kafka并没有给这种记录起个专门的名字,但是需要记住的是这个“记录”并不等于”Message”。Offset Message + Size Message加在一起,构成一条记录。而在Kafka Protocol中,Message具体的格式为

  1. Message => Crc MagicByte Attributes Key Value
  2. Crc => int32
  3. MagicByte => int8
  4. Attributes => int8
  5. Key => bytes
  6. Value => bytes

各个部分的含义是

Field Description
Attributes This byte holds metadata attributes about the message. The lowest 2 bits contain the compression codec used for the message. The other bits should be set to 0.
Crc The CRC is the CRC32 of the remainder of the message bytes. This is used to check the integrity of the message on the broker and consumer.
Key The key is an optional message key that was used for partition assignment. The key can be null.
key的作用是用来分区,可以为null
MagicByte This is a version id used to allow backwards compatible evolution of the message binary format. The current value is 0.
Offset This is the offset used in kafka as the log sequence number. When the producer is sending messages it doesn’t actually know the offset and can fill in any value here it likes.
Value The value is the actual message contents as an opaque byte array. Kafka supports recursive messages in which case this may itself contain a message set. The message can be null.