组成:
Crc
MagicByte
Attributes
Key
Value
需要注意的是,在Kafka的文档以及源码中,消息(Message)并不包括它的offset。Kafka的log是由一条一条的记录构成的,Kafka并没有给这种记录起个专门的名字,但是需要记住的是这个“记录”并不等于”Message”。Offset Message + Size Message加在一起,构成一条记录。而在Kafka Protocol中,Message具体的格式为
Message => Crc MagicByte Attributes Key Value
Crc => int32
MagicByte => int8
Attributes => int8
Key => bytes
Value => bytes
各个部分的含义是
Field | Description |
---|---|
Attributes | This byte holds metadata attributes about the message. The lowest 2 bits contain the compression codec used for the message. The other bits should be set to 0. |
Crc | The CRC is the CRC32 of the remainder of the message bytes. This is used to check the integrity of the message on the broker and consumer. |
Key | The key is an optional message key that was used for partition assignment. The key can be null. key的作用是用来分区,可以为null |
MagicByte | This is a version id used to allow backwards compatible evolution of the message binary format. The current value is 0. |
Offset | This is the offset used in kafka as the log sequence number. When the producer is sending messages it doesn’t actually know the offset and can fill in any value here it likes. |
Value | The value is the actual message contents as an opaque byte array. Kafka supports recursive messages in which case this may itself contain a message set. The message can be null. |