1 Matching Annotations
- May 2017
The Kafka cluster retains all published records—whether or not they have been consumed—using a configurable retention period. For example, if the retention policy is set to two days, then for the two days after a record is published, it is available for consumption, after which it will be discarded to free up space. Kafka's performance is effectively constant with respect to data size so storing data for a long time is not a problem.
irrespective of the fact that the consumer has consumed the message that message is kept in kafka for the entire retention policy duration.
You can have two or more consumer groups: 1 -> real time 2 -> back up consumer group