How to count active Kafka consumers?

Solution for How to count active Kafka consumers?
is Given Below:

We’re collecting some anonymised analytics to tack monthly active users with our current REST-based solution and considering migrating to Apache Kafka. Is it possible to have some stats on the consumers (count, connections, messages delivered)? If yes, is it GDPR-friendly?

Kafka doesn’t really provide this out-of-the-box.

None of these have identifiable data attached to them

  • TCP connections can be counted via standard network tools.
  • Message bytes delivered requires external JMX metric aggregation.
  • Message count is not available without custom code (since consumers can optionally not track offsets at all, or they can rewind themselves)

To count active consumers, you’ll need to fetch all consumer groups, then iterate consumers of the groups.