...

• Step1: Adding the first PID for user will create a bloom filter for this user in the cached map (let call it bloom_filter_1)
  • Any new PIDs will be added to this cache for the first half of producer.id.quota.window.size.seconds 
• Step2: A new bloom filter will be created along side the old one for the user for the second half of producer.id.quota.window.size.seconds (let's call it bloom_filter_2).
  • All new PIDs from this point will be added to the new filter. 
  • Both bloom filters will be used to check if we came across the same PID before or not. 
• Step3: The old bloom filter (bloom_filter_1) will be disposed once we reach  producer.id.quota.window.size.seconds .
• Step4: Repeat steps 2, and 3 on bloom_filter_2. 

User entry in the cached map will be entirely removed from the caching layer if it doesn't have any active bloom filters attached to it anymore. 

This way in each user within a window will have 2 bloom filters (both are available for read while only one of them active for write) and when we need to check if we came across PID before or not for a given user we will check if any of the bloom filters contains the PID. 

Note: User entry in the cached map will be entirely removed from the caching layer if it doesn't have any active bloom filters attached to it anymore. Performing a cleanup for inactive users.

Public Interfaces

New Broker Configurations

...