THIS IS A TEST INSTANCE. ALL YOUR CHANGES WILL BE LOST!!!!
...
With this end goal in mind, now let's see how we should bound the memory usage for the above cases, especially 3), 4) and 5).
Triggering based Caches
...
The total memory used for this part (denoted as Cache) can be calculated as:
...
- We only need in-memory caching if persistent stores are used for aggregates, which will introduce extra serde costs as mentioned above.
- If the state stores used are already in-memory (and this should be in deserialized objects), we do not need the caching in bytes any more, while we still keep the dirty map for triggered flushing.
Deserialized Objects Buffering
...
The total memory used for this part (denoted as Buffer) can be calculated as:
...
buffered.bytes.per.partition
Persistent State Store Buffering
...
The total memory used for this part (denoted as Store) as:
...