Page History

...

The design of the current file based wave store is described here:

http://www.waveprotocol.org/protocol/design-proposals/wave-store-design-for-wave-in-a-box^{Image Removed}

The current implementation is very simplistic and inefficient in most ways, preventing WIAB from scaling beyond thousands of waves and dozens of users. The biggest problems are:
P1. Wavelets are reconstructed from scratch from their entire delta history when loaded from disk. This is expensive and causes excessive latency when loading a wave with a long edit history from disk.
P2. The wave server keeps the entire wavelet delta history in memory for all the wavelets it has loaded into memory. This makes the memory footprint for waves with long edit histories excessive.
P3. All wavelets are kept in memory in the wave server, and the wave server loads all wavelets from disk on startup. Obviously, the memory footprint and startup latency of this solution doesn't scale to large number of waves.
P4. The search implementation is tightly coupled with the in-memory wave store, in particular, user searches contend for the wavelet lock in the wave store. This makes searches slow and expensive and doesn't scale beyond dozens of simultaneous users.

...

3.1 DELTA HISTORY EVICTION
The excessive footprint can be avoided if the wave server evicts the oldest deltas from memory, that is, it only keeps the most recent deltas6 deltas in memory and loads older deltas from disk only upon request. 19 Since we don't want to perform any disk operations synchronously under the wavelet lock in the wave store, we need to make all the methods that access delta history asynchronous. Specifically, the following methods need to be made asynchronous:
box.server.waveserver.WaveletContainer#requestHistory()
box.server.waveserver.WaveletContainer#requestTransformedHistory()
box.server.waveserver.WaveletProvider#getHistory()
which requires changes to all the places they are called, of course.30.
Implemented partially. Deltas are evicted, but without making the methods asynchronous

3.2 PERSISTENT WAVELET SNAPSHOTS

...