...
The Interim Caching Layer
From V6.0.0, the interim caching feature is removed, please consider other options.
When we thinking about the storage of ATS, the original desigin is to support many disks, with the same size(best work with block devices without raid), and build up the volumes(partitions), then assign each of the volumes to some domain(hostname), we find out that if we don't make big change in the storage design, we don't have a easy way to archive the multi-tiered caching storage, then we come to the following INTERIM solution:
...
the change of Dir:
Code Block |
---|
@@ -155,15 +157,42 @@ struct FreeDir
unsigned int reserved:8;
unsigned int prev:16; // (2)
unsigned int next:16; // (3)
+#if TS_USE_INTERIM_CACHE == 1
+ unsigned int offset_high:12; // 8GB * 4K = 32TB
+ unsigned int index:3; // interim index
+ unsigned int ininterim:1; // in interim or not
+#else
inku16 offset_high; // 0: empty
+#endif
#else
uint16_t w[5];
FreeDir() { dir_clear(this); }
|
we split the stat of read_success into disk, interim and ram:
Code Block |
---|
@@ -2633,6 +2888,11 @@ register_cache_stats(RecRawStatBlock *rsb, const char *prefix)
REG_INT("read.active", cache_read_active_stat);
REG_INT("read.success", cache_read_success_stat);
REG_INT("read.failure", cache_read_failure_stat);
+ REG_INT("interim.read.success", cache_interim_read_success_stat);
+ REG_INT("disk.read.success", cache_disk_read_success_stat);
+ REG_INT("ram.read.success", cache_ram_read_success_stat);
REG_INT("write.active", cache_write_active_stat);
REG_INT("write.success", cache_write_success_stat);
REG_INT("write.failure", cache_write_failure_stat);
|
...
we have a system with 160G SSD + 3 * 500G SAS, 16G RAM, 4 cores, here is the result of tsar and iostat -x:
```
Code Block |
---|
Time --------------------ts------------------ -------------ts_cache-----------
Time qps cons Bps rt rpc hit ramhit band ssdhit
24/06/13-10:30 901.83 18.89 22.6M 17.36 47.74 87.30 68.08 88.90 22.49
24/06/13-10:35 934.12 18.88 22.0M 14.34 49.47 87.60 68.53 90.70 22.21
24/06/13-10:40 938.14 18.92 21.7M 15.36 49.58 87.70 68.02 89.50 22.45
avg-cpu: %user %nice %system %iowait %steal %idle
5.47 0.00 15.62 25.09 0.00 53.82
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await svctm %util
sda 0.00 7.33 25.67 3.33 1600.00 1438.00 104.76 0.45 15.46 12.17 35.30
sdb 0.00 0.00 28.67 11.33 1461.00 8723.00 254.60 0.74 18.47 11.21 44.83
sdc 0.00 0.00 25.67 2.00 2178.00 1373.33 128.36 0.40 14.05 11.04 30.53
sdd 0.00 0.00 196.00 4.00 14790.00 2823.00 88.06 0.13 0.66 0.41 8.30
|
...