1. Introduction

This page describes the implementation of an adapter to Redis.
2. Related Documents
Redis website: http://redis.io/
3. Terminology
Redis - an open-source, networked, in-memory, key-value data store

4. Requirements



5. Architectural Description

The adapter, called GemFireRedisServer, understands the Redis protocol but uses Geode underneath as the data store. This allows Redis clients to seamlessly switch to Geode allowing them to obtain the same functionality as before but now on a distributed system. The point of emphasis in this server adapter is to handle all of the data types normally implemented in Redis. This includes Strings, Lists, Hashes, Sets and SortedSets and HyperLogLogs. It is important to note that these terms are Redis specific and the server's handling of the data types will be as such. The specifics of each type will be detailed below.

5.1 Strings

Redis Strings are just strings of data. These are binary safe implying that they can be used to store any kind of data and have a maximum size of 512MB. All Strings of a cache will be stored in one Geode region.

Supported Redis Strings commands: APPEND, BITCOUNT, BITOP, BITPOS, DECR, DECRBY, GET, GETBIT, GETRANGE, GETSET, INCR, INCRBY, INCRBYFLOAT, MGET, MSET, MSETNX, PSETEX, SET, SETBIT, SETEX, SETNX, STRLEN

5.2 Lists

Redis Lists are generally linked lists of Strings and the operations for the lists are designed around heavy access at the extremities of the list. Lists are stored in Geode by loose indexing and retrieved by querying. It is important to understand the limitations of Lists as they are implemented in this adapter. Geode has no implicit notion of lists and therefore lists are simulated in Regions. For this reason lists become data scalable at the expense of increased latency. As opposed to Sets or Hashes, which have no notion of order, Lists require extra care to ensure order and consistency which takes a heavy toll on performance. For example, the LPUSH command requires to determine the left most index of the list, append to the beginning of the list and update the index of the list. In Redis, lists do not scale and therefore these tasks are trivial but in Geode, even a simple pop/push on a distributed list can introduce multiple network hops. Also, because indexes are mapped to list elements, shifting list elements is unrealistic and LINSERT is not supported.

Supported Redis Lists commands: LINDEX, LLEN, LPOP, LPUSH, LPUSHX, LRANGE, LREM, LSET, LTRIM, RPOP, RPUSH, RPUSHX

5.3 Hashes
Redis Hashes are String to String maps. This is the most straightforward implementation of the group because a region in a Geode cache is a map so the data can be added without any reconfiguration.

Supported Hashes commands: HDEL, HEXISTS, HGET, HGETALL, HINCRBY, HINCRBYFLOAT, HKEYS, HMGET, HMSET, HSCAN, HSETNX, HLEN, HSET, HVALS

5.4 Sets

Redis Sets are unordered sets of Strings. This will be implemented in Geode by holding the set data as keys in the region which will implicitly perform the necessary actions to simulate a set.

Supported Sets commands: SADD, SCARD, SDIFF, SDIFFSTORE, SINTER, SINTERSTORE, SISMEMBER, SMEMBERS, SMOVE, SREM, SPOP, SRANDMEMBER, SSCAN, SUNION, SUNIONSTORE

5.5 SortedSets

Redis SortedSets are Sets that are ordered by an explicit "score". The ordering does not take into account ordering such as lexicographical or hashing functionality but instead uses the score for sorting. Members of the SortedSet are unique but scores are not, therefore sorted sets are member to score mappings in Geode.

Supported SortedSets commands: ZADD, ZCARD, ZCOUNT, ZINCRBY, ZLEXCOUNT, ZRANGE, ZRANGEBYLEX, ZRANGEBYSCORE, ZRANK, ZREM, ZREMRANGEBYLEX, ZREMRANGEBYRANK, ZREMRANGEBYSCORE, ZREVRANGE, ZREVRANGEBYSCORE, ZREVRANK, ZSCAN, ZSCORE

5.6 HyperLogLogs

Redis HyperLogLog's are C implementations of the data structure, whereas this implementation is done in Java within Geode. The only difference is that Geode implementation does not utilize the sparse implementation for small cardinalities but is otherwise the same.

Supported HyperLogLog commands: PFADD, PFCOUNT, PFMERGE

5.6 Additional Information

The GemFire Redis Server will also implement other commands to be a full fledged Redis server. However it is worth noting that while Redis has 160+ commands in their protocol, the GemFireRedisServer will support roughly 110 of them. These will cover all basic commands as well and other practical ones that should not be left out. Redis has many commands for server configurations that are set up differently for Geode as well as scripting that isn't supported.

Another important note about keys is that key support within GemFireRedisServer is not the same as Redis. While all values are binary safe, most keys are not as they will have their own Regions in Geode. The Region namespace is defined by Java Strings, and Geode's OQL engine does not support all characters. Therefore some keys may cause failures with Geode if attempting to create a key using non printable characters such as UTF-8 0x01, 0x02, etc.
The backing Regions for each key are partitioned by default, but the default region type can be configured by system property. If different keys are desired to have different types of Regions, these Regions must be defined in cache.xml. The other configuration to strongly consider per use case is defining the number of worker threads to use for Redis clients. This is also defined by system property and more information can be found in the GemFireRedisServer javadoc. To put it all together, this adapter can be instantiated through invoking com.gemstone.gemfire.redis.GemFireRedisServer independently or programmatically, or through GFSH when the property redis-port is set.

Supported Key commands: DEL, DBSIZE, EXISTS, EXPIRE, EXPIREAT, FLUSHALL, FLUSHDB, KEYS, PERSIST, PEXPIRE, PEXPIREAT, PTTL, SCAN, TTL
Supported Server commands: AUTH, ECHO, PING, QUIT
Supported Transaction commands: DISCARD, EXEC, MULTI

6. Comparison to Redis

The following points are the main differentiators between Geode/GemFireRedisServer and Redis: 
6.1 Concurrency
  • Redis is a single-threaded server. It is not designed to benefit from multiple CPU cores. People are supposed to launch several Redis instances to scale out on several cores if needed. It is not really fair to compare one single Redis instance to a multi-threaded data store.
We have leveraged the highly concurrent nature of Geode to make GemFireRedisServer concurrent. Each server instance will start 4 * (number of processor cores) threads for processing client requests,  but this can be configured by system property where either one thread per connection can be created or a specific number of client handler threads can be requested.
6.2 Scalability
In Redis, a single data structure cannot scale beyond the memory available on a single box. From: http://redis.io/topics/partitioning
  • The partitioning granuliary is the key, so it is not possible to shard a dataset with a single huge key like a very big sorted set.
(In Redis, key is the name of the data structure, so a hashmap.get() take the form HGET key field)
This limitation is not expected to go away even with the introduction of redis-cluster (which is going to partition the key space i.e. data structure name space). So, with Redis cluster you can scale up the number of data structures, not the data structures themselves.
In GemFireRedisServer, all Redis data structures are backed by PartitionedRegions, so each Redis data structure is horizontally scalable. 
6.3 Replication
In Redis, you can have any number of slaves for a master server. Replication from master to slave is always asynchronous, which may lead to data loss when master crashes before replicating.
When a slave is started, it gets entire data set from the master (even though the salve was persistent). It does, however, support a "Partial resynchronization" from master to slave, if the link between master and slave goes down.
Slaves can be used for scaling read operations.
In Geode, you can have upto 3 redundant copies (for partitioned regions). When persistent, these replicas will first recover from local disk and only get the delta from the primary.  These copies can be used for read operations. 
6.4 Persistence
Redis supports Snapshots as well as AOF (Append Only) persistence. AOF is a log of all the operations, which needs to be rebuilt on re-starts. AOF is automatically re-written when it gets too large.
Geode persistence is also append only, however keys and values are kept in separate files on disk. On restarts, only the key's file needs to be read.
6.5 HA
Redis uses Sentinel for managing HA. User starts a number of sentinel processes by providing them a list of all the masters (the slaves are automatically detected). When a primary crashes, the sentinels gossip and elect a new primary from among the slaves.
In Geode, all members are connected to all other members, failover is automatic. (no need to provide a list of all members, which is error prone)
6.6 Network Partition
With the sentinel approach, there is no real protection from network partition. The documentation mentions that write quorum should be used to guard against writing to a primary on the loosing side, however, since the replication is asynchronous, there will still be some amount of data loss. (This will be fixed with redis-cluster, no more need of sentinels for partition detection)
Geode has network partition detection built in. The loosing side servers will shutdown/fence themselves, so that clients cannot connect to them.

7. Performance and Scalability
These are some performance numbers acquired using the redis-benchmark utility. All these tests are run using separate bare metal machines for each node and client.

Run using no pipelining and 1KB payloads

Redis

SET: 100894.94 requests per second

GET: 103504.02 requests per second

INCR: 99662.14 requests per second

SADD: 99559.35 requests per second 

GemFireRedisServer

SET: 87627.06 requests per second

GET: 102988.52 requests per second

INCR: 92251.61 requests per second

SADD: 92254.50 requests per second

 

Pipelining 16 requests at a time

SET: 109277.91 requests per second

GET: 113583.70 requests per second

INCR: 1061300.75 requests per second

SADD: 989119.69 requests per second

GemFireRedisServer

SET: 109109.55 requests per second

GET: 113523.87 requests per second

INCR: 575023.25 requests per second

SADD: 644678.81 requests per second


Scalability is not within the framework of this specification as Geode handles the scaling independently of this implementation.

8. Testing

TBD

9. Documentation
TBD

Updated 07/08/2015
  • No labels

1 Comment

  1. linving

    use gfsh can be set  --J=-Dgemfireredis.regiontype=PARTITION_PERSISTENT

    gfsh> start server --name=server1 --redis-bind-address=localhost \
 --redis-port=11211 --J=-Dgemfireredis.regiontype=PARTITION_PERSISTENT

    how can i set properties  --J=-Dgemfireredis.regiontype=PARTITION_PERSISTENT  in java code？