This is an experimental Cascading Sink to Redis.
It uses the excellent Jedis client for most of its work.
The skeleton (incl. the build.xml) is copied from cascading/memcached.
cascading.redis is pretty minimal. It supports:
- sinking to hashes (RedisHashScheme)
- sinking simple key / value string pairs (RedisSimpleScheme)
This release requires at least Cascading 1.2.x. Hadoop 0.19.x or later, and a Redis cluster.
To build a jar,
ant -Dcascading.home=... -Dhadoop.home=... jar
To test (requires a localhost Redis instance),
ant -Dcascading.home=... -Dhadoop.home=... test
Add the .jar to the lib/ directory of your Hadoop application jar (like a normal dep).
Jedis/Redis are extremely feature rich and coupled with the flexibility of Cascading, there's many possibilities.
- misc. cleanups, better logging and error handling (some sort of automatic retry like the memcached client?)
- license-ify
- a scheme for sets, sorted sets and other datatypes
- a scheme for pubsub. It would be neat to have Redis clients listening for data updates pushed via Cascading.
- support sourcing from Redis? Redis can usually be about 80% of a analytics / moments server.
- remove the RedisCommand object and use something fun like reflection?