Skip to content

Latest commit

 

History

History
111 lines (74 loc) · 3.57 KB

README.md

File metadata and controls

111 lines (74 loc) · 3.57 KB
Raw

Clj-hazelcast

Clojure library for Hazelcast, an open source clustering and highly scalable data distribution platform for Java.

TravisCI Status

Build Status

Usage

Clj hazelcast comes preconfigured out of the box and allows adding members as an option. For example, The init function takes a map, {:peers [hostname1 hostname2 ...]}, and starts a peer node on them as the member of the cluster.

This library supports synchronization of clj-kryo (another open source library from Runa labs) serializable data structures across network. It also supports event driven communication among them. A typical use case would to sync key/value pair among the peers. For example:

Suppose you have following data in all nodes: (def test-map (atom nil)) You can put new values to this map on host 1: (hazelcast/put! @test-map :baz "foobar")

Then you can retrieve "foobar" from host 2: (:baz (hazelcast/get-map "your.name.space.test-map"))

Furthermore, you can add listener to that data to perform all sorts of callbacks. For example:

(let [events (atom [])
      listener-fn (fn [& event]
                    (swap! events conj event))
      listener (hazelcast/add-entry-listener! @test-map listener-fn)
      result (do
               (hazelcast/put! @test-map :baz "foobar")
               (hazelcast/put! @test-map :foo "bizbang")
               (Thread/sleep 5)
               (count @events))]
  (hazelcast/remove-entry-listener! @test-map listener)
  result)

Please refer to the inlcuded test for more detail.

#Using the MapReduce Framework The namespace clj-hazelcast.mr contains an abstraction to make it easier when dealing with mapreduce jobs.

##Mapper Runs the function f over the content.
f is a function of two arguments: key and value.
f must return a sequence of pairs like

[[key1 value1] [key2 value2] ...]

Sample Mapper

(defmapper sample-mapper [k v] [[k (+ 1 v)])

This is a simple mapper that gets each key-value pair and returns a pair of k and value+1

##Reducer Runs the reducer function rf over the content.
rf is a function of three arguments, key,value and an accumulator.

rf should return the new accumulator.

Sample Reducer

(defreducer sample-reducer [k v acc] (if (nil? acc) 1 (inc acc)))

This is a simple counting reducer.

##Collator

Runs the collactor function f over the content.
f is a function of one argument, a sequence of Entry objects, where Entry's are generated by the reducers output:

[Entry1 Entry2 ...]

Sample Collator

(defcollator sample-collator [seq] (reduce + (vals seq)))

This is a simple collator that counts aggregates all values from a map.

##Combiner

Combiners are mostly the same as the reducers

##Wordcount Example Check 'clj-hazelcast.test.mr-test

#Distributed Query You can use the defpredicate macro to use clojure functions as predicates. A predicate function is a function with 2 parameters, a key and a value and returns a boolean value.

(require '[clj-hazelcast.query :as q])
(q/defpredicate stark? [k v] (.contains v "Stark"))
(q/values @query-map stark?)

License

  • The use and distribution terms for this software are covered by the
  • Eclipse Public License 1.0 (http://opensource.org/licenses/eclipse-1.0.php)
  • which can be found in the file epl-v10.html at the root of this distribution.
  • By using this software in any fashion, you are agreeing to be bound by
  • the terms of this license.
  • You must not remove this notice, or any other, from this software.