Overriding Protocol Methods with ns-utils/immigrate

Recently I found myself experimenting with Lau Jensen's ClojureQL. Based on relational algebra, all queries to the database are expressed as relations. The user of the library applies functions on these relations to transform them via limit, join, order etc.

Internally, these relations are implemented in a record called RTable, which implements the Relation protocol. Of course the RTable record type is inaccessible to users of the library, they simply call the table function which returns a simple relation on the requested SQL table.

The reason I wanted to discuss ClojureQL's records is that they were preventing me from enhancing the RTable's implementation of the Relation protocol's methods -- or so I thought.

From my limited understand, protocol methods implemented in a record are dynamically compiled into a corresponding class which implements the protocol. This is what allows them to use primitives in the arguments. However - this is what I didn't realize - these methods are still accessible as vars in the parent namespace and thus can be referenced from elsewhere.

Using ns-utils/immigrate, references to all vars in a specified namespace are added to the current namespace. The effect is sort of like the use function, but with the immigrated functions appearing to other name spaces as though they were defined there.

This may be more clear with an example. The following is the external namespace containing the protocol and its implementation.

(ns override.external)

(defprotocol FooProtocol
  (foo [this])
  (bar [this]))

(defrecord MyRecord [val] FooProtocol
           (foo [this] (str "Foo of " val))
           (bar [this] (str "Bar of " val)))

(defn make-record [val]
  (MyRecord. val))

Now say that you want to wrap any calls to foo with your own custom wrapper defined in a test namespace. Just immigrate the external namespace and implement a new foo function with a signature which matches that of the original protocol method.

(ns override.test
  (:require override.external)
  (:use [clojure.contrib.ns-utils :only [immigrate]]))

(immigrate 'override.external)

(defn foo [rec]
  (str "Wrapping foo (" (override.external/foo rec) ")"))

Now anyone who references the override.test namespace will get all the functionality of the external namespace but with the custom behavior of the test namespace added for foo.

override.test> (def r (make-record "cucumber"))
#'override.test/r
override.test> (foo r)
"Wrapping foo (Foo of cucumber)"
override.test> (bar r)
"Bar of cucumber"
override.test> 

I am unsure of the performance cost of the indirection necessary to call a namespace function followed by a compiled method, so users concerned with performance should tread carefully.

Finally, the beauty of protocols is that this should work for any type implementing FooProtocol, not just MyRecord.

Mocking with clojure.contrib.mock

Edit: In the upcoming Clojure 1.3.0, binding can only be used with vars which are marked dynamic, so the new with-redefs is used instead. This functions identically except for the fact that bindings are not thread local.

This is a brief introduction to using clojure.contrib.mock in concert with clojure.test in order to achieve isolated, efficient and side-effect free tests of your clojure code. I will be basing the sample code here off the latest snapshot releases of both clojure and clojure.contrib, but there should not be anything that is unavailable in version 1.1.

If you are using leiningen, you can copy my project.clj from below to get started up quickly. I just ran lein new example and then altered the project.clj file as follows. You can of course omit the swank-clojure dev dependency if you are not using emacs.

(defproject example "1.0.0-SNAPSHOT"
  :description "A brief example on how to use clojure.contrib.mock for all
your functional mocking needs."
  :dependencies [[org.clojure/clojure "1.2.0-master-SNAPSHOT"]
                 [org.clojure/clojure-contrib "1.2.0-SNAPSHOT"]]
  :dev-dependencies [[swank-clojure "1.2.1"]])

In the world of Clojure, most people use Stuart Sierra's excellent clojure.test api for writing their unit tests. For this reason, I decided to use clojure.contrib.mock.test-adapter instead of the vanilla c.c.mock. Using the adapter namespace automatically imports all the vanilla mock functions, with the notable exception of overriding mock's error reporting functions with new ones that hook into clojure.test.

So to start, I created a new file called my-mock.clj in src/example and declared the namespace as follows:

(ns example.my-mock
  (:use clojure.contrib.mock.test-adapter)
  (:require [clojure.test :as test]))

This should give us access to all the functions and macros defined in clojure.contrib.mock. Calling one of the mock functions from the repl verifies this.

example.my-mock> (times 5)
  {:times #<mock$make_count_checker$fn__1717 clojure.contrib.mock$make_count_checker$fn__1717@1d41677>}
example.my-mock>

Good. Now let's write a couple simple functions that need to be tested.

(defn slow-square [x]
  (Thread/sleep 2000)
  (* x x))

(defn foo [y]
  (slow-square (+ y 2)))

The slow-square function represents the function we want to be mocked out (it's too slow to run in unit tests!) and foo represents the code we are testing. Here's a test using c.c.mock:

(test/deftest test-foo
  (expect [slow-square (times once (has-args [8]))]
    (foo 5)))

The outer macro, expect, is the entry point to the mocking scope. All code inside of the expect is executed with the specified function(s) overridden via clojure.core/binding and replaced with specialized mock functions. As guaranteed by the binding macro, these overrides are thread local and only in place within the the scope of the s-expression.

The first argument to expect is a vector of pairs of function names and mock descriptions represented by maps. The functions listed here are the ones that will be replaced by the generated mock function. The maps are typically created by invoking the provided functions such as times and has-args, as seen here. If you were to try running the code above, you would get the following error.

example.my-mock> (test/run-tests)

Testing example.my-mock

FAIL in (test-foo) (test_adapter.clj:26)
Argument 0 has an unexpected value for function. Function name: slow-square
expected: 8
actual: 7

Ran 1 tests containing 1 assertions.
1 failures, 0 errors.
{:type :summary, :test 1, :pass 0, :fail 1, :error 0}
example.my-mock>

This shows us that c.c.mock has properly integrated itself with clojure.test and it demonstrates the output. To make the test pass, just change the 8 to a 7 in has-args.

This is just the beginning of what you can do with mocking in Clojure. Each element in the has-args array can be a single parameter predicate function, allowing you to validate the parameters passed to the mock function however you wish. You will also note that the times function was passed the once function, but it could have been any predicate as well (or even just an integer).

For most purely functional mocks you will want the mock to return a value as well. For this, you can use the returns function. So, in the code above it might look something like this:

(test/deftest test-foo2
  (expect [slow-square (returns "foobar!")]
    (test/is (= (foo 45) "foobar!"))))

For even more advanced replacement functions you can use calls instead of returns and point to a function with an argument count matching that of the function being replaced, although I'm told this is kind of pushing what a mock is supposed to be. If both calls and returns are specified in a function replacement definition, the calls takes precedence. Both of these functions can of course be combined with times and has-args.

Well that about wraps up all I had to say about c.c.mock. I hope this little overview was helpful, and I hope the mock library was helpful too! If you have any questions about either, I'd be more than happy to hear from you.

Happy testing!