Module eqc_mocking

This module provides functionality for mocking Erlang modules.

Copyright © Quviq AB, 2013-2015

Version: 1.36.1

Authors: Hans Svensson.

Description

This module provides functionality for mocking Erlang modules. It is designed to work together with eqc_component where callouts can be mocked conveniently using the functionality in this module. However, it is also possible to use this module for traditional mocking, but the documentation focus on the callout use-case.

The mocking technique we use in this module is based on research results from the Prowess project - An Expressive Semantics of Mocking. In short we use a CSP-like language to specify the behaviour of mocked modules. The behaviour can be described using the following macros:

How to mock a module

To mock a module we need to specify its API in an API specification. Given the API we can also specify the particular behaviour of the defined functions, by defining a mocking langauage specification. As an example we give the mocking specification of a stack; its API consists of three functions new/0, push/2, and pop/1. Its API specification looks like:
  api_spec() -> #api_spec{
    language = erlang,
    modules  = [ #api_module{
      name = stack,
      functions = [ #api_fun{ name = new,  arity = 0 },
                    #api_fun{ name = push, arity = 2 },
                    #api_fun{ name = pop,  arity = 1 } ]
      }]}.
  
In one test case we expect the stack to be called four times; (1) creating a new stack, (2) pushing the integer 4 onto the stack, (3) pushing 6 onto the stack, and (4) popping one element from the stack expecting 6 to be returned. This could be expressed as:
  lang() -> ?SEQ([?EVENT(stack, new,  [],             stack_ref),
                  ?EVENT(stack, push, [stack_ref, 4], ok),
                  ?EVENT(stack, push, [stack_ref, 6], ok),
                  ?EVENT(stack, pop,  [stack_ref],    6)
                 ]).
  

Where we use an abstract stack reference stack_ref to represent the stack during the execution.

The stack can now be 'used' as follows (assuming that the functions defined above reside in the module stack_mock):
  2> eqc_mocking:start_mocking(stack_mock:api_spec()).
  {ok,<0.116.0>}
  3> eqc_mocking:init_lang(stack_mock:lang(), stack_mock:api_spec()).
  ok
  4> S = stack:new(), stack:push(S, 4), stack:push(S, 6), stack:pop(S).
  6
  5> eqc_mocking:check_callouts(stack_mock:lang()).
  true
  6> eqc_mocking:init_lang(stack_mock:lang(), stack_mock:api_spec()).
  ok
  7> f(S), S = stack:new(), stack:push(S, 4), stack:push(S, 6).
  ok
  8> eqc_mocking:check_callouts(stack_mock:lang()).
  {expected,{call,stack,pop,[stack_ref]}}
  9> eqc_mocking:init_lang(stack_mock:lang(), stack_mock:api_spec()).
  ok
  10> f(S), S = stack:new(), stack:pop(S).
  ** exception exit: {{mocking_error,{unexpected,{call,stack,pop,[stack_ref]}}}},
                      [{eqc_mocking,f5735660_0,
                                   [stack,pop,[stack_ref]],
                                   [{file,"../src/eqc_mocking.erl"},{line,375}]},
                       ... ]}
       in function  eqc_mocking:do_action/3 (../src/eqc_mocking.erl, line 371)
  
Noteable in the usage above is that we can call init_lang/2 many times without restarting the mocking framework in between. Also note that the check check_callouts/1 checks both that all expected calls where made, and that the calls were made with the expected arguments. If a call is made out of order a failure is reported immediately. If a call is made where the arguments does not match, the default behavior is to also report a failure immediately:
  19> eqc_mocking:init_lang(stack_mock:lang(), stack_mock:api_spec()).
  ok
  20> f(S), S = stack:new(), stack:push(S, 7), stack:push(S, -3), stack:pop(S).
  ** exception exit: {{mocking_error,{unexpected,{call,stack,push,[stack_ref,7]}}},
                     [{eqc_mocking,f5735660_0,
                                   [stack,push,[stack_ref,7]],
                                   [{file,"../src/eqc_mocking.erl"},{line,375}]},
                      ...]}
       in function  eqc_mocking:do_action/3 (../src/eqc_mocking.erl, line 371)
  
However, it is sometimes useful not to abort tests prematurely, and a small change to the API specification enables this:
  api_spec() -> #api_spec{
    language = erlang,
    modules  = [ #api_module{
      name = stack,
      functions = [ #api_fun{ name = new,  arity = 0 },
                    #api_fun{ name = push, arity = 2, matched = [] },
                    #api_fun{ name = pop,  arity = 1 } ]
      }]}.
  
Note the added matched = [] that tells eqc_mocking to not check any of the arguments at call time. (Default is matched = all). With this API specification the previous example behaves as follows:
  21> c(stack_mock).
  {ok, stack_mock}
  22> eqc_mocking:start_mocking(stack_mock:api_spec()).
  {ok,<0.116.0>}
  23> eqc_mocking:init_lang(stack_mock:lang(), stack_mock:api_spec()).
  ok
  24> f(S), S = stack:new(), stack:push(S, 7), stack:push(S, -3), stack:pop(S).
  6
  25> eqc_mocking:check_callouts(stack_mock:lang()).
  {unexpected,{call,stack,push,[stack_ref,7]},
              expected,
              {call,stack,push,[stack_ref,4]}}
  

Stop mocking

Once the mocked modules are no longer needed, the function stop_mocking/0 stops the mocking server and restores (i.e. unloads previously non-existing modules and reverting previously existing modules) the modules to their pre-mocking state.

Advanced usage - matching arguments

As we saw in the example above, strict argument checking is normally done during execution. This default behaviour is not always the wanted one, it is possible in the API specification to give the matching function that one wants to use!
  ... #api_fun{ name = foo, arity = 2, matched = fun([X1, _Y1], [X2, _Y2]) -> X1 == X2 end }
  
Where the first list of arguments is the actual arguments used in the call of the mocked function and the second list of arguments comes from the language specification. A particular use-case is to check for equality of (some of) the arguments, there is a short-hand notation for this, namely:
  ... #api_fun{ name = bar, arity = 3, matched = [1,2] }
  

where we are going to match on the first two arguments of bar during execution.

Advanced usage - ?WILDCARD arguments

Sometimes it is not practical to fully describe the expected arguments of a function call. Therefore, it is possible to use ?WILDCARD as an argument in a mocking language specification. This means that any argument is accepted in this position. (Note, that the user has to be very careful when mixing ?WILDCARD's and custom made matching functions.) For example:
?EVENT(module1, fun1, [Arg1, ?WILDCARD, Arg3], Result)

Advanced usage - fallback module

In some cases it is preferable to only mock some functions from an API, and let the mocked module fall back to a fallback module for all other calls. This can be done by specifying a fallback in the #api_module{} record for the mocked module. For example given the API specification:
  api_spec() -> #api_spec{
    language = erlang,
    modules  = [
      #api_module{ name = modX, fallback = modY,
                   functions = [#api_fun{ name = f1, arity = 1},
                                #api_fun{ name = f2, arity = 2}] } ]}.
  

When used, calls to modX:f1/1 and modX:f2/2 are handled by the mocking framework, while calls to, for example, modX:g/2 would be forwarded to modY:g/2.

Note that it is possible to have a function that is mocked (i.e. is in the API specification) and also exists in the fallback module. For this case there is an attribute in the #api_fun-record: fallback.
  ... #api_fun{ name = foo, arity = 1, fallback = true }
  

If fallback = true the call is first handled by the mocking framework, and only if the call can not be handled (i.e. it is not expected to be called at that point) is it forwarded to the fallback module. If fallback = false the fallback is ignored for this function.

Known limitations

Data Types

api_arg_c()

api_arg_c() = #api_arg_c{type = undefined | atom() | string(), stored_type = atom() | string(), name = undefined | atom() | string() | {atom(), string()}, dir = in | out, buffer = false | true | {true, non_neg_integer()} | {true, non_neg_integer(), string()}, phantom = boolean(), matched = boolean(), default_val = no | string(), code = no | string()}

api_fun_c()

api_fun_c() = #api_fun_c{name = undefined | atom(), classify = undefined | any(), ret = atom() | api_arg_c(), args = [api_arg_c()], silent = false | {true, any()}}

api_fun_erl()

api_fun_erl() = #api_fun{name = undefined | atom(), classify = undefined | any(), arity = undefined | non_neg_integer(), fallback = boolean(), matched = [non_neg_integer()] | fun((any(), any()) -> boolean()) | all}

api_module()

api_module() = #api_module{name = undefined | atom(), fallback = undefined | atom(), functions = [api_fun_erl()] | [api_fun_c()]}

api_spec()

api_spec() = #api_spec{language = erlang | c, mocking = atom(), config = undefined | any(), modules = [api_module()]}

event()

event(Action, Result) = {event, Action, Result}

An event. When the language is 'run' an action is matched, and the result (of type Result) is returned.

lang()

lang(Action, Result) = seq(lang(Action, Result), lang(Action, Result)) | xalt(lang(Action, Result), lang(Action, Result)) | event(Action, Result) | repl(lang(Action, Result)) | par(lang(Action, Result), lang(Action, Result)) | perm([lang(Action, Result)]) | success

The type of a mocking language, parameterized on the Action and the Result.

par()

par(L1, L2) = {par, L1, L2}

Parallel composition of two languages.

perm()

perm(Ls) = {perm, Ls}

Permutation, the language accepts any non-interleaving execution of the languages in Ls.

repl()

repl(L) = {repl, L}

Replication of a language, corresponds rougly to (*) in a regular expression.

seq()

seq(L1, L2) = {seq, L1, L2}

Sequential composition of two languages.

xalt()

xalt(L1, L2) = {xalt, L1, L2} | {xalt, term(), L1, L2}

Choice between two languages. Possibly tagged with a term. All conditionals tagged with the same term must make the same choice.

Function Index

callouts_to_mocking/1Translate a callout-term, as return by eqc_component, to a mocking-language.
check_callouts/1Checks that the correct callouts have been made.
get_choices/0Returns the branches taken in any tagged choices.
get_trace/1Returns the trace, i.e.
get_trace_within/1Same as 'get_trace' but waits (for Timeout ms) for enough actions to match the current language.
info/0Information about what is currently being mocked.
init_lang/2Initialize the mocking language.
is_lang/1Recognizer for the mocking language.
merge_spec_modules/1Safely merge multiple specifications of the same module.
possible_next_events/2
provide_return/2Equivalent to provide_return(L, As, fun (X, Y) -> X == Y end).
provide_return/3Given a language and a sequence of actions tries to provide a list of the respective results.
small_step/2Equivalent to small_step(A, L, fun (X, Y) -> X == Y end).
small_step/3Implementation of the small step semantics for language.
start_mocking/1Equivalent to start_mocking(Spec, []).
start_mocking/2Initialize mocking, mocked modules are created as specified in the supplied callout specification.
stop_mocking/0Gracefully stop mocking.
trace_verification/2Equivalent to trace_verification(L, As, fun (X, Y) -> X == Y end).
trace_verification/3Similar to provide_return/3, but returns a boolean if the sequence of actions leads to an accepting state.
validate/1Validate a mocking language.

Function Details

callouts_to_mocking/1

callouts_to_mocking(CalloutTerm::term()) -> lang(_Action, _Result)

Translate a callout-term, as return by eqc_component, to a mocking-language.

check_callouts/1

check_callouts(MockLang) -> true | Error

Checks that the correct callouts have been made. Given a language, checks that the mocked functions called so far matches the language (the argument checking during execution is normaly relaxed, so errors could be undiscovered until here). Also checks that the language is in an "accepting state", i.e. a state where it is ok to stop.

get_choices/0

get_choices() -> [{term(), left | right}]

Returns the branches taken in any tagged choices.

get_trace/1

get_trace(APISpec::api_spec()) -> [_Action]

Returns the trace, i.e. the {Action, Result}-pairs seen so far for the current language.

get_trace_within/1

get_trace_within(Timeout::integer()) -> reference()

Same as 'get_trace' but waits (for Timeout ms) for enough actions to match the current language. The trace is returned in a message {trace, Ref, Trace} sent to the calling process, where Ref is the result of the call.

info/0

info() -> [{atom(), #linfo{}}]

Information about what is currently being mocked.

init_lang/2

init_lang(MockLang, APISpec) -> ok | no_return()

Initialize the mocking language. This sets the language describing how the mocked modules should behave. Calling this function also resets the collected trace. Some sanity checks are made, throws an error if the language and the callout specification are inconsistent.

is_lang/1

is_lang(MockLang::lang(_Action, _Result)) -> boolean()

Recognizer for the mocking language.

merge_spec_modules/1

merge_spec_modules(MockMods::[api_module()]) -> [api_module()] | no_return()

Safely merge multiple specifications of the same module.

possible_next_events/2

possible_next_events(E, WithOptional) -> any()

provide_return/2

provide_return(MockLang, Actions) -> [Result] | Error

Equivalent to provide_return(L, As, fun (X, Y) -> X == Y end).

provide_return/3

provide_return(MockLang, Actions, MatchFun) -> [Result] | Error

Given a language and a sequence of actions tries to provide a list of the respective results. If the actions does lead to an inconsistent state or if the sequence of actions does not lead to an accepting state a descriptive error is returned.

small_step/2

small_step(Step, MockLang) -> Res

Equivalent to small_step(A, L, fun (X, Y) -> X == Y end).

small_step/3

small_step(Step, MockLang, MatchFun) -> Res

Implementation of the small step semantics for language.

start_mocking/1

start_mocking(APISpec) -> Result

Equivalent to start_mocking(Spec, []).

start_mocking/2

start_mocking(APISpec, Components) -> Result

Initialize mocking, mocked modules are created as specified in the supplied callout specification. Each mocked function is merely calling do_action/3. If the specification contains an error, an error is thrown. Functions that are modeled by a component in Components are not mocked!

stop_mocking/0

stop_mocking() -> ok

Gracefully stop mocking. Shutdown the mocking server, will try to restore mocked modules to its pre-mocking version.

trace_verification/2

trace_verification(MockLang, Actions) -> boolean()

Equivalent to trace_verification(L, As, fun (X, Y) -> X == Y end).

trace_verification/3

trace_verification(MockLang, Actions, MatchFun) -> boolean()

Similar to provide_return/3, but returns a boolean if the sequence of actions leads to an accepting state.

validate/1

validate(MockLang::lang(_Action, _Result)) -> boolean()

Validate a mocking language. Checks that the mocking language is not ambiguous.


Generated by EDoc, Sep 8 2015, 11:11:11.