mORMot Open Source

ab

Administrator

From: France

Registered: 2010-06-21

Posts: 14,655

Website

Stubs and Mocks internal

A question from Stefan (by email):

first I have to admit that I haven't looked much into the mormot sourcecode yet.

I was wondering if you are using some framework kind of solution for the interface interception and if so I am interested in the internals that obviously only rely on "old" rtti
and how you are passing parameters (as you don't use TValue) and how it compares to the enhanced rtti solution (which most likely is way slower due to some overhead).

I just recently added an interception "namespace" to dsharp that is now used by my mock implementation (which before was based on direct usage of TVirtualInterface and TVirtualMethodInterceptor) to keep the interception internals at one central place and just use it at the other places (like mocks and aop).

We are mocking / intercepting only interfaces, not classes.
In fact, mORMot implementation uses its own optimized factory:
- TInterfaceFactory is handling interface RTTI and fake implementation class (with a global list for performance reasons);
- TInterfacedObjectFake is able to emulate a given interface;
- TServiceFactoryClient on the client side to create a fake interface calling the remote server with fast JSON marshalling;
- TServiceFactoryServer on the server side to call the actual service method - see ExecuteMethod().

We do not use TValue, but all the JSON serialization process included in SynCommons.pas and mORMot.pas.
This allows for instance to handle "out" and "var" kind of parameters during the mocking (whereas the default RTTI unit of newer Delphi does not allow this).
mORMot from scratch implementation also much faster than the current RTTI.pas implementation of Delphi 2010+, which does not cache anything and create the VTable and asm stubs for each instance...
See http://blog.synopse.info/post/2012/03/0 … on-details

The same TInterfaceFactory is used by TInterfaceStub/TInterfaceMock/TInterfaceMockSpy to create fake objects implementing a given interface.
All process is also using JSON. That is why it was so easy adding a "spy/verify" feature to our classes.

Use the source, Luke!

Stefan

Member

Registered: 2012-11-23

Posts: 26

Re: Stubs and Mocks internal

Thanks for explanation.

I made a quick test to see how it feels compared to my solution.

Here is what i wrote:

unit TestFoo;

interface

uses
  TestFramework, mORMot;

type
  TFooTestCase = class(TTestCase)
  private
    function ExecuteBar(var Params: TOnInterfaceStubExecuteParams): Boolean;
  published
    procedure TestBar;
  end;

implementation

uses
  Diagnostics,
  DSharp.Testing.Mock,
  SynCommons,
  SysUtils;

type
  {$M+}
  IFoo = interface
    ['{26D6D411-5994-4EEB-8EE1-9D6DCA8D5EA8}']
    function Bar(var i: Integer): Integer;
  end;

function TFooTestCase.ExecuteBar(var Params: TOnInterfaceStubExecuteParams): Boolean;
var
  P: PUTF8Char;
begin
  P := Pointer(Params.Params);
  Params.Params := IntToString(GetNextItemCardinal(P) + 1);
  Params.Result := '[42]';
  Result := True;
end;

{ TFooTestCase }

procedure TFooTestCase.TestBar;
const
  CallCount = 100000;
var
  foo: IFoo;

  fooMock: Mock<IFoo>;

  sw: TStopwatch;
  i, k: Integer;
begin
  k := 0;

  sw := TStopwatch.StartNew;

  TInterfaceStub.Create(TypeInfo(IFoo), foo).Executes('Bar', ExecuteBar);

  for i := 1 to CallCount do
  begin
    CheckEquals(42, foo.Bar(k));
    CheckEquals(i, k);
  end;

  Status(Format('mORMot: %d ms', [sw.ElapsedMilliseconds]));

  // -------------------------------------------

  k := 0;

  sw := TStopwatch.StartNew;

  fooMock.WillExecute(
    function(var Args: array of TValue): TValue
    begin
      Result := 42;
      Args[0] := Args[0].AsInteger + 1;
    end).Any.WhenCallingWithAnyArguments.Bar(k);
  foo := fooMock;

  for i := 1 to CallCount do
  begin
    CheckEquals(42, foo.Bar(k));
    CheckEquals(i, k);
  end;

  Status(Format('DSharp: %d ms', [sw.ElapsedMilliseconds]));
end;

initialization
  RegisterTest(TFooTestCase.Suite);
  
end.

You said, RTTI cannot handle var and out parameters. That is not true (at least not with my implementation).

This code does not run it raises "EInterfaceFactoryException at  $0052CF3B Invalid "IFoo" interface call: returned item".

mORMot is approx 4-5 times faster than the DSharp implementation.

While it's great that mORMot runs on older Delphi versions its handling is really ugly imho as you can see in the ExecuteBar method (if it would work).
I think the TOnInterfaceStubExecuteParams should provide the params and the result in an easier to use way (like Variants or something like that).

Last edited by Stefan (2012-12-07 10:03:10)

ab

Administrator

From: France

Registered: 2010-06-21

Posts: 14,655

Website

Re: Stubs and Mocks internal

Your implementation does not work since all returned items have to be part of the result array.
See what documentation says about Result member:

/// parameters used by TInterfaceStub.Executes() events callbacks
  TOnInterfaceStubExecuteParams = record
    /// the stubbing / mocking generator
    Sender: TInterfaceStub;
    /// pointer to the method which is to be executed
    Method: PServiceMethod;
    /// incoming parameters array, i.e. defined as const or var
    Params: RawUTF8;
    /// a custom message, defined at TInterfaceStub.Executes() definition
    EventParams: RawUTF8;
    /// outgoing values array, i.e. defined as var, out or function result
    Result: RawUTF8;
  end;

Something like this:

function TFooTestCase.ExecuteBar(var Params: TOnInterfaceStubExecuteParams): Boolean;
begin
  Params.Result := JSONEncodeArrayOfConst([GetCardinal(pointer(Params.Params))+1,42],true);
  // or Params.Result := FormatUTF8('[%,%]'],[GetCardinal(pointer(Params.Params))+1,42]);
  result:= true;
end;

and do not touch Params.Params.

Executes() is pretty basic, and potentially expect "ugly" code (as you wrote), i.e. manual marshalling into a JSON array.
Using variants or such will make it slower, but could be easier to work with.
Is JSONEncodeArrayOfConst() so "ugly"?

I've changed TOnInterfaceStubExecuteParams to have a method, to make it easier to use, and enhanced the documentation:

procedure TOnInterfaceStubExecuteParams.Returns(const Values: array of const);
begin
  Result := JSONEncodeArrayOfConst(Values);
end;

Perhaps using a property array and variants could help, here...
Nice idea.
We would probably change TOnInterfaceStubExecuteParams from a record/object to a regular class.

Stefan

Member

Registered: 2012-11-23

Posts: 26

Re: Stubs and Mocks internal

Is JSONEncodeArrayOfConst() so "ugly"?

"Ugly" in the sense of I cannot handle parameters as I would do it in the real implementation and it easily can break because of handling them as strings.

Change the var to const for example. Suddenly i is not part of the result anymore and the method is returning the wrong value.

Btw when just testing TVirtualInterface vs TInterfaceStub (so just raw Rtti.pas stuff with no DSharp.Interception) on a parameterless method the performance is very close. Rtti implementation gets slower the more parameters are used - most likely because of copying around TValue.

Last edited by Stefan (2012-12-07 12:14:19)

ab

Administrator

From: France

Registered: 2010-06-21

Posts: 14,655

Website

Re: Stubs and Mocks internal

Change the var to const for example.

... won't compile.

I'll make two versions of the Execute() callbacks.
One with JSON arays (faster), and one with variant array properties (a bit slower).

Btw when just testing TVirtualInterface vs TInterfaceStub (so just raw Rtti.pas stuff with no DSharp.Interception) on a parameterless method the performance is very close.

I suspect you don't recreate the TVirtualInterface class before each call.
Just try to do benchmark whole fake class instance creation, and you will see a bigger time difference in favor of TInterfaceStub.
Execution should be more or less the same. What is pretty slow is the RTTI method information retrieval and the interceptor as implemented by Embarcadero.

Stefan

Member

Registered: 2012-11-23

Posts: 26

Re: Stubs and Mocks internal

... won't compile.

I was talking about the IFoo.Bar signature. If you have a unit test which just includes the interface definition which has been changed, it will compile for sure.

I suspect you don't recreate the TVirtualInterface class before each call.
Just try to do benchmark whole fake class instance creation, and you will see a bigger time difference in favor of TInterfaceStub.
Execution should be more or less the same. What is pretty slow is the RTTI method information retrieval and the interceptor as implemented by Embarcadero.

Yes, I was just referring to the execution. The caching of the rtti information could be done in my implementation aswell but currently I am more focused on the execution speed (since I am using this whole mechanism for AOP provided by a DI container).

Last edited by Stefan (2012-12-07 15:04:39)

ab

Administrator

From: France

Registered: 2010-06-21

Posts: 14,655

Website

Re: Stubs and Mocks internal

I am using this whole mechanism for AOP provided by a DI container

This is indeed a mandatory pattern.
mORMot has such DI container in its interface-based service, with the aspect of client-server access, and nice including security features.
You can use the TServiceFactory instances as service container on each side (client or server), and let it manage life time of each instance.

If your dependency injection pattern is to create a new class instance at every time (which is a potential behavior), having fast creation of fake instances does make sense, especially as in mORMot when you need to deal with it on the server side, with potentially a lot of instances to be created.
It is always the choice between fast creation/release cycles and in-memory caching. mORMot can do both, but I suspect that fast creation/release does make sense on server side.

In mORMot, dependency injection is not automated at instance creation time.
It is up to the constructor to provide the injected interfaces manually, as parameters.
Some axis of improvement is perhaps needed here in the future, to make it easier. But at constructor level, we begin to lack of RTTI in older versions of Delphi.

ab

Administrator

From: France

Registered: 2010-06-21

Posts: 14,655

Website

Re: Stubs and Mocks internal

I've added an alternative way of implementing callbacks, using variants, as you propose.
I suspect this implementation won't be much slower than the initial one. Probably faster than TValue based version.

See http://synopse.info/fossil/info/b61f3fb992

Stefan

Member

Registered: 2012-11-23

Posts: 26

Re: Stubs and Mocks internal

Looks better now however I still prefer an extra property for the Result of a function (instead of having it be the last item of the Output array) since changing the signature of the method can break existing test code just because Output[1] is not the result anymore but Output[2] for example.

Also why not return the values of var/out parameters the same value it was passed in (instead of having them separate in Input and Output which does not have matching indizes)?

ab

Administrator

From: France

Registered: 2010-06-21

Posts: 14,655

Website

Re: Stubs and Mocks internal

Also why not return the values of var/out parameters the same value it was passed in (instead of having them separate in Input and Output which does not have matching indizes)?

It was coded as such:
- For safety, to ensure that your intput values are read/only and output values are write/only;
- For small speed benefit and easier to code / test at implementation level.

We may easily add named parameters index, to make it easier to work with.

ab

Administrator

From: France

Registered: 2010-06-21

Posts: 14,655

Website

Re: Stubs and Mocks internal

I've introduced named parameters in TOnInterfaceStubExecuteParamsVariant kind of Executes() callback.
This is the new recommended implementation pattern of Executes() fluent interface.
See http://synopse.info/fossil/info/f2408976e7

Documentation has been updated:

Stubbing via a custom delegate or callback

In some cases, it could be very handy to define a complex process for a given method, without the need of writing a whole implementation class.

A delegate or event callback can be specified to implement this process, with three parameters marshaling modes:
- Via some Named[] variant properties (which are the default for the Ctxt callback parameter) - the easiest and safest to work with;
- Via some Input[] and Output[] variant properties;
- Directly as a JSON array text (the fastest, since native to the mORMot core).

Let's emulate the following behavior:

function TServiceCalculator.Subtract(n1, n2: double): double;
begin
  result := n1-n2;
end;

Delegate with named variant parameters

You can stub a method using a the Named[] variant arrays as such:

TInterfaceStub.Create(TypeInfo(ICalculator),ICalc).
    Executes('Subtract',IntSubtractVariant);
  (...)
  Check(ICalc.Substract(10.5,1.5)=9);

The callback function can be defined as such:

procedure TTestServiceOrientedArchitecture.IntSubtractVariant(
  Ctxt: TOnInterfaceStubExecuteParamsVariant);
begin
  Ctxt['result'] := Ctxt['n1']-Ctxt['n2'];
end;

That is, callback shall use Ctxt[''] property to access the parameters and result as variant values.

In fact, we use the Ctxt.Named[] default property, so it is exactly as the following line:

Ctxt.Named['result'] := Ctxt.Named['n1']-Ctxt.Named['n2'];

If the execution fails, it shall execute Ctxt.Error() method with an associated error message to notify the stubbing process of such a failure.

Using named parameters has the advantage of being more explicit in case of change of the method signature (e.g. if you add or rename a parameter). It should be the preferred way of implementing such a callback, in most cases.

Delegate with indexed variant parameters

There is another way of implementing such a callback method, directly by using the Input[] and Output[] indexed properties. It should be (a bit) faster to execute:

procedure TTestServiceOrientedArchitecture.IntSubtractVariant(
  Ctxt: TOnInterfaceStubExecuteParamsVariant);
begin
  with Ctxt do
    Output[0] := Input[0]-Input[1]; // result := n1-n2
end;

Just as with TOnInterfaceStubExecuteParamsJSON implementation, Input[] index follows the exact order of const and var parameters at method call, and Output[] index follows the exact order of var and out parameters plus any function result.

That is, if you call:

function Subtract(n1,n2: double): double;
...
  MyStub.Substract(100,20);

you have in TOnInterfaceStubExecuteParamsJSON:

Ctxt.Params = '100,20.5'; // at method call
Ctxt.Result = '[79.5]';   // after Ctxt.Returns([..])

and in the variant arrays:

Ctxt.Input[0] = 100;      // =n1 at method call
Ctxt.Input[1] = 20.5;     // =n2 at method call
Ctxt.Output[0] = 79.5;    // =result after method call

In case of additional var or out parameters, those should be added to the Output[] array before the last one, which is always the function result.

If the method is defined as a procedure and not as a function, of course there is no last Output[] item, but only var or out parameters.

Delegate with JSON parameters

You can stub a method using a JSON array as such:

TInterfaceStub.Create(TypeInfo(ICalculator),ICalc).
    Executes('Subtract',IntSubtractJSON);
  (...)
  Check(ICalc.Substract(10.5,1.5)=9);

The callback shall be defined as such:

procedure TTestServiceOrientedArchitecture.IntSubtractJSON(
  Ctxt: TOnInterfaceStubExecuteParamsJSON);
var P: PUTF8Char;
begin // result := n1-n2
  P := pointer(Ctxt.Params);
  Ctxt.Returns([GetNextItemDouble(P)-GetNextItemDouble(P)]);
  // Ctxt.Result := '['+DoubleToStr(GetNextItemDouble(P)-GetNextItemDouble(P))+']';
end;

That is, it shall parse incoming parameters from Ctxt.Params, and store the result values as a JSON array in Ctxt.Result.

Input parameter order in Ctxt.Params follows the exact order of const and var parameters at method call, and output parameter order in Ctxt.Returns([]) or Ctxt.Result follows the exact order of var and out parameters plus any function result.

Accessing the test case when mocking

In case of mocking, you may add additional verifications within the implementation callback, as such:

TInterfaceMock.Create(TypeInfo(ICalculator),ICalc,self).
    Executes('Subtract',IntSubtractVariant,'toto');
(...)
procedure TTestServiceOrientedArchitecture.IntSubtractVariant(
  Ctxt: TOnInterfaceStubExecuteParamsVariant);
begin
  Ctxt.TestCase.Check(Ctxt.EventParams='toto');
  Ctxt['result'] := Ctxt['n1']-Ctxt['n2'];
end;

Here, an additional callback-private parameter containing 'toto' has been specified at TInterfaceMock definition. Then its content is checked on the associated test case via Ctxt.Sender instance. If the caller is not a TInterfaceMock, it will raise an exception to use the Ctxt.TestCase property.

Thanks for your feedback!
Current implementation sounds easier and safer to work with, now.