Thursday 11 January 2018

Fakes

Faking it

As well as asserting, one technique that is particular to test code is the use of fakes. Fakes generally fall into two categories based on what we need from them. We either want to fake data coming into the system (stub) or we want to fake a component of the system to see data on the way out (mock).

There are lots of reasons that we may need to do this, such as:

  • Consistency; we need to ensure that the data coming into the system is always the same. If we rely on an external source like a file or database there is always a risk on the data changing
  • Performance; the sooner we get feedback from a test the better. Tests that access an external resource for data will be much slower than a test that initializes the data itself
  • Simplicity; creating the input data within the test makes it visible to anyone reading the test and therefore easier to understand
  • Side Effects: the whole point of doing something with software in production is to produce a side effect, something like inserting a row into a database table or sending an order to an exchange. We definitely do not want to do this each time we run a test so we will find some way to fake the side effect
  • Autonomy; Having the test access an external resource creates a coupling to that resource. This is a good thing in a production system, but bad for a test. You can think of a test as a small executable that must always run in the same way. We achieve this by making each test as autonomous as possible.

Stubs

A stub is a way of faking data coming into the system.

In our example of tracking changes to a PnL, lets say that each account is only allowed to trade certain instruments. These instruments would normally be stored in a database and loaded into the account when it is initialised. Wecan avoid all of that hassle using a stub.

First of all we need a way for the account to access the allowed instruments, so we will give it a class that can do this.
 
    public class Account
    {
        public decimal PnL { get; private set; }
        private readonly List<string> _allowedInstruments;

        public Account(ProdInstrumentLoader instrumentLoader)
        {
            _allowedInstruments = instrumentLoader.Load(this);
        }

        public void Sell(int quantity, string bbg)
        {
            if (!_allowedInstruments.Contains(bbg))
            {
                throw new Exception($"Invalid inistrument: {bbg}");
            }
        }
    }

    public class ProdInstrumentLoader
    {
        public List<string> Load(Account acc)
        {
            // loads from database
        }
    }

This still won't work because we have coupled the class to loading the instruments from the database, so we need a way to replace ProdInstrumentLoader with a stub. We can do this by extracting an interface and coupling the Account class to the instrface instead.


    public class Account
    {
        public decimal PnL { get; private set; }
        private readonly List<string> _allowedInstruments;

        public Account(InstrumentLoader instrumentLoader)
        {
            _allowedInstruments = instrumentLoader.Load(this);
        }

        public void Sell(int quantity, string bbg)
        {
            if (!_allowedInstruments.Contains(bbg))
            {
                throw new Exception($"Invalid inistrument: {bbg}");
            }
        }
    }

    public class ProdInstrumentLoader : InstrumentLoader
    {
        public List<string> Load(Account acc)
        {
            // loads from database
        }
    }

    public interface InstrumentLoader
    {
        List<string> Load(Account acc);
    }

Now we can create a stub version of the instrument loader that will be suitable for our test. Our stub implementation will take whatever list of instrument it is initialised with and always return them when Load is called.

public class StubInstrumentLoader : InstrumentLoader
{
    private List<string> _instruments;

    public StubInstrumentLoader(List<string> instruments)
    {
      _instruments = instruments;
    }

public List<string> Load(Account acc)
{
       return _instruments;
}
}

The test can now load the stubbed instrument instead of hitting the database.

[Test]
public void selling_decreases_the_pnl()
{
  var instrumentLoader = new StubInstrumentLoader(new List<Instrument> { "VOD LN" });
  var tradingAccount = new Account(instrumentLoader) { PnL = 0 };

  tradingAccount.Sell(200, "VOD LN");

  Assert.That(tradingAccount.PnL, Is.EqualTo(-200));
}

Mocks

There are lots of different ways that we can assert (see xxx for an exhaustive list), but they all fall into one of two categories.

One, the most common, is checking that a value is what we expect. The other is checking that a method was called in the way that we expect and we do this with a technique called Mocking.

Continuing with our previous example, let's say that part of the requirement for creating an order is to publish its details. This will involve our account class calling a service with the order details.

public class Account
{
public decimal PnL { get; private set; }
private OrderPublisher _publisher;

public Account(OrderPublisher publisher)
{
_publisher = publisher;
}

public void Sell(int quantity, string bbg)
{
...
_publisher.NewOrder(NewOrder(quantity, bbg));
}
}

We need some way to verify that the publisher published a NewOrder with the details that we gave. We can achieve this by creating a mock version of OrderPublisher. Our mock version of the publisher doesn't actually do any publishing, instead it records whatever information is necessary for out verification.

public class MockOrderPublisher : OrderPublisher
{
public List<NewOrder> Published { get;set; } = new List<NewOrder>();

public void NewOrder(NewOrder order)
{
Published.Add(order);
}
}

We can now verify the details of the published order in a test.

[Test]
public void ensure_that_order_details_are_published()
{
    var publisher = new MockOrderPublisher
var tradingAccount = new Account() { PnL = 0 };

tradingAccount.Buy(200, "AAPL");

    Assert.That(publisher, Contains(1));

    Assert.That(publisher.Published[0].Quantity, Is.EqualTo(200));
    Assert.That(publisher.Published[0].Bbg, Is.EqualTo("AAPL"));
}

That's the basic setup for a mock. You need some way of replacing the real functionality with something that acts like it. We used a custom rolled mock here, but there are no shortage of frameworks in .Net to do this for you, Moq being one of the most popular.

 Side Note: This section comes with a caveat. 

Mocking is one of those things that looks great on the face of it, but in practice its more trouble that it's worth. The example above is probably one of the few situations that I would use a mocking framework, when its way out at the boundaries of the system. If you find yourself using mocks regularly you will more that likely end up with an overly complex system and tests. So if you can avoid it, do, and if you have to do it try to avoid using a framework.



Wednesday 19 July 2017

Introduction

Problem

Manual testing of software systems is time consuming and expensive. As a system grows it becomes unfeasible to manually test the system after each change.

Solution

Use software that can execute a piece of functionality and check the result, i.e. a testing framework. We now have a fast way of validating that new changes have not broken existing functionality.

This allows us to change faster and at a lower cost.

Note: this is the bottom line in software, every technique we learn targets getting things done faster and/or cheaper.

How does it work?






The main component of any testing framework is the test runner, a component that is responsible for taking a series of tests, running them and gathering the results. Your responsibility in this scenario is to write a test that does something with your system and makes some assertion about the change that occurred.

This is easiest to understand by working through an example.

Writing a simple test with NUnit

My choice of testing framework for this example is NUnit. There are no shortage of choice when it comes to testing frameworks, but I prefer NUnit because it is one of the oldest, simplest and most standard available.

Here's an example of an NUnit test.

namespace Trading
{
  using NUnit.Framework;
  [TestFixture]
  public class how_trading_affects_the_pnl
  {
    [Test]
    public void buying_increases_the_pnl()
    {
      var tradingAccount = new Account { PnL = 0 };

      tradingAccount.Buy(200, "AAPL");

      Assert.That(tradingAccount.PnL, Is.EqualTo(200));
    }

    [Test]
    public void selling_decreases_the_pnl()
    {
      var tradingAccount = new Account { PnL = 0 };

      tradingAccount.Sell(200, "AAPL");

      Assert.That(tradingAccount.PnL, Is.EqualTo(-200));
    }
  }
}

The automated test structure in C# consists of a 'TestFixture' describing the area that we are writing tests around, which in this case is 'how trading affects the PnL'. Each 'TestFixture' is made up of multiple 'Test's that are relevant to that area.


This is the typical structure of automated tests. Seeing as we are using C# and everything must be a class, we model this structure with classes as the TestFixture and methods as the Test's.

You can see that the class and methods have TestFixture and Test attributes above them. This is how the NUnit test runner identifies tests, without these attributes the tests would be ignored.

Structure of a test

You are free to create your tests in any way that C# allows, but it's handy to have some kind of guide in how to structure them. The most typical is 'Arrange, act, assert' (AAA). The idea with this guide is to keep your tests small, focused and easy to read.
  • Arrange - perform all of the setup that you need for the test.
  • Act - perform the action that you are testing
  • Assert - check that the result of performing the action is what you expect
This is how one of the previous tests looks in terms of the AAA guide. I've commented the lines purely for clarity.

[Test]
public void selling_decreases_the_pnl()
{
  // arrange - setup up the account
  var tradingAccount = new Account { PnL = 0 };

  // act - sell the stock
  tradingAccount.Sell(200, "VOD LN");

  // assert - the PnL should have gone down
  Assert.AreEqual(tradingAccount.PnL, -200);
}

The arranging and acting part of the test should be very familiar to anyone that has written some C# code, but the assertion stands out as being unique to unit tests. The assertion acts as a barrier, where code that can pass the test gets through and code that fails the test gets stops in it's tracks.
The assertion used above covers one of the most common cases, equality, but there are many other useful ways of asserting. It's definitely worthwhile getting familiar with the assertion techniques available in NUnit, because a well written assert really improves the readability of the test.

For example, the assertion above will work fine, but there is another style that uses constraints that I think improves readability. The above assertions written using the constraints style would look like this:

   Assert.That(tradingAccount.PnL, Is.EqualTo(-200));

Side Note: Guides in automated testing techniques

Unit testing is full of guides like 'Arrange, Act, Assert'. They are very useful, especially when you are beginning or stuck, but it's important to remember that they are only a guides and not strict rules.

Any guide like this has a underlying goals and it's more important to understand them. In this case it's that the test is a cohesive unit and easy to read. If that's been achieved then we have reached our goal.

Some more reading

This post should have given you an overview of the why's and what's of automated testing. Your goal now should be to get more familiar with NUnit. You can find all you'll need to know in the NUnit documentation here http://nunit.org/docs/2.6/getStarted.html.

 

 

 









Choice of testing frameworks

There are no shortage of choices when it comes to testing frameworks (see: wikipedia). The one I would recommend is NUnit, which is part of the xUnit family of unit testing frameworks. These frameworks offer a simple, minimal interface that is ideal for almost all of your testing needs.

Other frameworks available trade this simplicity for a rich set of features. These frameworks are moving more into the area of Specification by Example and BDD. That's outside the scope of this lesson, but still worth looking at if you want a full understanding of the tools available. 

Exercise #1 - Create a testing framework

With so many free testing frameworks available so you wouldn't normally need to create one from scratch. But, it's a good way to understand how they work so we're going to create a really basic one. Use this specification to create the framework:
  1. it should be able to run a test
  2. it should provide the ability to assert equality
  3. it should give back the result 'test passed' when a test passes
  4. it should give back the result 'test failed' when a test fails

You need some way to verify that your framework is working correctly so I'd like you to create some tests around it.
Assertions
 - assert correct value is returned
 - assert that correct value is passed to another object

Structure of a test

we will use NUnit, here's a quickstart http://nunit.org/docs/2.2/quickStart.html

Here's an example with NUnit that shows the typical structure of a unit test.

Download and install NUnit.
Using NUnit, Create a test for points 3 and 4 of the above specification. 
Bonus# write these tests using the testing framework you just wrote.

See Also

Good unit testing blog posts?

Also
 - why unit test? discourages overly coupled design
  - naming tests? don't underestimate the importance
  - value of tests? incorporates when should I test etc.
  - start with a failing test
  - arrange act assert
  - simplicity
        - test one thing
        - avoid inheritance (C# is primarily an OO language, but just because we are restricted to C# for writing the tests does not mean that we have to use OO. In fact, OO is not suited to writing tests because it introduces a lot of unnecessary complexity, such as coupling)
        - avoid coupling; make each test a cohesive unit
  - types of tests, unit vs integration
        - try not to care
        - don't get misled by the name (NUnit), it can be used for integration tests also
        - try not to think about the implementation that you are testing, focus instead on the domain concept
  - the wars, anti unit test
        - https://techbeacon.com/1-unit-testing-best-practice-stop-doing-it
        - https://rbcs-us.com/documents/Why-Most-Unit-Testing-is-Waste.pdf
        - https://rbcs-us.com/documents/Segue.pdf
        - https://henrikwarne.com/2014/09/04/a-response-to-why-most-unit-testing-is-waste/#comment-12875
        - end result (use your own head, listen to all sides and make a decision, there is no answer that is right all of the time)
- the scope of tests; testing from a high level vs testing from a low level
 - high level means refactoring is easier, but test can be more complex to set up

  - Loose coupling and testability go hand in hand
  - test the domain, not the class; you can't write unit tests unless you understand the domain
  - https://en.wikipedia.org/wiki/Unit_testing
  - http://jamesgolick.com/2007/8/22/we-dont-write-tests-there-just-isnt-time-for-luxuries.html
  - https://exubero.com/junit/anti-patterns/
  - https://www.codeproject.com/Articles/5772/Advanced-Unit-Test-Part-V-Unit-Test-Patterns
  - https://henrikwarne.com/2014/02/19/5-unit-testing-mistakes/
  - maintainability
        - changes to constructor or method signature forces a change to all tests that call it
 - https://dzone.com/articles/writing-your-first-unit-tests
- https://martinfowler.com/bliki/UnitTest.html
- https://dannorth.net/introducing-bdd/

- http://csharpindepth.com/CodeFormatterTool.aspx

Fakes

Faking it As well as asserting, one technique that is particular to test code is the use of fakes. Fakes generally fall into two categorie...