Wednesday 19 July 2017

Introduction

Problem

Manual testing of software systems is time consuming and expensive. As a system grows it becomes unfeasible to manually test the system after each change.

Solution

Use software that can execute a piece of functionality and check the result, i.e. a testing framework. We now have a fast way of validating that new changes have not broken existing functionality.

This allows us to change faster and at a lower cost.

Note: this is the bottom line in software, every technique we learn targets getting things done faster and/or cheaper.

How does it work?






The main component of any testing framework is the test runner, a component that is responsible for taking a series of tests, running them and gathering the results. Your responsibility in this scenario is to write a test that does something with your system and makes some assertion about the change that occurred.

This is easiest to understand by working through an example.

Writing a simple test with NUnit

My choice of testing framework for this example is NUnit. There are no shortage of choice when it comes to testing frameworks, but I prefer NUnit because it is one of the oldest, simplest and most standard available.

Here's an example of an NUnit test.

namespace Trading
{
  using NUnit.Framework;
  [TestFixture]
  public class how_trading_affects_the_pnl
  {
    [Test]
    public void buying_increases_the_pnl()
    {
      var tradingAccount = new Account { PnL = 0 };

      tradingAccount.Buy(200, "AAPL");

      Assert.That(tradingAccount.PnL, Is.EqualTo(200));
    }

    [Test]
    public void selling_decreases_the_pnl()
    {
      var tradingAccount = new Account { PnL = 0 };

      tradingAccount.Sell(200, "AAPL");

      Assert.That(tradingAccount.PnL, Is.EqualTo(-200));
    }
  }
}

The automated test structure in C# consists of a 'TestFixture' describing the area that we are writing tests around, which in this case is 'how trading affects the PnL'. Each 'TestFixture' is made up of multiple 'Test's that are relevant to that area.


This is the typical structure of automated tests. Seeing as we are using C# and everything must be a class, we model this structure with classes as the TestFixture and methods as the Test's.

You can see that the class and methods have TestFixture and Test attributes above them. This is how the NUnit test runner identifies tests, without these attributes the tests would be ignored.

Structure of a test

You are free to create your tests in any way that C# allows, but it's handy to have some kind of guide in how to structure them. The most typical is 'Arrange, act, assert' (AAA). The idea with this guide is to keep your tests small, focused and easy to read.
  • Arrange - perform all of the setup that you need for the test.
  • Act - perform the action that you are testing
  • Assert - check that the result of performing the action is what you expect
This is how one of the previous tests looks in terms of the AAA guide. I've commented the lines purely for clarity.

[Test]
public void selling_decreases_the_pnl()
{
  // arrange - setup up the account
  var tradingAccount = new Account { PnL = 0 };

  // act - sell the stock
  tradingAccount.Sell(200, "VOD LN");

  // assert - the PnL should have gone down
  Assert.AreEqual(tradingAccount.PnL, -200);
}

The arranging and acting part of the test should be very familiar to anyone that has written some C# code, but the assertion stands out as being unique to unit tests. The assertion acts as a barrier, where code that can pass the test gets through and code that fails the test gets stops in it's tracks.
The assertion used above covers one of the most common cases, equality, but there are many other useful ways of asserting. It's definitely worthwhile getting familiar with the assertion techniques available in NUnit, because a well written assert really improves the readability of the test.

For example, the assertion above will work fine, but there is another style that uses constraints that I think improves readability. The above assertions written using the constraints style would look like this:

   Assert.That(tradingAccount.PnL, Is.EqualTo(-200));

Side Note: Guides in automated testing techniques

Unit testing is full of guides like 'Arrange, Act, Assert'. They are very useful, especially when you are beginning or stuck, but it's important to remember that they are only a guides and not strict rules.

Any guide like this has a underlying goals and it's more important to understand them. In this case it's that the test is a cohesive unit and easy to read. If that's been achieved then we have reached our goal.

Some more reading

This post should have given you an overview of the why's and what's of automated testing. Your goal now should be to get more familiar with NUnit. You can find all you'll need to know in the NUnit documentation here http://nunit.org/docs/2.6/getStarted.html.

 

 

 









Choice of testing frameworks

There are no shortage of choices when it comes to testing frameworks (see: wikipedia). The one I would recommend is NUnit, which is part of the xUnit family of unit testing frameworks. These frameworks offer a simple, minimal interface that is ideal for almost all of your testing needs.

Other frameworks available trade this simplicity for a rich set of features. These frameworks are moving more into the area of Specification by Example and BDD. That's outside the scope of this lesson, but still worth looking at if you want a full understanding of the tools available. 

Exercise #1 - Create a testing framework

With so many free testing frameworks available so you wouldn't normally need to create one from scratch. But, it's a good way to understand how they work so we're going to create a really basic one. Use this specification to create the framework:
  1. it should be able to run a test
  2. it should provide the ability to assert equality
  3. it should give back the result 'test passed' when a test passes
  4. it should give back the result 'test failed' when a test fails

You need some way to verify that your framework is working correctly so I'd like you to create some tests around it.
Assertions
 - assert correct value is returned
 - assert that correct value is passed to another object

Structure of a test

we will use NUnit, here's a quickstart http://nunit.org/docs/2.2/quickStart.html

Here's an example with NUnit that shows the typical structure of a unit test.

Download and install NUnit.
Using NUnit, Create a test for points 3 and 4 of the above specification. 
Bonus# write these tests using the testing framework you just wrote.

See Also

Good unit testing blog posts?

Also
 - why unit test? discourages overly coupled design
  - naming tests? don't underestimate the importance
  - value of tests? incorporates when should I test etc.
  - start with a failing test
  - arrange act assert
  - simplicity
        - test one thing
        - avoid inheritance (C# is primarily an OO language, but just because we are restricted to C# for writing the tests does not mean that we have to use OO. In fact, OO is not suited to writing tests because it introduces a lot of unnecessary complexity, such as coupling)
        - avoid coupling; make each test a cohesive unit
  - types of tests, unit vs integration
        - try not to care
        - don't get misled by the name (NUnit), it can be used for integration tests also
        - try not to think about the implementation that you are testing, focus instead on the domain concept
  - the wars, anti unit test
        - https://techbeacon.com/1-unit-testing-best-practice-stop-doing-it
        - https://rbcs-us.com/documents/Why-Most-Unit-Testing-is-Waste.pdf
        - https://rbcs-us.com/documents/Segue.pdf
        - https://henrikwarne.com/2014/09/04/a-response-to-why-most-unit-testing-is-waste/#comment-12875
        - end result (use your own head, listen to all sides and make a decision, there is no answer that is right all of the time)
- the scope of tests; testing from a high level vs testing from a low level
 - high level means refactoring is easier, but test can be more complex to set up

  - Loose coupling and testability go hand in hand
  - test the domain, not the class; you can't write unit tests unless you understand the domain
  - https://en.wikipedia.org/wiki/Unit_testing
  - http://jamesgolick.com/2007/8/22/we-dont-write-tests-there-just-isnt-time-for-luxuries.html
  - https://exubero.com/junit/anti-patterns/
  - https://www.codeproject.com/Articles/5772/Advanced-Unit-Test-Part-V-Unit-Test-Patterns
  - https://henrikwarne.com/2014/02/19/5-unit-testing-mistakes/
  - maintainability
        - changes to constructor or method signature forces a change to all tests that call it
 - https://dzone.com/articles/writing-your-first-unit-tests
- https://martinfowler.com/bliki/UnitTest.html
- https://dannorth.net/introducing-bdd/

- http://csharpindepth.com/CodeFormatterTool.aspx

No comments:

Post a Comment

Fakes

Faking it As well as asserting, one technique that is particular to test code is the use of fakes. Fakes generally fall into two categorie...