HowTo – NUnit – Features

Source Code

All source code can be found on GitHub here.

My cheat sheet for NUnit can be found here.

This is part of my HowTo in .NET seriies. An overview can be seen here,

Intro

NUnit is a unit test framework for .net; it is very mature and contains lots of features when compared to XUnit.

This post is part of a 3 part post on NUnit:

  1. NUnit Features
  2. Assertions
  3. Fluent Assertions

The Basic Test

Any method which is decorated with the [Test] attribute which sits within a class decorated with the [TestFixture] attribute should be automatically picked up by the test runner: Resharper, NUnit-MonoDevelop etc.

[TestFixture]
public cass MyTests
{
	[Test]
	public void BasicTest()
	{
		Assert.IsTrue (true);

	}
}

The test above will pass without error as the assertion condition is satisfied; true is true. The test below fails the assertion and as such throws an exception. The test runner handles the exception marks the test as failed and continues execution of the next test; all tests will be run independently of the execution of any other tests.

[Test]
public void BasicTest()
{
	Assert.IsTrue (false);

}

TestFixture

The [TestFixture] attribute decorates a class which contains one or more tests. We can also provide a category and description to annotate our collection of tests.

[TestFixture(), Category("Example Tests"), Description("Some examples tests")]
public class Test

Ignore & Explicit

Tests can be marked as Ignored and Explicit. Both tests will not be run by the test runner though Explicit tests can be run when explicitly run; ie you run them yourself.

The provided explanation is optional and is displayed in the test runner as the reason why the test was ignored.

[Test, Explicit("Test has to be run explicitly as it does not like mondays!")]
public void ExplicitAttributeTest()
{
}

[Test, Ignore("This test is not to be run!")]
public void IgnoreAttribueTest()
{
}

Timeout & MaxTimeAttribute

Both attributes allow configuration of a max time in milliseconds.

Timeout will stop and fail a test when the test has been running for the amount of configured time.
MaxTime will fail the test once the time has been reached but the test will be aloud to complete.

[Test, MaxTimeAttribute(2000)]
public void MaxtimeAttributeTest() { }

[Test, Timeout(200)]
public void TimeOutTest() { }

Categorising Tests

The property flag can be used to categorise the test within the result output XML.

[Test, Property("ImportantTest", "Very")]
public void PropertyAttributeTest() {}

You can also define your own properties to ensure consistency. We can create an enumerator as an argument to this attribute and don’t forget you can have multiple attributes.

In the example below we define our own attribute subclassing PropertyAttribute which takes a defined enumerator. We the decorate a the test with this attribute and an enum value.


[AttributeUsage(AttributeTargets.Method, AllowMultiple = false)]
public class CustomPropertyAttribute : PropertyAttribute
{
	public CustomPropertyAttribute( CustomPropertyValue value ) : base( "Custom", value.ToString() )
	{
	}
}

public enum CustomPropertyValue
{
	One,
	Two,
}

[Test, CustomPropertyAttribute(CustomPropertyValue.One)]
public void CustomPropertyAttributeTest() {}

SetUp & TearDown

[TestFixtureSetUp] marks a method which is called once prior to all unit tests whiting a [TestFixture] class. No matter how many [Test]’s are found whiting the test fixture the method is called only once. It provides a common point for placing code required to set up the test fixture.

TestFixtureSetUp does not promote proper test isolation. Initiating code common between unit tests should be used lightly and only when required. We don’t want one unit test failure to bring down other unit tests nor do we want a required order for unit tests. It is preferred to take the performance hit for initiating required classes for each unit test. Trying to find out why numerous unit tests have failed instead of one is a lot more work to do! XUnit has no such feature and only provide a test set up to promote correct test isolation. Some people actually prefer one test per source file.

If a [TestFixtureSetup] method throws an exception then no tests within it are run.

[SetUp] marks a method which is called before every test in the test fixture. If a test fixture contains 100 tests, this method is called 100 times. If the method throws an error no tests in the test fixture are run.

[TearDown] marks a method which is called after every test in the test fixture. If a test fixutre contains 100 tests, this method is called 100 times.

[TestFixtureTearDown] marks a method which is called once after all tests within a test fixture have been run. If a [TearDown] has been provided [TestFixtureTearDown] will be run after the last call to [TearDown]

[TestFixtureSetUp] public void FixtureSetUp() { }

[TestFixtureTearDown] public void FixtureTearDown() { }

[SetUp] public void SetUp() { }

[TearDown] public void TearDown() { }

SetUpFixture

The [TestFixtureSetUp] attribute marks a config class for providing a test fixture. It provides with [SetUp] and [TearDown] methods; you can have a maximum of one of each type. The class is required to be public and have a parameter-less constructor.

[SetUp] will be called once before any test fixtures tests, [SetUp] [TestFixtureSetUp] methods are called.
[TearDown] will be called after all tests, [TearDown] and [TestFixture] methods are called.

The methods will be called at most once each irrelevant of how many test fixtures it will be used for.

[SetUpFixture]
public class TestFixtureSetUp
{
    [SetUp]
	public void RunBeforeAnyTests() { }

    [TearDown]
	public void RunAfterAnyTests() { }
}

TestCase

The TestCase attribute provides a way of specifying a test’s methods parameter and required return value or result. This allows one test to be called with a varied range of test values.

In our example method it takes takes two integers and returns an integer though any amount or types of parameters and return results can be used. You simply need to ensure that the number of types match.

[TestCase(1,1, Result=2)] defines that the method will be called with parameters a and b with a value of 1 and test iteration will only pass if the return value is 2.

[TestCase(1,1, Result=2)]
[TestCase(2,2, Result=4)]
[TestCase(3,3, Result=6)]
public int TestCaseTest(int a, int b)
{
  return( a + b);
}

Range

The Range attribute allows us to define a list of possible values which are passed each time into a parameter of our test method. This is done by specifying a min, max and increment.

The following example has a [Range(0.0, 1, 0.5)]. The test method will be called with values of 0, 0,5 and 1.

[Test]
public void RangeAttributeTest(
[Range(0.0, 1, 0.5)] decimal value) { }

Random

The Random attribute allows the function to be called x times with a random value between a specified min and max.

The example below has Random(1, 10, 2)] int value: The method will be called twice with a value equal to or between 1 and 10.

[Test]
public void RandomAttributeTest(
[Random(1, 10, 2)] int value) { }

Sequential

The sequential attribute allows us to define a range of values for each parameter. The sequential attribute decorates the Test while the Value attribute decorates each parameter.

Our method takes an int x and a string s:

[Values(1,2,3)] int x: means x will be set to 1, 2 and then 3.
[Values(“A”,”B”)] string s: means that s will be set to “A” and “B” and null.

Because x has three defined values and s will have two, s will be called the final time with null or the default(T) value; if s was an int 0 would be specified as the last called value.

To clarify our parameter pairs will be run with: {1,A}, {2,B}, {3, null}

[Test, Sequential]
public void SequentialAttributeTest(
[Values(1,2,3)] int x,
[Values("A","B")] string s) { }

The combinatorial allows us to specify a combination of values for each parameter.

By specifying the first parameter as having [Values(1,2,3)] and the second parameter as having Values(“a”, “b”, “c”)]; the first parameter will be 1,2 or 3, the second parameter will be “a”, “b”, or “c”. The method will be called once with every possible combination of parameters defined. In this case it will be 9 times: {1,a}, {1,b}, {1,c}, {2,a}, {2,b}…. {3,c}.

[Test, Combinatorial]
public void CombinatorialAttributeTest ([Values(1,2,3)] int a, [Values("a", "b", "c")] string b) { }

TestCaseSource: Public Property

TestCaseSource allows us to provide test data for a function by defining a public property which returns an array of arrays. Each internal array is a defined set of parameter values for a test iteration.

In the example below the public property CaseSourceTestData defines our parameters to be used. We then tell our test to use this as our parameters source be passing the name of the property as a string into the TestCaseSource attribute: [Test, TestCaseSource(“CaseSourceTestData”)].

Any number of parameters and types can be used along as they match the method signature.

public static object[] CaseSourceTestData =
{
    new object[] { 1, 1.1m, "2.1" },
	new object[] { 2, 2.2m, "4.2" },
	new object[] { 3, 3.3m, "6.3" },
};

[Test, TestCaseSource("CaseSourceTestData")]
public void CaseSourceTest(int a, decimal b, string c) { }

TestCaseSource can also be defined by providing a factory class and the name of a public property which returns IEnumerable.

The yield keyword is used to return a test iterations parameter values.

Each iteration requires to return an instance of TestCastData initiated with a value for each parameter in the test method.

yield return new TestCaseData( 1, "1" );

TestCastData has fluent syntax and be used to define a require return result

yield return new TestCaseData( 1, "1" ).Returns( 2 );

or expected exception to be throw

yield return new TestCaseData( 0, "a" ).Throws(typeof(ArgumentException));

The test is configured to use the factory class and property which returns the above information by:

[Test,TestCaseSource(typeof(TestCaseDataFactory),"TestCasesDataForTestCaseSourceTest")]

In this example the class TestCaseDataFactory is our provider class and TestCasesDataForTestCaseSourceTest is the name of the property which has out test data.

The has to be public and static or the class has to have a paramaterless constructor.

In the following example our method takes an int, a string and returns a decimal. The test is called three times:

(1,”1″) with expected result 1
(2,”2″) with expected result 2
(0,”a”) with expected result of an ArgumentException being thrown.

[Test,TestCaseSource(typeof(TestCaseDataFactory),"TestCasesDataForTestCaseSourceTest")]
public decimal TestCaseSourceTest(int a, string b)
{
	int bInt;
	if( !int.TryParse(b, out bInt))
		throw new ArgumentException(string.Format("Can not parse '{0}' to an integer", b), "b");

	return a + bInt;
}

public class TestCaseDataFactory
{
  	public static IEnumerable TestCasesDataForTestCaseSourceTest
	{
	    get
	    {
	      yield return new TestCaseData( 1, "1" ).Returns( 2 ); // Defines the test data and the expected return
	      yield return new TestCaseData( 2, "2" ).Returns( 4 );
	      yield return new TestCaseData( 0, "a" )
					.Throws(typeof(ArgumentException)); // Defines the test data and the expected throw exception
	    }
	}
}

Exceptions

Unit testing requires testing “The Good, The Bad and the Ugly”. Exceptions are valid results when we are expecting them. If our test throws an expected exception we can cater for a passing test. We can even specify the type of exception and the internal message.

See the next post for expecting exceptions with assertions; this covers only the attribute decorations.

The internal message can be asserted with a few different techniques via an enum.

MessageMatch.Contains: The inner messages contains the ExcpectedMessage anywhere within the string (like ‘%Foo%’).
MessageMatch.Exact: The inner message matches the ExpectedMessage exact ( = ‘Foo’ ).
MessageMatch.StartsWith: The inner message starts with the ExpectedMessage ( like ‘Foo%’ ).
MessageMatch.Regex: The ExpectedMessage is a regular expression which describes the inner message.

The ExpectedMessage is optional.

[Test, ExpectedException( typeof(Exception), ExpectedMessage = "Foo", MatchType = MessageMatch.Exact)]
public void ExpectedExceptionAttributeTest()
{
	throw new Exception("Foo");
}
Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s