Unit Tester Tutorial

Contents

• Introduction
• Testing Environment
• Creating a Unit Test
• Running a Unit Test
• Test Suite Class
• Running a Test Suite
• Conclusion

Expand All

Introduction

Blayd Software's Unit Tester component provides a framework that you can utilize to create, manage and run unit tests. This tutorial is not about how to code effective unit tests, that subject is worthy of an article of its own, we are going to describe how to incorporate the Unit Tester component into your testing regime and how to get the most out of it. To follow the article you will need to have a copy of the Unit Tester component installed. If you do not already have a copy of the Unit Tester you can download an evaluation version from www.blayd.co.uk. However, you should note that some of the features described in this article are not available in the evaluation version.

Testing Environment

We are going to use Visual Studio 2005 in this article to demonstrate the Unit Tester component, however, you can use the testing framework with any .Net Framework development environment. Therefore, once we get past setting up the test harness, the article and the code samples are not Visual Studio specific.

For the purpose of this article we have created a Visual Studio 2005 solution containing two projects. The first project is a class library component named SampleComponent, this project represents the component that we are going to test and contains two classes SampleClass1 and SampleClass2. The second project is a Windows Forms project named SampleComponent.TestHarness, this project, unsurprisingly, represents the test harness which is to contain the unit tests to test the two classes in SampleComponent. The test harness project requires a reference to the SampleComponent project and is set as the 'Start Up' project. The test harness project requires a reference to the Unit Tester component so that we can make use of it to create, manage and run the unit tests. The Unit Tester component's assembly is named Blayd.UnitTesting.dll and by default is, typically, installed in the following folder:

C:\Program Files\Blayd Software\Unit Tester 2

Finally, the unit test classes that we create in the test harness require a using (Imports in Visual Basic) statement for the SampleComponent namespace and the Unit Tester component namespace. The test harness Form that we are going to use to run the tests also requires a using statement for the Unit Tester component namespace. The Unit Tester component's namespace is as follows:

Blayd.UnitTesting

Creating a Unit Test

The Unit Tester component provides a testing framework that is used to create, manage and run unit tests. The testing framework defines a single unit test as a 'test case' and provides an abstract base class, unsurprisingly, named TestCase that we can use as the base class for our unit tests. Therefore, to create a test case we add a class to the test harness project and extend the testing framework's TestCase class as outlined below.

Test Case Class

using System;
using Blayd.UnitTesting;

namespace SampleComponent.TestHarness
{
   internal class SampleClass1TestCase : TestCase
   {
      public SampleClass1TestCase(string name) : base(name) { }
   }
}

We could have used whatever name we wanted for the class but by convention the text TestCase is added to the name of the class that is being tested, in this example we have created a test case to test the SampleClass1 class. The test case class may be public or internal but as we will see later it is important to note that the test methods that we want the testing framework to execute must be public instance methods. The framework's TestCase base class has a single protected constructor that requires the name of the test case as it's only parameter. Again, we can use whatever name we want, however, the name is used to identify the test case in the result report so it should be descriptive and it makes sense to make it unique within the test harness, we often use the test case class name as the value for this parameter.

The TestCase base class has two protected virtual methods, OnTestCaseStarting and OnTestCaseComplete, that can be overridden in a derived class to set up and tear down a test case. For example, if we want to use the same SampleClass1 instance in all of the test methods, the code would be similar to that outlined below.

Test Case Class Initialization and Tear Down

using System;
using Blayd.UnitTesting;

namespace SampleComponent.TestHarness
{
   internal class SampleClass1TestCase : TestCase
   {
      private SampleClass1 testClass;

      public SampleClass1TestCase(string name) : base(name) { }

      protected override void OnTestCaseStarting()
      {
         testClass = new SampleClass1();
      }
      protected override void OnTestCaseComplete()
      {
         if (testClass != null)
            testClass.Dispose();
      }
   }
}

The testing framework calls the OnTestCaseStarting method before executing the first test method and the OnTestCaseComplete method after executing the last test method. There is no need to call the base class implementation of either of these two methods as they do not contain any code and therefore, do not do anything, they exist solely to be overridden, however as they are not abstract they do not have to be overridden.

To make use of our test case we need to add test methods. The test methods that we create will test the members of the class that we are testing, in our example we will be testing the members of the Sample1Class. We want the testing framework to execute each of our test methods by calling them in turn during a test run. To get the testing framework to call a test method, the method's name must begin with the text 'Test' and it must be a public instance method of a TestCase derived class. The Sample1Class class has a constructor that takes a single String parameter which we will need to test in the test case. The code below shows how we might test the constructor in our test case.

Test Methods

using System;
using Blayd.UnitTesting;

namespace SampleComponent.TestHarness
{
   internal class SampleClass1TestCase : TestCase
   {
      public SampleClass1TestCase(string name) : base(name) { }

      public void TestConstructorNull()
      {
         ResultExpected(typeof(ArgumentNullException));

         SampleClass1 sampleClass = new SampleClass1(null);

         Fail("SampleClass1 constructor allowed null value.");
      }
      public void TestConstructorEmpty()
      {
         ResultExpected(typeof(ArgumentException));

         SampleClass1 sampleClass = new SampleClass1(string.Empty);

         Fail("SampleClass1 constructor allowed empty value.");
      }
      public void TestConstructor()
      {
         ResultExpectedNoError();

         SampleClass1 sampleClass = new SampleClass1("A Sample Class");

         AssertEqual("A Sample Class", sampleClass.Name);
      }
   }
}

This version of our test case does not make use of the OnTestCaseStarting method or the OnTestCaseComplete method, instead we create a new instance of the Sample1Class in each of the test methods. We have coded three methods that we want the testing framework to call so that we can test the Sample1Class constructor, each method begins with the text 'Test' and each method is a public instance method. Test method names can be whatever you like, as long as they start with 'Test' but you may find it useful when analysing test results if you use a descriptive or meaningful name. A point worth noting is that the testing framework is case sensitive therefore, for example, it will ignore methods that begin with 'TEST' or 'test'.

The first test method ensures that the SampleClass1 constructor does not allow a null value to be passed in as it's name. This method introduces two methods that our test case inherits from the testing framework's TestCase class. The ResultExpected method, inherited directly from TestCase, tells the testing framework that we are expecting the member that we are testing to throw an ArgumentNullException. If it does not throw an exception, the Fail method inherited from the Assert class, from which the TestCase class derives, will, unsurprisingly, cause the test case to fail. Therefore, if the SampleClass1 constructor throws an exception, as expected, the test will continue, however, if it does not throw an exception, the test case will terminate due to the testing framework throwing an AssertionFailedException from the Fail method.

The second test method ensures that the SampleClass1 constructor does not allow an empty value to be passed in as it's name. This method is the same as the first method, apart from expecting the member being tested to throw an ArgumentException, so we will not discuss it further.

Having tested the SampleClass1 constructor with invalid input to make sure that it works correctly the third test method passes a valid parameter value to the constructor. In this example we have only tested for two possible forms of invalid input, there may be more, for instance, the SampleClass1 name may have a maximum length and/or a minimum length in which case we would also need to test the constructor with values of various lengths. However, for the purpose of this article we will assume that the invalid input tests are complete and test to make sure that valid input is handled correctly by the constructor. The third test method introduces two more methods inherited from the testing framework's TesCase class. The ResultExpectedNoError method, inherited directly from TestCase, tells the testing framework the we are not expecting any exceptions to be thrown from the member being tested. The method then initializes an instance of the SampleClass1 class with a valid name and then performs an assertion to ensure that the SampleClass1.Name property contains the value that we have just passed to the constructor. The AssertEqual method, inherited from the Assert class, from which the TestCase class derives, checks if the expected value i.e. 'A Sample Class' is equal to the actual value of the property. If it is, the test has passed and the test case continues, if it is not, the testing framework will throw an AssertionFailedException and terminate the test case with a status of failed.

Running a Unit Test

Now that we have coded the first set of test methods, to test the SampleClass1 constructor, in our test case we need to execute or run the SampleClass1TestCase test case. The easiest way to do this is to call the TestCase.Run method. As our SampleClass1TestCase class inherits from the TestCase class we can make use of the inherited Run method. The code example below shows a helper method that we have created in the test harness project's main Form. The method is called from a command button that we have added to the Form.

Test Case Run Method

private void RunTest()
{
   SampleClass1TestCase testCase = new SampleClass1TestCase("Test Case 1");
   testCase.Run();
   testCase.Results[0].GetTestResult(Console.Out);
}

The code example above is a very quick and easy way to run a test case and produce a test result report. If you only have one or two test cases to run this type of code will be fine, however, when testing a 'real' component you are likely to have quite a few test cases to manage and run, so we will be expanding on this code later in the article to demonstrate the other options available for running test cases.

Our first attempt at running a test case creates an instance of the SampleClass1TestCase, passing in a test case name of 'Test Case 1'. It then calls the Run method which our test case inherits from the TestCase class. When the Run method is called, the testing framework first calls the OnTestCaseStarting method, therefore, if we have overridden this method, we get the opportunity to perform any initialization required by our test case. The testing framework then looks for public instance methods in our test case that have names beginning with 'Test' and calls each method that it finds one after the other. When all of the test methods in the test case have been called by the testing framework or when the test case fails, the testing framework calls the OnTestCaseComplete method, therefore, if we have overridden this method we get the opportunity to perform any tear down required by our test case. When the test case run has completed, our sample code then retrieves the test result report and outputs it to the Console, there are other output options which we will show later in the article. Each time that a test case runs it compiles a test result report and adds the report to it's report collection. The code above demonstrates how to retrieve the test result report for the first run i.e. index position zero in the result collection. The test result report for our SampleClass1TestCase is shown below.

Test Result Report

Test case:        Test Case 1
Test run number:  1.
Start time:       172.11:11:52.7960000.
End time:         172.11:11:52.9210000.
Duration:         00:00:00.1250000.
* TestConstructorNull method called.
*     Result expected: System.ArgumentNullException
Exception:
    Thrown from test method: TestConstructorNull.
    Type:     System.ArgumentNullException
    Source:   SampleComponent
    Target:   Void .ctor(System.String)
    Message:  Argument cannot be null.
    Parameter name: name
* TestConstructorNull method complete.
* TestConstructorEmpty method called.
*     Result expected: System.ArgumentException
Exception:
    Thrown from test method: TestConstructorEmpty.
    Type:     System.ArgumentException
    Source:   SampleComponent
    Target:   Void .ctor(System.String)
    Message:  Argument cannot be empty.
    Parameter name: name
* TestConstructorEmpty method complete.
* TestConstructor method called.
*     Result expected: no error.
* TestConstructor method complete.
Number of exceptions: 2.
Test case succeeded.

The test result report shows that the testing framework has called each of our three test methods and we can see that two of the test methods have resulted in an exception being thrown from the member being tested. The third test method has executed without error, as we expected. As all three test methods produced the result we were expecting them to, the test case result has a status of succeeded. It may seem strange at first that a test case with two exceptions should have a result status of succeeded, however, we were expecting the constructor that we are testing to throw an exception when it received invalid input, therefore if the constructor does throw an exception the test has passed and if it does not throw an exception the test has failed.

The Unit Tester component provides the TestResultFormatter class which has various options for formatting test result reports. For example, the test method entry and exit comments can be disabled if required, the comment prefix can be changed, test run timings can be disabled, etc.

We can make the simple test case run method, shown above, a bit more useful and more importantly reusable by making just a few changes to the code. Our second attempt is shown below.

Reusable Test Case Run Method

private void RunTest(TestCase testCase, TextWriter writer)
{
   testCase.Run();
   testCase.Results[0].GetTestResult(writer);
}

By adding a TestCase and a TextWriter parameter to the method we can run any test case and can specify an output destination for each run. To run our test case and output the test result report to the Console we would call the method as shown below.

Output Result Report to Console

RunTest(new SampleClass1TestCase("Test Case 1"), Console.Out);

To run our test case and output the test result report to a file we would call the method as shown below.

Output Result Report to File

string file = @"C:\TestCase1.rpt";
using(StreamWriter writer = new StreamWriter(file))
{
   RunTest(new SampleClass1TestCase("Test Case 1"), writer);
}

You will, or course, have to specify a file path that is relevant to your own system rather than the one shown. We can call the test case run method in this way because all of our test cases inherit from the testing framework's TestCase class and they can, therefore, be passed to our RunTest method as their base type i.e. TestCase. Test result reports can be output to any TextWriter derived class and the Console.Out property type and the StreamWriter type we have used in the examples both derive from TextWriter and can therefore be passed as the second parameter to the RunTest method.

Using this technique provides a better solution for running just one or two test cases but if you have more than one or two test cases to run or if you want to run your test cases many times, as part of a soak or stress test for example, another solution is required, the Unit Tester component's TestSuite class.

Test Suite Class

The Unit Tester component provides the TestSuite class to manage and execute a collection of test cases. Rather than run each test case individually, TestCase derived class instances can be added to an instance of the TestSuite class and executed sequentially by calling the TestSuite.Run method. The TestSuite class can be used to execute a collection of test cases once, a specified number of times or for a specified period of time. Test cases can also be executed asynchronously once, a specified number of time or for a specified period of time.

Running a Test Suite

We have already discussed the coding of test cases so we won't repeat the code here, we will use the SampleClass1TestCase created to test the SampleClass1 class together with the SampleClass2TestCase which we will assume has been coded to test the SampleClass2 class in the SampleComponent class library. The code example below, like the previous example, shows a helper method that we have created in the test harness project's main Form. The method is called from a command button that we have added to the Form.

Test Suite Run Method

private void RunTestSuite()
{
   TestSuite suite = new TestSuite("Sample Test Suite");

   SampleClass1TestCase testCase1 = new SampleClass1TestCase("Test Case 1");
   suite.TestCases.Add(testCase1);

   SampleClass2TestCase testCase2 = new SampleClass2TestCase("Test Case 2");
   suite.TestCases.Add(testCase2);

   suite.Run();
   suite.GetTestResult(Console.Out);
}

The code example above is a very quick and easy way to run a test suite and produce a test result report. Our RunTestSuite method creates an instance of the TestSuite class and passes a name for the suite to the constructor. The code then initializes an instance of each of our test cases and adds them to the test suite's TestCaseCollection using the TestSuite.TestCases property, it then calls the TestSuite.Run method and finally outputs the test result report to the Console. When the TestSuite.Run method is called the testing framework iterates through the test case collection and calls the Run method of each TestCase. The TestSuite.GetTestResult method outputs the test result of each TestCase instance in the test case collection sequentially, therefore a test suite result report is made up of a series of test case result reports. As with the TestCase example, we could have chosen to output the test result report to a file, for example, simply by passing a different TextWriter derived class instance to the GetTestResult method.

Unlike a test case, which can only be executed once, a test suite can be executed a specified number of times or for a specified period of time. The code to execute a test suite for a specified number of times or for a specified period of time would be very similar to the example above, the only change that is required is to call the relevant overload of the TestSuite.Run method i.e. the overload that takes an Int32 for the number of runs or the overload that takes a TimeSpan for the period of time. As the code is more or less the same we will not show it here, instead we will move on to examine how we can execute a test suite asynchronously.

The TestSuite class has an overloaded BeginRun method that can be used to run a test suite asynchronously for a specified number of times or for a specified period of time. To make use of this method we need to specify a method that the testing framework will call when the test run is complete. When the specified callback method is called by the testing framework the code in the method needs to call the TestSuite.EndRun method to end the asynchronous run and retrieve the test result report.

Test Suite Asynchronous Run

private TestSuite testSuite;
private void RunTestSuite(int numberOfRuns)
{
   testSuite = new TestSuite("Sample Test Suite");

   SampleClass1TestCase testCase1 = new SampleClass1TestCase("Test Case 1");
   testSuite.TestCases.Add(testCase1);

   SampleClass2TestCase testCase2 = new SampleClass2TestCase("Test Case 2");
   testSuite.TestCases.Add(testCase2);

   AsyncCallback callback = new AsyncCallback(this.EndRun);
   IAsyncResult result = testSuite.BeginRun(numberOfRuns, callback);
}
private void EndRun(IAsyncResult asyncResult)
{
   testSuite.EndRun(asyncResult, Console.Out);
}

We have altered our RunTestSuite method by adding a parameter to take the number of runs for the test suite and we have also added the EndRun callback method which will receive the call from the testing framework to signify that the test run is complete. When the EndRun method is called it calls the TestSuite.EndRun method to output the test result report to the Console. Using this technique we can now run out test suite as many times as we want by passing in the required number of runs to our RunTestSuite method. However, our test harness user interface does not reflect what is happening, so once we call the RunTestSuite method we have no indication in the test harness that the test suite is running and we can only tell when it has finished by watching the Console for the test result report to be displayed. Therefore, we need to sink the TestSuite.Progress event to provide some feedback in the test harness user interface.

Test Suite Asynchronous Run with Progress Event

private void RunTestSuite(int numberOfRuns)
{
   testSuite = new TestSuite("Sample Test Suite");
   testSuite.EnableProgressEvent = true;
   testSuite.Progress += new ProgressEventHandler(this.TestSuiteProgress);

   SampleClass1TestCase testCase1 = new SampleClass1TestCase("Test Case 1");
   testSuite.TestCases.Add(testCase1);

   SampleClass2TestCase testCase2 = new SampleClass2TestCase("Test Case 2");
   testSuite.TestCases.Add(testCase2);

   AsyncCallback callback = new AsyncCallback(this.EndRun);
   IAsyncResult result = testSuite.BeginRun(numberOfRuns, callback);
}
private void EndRun(IAsyncResult asyncResult)
{
   testSuite.EndRun(asyncResult, Console.Out);
}
private delegate void ProgressDelegate(object sender, ProgressEventArgs e);
private void UpdateProgress(object sender, ProgressEventArgs e)
{
   progressTestSuite.Value = e.PercentageComplete;
}
private void TestSuiteProgress(object sender, ProgressEventArgs e)
{
   if (this.InvokeRequired)
   {
      this.Invoke(new ProgressDelegate(this.UpdateProgress),
                  new object[]{ sender, e });
   }
   else
   {
      UpdateProgress(sender, e);
   }
}

Again we have made some minor changes to our RunTestSuite method by adding code to enable the TestSuite.Progress event and code to sink the event, specifying the TestSuiteProgress method as the event handler. We have added a ProgressBar control to the main Form in the test harness project so that we can provide visual feedback on the progress of the test suite run. To ensure that we are updating the ProgressBar control on the main UI thread we have implemented a check to see if we need to use the Form's inherited Invoke method to call our UpdateProgress method on the main UI thread.

The TestCase class also has a Progress event, so we could implement feedback for the progress of each test case within the test suite as well as the test suite's progress. However, the test case Progress event is only worth sinking for large or relatively slow running test cases as, typically, a test case is completed so quickly that any visual feedback just becomes a blur and can be more annoying than informative!

Conclusion

We have, in this article, tried to describe and step through the fundamental requirements for getting started with the Blayd Software Unit Tester component. So you should now be in a position to introduce unit testing into your development cycle. The Unit Tester component aims to enable you to work the way that suits you best when implementing and running your test cases and does not require you to have a specific version or make of development environment, so you can get started immediately!