Otter Unit Test Framework User Guide

Version 00.05

Applicaton otter image

Index

Introduction
Big Picture
Setting Up a Basic Application
    Downloading the Framework
    Organizing the Application
         Creating a Windows Application
         Installing the MS Visual Studio Add-in
         Adding the Otter Unit Test .dll Reference
    Creating Test Classes
    Running the Framework
Using the Framework
Assert methods
Handling Exceptions expectations
Web Application Issues
Contact information

Introduction

This document aims to explain how to use the Otter Unit Test Framework. You will notice that this tool is quite similar to the JUnit Framework. Therefore, whoever is accustomed with JUnit will have no problem adapting to this framework.

Big Picture

Here is a summary on how to put the framework to work. For that, two approaches can be used:

The first approach has the advantage of usability. It is quite easy to use it, since one only has to install the add-in, create the unit tests and run it via menu command. As a drawback, when using the add-in, the application context is lost. It means the application location, external files, environmental variables; all of it is lost when running the tests because only the test class (and objects it refers) are used in the tests execution. This obviously consists in a problem since it is quite desirable that context aspects are taken into consideration when running unit tests. Thus a solution to this problem is being studied and as soon as it is found, a new release containing it will be launched. A more detailed explanation of this problem is reported at the Project's Wiki.

The second approach has the advantage of using all solution context since the tests are placed as a new application in the solution. As a drawback, one has to program the tests insertion in the framework and its execution. This however, is a quite easy to do, as described further in this material.

Setting Up a Basic Application

Now, a detailed explanation on how to use the framework will be given. In order to ease the learning process, the explanation will be based on an small example which you can get by clicking here.

Downloading the Framework

Go to the sourceforge project web site, and download the Otter Unit Test Framework version 00.04.

Organizing the Application

As mentioned before, in order run the application, you can either, install the framework MS Visual Studio 2005 add-in, or create a Windows Application project, programmatically add the Test classes to the framework and launch it. Here both manners are presented in details.

Creating a Windows Application

Follow the steps below in order to add a Windows Application Solution to your application. Note that the sample application (OtterUsageSample) already had this implemented as its only project.

  1. Open the application to be tested;
  2. Select the Solution Explorer View;
  3. Right-click in the Solution, select the option Add... and click New Project...;
  4. Choose the Windows Application project-type, write the name OtterUsageSample in the Name text field and press ENTER;

Installing the MS Visual Studio Add-in

In order to install the Visual Studio Add-in follow the steps below:

  1. In the application .zip package you have downloaded, execute either ./bin/Add-in_for_Visual_Studio_2005/Otter Unit Test Add-in for VS 2005 Setup.msi or ./bin/Add-in_for_Visual_Studio_2005/setup.exe.
  2. Proceed with the setup process without changing the destination directory suggested to install the add-in. The suggested directory is the one Visual Studio gathers information regarding add-ins.
  3. After the installation is finished, open the MS Visual Studio 2005. Continuing, go to Tools..., Add-in Manager.... In the screen which will be opened you will find the Otter Unit Test Framework add-in on the list. Check its combo-box and also check the one labeled Startup.
  4. Load the project to be tested (OtterUssageSample for this tutorial purpose).
  5. Load the Solution Explorer View and right click in any Project of your solution. You should see the Otter Unit Test option in the context menu. Going there you will find the button to launch the framework - Execute Tests.... Should you click there, the framework will be executed showing the created tests, in case there are any.
  6. In case the Otter Unit Test Framework does not appear in the context menu, close and open the MS Visual Studio 2005 again. Load the project where add-in ought to be displayed.

Adding the Otter Unit Test .dll Reference

Here the OterUsageSample application will be used to demonstrate the procedure. In order to add the OtterUnitTestFramework.dll as a reference to the application, right-click in the OtterUsageSample application and select Add Reference.... Then select the Browse tab and fetch the OtterUnitTestFramework.dll. In the sample application you can find the OtterUnitTestFramework.dll at the lib directory. In the .zip file you downloaded from Sourceforge you can find the OtterUnitTestFramework.dll at: ./bin/dll.

Creating Test Classes

Now that the framework is set up, it is time to create a Test class. In the sample application, it was created in the directory test and was called TestCaseSample.cs. The implementation is shown in Figure 1.

OtterUsageSample.test.TestCaseSample class
Figure 1: Implementation of TestCaseTemplate

Note that the class TestCaseSample implements OtterUnitTestFramework.Public.Test.TestCaseTemplate. By doing so, the implemented class is considered a Test Case class. After that, test methods are needed. In order to create them, make methods which follow the rules below:

Note that there is nothing unusual about the implemented methods (tester-methods). They simply call other methods - the ones to be tested. Also observe that OtterUnitTestFramework.Public.Assert class calls several static methods. They serve the purpose of testing expressions. For instance, if you call Assert.IsTrue(EXPRESSION> and the tested EXPRESSION is false, the tester-method will fail (otherwise it will succeed). This is how methods, or their results are tested, they are evaluated as an expression.

See in Figure 1 that the method TestCaseExecuteMethod runs the method DoNothing of the class Sample. In case no exception is thrown, the test will pass, otherwise it will fail. Note that the method DoNothing was tested.

Again, in Figure 1 the method TestCasePassAssertions makes several assertions using Assert methods. In case all of them pass, the test will succeed, otherwise it will fail. In this case, all evaluated expressions are successful resulting in a tester-method which has passed. Again, observe that methods (and object values) called by the Assert methods were tested.

On the other hand, the method TestCaseFailAssertions executes only faulty assertions. This obviously will result in a test failure.

An interesting tester-method is the TestCaseExpectException. Its goal is to test a method which throws an exception, more specifically, SampleException. In case SampleException is thrown the method passes, otherwise it fails. Thus, in this case the test is successful.

You should have observed that the there are two methods which do not follow the tester-methods rules - they are the StartUp and TearDown methods. You must have also perceived that they are overridden methods. They have the following purpose.

Note that in the example of Figure 1 the Startup method always creates an instance of Sample whereas the TearDown set it to null. In real-life-testing-situations, these methods are useful for opening and closing resources which are used in every tester-method.

Running the Framework

Running the framework by using the add-in is fairly simple. Right-click in the project where the Test classes have been created - in the OtterUsageSample Project - select Otter Unit Test and click Execute Tests.... After that, the application will be launched with all created tests. Next, will be shown how to run the project programmatically.

Register all Test classes in the framework and then execute it. This is pretty straightforward as displayed in Figure 2.

OtterUsageSample.Program Class
Figure 2: Registration of Test classes and framework execution

In Figure 2 app instance was retrieved from a Factory (singleton element). Also, see that the Test class were registered simply by using the property TestRegistration which has the register method RegisterTest. Finally, the application was executed using the method ExecuteApplication. Note that these steps were taken in the Main method.

Now you need to run the application. In order to do so, right-click on the OtterUsageSample project, click Debug and Start new instance. This will launch the Otter Unit Test Framework. The explanation on how to interact with the application is given in the next topic.

Using the Framework

In this session it is described how to use the Otter Unit Test Framework. You will notice that the interface is quite intuitive and its usage is quite simple. Figure 3 shows the application.

Otter Unit Test Application
Figure 3: Framework

As you can see in Figure 3, you have the option of selecting and unselecting the tests to be run by checking or unchecking the check boxes for each test, respectively. Also, note that you can check and uncheck all tests by clinking "Select All" or "Unselect All" respectively. In addition to that, you can see in each row of the shown table: the test class, class method and the test status. Note the list box which is open, displaying the Test class name. In case you wish to execute tests related only to one class, select it, if you want to run tests from all classes, choose the empty option (first one). Finally, in order to start the tests, click "Start Test" and should you want to quit the framework, click "Close".

Display after the tests were executed
Figure 4: Display after the tests were executed

After a executing the test, something resembling Figure 4 is displayed. Observe that the statistics of the tests execution are listed in the OK message. In Figure 5, you can see the results of the tests execution without the OK message. Note that are tests which have passed and others which have not. All of them represent the ones in the sample application mentioned before.

Tests Results
Figure 5: Tests execution result

In Figure 5 you can also see a log in the middle-below part of the panel. This log shows all information regarding tests execution (i.e. tests which have passed, tests which have not passed and the reason for that).

Tests log message
Figure 6: Test message after its execution

Another way to see test messages is the following: double-click the row of a certain executed test and a screen like Figure 6 is displayed.

This covers framework functionalities. As you may have noticed, it is quite straightforward to use the Otter Unit Test Framework which is the main goal of this framework: to be simple. Next, will be covered all Assert methods and the possibilities for the ExpectedException tag.

Assert methods

Assert methods basically test methods and their results by applying a condition (e.g.: is true, is false, is null, etc.). Below are all Assert methods and their usage.

Handling Exceptions expectations

A common situation of testing is to verify whether a method throws an exception. In this framework this can be verified with the aid of the method tag ExpectedException. Suppose you have the method ThrowSampleException and in order to successfully test this method, the SampleException must be thrown. Therefore, to program this test, call ThrowSampleException within the tester-method and mark it (the tester-method) with the tag ExpectedException(typeof(SampleException) as shown in Figure 1. By doing this, the test will pass in case an exception of type (typeof) SampleException is thrown and fail otherwise.

One important observation about the ExpectedException tag the following. Consider the example just given above. In case an exception which has inherited from SampleException was thrown, the test would still have succeeded because inheritance is considered when using the ExpectedException tag.

You should also be aware that you can enter several exceptions within the ExpectedException tag, For instance, if you make a tester-method with the following tag: ExpectedException(new Type[] { typeof(OtherException), typeof(SampleException) }), then if either OtherException, SampleException or any inherited exception were thrown, the test would be successful, otherwise it would fail.

Web Application Issues

In MS Visual Studio 2005, Web Applications by default are not Projects, which terribly hardens the creation of any add-in that can interact with them (Web Projects) at the level this framework does. Therefore, in order to use the Otter Unit Test Framework with a Web Application, there will be needed a manner to build a Web Application as a Project. To accomplish such a goal, one has to follow the steps below:

  1. Install the Microsoft Visual Studio 2005 - Update to Support Web Application Projects;
  2. Install the Microsoft® Visual Studio® 2005 Team Suite Service Pack 1;

After the installation of these packages (which will take a while), you now have the option of creating a ASP.NET Web Application as if it were a Project. In order to accomplish it, follow the steps below:

  1. In the MS Visual Studio 2005, open New and click Project.... Not that here you had not opened New, Web Site... because it would have led you to the old non-project Web Application.
  2. Search and select the ASP.NET Web Application which will be an ordinary Project, however, a Web Application one.

For more information regarding this matter, please consult Visual Studio 2005 Web Application Projects.

Contact information

If you wish to send any comments, suggestions, etc., please feel free to do so by e-mail (link below).

e-mail contact: otterunittestframework@gmail.com



___________________________________________________________
©Copyright - 2007 - All rights reserved by Eduardo Antonio Cecilio Fernandes Fernandes