Unit test routines. More...

#include <stdarg.h>
#include "compilerDependencies.h"
#include "libComAPI.h"

Include dependency graph for epicsUnitTest.h:

This graph shows which files directly or indirectly include this file:

Functions
LIBCOM_API void	testPlan (int tests)
	Declare the test plan, required. More...

LIBCOM_API void LIBCOM_API int	testDiag (const char *fmt,...) EPICS_PRINTF_STYLE(1
	Output additional diagnostics. More...

LIBCOM_API void LIBCOM_API int LIBCOM_API int	testDone (void)
	Mark the end of testing. More...

LIBCOM_API int	testImpreciseTiming (void)
	Return non-zero in shared/oversubscribed testing envrionments. More...

Missing or Failing Tests
Routines for handling special situations.
LIBCOM_API void	testSkip (int skip, const char *why)
	Place-holders for tests that can't be run. More...

LIBCOM_API void	testTodoBegin (const char *why)
	Mark the start of a group of tests that are expected to fail. More...

LIBCOM_API void	testTodoEnd (void)
	Mark the end of a failing test group. More...

LIBCOM_API void	testAbort (const char *fmt,...) EPICS_PRINTF_STYLE(1
	Stop testing, program cannot continue. More...

Announcing Test Results
Routines that declare individual test results.
#define	testOk1(cond) testOk(cond, "%s", #cond)
	Test result using condition as description. More...

LIBCOM_API int	testOk (int pass, const char *fmt,...) EPICS_PRINTF_STYLE(2
	Test result with printf-style description. More...

LIBCOM_API int	testOkV (int pass, const char *fmt, va_list pvar)
	Test result with var-args description. More...

LIBCOM_API void	testPass (const char *fmt,...) EPICS_PRINTF_STYLE(1
	Passing test result with printf-style description. More...

LIBCOM_API void LIBCOM_API void	testFail (const char *fmt,...) EPICS_PRINTF_STYLE(1
	Failing test result with printf-style description. More...

Test Harness for Embedded OSs
These routines are used to create a test-harness that can run multiple test programs, collect their names and results, then display a summary at the end of testing.
#define	runTest(func) runTestFunc(#func, func)
	Run a test program. More...

#define	testHarnessDone() testHarnessExit(0)
	Declare all test programs finished. More...

typedef int(*	TESTFUNC) (void)

LIBCOM_API void	testHarness (void)
	Initialize test harness. More...

LIBCOM_API void	testHarnessExit (void *dummy)

LIBCOM_API void	runTestFunc (const char *name, TESTFUNC func)

Detailed Description

Unit test routines.

Author: Andrew Johnson

The unit test routines make it easy for a test program to generate output that is compatible with the Test Anything Protocol and can thus be used with Perl's automated Test::Harness as well as generating human-readable output. The routines detect whether they are being run automatically and print a summary of the results at the end if not.

A test program starts with a call to testPlan(), announcing how many tests are to be conducted. If this number is not known a value of zero can be used during development, but it is recommended that the correct value be substituted after the test program has been completed.

Individual test results are reported using any of testOk(), testOk1(), testOkV(), testPass() or testFail(). The testOk() call takes and also returns a logical pass/fail result (zero means failure, any other value is success) and a printf-like format string and arguments which describe the test. The convenience macro testOk1() is provided which stringifies its single condition argument, reducing the effort needed when writing test programs. The individual testPass() and testFail() routines can be used when the test program takes a different path on success than on failure, but one or other must always be called for any particular test. The testOkV() routine is a varargs form of testOk() included for internal purposes which may prove useful in some cases.

If some program condition or failure makes it impossible to run some tests, the testSkip() routine can be used to indicate how many tests are being omitted from the run, thus keeping the test counts correct; the constant string why is displayed as an explanation to the user (this string is not printf-like).

If some tests are expected to fail because functionality in the module under test has not yet been fully implemented, these tests may still be executed, wrapped between calls to testTodoBegin() and testTodoEnd(). testTodoBegin() takes a constant string indicating why these tests are not expected to succeed. This modifies the counting of the results so the wrapped tests will not be recorded as failures.

Additional information can be supplied using the testDiag() routine, which displays the relevent information as a comment in the result output. None of the printable strings passed to any testXxx() routine should contain a newline '
' character, newlines will be added by the test routines as part of the Test Anything Protocol. For multiple lines of diagnostic output, call testDiag() as many times as necessary.

If at any time the test program is unable to continue for some catastrophic reason, calling testAbort() with an appropriate message will ensure that the test harness understands this. testAbort() does not return, but calls the ANSI C routine abort() to cause the program to stop immediately.

After all of the tests have been completed, the return value from testDone() can be used as the return status code from the program's main() routine.

On vxWorks and RTEMS, an alternative test harness can be used to run a series of tests in order and summarize the results from them all at the end just like the Perl harness does. The routine testHarness() is called once at the beginning of the test harness program. Each test program is run by passing its main routine name to the runTest() macro which expands into a call to the runTestFunc() routine. The last test program or the harness program itself must finish by calling testHarnessDone() which triggers the summary mechanism to generate its result outputs (from an epicsAtExit() callback routine).

IOC Testing

Some tests require the context of an IOC to be run. This conflicts with the idea of running multiple tests within a test harness, as iocInit() is only allowed to be called once, and some parts of the full IOC (e.g. the rsrv CA server) can not be shut down cleanly. The function iocBuildIsolated() allows to start an IOC without its Channel Access parts, so that it can be shutdown quite cleanly using iocShutdown(). This feature is only intended to be used from test programs, do not use it on productional IOCs. After building the IOC using iocBuildIsolated() or iocBuild(), it has to be started by calling iocRun(). The suggested call sequence in a test program that needs to run the IOC without Channel Access is:

#include "iocInit.h"
MAIN(iocTest)
{
    testPlan(0);
    testdbPrepare();
    testdbReadDatabase("<dbdname>.dbd", 0, 0);
    <dbdname>_registerRecordDeviceDriver(pdbbase);
    testdbReadDatabase("some.db", 0, 0);
    ... test code before iocInit().  eg. dbGetString() ...
    testIocInitOk();
    ... test code with IOC running.  eg. dbGet()
    testIocShutdownOk();
    testdbCleanup();
    return testDone();
}

The part from iocBuildIsolated() to iocShutdown() can be repeated to execute multiple tests within one executable or harness.

To make it easier to create a single test program that can be built for both the embedded and workstation operating system harnesses, the header file testMain.h provides a convenience macro MAIN() that adjusts the name of the test program according to the platform it is running on: main() on workstations and a regular function name on embedded systems.

Example

The following is a simple example of a test program using the epicsUnitTest routines:

#include <math.h>
#include "epicsUnitTest.h"
#include "testMain.h"
MAIN(mathTest)
{
    testPlan(3);
    testOk(sin(0.0) == 0.0, "Sine starts");
    testOk(cos(0.0) == 1.0, "Cosine continues");
    if (!testOk1(M_PI == 4.0*atan(1.0)))
        testDiag("4 * atan(1) = %g", 4.0 * atan(1.0));
    return testDone();
}

The output from running the above program looks like this:

1..3
ok  1 - Sine starts
ok  2 - Cosine continues
ok  3 - M_PI == 4.0*atan(1.0)
    Results
    =======
       Tests: 3
      Passed:  3 = 100%

Definition in file epicsUnitTest.h.

Macro Definition Documentation

#define runTest ( func ) runTestFunc(#func, func)

Run a test program.

Parameters

func	Name of the test program.

Definition at line 269 of file epicsUnitTest.h.

#define testHarnessDone ( ) testHarnessExit(0)

Declare all test programs finished.

Definition at line 272 of file epicsUnitTest.h.

#define testOk1 ( cond ) testOk(cond, "%s", #cond)

Test result using condition as description.

Parameters

cond	Condition to be evaluated and displayed.

Returns: The value of cond.

Definition at line 182 of file epicsUnitTest.h.

Typedef Documentation

typedef int(* TESTFUNC) (void)

Definition at line 259 of file epicsUnitTest.h.

Function Documentation

LIBCOM_API void runTestFunc	(	const char *	name,
		TESTFUNC	func
	)

Definition at line 310 of file epicsUnitTest.c.

                                                   {
     printf("\n***** %s *****\n", name);
     testing = name;
     func();     /* May not return */
     epicsThreadSleep(1.0);
 }

LIBCOM_API void testAbort	(	const char *	fmt,
			...
	)

Stop testing, program cannot continue.

Parameters

fmt	A printf-style format string giving the reason for stopping.
...	Any parameters required for the format string.

LIBCOM_API void LIBCOM_API int testDiag	(	const char *	fmt,
			...
	)

Output additional diagnostics.

Parameters

fmt	A printf-style format string containing diagnostic information.
...	Any parameters required for the format string.

LIBCOM_API void LIBCOM_API int LIBCOM_API int testDone ( void )

Mark the end of testing.

Definition at line 209 of file epicsUnitTest.c.

                    {
     int status = 0;
 
     epicsMutexMustLock(testLock);
     if (perlHarness) {
         if (!planned) printf("1..%d\n", tested);
     } else {
         if (planned && tested > planned) {
             printf("\nRan %d tests but only planned for %d!\n", tested, planned);
             status = 2;
         } else if (planned && tested < planned) {
             printf("\nPlanned %d tests but only ran %d\n", planned, tested);
             status = 2;
         }
         printf("\n    Results\n    =======\n       Tests: %-3d\n", tested);
         if (tested) {
             testResult("Passed", passed);
             if (bonus) testResult("Todo Passes", bonus);
             if (failed) {
                 testResult("Failed", failed);
                 status = 1;
             }
             if (skipped) testResult("Skipped", skipped);
         }
     }
     if (Harness) {
         if (failed) {
             testFailure *fault = callocMustSucceed(1, sizeof(testFailure),
                 "testDone calloc");
             fault->name     = testing;
             fault->tests    = tested;
             fault->failures = failed;
             fault->skips    = skipped;
             ellAdd(&faults, &fault->node);
         }
         Programs++;
         Tests += tested;
     }
     epicsMutexUnlock(testLock);
     return (status);
 }

LIBCOM_API void LIBCOM_API void testFail	(	const char *	fmt,
			...
	)

Failing test result with printf-style description.

Parameters

fmt	A printf-style format string describing the test.
...	Any parameters required for the format string.

LIBCOM_API void testHarness ( void )

Initialize test harness.

Definition at line 300 of file epicsUnitTest.c.

                        {
     epicsThreadOnce(&onceFlag, testOnce, NULL);
     epicsAtExit(testHarnessExit, NULL);
     Harness = 1;
     Programs = 0;
     Tests = 0;
     ellInit(&faults);
     epicsTimeGetCurrent(&started);
 }

LIBCOM_API void testHarnessExit ( void * dummy )

Definition at line 265 of file epicsUnitTest.c.

                                   {
     epicsTimeStamp ended;
     int Faulty;
 
     if (!Harness) return;
 
     epicsTimeGetCurrent(&ended);
 
     printf("\n\n    EPICS Test Harness Results"
              "\n    ==========================\n\n");
 
     Faulty = ellCount(&faults);
     if (!Faulty)
         printf("All tests successful.\n");
     else {
         int Failures = 0;
         testFailure *f;
 
         printf("Failing Program           Tests  Faults\n"
                "---------------------------------------\n");
         while ((f = (testFailure *)ellGet(&faults))) {
             Failures += f->failures;
             printf("%-25s %5d   %5d\n", f->name, f->tests, f->failures);
             if (f->skips)
                 printf("%d subtests skipped\n", f->skips);
             free(f);
         }
         printf("\nFailed %d/%d test programs. %d/%d subtests failed.\n",
                Faulty, Programs, Failures, Tests);
     }
 
     printf("Programs=%d, Tests=%d, %.0f wallclock secs\n\n",
            Programs, Tests, epicsTimeDiffInSeconds(&ended, &started));
 }

LIBCOM_API int testImpreciseTiming ( void )

Return non-zero in shared/oversubscribed testing envrionments.

May be used to testSkip(), or select longer timeouts, for some cases when the test process may be preempted for arbitrarily long times. This is common in shared CI environments.

The environment variable $EPICS_TEST_IMPRECISE_TIMING=YES should be set in by such testing environments.

Definition at line 253 of file epicsUnitTest.c.

 {
     if(impreciseTiming==0) {
         const char* env = getenv("EPICS_TEST_IMPRECISE_TIMING");
 
         impreciseTiming = (env && strcmp(env, "YES")==0) ? 1 : -1;
     }
     return impreciseTiming>0;
 }

LIBCOM_API int testOk	(	int	pass,
		const char *	fmt,
			...
	)

Test result with printf-style description.

Parameters

pass	True/False value indicating result.
fmt	A printf-style format string describing the test.
...	Any parameters required for the format string.

Returns: The value of pass.

LIBCOM_API int testOkV	(	int	pass,
		const char *	fmt,
		va_list	pvar
	)

Test result with var-args description.

Parameters

pass	True/False value indicating result.
fmt	A printf-style format string describing the test.
pvar	A var-args pointer to any parameters for the format string.

Returns: The value of pass.

Definition at line 112 of file epicsUnitTest.c.

                                                      {
     const char *result = "not ok";
     epicsMutexMustLock(testLock);
     tested++;
     if (pass) {
         result += 4;    /* skip "not " */
         passed++;
         if (todo)
             bonus++;
     } else {
         if (todo)
             passed++;
         else
             failed++;
     }
     printf("%s %2d - ", result, tested);
     vprintf(fmt, pvar);
     if (todo)
         printf(" # TODO %s", todo);
     putchar('\n');
     fflush(stdout);
     epicsMutexUnlock(testLock);
     return pass;
 }

LIBCOM_API void testPass	(	const char *	fmt,
			...
	)

Passing test result with printf-style description.

Parameters

fmt	A printf-style format string describing the test.
...	Any parameters required for the format string.

Returns: The value of pass.

LIBCOM_API void testPlan ( int tests )

Declare the test plan, required.

Parameters

tests Number of tests to be run. May be zero if not known but the test harness then can't tell if the program dies prematurely.

Definition at line 102 of file epicsUnitTest.c.

                         {
     epicsThreadOnce(&onceFlag, testOnce, NULL);
     epicsMutexMustLock(testLock);
     planned = plan;
     tested = passed = failed = skipped = bonus = 0;
     todo = NULL;
     if (plan) printf("1..%d\n", plan);
     epicsMutexUnlock(testLock);
 }

LIBCOM_API void testSkip	(	int	skip,
		const char *	why
	)

Place-holders for tests that can't be run.

Parameters

skip	How many tests are being skipped.
why	Reason for skipping these tests.

Definition at line 159 of file epicsUnitTest.c.

                                          {
     epicsMutexMustLock(testLock);
     while (skip-- > 0) {
         tested++;
         passed++;
         skipped++;
         printf("ok %2d # SKIP %s\n", tested, why);
     }
     fflush(stdout);
     epicsMutexUnlock(testLock);
 }

LIBCOM_API void testTodoBegin ( const char * why )

Mark the start of a group of tests that are expected to fail.

Parameters

why	Reason for expected failures.

Definition at line 171 of file epicsUnitTest.c.

                                     {
     epicsMutexMustLock(testLock);
     todo = why;
     epicsMutexUnlock(testLock);
 }

LIBCOM_API void testTodoEnd ( void )

Mark the end of a failing test group.

Definition at line 177 of file epicsUnitTest.c.

                        {
     testTodoBegin(NULL);
 }

Functions

Announcing Test Results

Test Harness for Embedded OSs

Detailed Description

IOC Testing

Example

Macro Definition Documentation

Typedef Documentation

Function Documentation