Deep Unit Testing?

Unit testing helps you make sure that your code is working properly but the black-box approach has its limits. In fact, in a complex program with (unsurprisingly) complex behavior, black-boxing becomes a major hindrance to testing. So what are the options? There are several options, but they all seem to have their inconveniences; some violate the basic tenets of object oriented programming, some introduce additional occasions for bugs. I think that there’s no easy solution.

Exposing Internals. One possibility would be to give access to data that would otherwise be hidden using extra accessors. That’s still better than adding public: here and there to (permanently) expose hidden data because, at least, if you are careful, you can make them read-only. But adding accessors is cumbersome; it adds a lot of code, which also needs to be maintained. They also add to the complexity of your code and they are necessarily implementation-specific; change the internals of your class and you must change all the accessors even if, paradoxically, the interface remains the same.

Befriending. In C++, there’s the additional mechanism of declaring a class or function friend to a class to grant access to private members. friend also breaks encapsulation in one major way: it doesn’t restrict the type of access; a friend class or function can read and modify the private members. So despite claims of friend improving encapsulation, friend doesn’t seems much of a solution either. It might have if, for example, const friend would grant const access to the friend; but that syntax or concept isn’t part of C++ yet.

Conditional Access. In a previous entry I described how we can use the CPP to conditionally activate pieces of code depending on whether we are in debug, unit testing, or release mode. But with this, again, you have to be careful. You can make a private: transform into a public: the time of unit testing: that’s still much better than adding a series of accessors, but you still have to be extra careful to make sure the resulting code still works as it should in the other modes.

Isn’t the CPP Evil?. Yes. Worse, it still doesn’t solve the problem! Consider the following code snippet:

template <typename T>
class A
 {
  private:
   
    T * copy;

  public:

  ...

    void set(const T * other)
     {
      delete copy;
      copy = other->clone();
     }

  ...
  A() : copy(0) {}
 };

How do you test if set works properly? Merely examining the pointer copy doesn’t tell you much; if it was previously null, delete doesn’t do anything; and if the program doesn’t crash it doesn’t tell you that the previous value was a cromulent one; just that it didn’t crash the test. From the interface void set(const T *) you can’t decide what exactly the function does. OK, this is a simple example, simple code, but let’s pretend we’re doing black-box unit testing again. How do we make sure that the function frees its previous copy? For all we know, we can remove the delete statement and the unit test would succeed just fine. Except that now, the function leaks memory.

*
* *

I know no clean solutions. The solutions I came up with necessitate that we modify T so that it can count instances of itself. The instances of T that are copied (via the clone function, which is necessary for copy-constructing classes that can be derived, as C++ doesn’t allow virtual copy constructors) must increase an internal counter; similarly, the destructor must decrease it.

This also is quite bad because it breaks the black-boxing of T as well as of A. It also introduces cross-cutting concerns which are also difficult to maintain in the large scale. Why would T need a static data member that counts its instances because somewhere else a class could, maybe, make copies? Should we have counters and related accessors available only in debug or unit testing mode? Should we overload operator new and operator delete to contain instance-tracking code? How do we report results?

To check if all instances have been properly deallocated, we must add yet more code. Just before exiting main—and should we suppose that the program has only one exit point? 1—we add code to check if all debugged classes have all their instances freed. That’s also pretty cumbersome.

*
* *

So, tell me, reader, what are your techniques.


1 You can use the atexit function to register functions to be called just before the program exits.

2 Responses to Deep Unit Testing?

  1. pete says:

    And you haven’t touched on exceptions. The above code, for example, is catastrophically broken if T::clone throws (and anything that calls operator new could throw std::bad_alloc).

    Anyway, there is no perfect solution: engineering is an art born in the choice of compromise. But a couple of special debugging accessors can work well, e.g.

    #ifdef INCLUDE_INSTRUMENTATION
    tuple get_stage1_params_UNIT_TEST_CODE_ONLY() const
    { return make_tuple( member1, member2, meber3 ); }
    #endif

    It’s not too hard to maintain. There’s not too much line noise to clog up the code. It may not be super-efficient but that’s not so much of an issue for test code, and even there, if you’re testing x, you’re probably testing y, and a modern optimising compiler can spot that z is untouched and elide it. And anybody trying to use T::get_stage1_params_UNIT_TEST_CODE_ONLY() in production is reproached by the name; in fact calling a method xxxx_hack_DO_NOT_USE is remarkably good protection against misuse when you need to break encapsulation.

  2. Steven Pigeon says:

    You’re right about exceptions… that’s another thing people do not grok much and, unfortunately, a lot of “C++ programmers” are merely doing C with classes-objects-thingies. And often those who actually know C++ have to deal with the others on the team that aren’t too hot about C++. (and you have to be mindful of them as well; if you change their C-ish code to more bona fide C++, they might see it as a personal attack.)

    But I like the trick of bundling values like that. That, indeed, minimizes the number of accessors, and you can also make them read-only. Good idea.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: