[Nobug] coverage testing, are all failures covered?

[Christian Thaeter]

Christian Thaeter wrote:
> Just a bit brainstorming for developers:
> 
> NoBug has the fault-injection features, we don't use it yet for Lumiera
> testing, I am thinking about making it little more useful/automatic.
> 
> The Plan:
>  - load an (initially empty) log which contains backtraces of
>    already tested fault-injection points
>  - at an fault injection point, check if this point was already tested
>    - if not, then log it and inject a fault
>    - if yes, do nothing
>  - rinse repeat with the new log .. towards disovering every possibly
>    way leading to a fault injection point
> 

I've implemented this now in a somewhat different way, it's currently
available in my 'coverage' branch. No (much) documentation written yet.

As soon I have time to write documentation I will merge this into the
master branch. It will be proclaimed as /experimental/ feature for the
time being. Anyways this fault-coverage cheking will become part of the
next NoBug release then.

In Lumiera I started to use it a bit to get a feeling about its
usefulness, it is quite fun to see the test permutate over all possible
faults. Actually I've discovered a few minor problems in Lumiera so I
feel confident that this becomes a useable tool. It also shown that
sometimes false positives are raised, to account for that i'd added
coverage DISABLE/ENABLE facilities to nobug.

	Christian