File Coverage

blib/lib/Test/LectroTest.pm

Criterion	Covered	Total	%
statement	27	27	100.0
branch	2	4	50.0
condition			n/a
subroutine	8	8	100.0
pod			n/a
total	37	39	94.8

package Test::LectroTest;
{
  $Test::LectroTest::VERSION = '0.5001';
}

use warnings;
use strict;

use Test::LectroTest::TestRunner;
use Filter::Util::Call;
require Test::LectroTest::Property;
require Test::LectroTest::Generator;

=head1 NAME

Test::LectroTest - Easy, automatic, specification-based tests

=head1 VERSION

version 0.5001

=head1 SYNOPSIS

    #!/usr/bin/perl -w

    use MyModule;  # contains code we want to test
    use Test::LectroTest;

    Property {
        ##[ x <- Int, y <- Int ]##
        MyModule::my_function( $x, $y ) >= 0;
    }, name => "my_function output is non-negative" ;

    Property { ... }, name => "yet another property" ;

    # more properties to check here

=head1 DESCRIPTION

This module provides a simple (yet full featured) interface to
LectroTest, an automated, specification-based testing system for Perl.
To use it, declare properties that specify the expected behavior
of your software.  LectroTest then checks your software to see whether
those properties hold.

Declare properties using the C function, which takes a
block of code and promotes it to a L:

    Property {
        ##[ x <- Int, y <- Int ]##
        MyModule::my_function( $x, $y ) >= 0;
    }, name => "my_function output is non-negative" ;

The first part of the block must contain a generator-binding
declaration.  For example:

        ##[  x <- Int, y <- Int  ]##

(Note the special bracketing, which is required.)  This particular
binding says, "For all integers I and I."  (By the way, you
aren't limited to integers.  LectroTest also gives you booleans,
strings, lists, hashes, and more, and it lets you define your own
generator types.  See L for more.)

The second part of the block is simply a snippet of code that makes
use of the variables we bound earlier to test whether a property holds
for the piece of software we are testing:

        MyModule::my_function( $x, $y ) >= 0;

In this case, it asserts that C returns
a non-negative result.  (Yes, C<$x> and C<$y> refer to the same I
and I that we bound to the generators earlier.  LectroTest
automagically loads these lexically bound Perl variables with values
behind the scenes.)

B If you want to use testing assertions like C from
L or C, C, or C from L
(and the related family of L-based testing modules),
see L, which lets you mix and match
LectroTest with these modules.

Finally, we give the whole Property a name, in this case "my_function
output is non-negative."  It's a good idea to use a meaningful name
because LectroTest refers to properties by name in its output.

Let's take a look at the finished property specification:

    Property {
        ##[ x <- Int, y <- Int ]##
        MyModule::my_function( $x, $y ) >= 0;
    }, name => "my_function output is non-negative" ;

It says, "For all integers I and I, we assert that my_function's
output is non-negative."

To check whether this property holds, simply put it in a Perl program
that uses the Test::LectroTest module.  (See the L for an
example.)  When you run the program, LectroTest will load the property
(and any others in the file) and check it by running random trials
against the software you're testing.

B If you want to place LectroTest property checks into
a test plan managed by L-based modules such as
L or L, see L.

If LectroTest is able to "break" your software during the property
check, it will emit a counterexample to your property's assertions and
stop.  You can plug the counterexample back into your software to
debug the problem.  (You might also want to add the counterexample to
a list of regression tests.)

A successful LectroTest looks like this:

  1..1
  ok 1 - 'my_function output is non-negative' (1000 attempts)

On the other hand, if you're not so lucky:

  1..1
  not ok 1 - 'my_function output is non-negative' falsified \
      in 324 attempts
  # Counterexample:
  # $x = -34
  # $y = 0

=head1 EXIT CODE

The exit code returned by running a suite of property checks is the
number of failed checks.  The code is 0 if all properties passed their
checks or I if I properties failed. (If more than 254 properties
failed, the exit code will be 254.)


=head1 ADJUSTING THE TESTING PARAMETERS

There is one testing parameter (among others) that you might wish to
change from time to time: the number of trials to run for each
property checked.  By default it is 1,000.  If you want to try more or
fewer trials, pass the C>I flag:

  use Test::LectroTest trials => 10_000;


=head1 TESTING FOR REGRESSIONS AND CORNER CASES

LectroTest can record failure-causing test cases to a file, and it can
play those test cases back as part of its normal testing strategy.
The easiest way to take advantage of this feature is to set the
I parameter when you C this module:

    use Test::LectroTest
        regressions => "regressions.txt";

This tells LectroTest to use the file "regressions.txt" for both
recording and playing back failures.  If you want to record and
play back from separate files, or want only to record I play
back, use the I and/or
I options:

    use Test::LectroTest
        playback_failures => "regression_suite_for_my_module.txt",
        record_failures   => "failures_in_the_field.txt";

See L for more.


=head1 CAVEATS

When you use this module, it imports all of the generator-building
functions from L into the your code's
namespace.  This is almost always what you want, but I figured I ought
to say something about it here to reduce the possibility of surprise.

A Property specification must appear in the first column, i.e.,
without any indentation, in order for it to be automatically loaded
and checked.  If this poses a problem, let me know, and this
restriction can be lifted.

=cut

our $r;
our @props;
our @opts;

sub import {
    my $self = shift;
    Test::LectroTest::Property->export_to_level(1, $self);
    Test::LectroTest::Generator->export_to_level(1, $self, ':all');
    @opts = @_;
    $r = Test::LectroTest::TestRunner->new( @_ );
    my $lines = 0;
    my $subfilter = Test::LectroTest::Property::_make_code_filter();
    filter_add( sub {
        my $status = filter_read();
        s{^(?=Test|Property)\b}{push \@Test::LectroTest::props, };
        $subfilter->( $status );
    });
}

sub _run {
    return @props - $r->run_suite( @props, @opts );
}

END {
    if ($r) {
        my $failed = Test::LectroTest::_run();
        $? = $failed > 254 ? 254 : $failed;
    }
}

1;

__END__