switching from 'prove' to 'perl' to run the test scripts when we're preparing
them to be submitted to smolder. Prove adds an additional summary at the end
of the test run which confuses the TAP parser in smolder. It thinks that an
additional test has been run, and it fails it.
Then, I added a '-' before the command in the Makefile. THat's so that even
if 'perl' returns a failing code (because some tess failed), the rest of the
rules are run and a report gets submitted. Othewise, only passing reports
get sent.
Signed-off-by: Galen Charlton <galen.charlton@liblime.com>
This patch adds a target in the t/Makefile and a script in t/smolder_smoke_signal
to submit the results of test runs to a smolder server. It will allow users to submit
their own test results to a central location by running "make submit" from the t/
directory.
Smolder is a web-based application that aggregates and presents the output of
tests. We have an instance running at:
http://arwen.metavore.com:8000/app/public_projects/smoke_reports/1
More information about using smolder for Koha is available at:
http://wiki.koha.org/doku.php?id=en:development:smoke_reports
I intend to have the test suite run each night automatically and the results
sent to the smolder server. This will help us keep track of the current state
of the the code and possibly catch bugs introduced by changes. I encourge others
to submit their own test results if they want, especially if they are on an unusual
platform or are with a non-standard configuration.
Signed-off-by: Galen Charlton <galen.charlton@liblime.com>
Added support for a new target in the test Makefile
to run a single test module. If you do (for example)
make test-single TEST_FILES=lib/KohaTest/Biblio.pm
only the tests in that module will be run. Unlike
the full test suite as run via 'make test', 'make test-single'
does not clear the test database before running the
tests.
Please note that "TEST_FILES=path/to/test/class.pm" is
required when using 'make test-single'.
Signed-off-by: Andrew Moore <andrew.moore@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
Defined a function attribute for KohaTest and subclasses
called 'Expensive'. When a test method has that attribute,
it is skipped unless the RUN_EXPENSIVE_TESTS environment
variable is true.
To mark a test method, expensive, do this:
sub test_foo : Tests(4) Expensive { ... }
To mark a whole class and its subclasses expensive,
define a SKIP_CLASS sub (with empty body) with the
Expensive attribute:
sub SKIP_CLASS : Expensive { }
Updated the t/Makefile so that 'make test' runs
nonexpensive tests, while 'make fulltest' runs both
cheap and expensive tests.
Marked KohaTest::Installer test class expensive.
Signed-off-by: Andrew Moore <andrew.moore@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
[1] When running the database-dependent tests (cd t, make test),
all tables in the test database are dropped prior to running
the installer and test cases. This means that the test
database will start with a clean slate.
[2] It is now possible to specify a single test class to run,
to avoid having to run all of them:
cd t
make test TEST_CLASS=Search
To run all DB-dependent tests, just do the usual
cd t
make test
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
This patch makes the test suite work even when $PERL5LIB isn't set to the install directory.
It replaces $PERL5LIB with '..' in order to help ensure that we're loading the C4 libraries
out of the installation directory.
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
* All configuration and run-time files created during
a test suite run now live under t/run
* made sure scripts copied to t/run/bin are executable
* added t/run and t/test-config.txt to .gitignore
Signed-off-by: Andrew Moore <andrew.moore@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
I'm adding a bunch of additions to the test suite. There is now a test
database that is created, and a test instance of the zebra server and
daemon that are run before the tests in t/database-dependent.pl are
run. This means that you can test things that insert (or destroy)
things in the database.
To use these tests, after you 'make' your koha installation, you can
change to the 't' directory and 'make test' there.
There is now an additional question asked during installation. It asks
whether you would like to run the data-dependent tests. If so, It asks
you for some login information to a test database. I recommend that
you do not not use your production database here. It will delete all
of the data in it.
Things that need improvement:
* I suspect that there are large parts that are not very platform
independent, so they will need to be improved.
* There are some parts of the installer that will probably be
refactored to let this work a little bit better, including being run
directly from a main-level 'make' target at some point.
* Lots more tests to add. Be bold! (see the t/lib directory)
* other tests in t/*.t can possibly benefit from being included in here.
There are two required perl modules for this part of the test suite. I
use:
'Test::Class' => 0.028,
'Test::Class::Load' => 0.02,
They are not listed as requirements in the top-level Makefile.PL
because they're not actually required to install or use Koha, but if
you want to run the test suite, you can install them from CPAN.
Signed-off-by: Galen Charlton <galen.charlton@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
