Bug 16155: Composite keys in TestBuilder and more

Bug 16155: Composite keys in TestBuilder and more

The number of tests using TestBuilder is gradually growing. Once you
are familiar with its use, you will appreciate it and find yourself
using it when writing new tests. Although it works quite well, some details
still needs some polishing.

While looking at the handling of composite keys in TestBuilder, a number
of related points came up too. This patch finally ended up in setting
the following design goals:

[1] TB should not only look at the first column of a composite FK.
[2] TB should optionally create records for fixed FK values.
[3] TB should create a new record, never change an existing record.
[4] TB should respect auto_increment columns.
[5] Passing a hash for a foreign key should always imply a new record.
[6] Explicitly passing undef for a column should mean NULL.
[7] Add a delete method; it will replace the clear method.
[8] Simplify by removing $default_values, hash key _fk and param only_fk.

The comments below further clarify these points. Note that they refer to
the old behavior (before this patch) unless stated otherwise.

Comments for point 1
====================
Look at:
    $builder->build({
        source => 'UserPermission',
        value => {
            borrowernumber => $borrowerno,
            module_bit => { flag => 'my flag' },
            code => 'will_be_ignored',
        },
    });
Module_bit and code here are a composite FK to permissions.
TB ignores the value for the code column here and creates a record with a
randomized code.

But if we would pass the hash in the second column of this compound key like:
            borrowernumber => $borrowerno,
            module_bit => 10,
            code => { code => 'new_code' },
TB would now crash when passing the hash for code thru to DBIx.

Comments for point 2
====================
Look at:
    $builder->build({
        source => 'UserPermission',
        value => {
            borrowernumber => $borrowerno,
            module_bit => 99,
            code => 'new_super_tool',
        },
    });
TB detects a fixed value for the module_bit, continues and will crash on
DBIx if the foreign keys are not found. In this case it would be friendly
to create a missing linked record.

Comments for point 3
====================
Look at:
    $builder->build({
        source => 'Branch',
        value  => { branchcode => 'CPL' },
    });
If this branch already exists, this call would modify the branch record and
overwrite all columns with randomized values (expected or not). In any case,
it would be safer here to return undef than modifying an existing record.

Comments for point 4
====================
Look at:
    $builder->build({
        source => 'Borrower',
        value  => { borrowernumber => '123456789' },
    });
If this number would exist, we would update (as earlier). But if this
number does not exist, we would create the record. Although that is
technically possible, I would prefer to have TB respect the auto
increment property of this column.

Comments for point 5
====================
Look at:
    $builder->build({
        source => 'Item',
        value  => { homebranch => { branchcode => 'MPL' } },
    });
As discussed under point 3, we should actually not pass primary key values
if we expect to build new records. The same also holds for the deeper
(recursive) calls to build when using hashes like here for homebranch.
In this case again an existing record might be overwritten.

Comments for point 6
====================
Look at:
    $builder->build({
        source => 'Reserve',
        value => { itemnumber => undef },
    });
As you know, a reserve without an itemnumber is a biblio level hold.
Unfortunately, TB did not care about passing undefs until now. So you would
get an item level hold.
In the new situation TB will respect these explicit undefs, as long as the
corresponding foreign key column is nullable of course.

Comments for point 7
====================
This patch will allow you to delete records created by TB:
    my $patron = $builder->build({ source => 'Borrower' });
    $builder->delete({ source => 'Borrower', records => $patron });
Or:
    $builder->delete({ source => 'Borrower', records => [ $patron, ... ] });
For safety, delete requires you to provide all primary key values in the
passed hashref(s).
Deleting all records in a table via clear is no longer supported and can
still be arranged in one statement.

Comments for point 8
====================
Current use of TestBuilder reveals that $default_values and only_fk
are not really needed. The current $default_values should imo not be in the
module anyway; if you want to use it, you could still pass it to TB:
    $builder->build({ ..., value => { %defa, %your_values } });

Only_fk stops at the very last step of saving the top level record while
storing all linked records at the lower levels. Practical use is not
very obvious; it can be easily simulated by one delete statement.

The hash key _fk is now used to store all linked records one or more levels
down. It is used in a few tests to retrieve a value one level down.
Why not retrieve that one value via the database and get rid of the
whole structure?

Implementation
==============
This highlights the main changes:

The $default_value hash (with some hardcoded values for UserPermission)
is removed from the module. It was used by a test in TestBuilder.t and has
been relocated.
The value of $gen_type is returned now by sub _gen_type. (See new.)

The main change in the build method is moving the foreign keys logic to a
new subroutine _create_links. This routine now looks at all columns of a
composite FK. It checks if a linked record exists for passed values, and
it looks at NULL values.

Routine _buildColumnValues is slightly adjusted to allow for passed undef
values. The theoretically endless loop is replaced by three tries. For
composite unique constraints we only check complete sets of values.

Routine _getForeignKeys contains a check to not return the same relation
twice in case of doubled belongs_to relations in the DBIx scheme.

The eval in _storeColumnValues is removed. The autoincrement check in sub
_buildColumnValue got more priority; the handling of foreign keys has been
adjusted and a check for not-nullable columns has been added.

TEST PLAN
=========
This patch only deals with the TestBuilder module itself. In the follow-up
patches TestBuilder.t and a few other unit tests are adjusted.

[1] Do not yet apply this patch. But apply the 'OldBehavior' patch.
    Verify that all tests pass. (They cover the first six design goals.)
[2] Apply this patch. Does TestBuilder still compile (perl -c)?
[3] Run the OldBehavior test again. Do all tests fail now? This means
    that we got rid of all unwanted side-effects in the list of goals.
[4] Run some other tests that use TestBuilder. (See below.)
    Skip the following tests; a follow-up patch deals with them.
    t/db_dependent/Accounts.t
    t/db_dependent/Barcodes.t
    t/db_dependent/Circulation/AnonymiseIssueHistory.t
    t/db_dependent/Circulation/CalcFine.t
    t/db_dependent/Holds.t
    t/db_dependent/Items/MoveItemFromBiblio.t
    t/db_dependent/Koha/BiblioFrameworks.t
    t/db_dependent/Members.t
    t/db_dependent/TestBuilder.t

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
The following tests pass:
    t/db_dependent/Acquisition/OrderUsers.t
    t/db_dependent/Auth/haspermission.t
    t/db_dependent/Barcodes_ValueBuilder.t
    t/db_dependent/Budgets.t
    t/db_dependent/Category.t
    t/db_dependent/Circulation.t
    t/db_dependent/Circulation/GetTopIssues.t
    t/db_dependent/Circulation/IsItemIssued.t
    t/db_dependent/Circulation/IssuingRules/maxsuspensiondays.t
    t/db_dependent/Circulation/Returns.t
    t/db_dependent/Circulation/TooMany.t
    t/db_dependent/Circulation_dateexpiry.t
    t/db_dependent/Circulation_transfers.t
    t/db_dependent/CourseReserves.t
    t/db_dependent/Creators/Lib.t
    t/db_dependent/DecreaseLoanHighHolds.t
    t/db_dependent/Exporter/Record.t
    t/db_dependent/Holds/LocalHoldsPriority.t
    t/db_dependent/Holds/RevertWaitingStatus.t:
    t/db_dependent/HoldsQueue.t
    t/db_dependent/Holidays.t
    t/db_dependent/Items.t
    t/db_dependent/Items_DelItem.t
    t/db_dependent/Koha/Acquisition/Currencies.t
    t/db_dependent/Koha/Authorities.t
    t/db_dependent/Koha/BiblioFrameworks.t
    t/db_dependent/Koha/Cities.t
    t/db_dependent/Koha/Libraries.t
    t/db_dependent/Koha/Objects.t
    t/db_dependent/Koha/Patron/Categories.t
    t/db_dependent/Koha/Patron/Images.t
    t/db_dependent/Koha/Patron/Messages.t
    t/db_dependent/Koha/Patrons.t
    t/db_dependent/Koha/SMS_Providers.t
    t/db_dependent/Koha_template_plugin_Branches.t
    t/db_dependent/Letters.t
    t/db_dependent/Members/AddEnrolmentFeeIfNeeded.t
    t/db_dependent/Members/GetUpcomingMembershipExpires.t
    t/db_dependent/Members_Attributes.t
    t/db_dependent/Patron/Borrower_Debarments.t
    t/db_dependent/Patron/Borrower_Discharge.t
    t/db_dependent/Patron/Borrower_Files.t
    t/db_dependent/Ratings.t
    t/db_dependent/Reports_Guided.t
    t/db_dependent/Reserves/GetReserveFee.t
    t/db_dependent/Review.t
    t/db_dependent/Serials_2.t
    t/db_dependent/ShelfBrowser.t
    t/db_dependent/Virtualshelves.t
    t/db_dependent/api/v1/patrons.t

Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>