Koha/t/db_dependent
Jonathan Druart 38f92df4e5 Bug 14722: Refactor the export tool
Why a refactoring was need for this script?
The export tool (tools/export.pl) can be called from the command line
and some parts of code were unnecessarity complicated (just look at the
code, you will understand).

Worse still, the script does not provide the same options for both
interface. For instance you cannot export records given a range of
biblionumbers, authids, callnumbers, etc. from the commandline.

What does this patch?
1/ Important: The script tools/export.pl does not work anymore if called from
the command line (should be in the release notes).
2/ The code used to generated a file (csv, iso2709 or xml) has been moved to a new
module (Koha::Exporter::Record) and tests have been provided.
3/ No change is done on the web interface
4/ Some new options have been added to the commandline script
(misc/export.pl):
    - starting_authid
    - ending_authid
    - authtype
    - starting_biblionumber
    - ending_biblionumber
    - itemtype
    - starting_callnumber
    - ending_callnumber
    - start_accession
    - end_accession
5/ There is a change in the behavior if an error occurs:
Can't call method "as_usmarc" on an undefined value at Koha/Exporter/Record.pm line 114.
record (number 5530) is invalid and therefore not exported because its reopening generates warnings above at Koha/Exporter/Record.pm line 117.

Before this patch, they were not displayed (using the command line).

What does not do this patch?
It does not provide the 'clean', 'timestamp' and 'deleted_barcodes' options to
the web interface (same as before).

What about the perfs?
With a DB with ~800 biblios (MARC21)
Before: perl tools/export.pl 14.79s user 0.83s system 71% cpu 21.905 total
After:  perl misc/export.pl  17.19s user 0.84s system 75% cpu 24.018 total

With a DB with ~6400 biblios (UNIMARC)
Before: perl tools/export.pl 26.55s user 0.76s system 76% cpu 35.498 total
After:  perl misc/export.pl  26.78s user 0.84s system 80% cpu 34.494 total

How to test this patch?
Test plan:
A. Web interface:
1/ On the current master, export some records, biblios and authorities (with
the 3 differents exports) playing with the different filters (item type,
libraries, callnumber, accession date, don't export items, remove
non-local items, don't export fields, etc.).
2/ Apply this patch, export again the same records, and compare the
generated files. They must be identical!
3/ Confirm that the export features on the checkout list
(circ/circulation.pl) works as before this patch.

B. The command line
1/ On the current master, export some records, biblios and authorities (with
the 2 differents exports) playing with the different options (date,
deleted_barcodes, clean).
2/ Apply this patch, export again the same records, and compare the
generated files. They must be identical!

Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2015-10-27 17:01:28 -03:00
..
Acquisition Bug 8417: Fix test 2015-10-21 08:46:20 -03:00
Circulation Bug 14788: Add unit tests for GetTopIssues 2015-10-22 11:35:06 -03:00
Creators Bug 14059: Deleting 'issues' table before processing the tests 2015-05-22 17:02:22 -03:00
data
Exporter Bug 14722: Refactor the export tool 2015-10-27 17:01:28 -03:00
Holds Bug 9809: Update unit tests 2015-08-26 10:26:29 -03:00
Items Bug 8723: Update the reserves table for item-level holds 2015-09-16 10:39:02 -03:00
Koha Bug 14888: Revert "Bug 14888: (QA followup) remove unused lib/var" 2015-10-05 15:05:44 -03:00
Labels Bug 14468: (QA followup) remove useless diags 2015-10-27 10:38:15 -03:00
LDAP
Letters Bug 13215: Fix GetLetterTemplates should return default templates if branchcode is not defined 2014-11-27 11:42:26 -03:00
Members Bug 15029: Make the IssueSlip tests pass even if launched after 23:00 2015-10-20 14:21:05 -03:00
OAI
Record Bug 12404: Allow equal sign '=' in the TT directive 2015-04-27 10:43:23 -03:00
Reports
Reserves Bug 14702: Unit tests for GetReserveFee and ChargeReservesFee 2015-09-07 12:04:49 -03:00
Search Bug 10933: (followup of the followup) fix the previous patch to work with master 2015-10-27 13:06:01 -03:00
Serials Bug 12671: Guess next serial date when there are several issues per unit. Added support for real month and year lenghts. 2015-05-05 15:22:10 -03:00
Template/Plugin
Utils Bug 14782: Add tests 2015-09-16 10:36:40 -03:00
www Bug 10363: (QA followups) Fix tests 2015-10-21 17:23:38 -03:00
XSLT_Handler
00-strict.t Bug 7844: (follow-up) don't test koha.psgi for ability to compile (yet) 2014-04-29 19:34:23 +00:00
Accounts.t Bug 11229: Make the tests pass even if issues exist in the DB 2015-08-18 13:18:21 -03:00
Acquisition.t
AdditionalField.t Bug 10855: Remove additional field values when subscription is removed 2015-10-02 15:10:31 -03:00
Amazon.t
Auth.t Bug 14439: Add test - template path should finish by .tt 2015-06-24 10:53:25 -03:00
Auth_with_cas.t Bug 14121: Silence warnings t/db_dependent/Auth_with_cas.t 2015-07-20 10:32:21 -03:00
Auth_with_ldap.t Bug 12831: (RM followup) regression test 2014-11-27 14:45:11 -03:00
AuthorisedValues.t Bug 10363: Use Koha::AuthorisedValue[s] in the admin page 2015-10-19 12:46:27 -03:00
AuthoritiesMarc.t
BackgroundJob.t
Barcodes.t Bug 13673: (QA followup) Add license 2015-05-15 16:00:28 -03:00
Barcodes_ValueBuilder.t Bug 14778: Make Barcodes_ValueBuilder.t db dependent 2015-10-23 12:01:19 -03:00
Biblio.t Bug 10961: (followup) Make query fields explicit and add regression tests 2015-07-28 10:19:51 -03:00
Bookseller.t
Borrower.t Bug 14778: Make 3 tests pass 2015-10-23 12:01:17 -03:00
Borrower_Debarments.t
Borrower_Discharge.t Bug 14368: [Follow-up] Fix workflow 2015-09-28 11:57:45 -03:00
Borrower_Files.t Bug 12417: adding the units tests of the module Koha::Borrower::Files.pm 2014-07-11 16:16:11 -03:00
Borrowers.t
Branch.t
Breeding.t Bug 12898: (regression test) test terms inside double quotes 2014-09-14 02:02:51 -03:00
Budgets.t
Category.t Bug 12604: refactoring Category.t with TestBuilder 2015-04-20 10:12:08 -03:00
Charset.t Bug 8218 : Add a maintenance script to sanitize biblio records 2014-11-11 15:38:36 -03:00
check_sysprefs.t
Circulation.t Bug 8236: (QA followup) fix test count error due to bad merge 2015-10-20 10:52:23 -03:00
Circulation_Branch.t Bug 14045: Make GetBranchBorrowerCircRule return maxonsiteissueqty 2015-10-13 11:13:24 -03:00
Circulation_dateexpiry.t Bug 14494: Add some unit tests too for CalcDateDue 2015-07-20 10:16:24 -03:00
Circulation_issue.t Bug 14640: Add tests 2015-09-16 11:00:17 -03:00
Circulation_Issuingrule.t Bug 13590: Add ability to charge fines at start of charge period 2015-10-22 14:51:24 -03:00
Circulation_issuingrules.t Bug 14778: Use mock_dbh where it should be used 2015-10-23 12:01:18 -03:00
Circulation_OfflineOperation.t Bug 10710: (follow-up) perltidy new code 2013-08-28 14:52:43 +00:00
Circulation_transfers.t Bug 12122: TransferSlip should accept both itemnumber and barcode 2015-03-05 11:47:12 +01:00
ClassSource.t Bug 10644: (follow-up) remove inadvertant dependence on Perl 5.14 2013-08-09 18:53:20 +00:00
ColumnsSettings.t
Context.t Bug 14778: Use mock_dbh where it should be used 2015-10-23 12:01:18 -03:00
Contract.t Bug 14693: Contract.t should generate dates based on current date 2015-08-21 10:43:38 -03:00
CourseReserves.t
Csv.t
default_search_class.pl
External_BakerTaylor.t Bug 5327: Unit tests for C4/Exteral/BakerTaylor.pm 2011-12-07 16:17:05 +01:00
Fines.t Bug 13590: Add ability to charge fines at start of charge period 2015-10-22 14:51:24 -03:00
FrameworkPlugin.t Bug 10480: New module and unit test for framework plugins 2015-04-16 14:48:35 -03:00
Heading_MARC21.t
Hold.t Bug 13853: Add tests for is_waiting 2015-08-11 14:48:44 -03:00
Holds.t Bug 5144: Add unit test 2015-10-06 10:01:45 -03:00
HoldsQueue.t Bug 14297: Unit Tests 2015-09-25 11:59:56 -03:00
Holidays.t Bug 14522: (QA followup) Remove POD and fix tests 2015-10-02 12:08:04 -03:00
ILSDI_Services.t
ImportBatch.t
Installer.t Bug 11541: (follow-up) move Installer.t to t/db_dependent 2014-01-14 21:43:56 +00:00
Items.t Bug 14778: Make 3 tests pass 2015-10-23 12:01:17 -03:00
Items_DelItem.t
KitchenSinkPlugin.kpz
Koha.t Bug 10937: cleanup and rename DOCTYPECAT to ITEMTYPECAT 2015-10-22 11:06:22 -03:00
Koha_Authority.t Bug 9978: Replace license header with the correct license (GPLv3+) 2015-04-20 09:59:38 -03:00
Koha_borrower_modifications.t Bug 12623: SQLHelper replacement - Borrower::Modifications 2014-08-11 11:42:58 -03:00
Koha_Database.t Bug 13645: Cache the DBIx connection 2015-02-20 14:37:08 -03:00
Koha_Misc_Files.t
Koha_template_plugin_Branches.t Bug 10900 - Follow up, since more has been added to master 2015-02-09 17:00:15 -03:00
Koha_template_plugin_KohaDates.t
Languages.t Bug 14030 - Added tests for duplicates. 2015-04-24 09:55:54 -03:00
Letters.t Bug 15028: Make t/db_dependent/Letters.t set the numbering pattern it needs 2015-10-19 11:52:10 -03:00
Linker_FirstMatch.t
Log.t
MarcModificationTemplates.t Bug 11413: Fix field_numbers 2014-11-14 12:05:47 -03:00
Members.t Bug 10020: Remove code related to ethnicity 2015-10-06 10:30:32 -03:00
Members_Attributes.t
Members_columns.t
MungeMarcPrice.t Bug 9593: (follow-up) don't make currency.isocode required 2014-05-04 22:21:17 +00:00
NewsChannels.t Bug 14248: Regression tests 2015-10-27 16:26:33 -03:00
Overdues.t Bug 9180: All branches should be returned if a default rule exists 2014-08-19 09:29:51 -03:00
PatronLists.t Bug 10565: (follow-up) don't assume borrowernumber 1 exists in unit test 2013-10-14 21:30:34 +00:00
Plugins.t
QueryParser.t Bug 12738: (regression tests) C4::Context should set keyword search as default for QueryParser 2014-08-11 10:05:59 -03:00
Ratings.t
Record.t Bug 12357: Following whitespace failures on koha qa test tools. 2015-09-07 13:59:07 -03:00
RecordProcessor_EmbedSeeFromHeadings.t Bug 14456: (regression test) Holdings fields should be skipped by EmbedSeeFromHeadings 2015-07-24 13:39:53 -03:00
Reports.t
Reports_Guided.t Bug 14778: Make 3 tests pass 2015-10-23 12:01:17 -03:00
ReportsGuided.t Bug 14778: Use mock_dbh where it should be used 2015-10-23 12:01:18 -03:00
Reserves.t Bug 5144: Add test on carp message 2015-10-06 10:02:06 -03:00
Review.t
rollingloans.t Bug 13315 - Add feedback for last item checked out to circulation.pl 2015-04-13 14:52:26 -03:00
RotatingCollections.t
Search.t Bug 14778: (QA followup) remove warnings 2015-10-26 11:18:04 -03:00
Serials.t Bug 13423: Fix number of tests in Serials.t 2015-05-14 11:48:17 -03:00
Serials_2.t
Service.t
ShelfBrowser.t Bug 10856: Improve the previous and next items on the shelf browser 2013-10-04 15:56:35 +00:00
SIP_ILS.t
Sitemapper.t Bug 14778: Example - Replace DBI::Mock with Test::DBIx::Class - Sitemapper.t 2015-10-23 12:01:18 -03:00
Stats.t Bug 11230 - Refactor C4::Stats::UpdateStats and add UT 2014-07-27 11:29:28 -03:00
SuggestionEngine_ExplodedTerms.t Bug 12653: remaining prog reference in t/db_dependent/SuggestionEngine_ExplodedTerms.t 2014-10-29 23:37:54 -03:00
Suggestions.t
sysprefs.t
Tags.t
Templates.t
TestBuilder.t Bug 15052: Add diagnostic for the TestBuilder tests 2015-10-23 09:35:13 -03:00
Upload.t Bug 14321: [QA Follow-up] Additional QA requests 2015-09-25 12:02:52 -03:00
VirtualShelves.t Bug 14383: Fix some typos in comments and documentation 2015-06-22 17:34:46 -03:00
VirtualShelves_Page.t Bug 5477 [SIGN-OFF] Fix test cases that require database access 2011-01-29 17:02:31 +13:00
XISBN.t Bug 12782: (followup) skip tests that fail because of provider's error 2014-08-20 12:29:27 -03:00
XSLT_Handler.t Bug 13262: (QA followup) fix duplicate variable definition 2015-02-05 15:25:50 -03:00
zebra_config.pl