Koha/t
Jonathan Druart 38f92df4e5 Bug 14722: Refactor the export tool
Why a refactoring was need for this script?
The export tool (tools/export.pl) can be called from the command line
and some parts of code were unnecessarity complicated (just look at the
code, you will understand).

Worse still, the script does not provide the same options for both
interface. For instance you cannot export records given a range of
biblionumbers, authids, callnumbers, etc. from the commandline.

What does this patch?
1/ Important: The script tools/export.pl does not work anymore if called from
the command line (should be in the release notes).
2/ The code used to generated a file (csv, iso2709 or xml) has been moved to a new
module (Koha::Exporter::Record) and tests have been provided.
3/ No change is done on the web interface
4/ Some new options have been added to the commandline script
(misc/export.pl):
    - starting_authid
    - ending_authid
    - authtype
    - starting_biblionumber
    - ending_biblionumber
    - itemtype
    - starting_callnumber
    - ending_callnumber
    - start_accession
    - end_accession
5/ There is a change in the behavior if an error occurs:
Can't call method "as_usmarc" on an undefined value at Koha/Exporter/Record.pm line 114.
record (number 5530) is invalid and therefore not exported because its reopening generates warnings above at Koha/Exporter/Record.pm line 117.

Before this patch, they were not displayed (using the command line).

What does not do this patch?
It does not provide the 'clean', 'timestamp' and 'deleted_barcodes' options to
the web interface (same as before).

What about the perfs?
With a DB with ~800 biblios (MARC21)
Before: perl tools/export.pl 14.79s user 0.83s system 71% cpu 21.905 total
After:  perl misc/export.pl  17.19s user 0.84s system 75% cpu 24.018 total

With a DB with ~6400 biblios (UNIMARC)
Before: perl tools/export.pl 26.55s user 0.76s system 76% cpu 35.498 total
After:  perl misc/export.pl  26.78s user 0.84s system 80% cpu 34.494 total

How to test this patch?
Test plan:
A. Web interface:
1/ On the current master, export some records, biblios and authorities (with
the 3 differents exports) playing with the different filters (item type,
libraries, callnumber, accession date, don't export items, remove
non-local items, don't export fields, etc.).
2/ Apply this patch, export again the same records, and compare the
generated files. They must be identical!
3/ Confirm that the export features on the checkout list
(circ/circulation.pl) works as before this patch.

B. The command line
1/ On the current master, export some records, biblios and authorities (with
the 2 differents exports) playing with the different options (date,
deleted_barcodes, clean).
2/ Apply this patch, export again the same records, and compare the
generated files. They must be identical!

Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2015-10-27 17:01:28 -03:00
..
Acquisition Bug 14778: Remove t/Acquisition/Invoices.t 2015-10-23 12:01:19 -03:00
Biblio Bug 11247: Improve tests 2015-09-02 14:36:33 -03:00
Budgets Bug 12019: ensure that it is optional to define an owner if a fund is restricted 2014-05-03 19:00:08 +00:00
Circulation Bug 13106 - Encapsulate Circulation::GetAgeRestriction() and modify it to check borrowers age as well. 2014-11-11 09:52:59 -03:00
DataTables Bug 9811: Patron search improvement 2014-07-01 09:57:09 -03:00
db_dependent Bug 14722: Refactor the export tool 2015-10-27 17:01:28 -03:00
External Bug 14147: Add unit tests to C4::External::OverDrive 2015-05-14 11:11:58 -03:00
Koha/Plugin Bug 12412: Add ability for plugins to convert arbitrary files to MARC from record staging tool 2015-05-04 15:33:51 -03:00
lib Bug 14778: Mocks Koha::Database->_new_schema instead of C4::Context->_new_dbh 2015-10-23 12:01:18 -03:00
Members Bug 14778: Use mock_dbh where it should be used 2015-10-23 12:01:18 -03:00
Number Bug 12844: Force the symbol place in the tests 2014-10-27 12:56:30 -03:00
Search Bug 10807: Add an authority search history for the OPAC 2014-05-05 02:32:27 +00:00
Serials Bug 7688: (follow-up) rewrite 'for' loop in unit tests 2013-10-30 00:38:52 +00:00
00-checkdatabase-version.t Bug 9978: Replace license header with the correct license (GPLv3+) 2015-04-20 09:59:38 -03:00
00-deprecated.t
00-load.t Bug 7904 Change SIP modules to use standard LIB path 2015-02-05 14:44:54 -03:00
00-merge-conflict-markers.t Bug 9978: Replace license header with the correct license (GPLv3+) 2015-04-20 09:59:38 -03:00
00-testcritic.t Bug 8798: (follow-up) remove DBIC_DONT_VALIDATE_RELS 2013-10-14 22:51:44 +00:00
00-valid-xml.t Bug 9978: Replace license header with the correct license (GPLv3+) 2015-04-20 09:59:38 -03:00
Auth_with_shibboleth.t Bug 14111 - More t/Auth_with_shibboleth.t silencing 2015-06-04 10:12:03 -03:00
AuthoritiesMarc_MARC21.t
AuthoritiesMarc_UNIMARC.t
AuthUtils.t Bug 9611: (follow-up) move new password hashing routines to separate module 2013-10-09 03:29:22 +00:00
Barcodes_annual.t
Barcodes_EAN13.t
Barcodes_hbyymmincr.t Bug 5327: More basic unit tests for C4 modules 2010-11-01 15:35:50 +13:00
Barcodes_incremental.t
Biblio.t Bug 14778: Install fixtures for t/Biblio.t 2015-10-23 12:01:18 -03:00
Bookseller.t
Boolean.t Bug 13814: (QA followup) test for generated warnings 2015-03-15 09:03:33 -03:00
Borrower.t Bug 14778: Use mock_dbh where it should be used 2015-10-23 12:01:18 -03:00
Branch.t
Breeding.t Bug 13279: t/Breeding.t shouldn't depend on the DB 2014-11-20 09:39:53 -03:00
Budgets.t
Cache.t Bug 12800: running unit tests that use Koha::Cache breaks cache usage from Apache 2014-08-24 12:48:34 -03:00
Calendar.t Bug 14778: Install fixtures for t/Calendar.t 2015-10-23 12:01:18 -03:00
Charset.t Bug 14112: Silence warnings in t/Charset.t 2015-05-27 14:32:41 -03:00
Circulation_barcodedecode.t Bug 14778: Use mock_dbh where it should be used 2015-10-23 12:01:18 -03:00
ClassSortRoutine.t
ClassSortRoutine_Dewey.t Bug 8837 [QA Followup] - Add unit tests 2014-10-28 09:39:44 -03:00
ClassSortRoutine_Generic.t
ClassSortRoutine_LCC.t bug 6281: introduce LC::CallNumber::LC for sorting LC call numbers 2013-06-10 07:56:53 -07:00
ClassSource.t
Context.t Bug 11848: Fix C4::Context::interface, add POD and UT 2014-05-05 04:32:31 +00:00
Contract.t
Creators.t Bug 12194: add more tests for pdf creation 2015-10-22 09:42:21 -03:00
Dates.t Bug 14113: Silence t/Dates.t warnings and cleanup output. 2015-06-01 14:13:54 -03:00
DateUtils.t Bug 13933: Fix t/DateUtils.t without DB 2015-04-01 13:10:16 -03:00
Debug.t Bug 14114: Silence warns and cleanup t/Debug.t 2015-05-22 09:33:34 -03:00
dummy.t
External_Syndetics.t
Form_MessagingPreferences.t
Heading.t
Images.t Bug 14778: Install fixtures for t/Images.t 2015-10-23 12:01:18 -03:00
ImportBatch.t Bug 13281: t/ImportBatch.t shouldn't depend on the DB 2014-11-20 09:53:36 -03:00
Installer_PerlDependencies.t
Installer_PerlModules.t bug 11544: increasing test coverage for Installer_PerlModules.pm 2014-01-14 21:48:43 +00:00
Installer_pm.t
ItemCirculationAlertPreference.t
ItemType.t Bug 14778: Install fixtures for t/ItemType.t 2015-10-23 12:01:18 -03:00
Koha.t Bug 14778: Install fixtures for t/Koha.t 2015-10-23 12:01:18 -03:00
Koha_Email.t Bug 13172 t/Koha_Email.t was database dependent 2014-10-31 12:10:32 -03:00
Koha_MetadataRecord.t Bug 14639: (QA followup) make schema mandatory 2015-09-07 12:06:21 -03:00
Koha_Template_Plugin_Cache.t Bug 11452: Add unit tests for Koha::Template::Plugin::Cache 2014-01-10 15:42:23 +00:00
Koha_Template_Plugin_Koha.t Bug 13758: (QA followup) revert case change that broke the tests 2015-05-07 14:12:14 -03:00
Koha_Util_FrameworkPlugin.t Bug 13437: Preliminary changes for marc21 plugins field 008 2015-06-10 12:51:26 -03:00
Koha_Util_MARC.t Bug 9978: Replace license header with the correct license (GPLv3+) 2015-04-20 09:59:38 -03:00
Labels.t Bug 9978: (followup) Replace license header with the correct license (GPLv3+) 2015-04-20 09:59:43 -03:00
Labels_split_ccn.t Bug 9978: (followup) Replace license header with the correct license (GPLv3+) 2015-04-20 09:59:43 -03:00
Labels_split_ddcn.t Bug 9978: (followup) Replace license header with the correct license (GPLv3+) 2015-04-20 09:59:43 -03:00
Labels_split_lccn.t Bug 9978: (followup) Replace license header with the correct license (GPLv3+) 2015-04-20 09:59:43 -03:00
Languages.t Bug 11944: use CGI( -utf8 ) everywhere 2015-01-13 13:07:21 -03:00
Letters.t Bug 14778: Install fixtures for t/Letters.t 2015-10-23 12:01:18 -03:00
Log.t Bug 11537: (follow-up) mock C4::Context::preference 2014-01-14 19:27:07 +00:00
Logger.t Bug 14167: (QA followup) use warn instead of just STDERR 2015-07-21 10:50:17 -03:00
Matcher.t Bug 14778: Install fixtures for t/Matcher.t 2015-10-23 12:01:18 -03:00
Members_Attributes.t
Members_AttributeTypes.t Bug 14778: Install fixtures for t/Members_AttributeTypes.t 2015-10-23 12:01:18 -03:00
Members_Messaging.t
Message.t Bug 13282: t/Message.t shouldn't depend on the DB 2014-11-20 09:38:02 -03:00
NorwegianPatronDB.t Bug 11401: (followup) make tests run on absent deps 2014-11-14 15:32:25 -03:00
Output.t Bug 11944: use CGI( -utf8 ) everywhere 2015-01-13 13:07:21 -03:00
Output_JSONStream.t Bug 11480: Fix invalid assumptions in JSONStream test 2014-01-10 15:42:17 +00:00
Overdues.t Bug 13283: t/Overdues.t shouldn't depend on the DB 2014-11-20 09:37:39 -03:00
Patroncards.t
Patroncards_Batch.t
Patroncards_Layout.t
Patroncards_Lib.t
Patroncards_Patroncard.t
Patroncards_Profile.t
Patroncards_Template.t
perlcriticrc
Prices.t Bug 12976: Fix some comment in tests 2015-02-19 09:46:44 -03:00
Print.t bug 11542: Increase test coverage for Print.pm 2014-01-14 20:41:31 +00:00
QueryParser.t Bug 12647: PQF QueryParser driver and unit tests fixes 2014-08-11 10:09:38 -03:00
RecordProcessor.t Bug 14646: (regression test) Koha::RecordProcessor only accepts one filter at a time 2015-08-20 13:44:15 -03:00
Review.t
Ris.t
RotatingCollections.t Bug 13284: t/RotatingCollections.t shouldn't depend on the DB 2014-11-20 09:37:20 -03:00
Scheduler.t
Scrubber.t Bug 14116: Silence noisy output for t/Scrubber. 2015-06-01 14:10:56 -03:00
Search.t Bug 13278: (QA followup) use t::lib::Mocks 2014-11-19 18:11:58 -03:00
Search_PazPar2.t Bug 14117: Silence warnings t/SearchPazPar2.t 2015-06-01 14:10:14 -03:00
SimpleMARC.t Bug 14098: Implement the copy_and_replace action for MTT 2015-09-07 11:17:13 -03:00
SIP_Sip.t Bug 7904 Change SIP modules to use standard LIB path 2015-02-05 14:44:54 -03:00
smolder_smoke_signal
SMS.t Bug 12642: t/SMS.t should not be db dependent 2014-07-29 11:27:55 -03:00
SocialData.t Bug 14778: Install fixtures for t/SocialData.t 2015-10-23 12:01:18 -03:00
Stats.t
SuggestionEngine.t
SuggestionEngine_AuthorityFile.t Bug 13277: (QA followup) use t::lib::Mocks 2014-11-19 18:12:03 -03:00
TmplToken.t
XSLT.t Bug 13276: use t::lib::Mocks::mock_dbh 2014-11-19 11:30:45 -03:00