Koha-community/Koha - Koha - Koha: The world's first free and open source library system

Commit Graph

Author	SHA1	Message	Date
Fridolin Somers	4d9ab04a93	Bug 26175: Remove warn if undefined barcode in misc/export_records.pl When using misc/export_records.pl --deleted_barcodes if barcode is NULL you get warn : Use of uninitialized value in say at /home/koha/src/misc/export_records.pl Also exporting barcode if empty string makes no sense. Test plan : 1) Delete an item with barcode = NULL 2) Run : misc/export_records.pl --date=`date +%d/%m/%Y` --deleted_barcodes --filename=/tmp/deleted_barcodes 3) Check you se no warn "Use of uninitialized value in say ..." 4) Delete an item with barcode = '' 5) Run 2) 6) Check there is no empty line in /tmp/deleted_barcodes Signed-off-by: Owen Leonard <oleonard@myacpl.org> Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>	3 years ago
Jonathan Druart	7d8b96803f	Bug 24545: Fix license statements Bug 9978 should have fixed them all, but some were missing. We want all the license statements part of Koha to be identical, and using the GPLv3 statement. Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>	4 years ago
David Gustafsson	88eb0c5904	Bug 20485: Don't account for items timestamps when items not included If combining "--dont_export_items" and "--date" options in export_records.pl, don't include items modification/deletion timestamp in biblio query conditions. How to test: 1) Delete or modify a biblio item. 2) Run export_records.pl with "--dont_export_items" and "--date" options with a date value before item was deleted or modified, but after the biblio was modified. 3) The biblio for which an item was changed should not be included in the export. Sponsored-by: Gothenburg University Library Signed-off-by: Maryse Simard <maryse.simard@inlibro.com> Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> Signed-off-by: Nick Clemens <nick@bywatersolutions.com>	6 years ago
Martin Renvoize	d2e189ca1c	Bug 22600: Set 'commandline' interface appropriately This patch change Koha::Cron to be a more generic Koha::Script class and update all commanline driven scripts to use it. Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> Signed-off-by: Josef Moravec <josef.moravec@gmail.com> Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> Signed-off-by: Nick Clemens <nick@bywatersolutions.com>	5 years ago
David Gustafsson	f3c51f99d1	Bug 20486: Add --marc_conditions option to export_records.pl Add --marc_conditions option to export_records.pl script, for excluding records based on conditions applied to MARC-record data. How to test: 1) Run tests in t/db_dependent/Exporter/Record.t 2) All tests should pass Sponsored-by: Gothenburg University Library Signed-off-by: Maksim Sen <maksim.sen@inlibro.com> Signed-off-by: Michal Denar <black23@gmail.com> Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz> Signed-off-by: Nick Clemens <nick@bywatersolutions.com>	6 years ago
Julian Maurice	b65f1f4c94	Bug 19730: (follow-up bug 17196) Use biblio_metadata.timestamp in export_records.pl Since bug 17196, biblioitems.timestamp is not always updated after a change in the MARC record. Filtering should be based on biblio_metadata.timestamp instead. REVISED TEST PLAN ----------------- 0. Do not apply patch 1. Find a biblio record, remember the biblionumber for step 3 2. Edit the record, modify a field (e.g. 003, 015$q) that is not mapped to a DB column, so biblio_metadata.timestamp will be modified but not biblioitems.timestamp 3. In MySQL with the koha database selected: > select timestamp from biblio where biblionumber=###; > select timestamp from biblio_metadata where biblionumber=###; -- you'll need to change the ###'s based on the biblionumber you remembered in step 1. -- the two timestamps will differ. -- Remember the timestamp of biblio_metadata for step 4. 4. Run this command: $ sudo koha-shell -c bash kohadev $ export DATE="YYYY-MM-DD HH:mm:SS" -- use the timestamp remembered in step 3. 5. Run this command: $ ./misc/export_records.pl --date="$DATE" $ ls -la koha.mrc -- the file should be 0 bytes. 6. Run this command: $ exit $ git bz apply 19730 $ restart_all $ sudo koha-shell -c bash kohadev $ export DATE="YYYY-MM-DD HH:mm:SS" -- use the timestamp remembered in step 3. 7. Run this command: $ ./misc/export_records.pl --date="$DATE" $ ls -la koha.mrc -- the file should be a lot more than 0 bytes. 8. Run this command: $ /home/vagrant/qa-test-tools/koha-qa.pl -v 2 -c 1 -- this should pass. Signed-off-by: Mark Tompsett <mtompset@hotmail.com> Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>	6 years ago
Jonathan Druart	b0dc5fc0f3	Bug 15498: Let the user choose the CSV profile to export circ history The way the export options are displayed at the bottom of the checkouts table was not consistent. Prior to this patch set, they are display if ExportRemoveFields or ExportWithCsvProfile is set. It does not make any sense, the user could want to export the checkouts in iso2709 format without having to define a csv profile and fill the pref. Moreover the behavior of this pref did not match its description: it's used as a default CSV profile when exporting records from the export tools or the command line. This patch set adds a new pref ExportCircHistory and remove ExportWithCsvProfile. The new pref is set if ExportWithCsvProfile or ExportRemoveFields were set. A new dropdown list with the CSV profile list will be displayed in the export area, at the bottom of the checkouts table. Note that now --csv_profile_id is mandatory for the export command line (misc/export_records.pl) if the export format is csv. Test plan: 0/ Do not execute the DB entry 1/ Clear both ExportWithCsvProfile and ExportRemoveFields prefs 2/ Execute the DB entry 3/ ExportCircHistory should not be set and the export options should not be displayed at the bottom of the checkouts table. 4/ Remove the pref DELETE FROM systempreferences WHERE variable='ExportCircHistory'; and reinsert the previous one, with a value: INSERT INTO systempreferences (variable, value) VALUES ('ExportWithCsvProfile', 'something'); Execute the DB entry again => The now pref should be now set 5/ Export some checkouts using the CSV entry 6/ Note that the export tool and commandline script still work using the csv format. You have to provide a --csv_profile_id option to make it work. Signed-off-by: Josef Moravec <josef.moravec@gmail.com> Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>	7 years ago
Fridolin Somers	8630e83ed1	Bug 17420 - record export fails when itemtype on biblio - followup Same as previous patch for misc/export_records.pl. Test plan : - Use syspref item-level_itypes = biblio record - Run misc/export_records.pl => Without patch you get an error : DBD::mysql::st execute failed: Unknown column 'biblioitems.itemtype' in 'where clause' ... => With patch you get a correct export file Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> Export Ok, no errors. Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>	7 years ago
Fridolin Somers	dbab260afc	Bug 16935 - launch export_records.pl with deleted_barcodes param fails When launching misc/export_records.pl with this command line : misc/export_records.pl --date=`date +%d/%m/%Y` --deleted_barcodes --filename=/tmp/koha.mrc You get this error message : DBD::mysql::db selectall_arrayref failed: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near '' at line 4 [for Statement " ( SELECT DISTINCT barcode FROM deleteditems WHERE deleteditems.biblionumber = ? "] at misc/export_records.pl line 189. This is because of a '(' after 'q\|', looks like a typo. Also, this patch removes useless var $q. Test plan : - Delete an item with barcode - Without patch, run : misc/export_records.pl --date=`date +%d/%m/%Y` --deleted_barcodes --filename=/tmp/koha.mrc => You get dirty MySQL - Without patch, run the same command => No error and the file is generated Signed-off-by: Nicolas Legrand <nicolas.legrand@bulac.fr> Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com> Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>	7 years ago
Josef Moravec	947ab63bb7	Bug 17444: Enable exporting by time, not only date Test plan: 1) Apply the patch 2) Edit a biblio 3) run export_records.pl with date time few minutes in the past for example: --format=xml --record-type=bibs --date="2016-10-14 10:00:05" --filename="koha.xml" 4) look in the file and cofirm that the right record was exported 5) Try the same but with time after the biblio was edited, it shouldn't be exported Signed-off-by: radiuscz <radek.siman@centrum.cz> Bug 17444: Follow-up, don't change the name of parameter "date" Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org> Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>	8 years ago
Jonathan Druart	0e9c3d446d	Bug 15451: Do not considered a Resultset as a Result The previous calls were wrong, but there is something bad with the DB structure: export_format.profile should be a unique key. This patch fixes the previous calls and add a FIXME not to forget to fix the DB structure. Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> Previous test where done with all patches applied, including this one, and all worked. No errors Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>	8 years ago
Jonathan Druart	f3d7f27107	Bug 15451: Koha::CsvProfiles - Remove the residue This patch erase all traces of C4::Csv since it's not used anymore. All occurrences have been replaced by previous patches to use Koha::CsvProfiles. Note that GetMarcFieldsForCsv was not used prior this patch set. Test plan: git grep 'C4::Csv' should not return any result. Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> No more traces of the file. This produces a koha-qa fail, due to the missing file. No other errors Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>	8 years ago
Jonathan Druart	bea95dace6	Bug 15451: Koha::CsvProfiles - Remove GetCsvProfileId This subroutine returned the export_format_id for a given profile name. This can be done easily with the Koha::CsvProfiles->search method. Test plan: Export records using the misc/export_records.pl script and the export tool. If you are exporting using the MARC format, the profile filled in the pref ExportWithCsvProfile will be used (or the one passed in parameter of misc/export_records.pl). If you are exporting using the CSV format, you can choose a profile in the dropdown list. Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> Exported using tool & cmd, marc & csv. Pref is used. No errors Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>	8 years ago
Jonathan Druart	cd9589681d	Bug 15923: Fix --id_list_file option for misc/export_records.pl Signed-off-by: Josef Moravec <josef.moravec@gmail.com> Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de> Signed-off-by: Brendan Gallagher brendan@bywatersolutions.com	8 years ago
Jonathan Druart	24e502e4a9	Bug 15923: Take the id list file into account when exporting records When exporting records (tools/export.pl or misc/export_records.pl), a file of ids (authid or biblionumber) can be passed to filter the results. Bug 14722 has broken this behavior. Test plan: Export records and specify a list of records to filter the results. Prior to this patch, the record with the id 1 was exported. Signed-off-by: Josef Moravec <josef.moravec@gmail.com> Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de> Signed-off-by: Brendan Gallagher brendan@bywatersolutions.com	8 years ago
Jonathan Druart	03c89fa02e	Bug 15380: Move Koha::Authority to Koha::MetadataRecord::Authority The existing Koha::Authority does not use Koha::Objects and does not exclusively deal with DB objects. It makes much sense to move it to let the space free for a Koha::Authority and Koha::Authorities modules based on Koha::Object. Signed-off-by: Marc Véron <veron@veron.ch> Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> Signed-off-by: Brendan Gallagher brendan@bywatersolutions.com	8 years ago
Jonathan Druart	03891d2edc	Bug 14722: CSV is only available for biblio records Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>	8 years ago
Jonathan Druart	d3c9dac7ac	Bug 14722: Add POD for some options Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>	8 years ago
Jonathan Druart	b99901136c	Bug 14722: rename the script Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>	8 years ago
Jonathan Druart	38f92df4e5	Bug 14722: Refactor the export tool Why a refactoring was need for this script? The export tool (tools/export.pl) can be called from the command line and some parts of code were unnecessarity complicated (just look at the code, you will understand). Worse still, the script does not provide the same options for both interface. For instance you cannot export records given a range of biblionumbers, authids, callnumbers, etc. from the commandline. What does this patch? 1/ Important: The script tools/export.pl does not work anymore if called from the command line (should be in the release notes). 2/ The code used to generated a file (csv, iso2709 or xml) has been moved to a new module (Koha::Exporter::Record) and tests have been provided. 3/ No change is done on the web interface 4/ Some new options have been added to the commandline script (misc/export.pl): - starting_authid - ending_authid - authtype - starting_biblionumber - ending_biblionumber - itemtype - starting_callnumber - ending_callnumber - start_accession - end_accession 5/ There is a change in the behavior if an error occurs: Can't call method "as_usmarc" on an undefined value at Koha/Exporter/Record.pm line 114. record (number 5530) is invalid and therefore not exported because its reopening generates warnings above at Koha/Exporter/Record.pm line 117. Before this patch, they were not displayed (using the command line). What does not do this patch? It does not provide the 'clean', 'timestamp' and 'deleted_barcodes' options to the web interface (same as before). What about the perfs? With a DB with ~800 biblios (MARC21) Before: perl tools/export.pl 14.79s user 0.83s system 71% cpu 21.905 total After: perl misc/export.pl 17.19s user 0.84s system 75% cpu 24.018 total With a DB with ~6400 biblios (UNIMARC) Before: perl tools/export.pl 26.55s user 0.76s system 76% cpu 35.498 total After: perl misc/export.pl 26.78s user 0.84s system 80% cpu 34.494 total How to test this patch? Test plan: A. Web interface: 1/ On the current master, export some records, biblios and authorities (with the 3 differents exports) playing with the different filters (item type, libraries, callnumber, accession date, don't export items, remove non-local items, don't export fields, etc.). 2/ Apply this patch, export again the same records, and compare the generated files. They must be identical! 3/ Confirm that the export features on the checkout list (circ/circulation.pl) works as before this patch. B. The command line 1/ On the current master, export some records, biblios and authorities (with the 2 differents exports) playing with the different options (date, deleted_barcodes, clean). 2/ Apply this patch, export again the same records, and compare the generated files. They must be identical! Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz> Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de> Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>	8 years ago

19 Commits (4d011bd9983926488f783730ca43e98e6e07dabe)