Commit graph

91 commits

Author SHA1 Message Date
2b1b168ec4 Bug 21183: Replace C4::Items::GetItemnumberFromBarcode calls
C4::Items::GetItemnumberFromBarcode calls can be replaced with
  Koha::Items->find({ barcode => $barcode });

We should make sure the barcode existed in DB and so that ->find
returns an object. Note that most of the time we just wanted to know if
the barcode existed.
The changes are very simple, the only one that need attention is
the one in batchMod.pl. It is basically reusing what we did on
bug 21141.

Test plan:
Use the batch item modification/deletion tools to modify/delete items
from their barcode (using the textarea or a file)

Signed-off-by: Josef Moravec <josef.moravec@gmail.com>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
2018-08-30 13:40:35 +00:00
294ea88614 Bug 20144: [sql_modes] Set import_records.marcxml_old to '' on INSERT
Fix for:
Field 'marcxml_old' doesn't have a default value

t/db_dependent/ImportBatch.t

Fixme - need to update existing DB?

Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2018-02-13 13:58:54 -03:00
Mark Tompsett
3524925631 Bug 19937: Silence warnings t/db_dependent/www/batch.t
TEST PLAN
---------
1) Run the following on a kohadevbox:
    git checkout -b bug_19937 origin/master
    sudo koha-shell -c bash kohadev
    prove t/db_dependent/www/batch.t
    cat /var/log/koha/kohadev/plack-error.log

    The following errors are triggered at the end of the log file:
        Use of uninitialized value in array element at
        /home/vagrant/kohaclone/tools/batch_records_ajax.pl line 50.
        Use of uninitialized value $results_per_page in numeric eq (==) at
        /home/vagrant/kohaclone/tools/batch_records_ajax.pl line 53.
        Use of uninitialized value in uc at
        /home/vagrant/kohaclone/C4/ImportBatch.pm line 1120.

2) Run the following on a kohadevbox:
    exit
    git bz apply 19937
    restart_all
    sudo koha-shell -c bash kohadev
    prove t/db_dependent/www/batch.t
    cat /var/log/koha/kohadev/plack-error.log

    The log file will end with the restarting of plack, not the
    errors.

3) run koha qa test tools

Signed-off-by: Lee Jamison <ldjamison@marywood.edu>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Amended handling of $results_per_page.

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2018-01-15 12:22:16 -03:00
a423fcff42 Bug 19049: [QA Follow-up] Mock config, default format
As requested by QA:
[1] Mock_config enable_plugins in the test.
[2] Fallback to MARC when format is empty. Remove die statement.
Added:
[3] Remove $marc. This variable got obsolete during development.
[4] Add test on $input_file and $plugin_class. Test $text before calling
    Handler or processing $text. No need to split undef if somehow Handler
    returned undef, etc. If the routine returns an empty arrayref,
    stage-marc-import will do fine.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2017-08-15 12:17:42 -03:00
d24568b672 Bug 19049: Fix regression on stage-marc-import with to_marc plugin
Bug 12412 added the use of to_marc plugins allowing arbitrary file formats
in stage-marc-import (as long as the plugins can handle them). The feature
was not very visible in the code, and when bug 10407 added the marcxml
format, it made some changes that broke the use of to_marc.

This patch restores the functionality by:
[1] Adding a sub RecordsFromMarcPlugin to ImportBatch.pm, specifically
    addressing the conversion from arbitrary formats to MARC::Record.
    The original to_marc interface is used: pass it the file contents,
    and it returns a string consisting of a number of MARC blobs separated
    by \x1D.
    Consequently, the call of to_marc is removed from routine
    BatchStageMarcRecords where it did not belong. The to_marc_plugin
    parameter is removed and two calls are adjusted accordingly.
[2] Instead of a separate combo with plugins, the format combo contains
    MARC, MARCXML and optionally some plugin formats.
[3] The code in stage-marc-import.pl now clearly shows the three main
    format types: MARC, MARCXML or plugin based.

Note: This patch restores more or less the situation after bug 12412, but
I would actually recommend to have the to_marc plugins return MARC::Record
objects instead of large text strings. In the second example I added a
to_marc plugin that actually converts MARC record objects to string format,
while RecordsFromMarcPlugin reconverts them to MARC::Records.

Test plan:
See second patch.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2017-08-15 12:17:41 -03:00
c73e269a13 Bug 18260: Koha::Biblio - Remove GetBiblio
C4::Biblio::GetBiblio can be replaced with Koha Biblio->find

Test plan:
Import batch, view issue history, search for items, see the image of a
bibliographic record, modify and delete records in a batch

Followed test plan, works as expected.
Signed-off-by: Marc Véron <veron@veron.ch>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2017-07-10 13:03:38 -03:00
b37df275b8 Bug 18611: [QA Follow-up] Make SQL query more readable
Make it more explicit by adding join statements.

Test plan:
See next patch for adding a unit test.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
2017-05-28 22:23:34 -04:00
092f02d227 Bug 18611 - Followup, remove tabs to make qa tools happy
Signed-off-by: Josef Moravec <josef.moravec@gmail.com>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
2017-05-28 22:23:34 -04:00
7249351f12 Bug 18611 - Create labels action fails in manage-marc-import.pl if an item has been deleted from the import batch.
To test:
1 - Import a batch with some items
2 - Delete one of the imported items
3 - Browse to Tools->Staged MARC record management
4 - Click (Create label batch) for the batch you imported
5 - Recieve an error
6 - Apply patch
7 - Click (Create label batch)
8 - Batch is created with remaning items from the import

Signed-off-by: Josef Moravec <josef.moravec@gmail.com>

Signed-off-by: Josef Moravec <josef.moravec@gmail.com>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
2017-05-28 22:23:34 -04:00
4c762ba69c Bug 9988: Merge should have a parameter hash
We will need a few additional parameters for merge later on. This patch
puts the original parameters in a parameter hash.
For the same reason DelAuthority gets a parameter hash here.

Note: We remove the second parameter from the DelAuthority call in
authorities/authorities-home.pl here. It was not used and could have
presented problems in the future.

Test plan:
[1] Run t/db_dependent/AuthoritiesMarc.t.
[2] Run t/db_dependent/Authorities/Merge.t.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Marc Véron <veron@veron.ch>
Signed-off-by: Jacek Ablewicz <abl@biblos.pk.edu.pl>
Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
2017-04-13 08:53:46 -04:00
17e84750d9 Bug 18070: [Follow-up] What should DelAuthority return?
Script tools/batch_delete_records.pl only checks the return value of
DelAuthority. The return value depends on DBI rows, which is not always
reliable. It may return -1 when it 'does not know'.

Testing $@ in tools/batch_delete_records.pl should actually be enough.
The return value was discarded in C4/ImportBatch.pm. Removing an unused
variable.

Test plan:
[1] Pick an authority record with a few linked biblios.
    Delete this authority record via tools/batch_delete_records.pl.
    Check if the linked biblio records are cleaned up.
[2] Bonus: Make a typo in the SQL statement of DelAuthority. Check
    if batch_delete_records shows you the error message.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
2017-03-03 18:12:09 +00:00
48aba1533c Bug 6852: Staged import reports wrong success for items
If we import items that have a wrong branch code, the items will not
be imported but manage-marc-import reports them as imported. (A wrong
branch code probably occurs most, but other causes are possible too.)

The underlying cause is that AddItem does not look at the error
returned from _koha_new_item in Items.pm.

This patch deals with that omission in the most economical way. It adjusts
AddItem so that it returns undef if no item was added.
In ImportBatch.pm I check if an item was added and adjust the totals
accordingly instead of just always counting them.

Note: Several scripts like additem.pl use AddItemFromMarc to call
AddItem. They do not report an error, but fail silently. With this patch,
these scripts get undef and will still fail silently. (No change.)
Adding error checks around calls of AddItemFromMarc is outside the scope of
this report. Here we are taking care of correct imported item numbers.

Test plan:
[1] Verify that additem.pl still works by adding a new item.
[2] Run t/db_dependent/Items.t
[3] Add a new branchcode, say XXX.
[4] Pick a biblio record with a few items (n) and set one item to branch XXX.
[5] Export this biblio with items to a MARC file.
[6] Change the XXX item to the original branch and remove branch XXX.
[7] Import the MARC file. Verify that one item was not imported and that
    the number of imported items reflects that (equals n-1).

Signed-off-by: Owen Leonard <oleonard@myacpl.org>

Signed-off-by: Katrin Fischer  <katrin.fischer@bsz-bw.de>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
2016-09-12 08:55:41 +00:00
3badc57c93 Bug 10407: Add marcxml import (follow-up)
This patch makes the following changes:

[1] Based on the groundwork of the former patch, add call to
    RecordsFromMARCXMLFile in stage-marc-import. Use format param.
[2] Add format to the template. Use file extension to determine.
    If you use .xml or .marcxml as extension, MARCXML is selected.
[3] In stage-marc-import.tt mark UTF-8 encoding as UTF-8 not as utf8.
[4] BatchStageMarcRecords: do not call plugin if you have no records.
[5] RecordsFromISO2709File: also return errors in an array.
[6] In misc/stage_file.pl also use UTF-8. Handling of errors from [5].

Test plan:
[1] Import an empty file as MARC or MARCXML (with Tools/Stage..import).
[2] Import an non-empty file with invalid contents as MARC or MARCXML.
[3] Export a few records with Tools/Export as MARC and MARCXML.
[4] Import these two files. Check selected format versus file extension.
[5] Import a MARCXML file with misc/stage_file.pl.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Josef Moravec <josef.moravec@gmail.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
2016-09-02 16:00:13 +00:00
Olli-Antti Kivilahti
aa5bf4d3cc Bug 10407: Allow MARCXML records to be imported via GUI (groundwork)
Patch from Olli, manual rebase by Marcel (July 7, 2016).

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Needs follow-up. Test plan in the third patch.

Signed-off-by: Josef Moravec <josef.moravec@gmail.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
2016-09-02 16:00:13 +00:00
Barton Chittenden
c02b33e425 bug 14504: (QA followup) fixing DelItemCheck arguments
Remove $dbh as argument to C4::Items::DelItemCheck
and C4::Items::ItemSafeToDelete, also change all
calls to these functions throughout the codebase.

Also remove remaining reference to 'do_not_commit' in
t/db_dependent/Items_DelItemCheck.t

Fixed doubled "$$" in C4/ImportBatch.pm

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
2016-08-26 12:07:26 +00:00
Aleisha
08e448ee4b Bug 9259: Ability to delete a staged file once it has been cleaned
To test:
1) Go to Tools -> Staged MARC Management and clean a file. If you have no files to clean, go to 'Stage MARC for import' and upload one to clean following the necessary steps.
2) Confirm that once the file has been cleaned, the Action column now shows a Delete button. Confirm this button only shows for cleaned files.
3) Click the Delete button.
4) Confirm that clicking Cancel exits the pop-up message and does not delete the file.
5) Confirm that clicking OK refreshes the list of staged records and the one you just deleted is no longer on it (has been deleted). You can confirm this by checking for the file in mysql (SELECT * FROM import_batches WHERE import_batch_id = X;)
6) Run prove -v t/db_dependent/ImportBatch.t (have written unit tests for CleanBatch and DeleteBatch)

Sponsored-by: Catalyst IT
Signed-off-by: Liz Rea <liz@catalyst.net.nz>
Catalyst sign off, so needs another one but YAY this is great.

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
2016-07-08 13:43:53 +00:00
c564f30155 Bug 16169: Change prototype for C4::Biblio::TransformMarcToKoha
subroutines should not take $dbh in parameter.
C4::Biblio::TransformMarcToKoha has it and does not use it.

Test plan:
Look at the patch and confirm that all occurrences of
TransformMarcToKoha have been modified.

Signed-off-by: Jacek Ablewicz <abl@biblos.pk.edu.pl>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

Signed-off-by: Brendan Gallagher brendan@bywatersolutions.com
2016-04-07 00:04:21 +00:00
798d38e4c7 Bug 16011: $VERSION - Remove comments
perl -p -i -e 's/^.*set the version for version checking.*\n//' **/*.pm

+ manual adjustements

Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@unc.edu.ar>

Signed-off-by: Brendan A Gallagher <brendan@bywatersolutions.com>
2016-03-24 17:20:29 +00:00
017699c345 Bug 16011: $VERSION - Remove the $VERSION init
Mainly a
  perl -p -i -e 's/^.*3.07.00.049.*\n//' **/*.pm
Then some adjustements

Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@unc.edu.ar>

Signed-off-by: Brendan A Gallagher <brendan@bywatersolutions.com>
2016-03-24 17:20:28 +00:00
3830d78d46 Bug 16011: $VERSION - remove use vars $VERSION
perl -p -i -e 's/^(use vars .*)\$VERSION\s?(.*)/$1$2/' **/*.pm

Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@unc.edu.ar>

Signed-off-by: Brendan A Gallagher <brendan@bywatersolutions.com>
2016-03-24 17:20:26 +00:00
19ae949e6a Bug 14597: (QA followup) Make Logger package level
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Tomas Cohen Arazi <tomascohen@unc.edu.ar>
2015-09-28 12:20:49 -03:00
9b0f065392 Bug 14597: Reverting a batch where a record overlaid is now deleted record will fail
If batch of records is used to overlay existing records, and one of
those records is deleted, any attempt to revert this batch will fail.

The unanimous consensus is that deleted records should stay deleted (
 https://lists.katipo.co.nz/public/koha/2015-June/043048.html ).

Test Plan:
1) Import a batch of records
2) Import the same batch again, using ISBN as a matcher for overlay
3) Find a record that overlayed a previous record and delete it
   ( the match type will be "match applied" and there will be a link in
     "match details" )
4) Attempt to revert the second batch ( that overlayed the first )
5) The progress will stop at some point and never complete
6) Apply this patch
7) Attempt to revert the second batch again
8) This time it should succeed!

Signed-off-by: Do, Tam T <tdo@albright.edu>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Tomas Cohen Arazi <tomascohen@unc.edu.ar>
2015-09-28 12:20:41 -03:00
ca167b32b4 Bug 12412: Add ability for plugins to convert arbitrary files to MARC from record staging tool
Many libraries would like to be able to import various types of files as
MARC records ( citations, csv files, etc ). We can add a new function to
the plugins system to allow that kind of behavior at a very custom
level.

Test Plan:
1) Ensure you have plugins enabled and configured correctly
2) Installed the attached version 2.00 of the Kitchen Sink plugin
3) Download the attached text file
4) Browse to "Stage MARC records for import"
5) Select the downloaded text file for staging
6) After uploading, you should see a new area "Transform file to MARC:",
   select "Example Kitchen-Sink Plugin" from the pulldown menu
7) Click 'Stage for import"
8) Click 'Manage staged records"
9) You should now see two new MARC records!

Signed-off-by: Aleisha <aleishaamohia@hotmail.com>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Works as described - interesting new feature.
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
2015-05-04 15:33:51 -03:00
Jonathan Druart
a6c9bd0eb5 Bug 9978: Replace license header with the correct license (GPLv3+)
Signed-off-by: Chris Nighswonger <cnighswonger@foundations.edu>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>

http://bugs.koha-community.org/show_bug.cgi?id=9987

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
2015-04-20 09:59:38 -03:00
Jonathan Druart
4d8dd344ae Bug 6520: Display items for staged record
When records are imported into Koha, the items is stored into the
import_items table.
This marcxml in this table is never retrieved to display items.

Test plan:
1/ Import a records with items
2/ Before importing the batch into the catalog, you can see the marc
of the records, in the table below.
3/ Verify that the items is correctly displayed.

QA note: This patch does not provide test for new subroutines but the
module (C4::ImportBatch) is not tested at all and it will be time
consuming to provide them.

Signed-off-by: Nicole Engard <nengard@bywatersolutions.com>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
2015-04-16 13:40:46 -03:00
Jacek Ablewicz
7c2fad1baf Bug 10906 - Reimported records into Koha are imported only as DEFAULT frameworks, not what they were originally
Existing framework code is currently not retained when local record
gets replaced during batch import, or when the restore/reverse function
is being used.

This patch fixes aforementioned issues by correcting outdated GetBiblio()
calls in C4/ImportBatch.pm

To test:

1/ try to replicate the issue: import some MARC records with
"Tools -> Stage MARC records for import" etc., using test setup,
matching rules and so on such that some existing records will get
replaced with imported ones
2/ observe that records replaced during import now open in the editor
with 'Default' framework, even if they have some other framework
set up previously
3/ apply patch
4/ redo 1/, confirming that this problem is no longer replicable
5/ try use 'restore' function with some freshly imported
records, ensure that original framework code got retained
in the records which had their imports reverted

NOTE: I confirmed this change by
1) git grep "=\s*GetBiblio\s*("
   -- this shows how GetBiblio is called elsewhere.
      It differed! So then...
2) vi C4/Biblio.pm
   /sub GetBiblio
   -- find the right one, notice it only returns a HASH ref,
      not an array.

Signed-off-by: Mark Tompsett <mtompset@hotmail.com>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
2015-04-09 15:05:11 -03:00
Olli-Antti Kivilahti
fc4e9ab62e Bug 13222 - Accessors for import_record.status have variable names suggesting overlay_status
TODO:: Somebody who knows what these are about could write some documentation about it.

Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
2015-01-14 21:27:49 -03:00
Jonathan Druart
46d3e94e66 Bug 11876: (qa-followup) some minor changes
- GetImportBiblios: prepare the query is useless here
- use Modern::Perl
- Remove useless indentation level in pl file and redirect to 404 if
  required params are not given.

Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
2014-10-31 14:25:13 -03:00
mbeaulieu
38742e0722 Bug 11876 - Add a diff view for staged MARC records.
New patch should apply correctly.

	modified:   C4/ImportBatch.pm
	modified:   koha-tmpl/intranet-tmpl/prog/en/modules/tools/manage-marc-import.tt

Signed-off-by: Owen Leonard <oleonard@myacpl.org>

Bug 11876 - Diff column is now displayed.

If a match is found, a 'View' link appears in the Diff column.

Old patch was not implementing datatables, and therefore was not compatible with this version of Koha.
	modified:   koha-tmpl/intranet-tmpl/prog/en/modules/tools/manage-marc-import.tt
	modified:   tools/batch_records_ajax.pl

Signed-off-by: Owen Leonard <oleonard@myacpl.org>

These patches have some issues but I'm going to submit a follow-up which
I think will address them all and offer some improvements as well.

Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
2014-10-31 14:24:58 -03:00
Fridolyn SOMERS
cac06afeb1 Bug 11254: make reservoir search normalize ISBNs
When importing records, the ISBN is normalized and stored
into database (see C4::ImportBatch::_add_biblio_fields).  But when
searching with ISBN into reservoir, it is not normalized
(see C4::Breeding::BreedingSearch).  So search does not match.

This patch adds the normalisation to reservoir search.  Also, it
replaces call private method _isbn_cleanup by GetNormalizedISBN,
the correct public method.  Also allows the reservoir search
on ISBN with hyphens.

This is intended to fix only reservoir searches.

Revised Test plan
-----------------
 1) Back up DB
 2) Save copy of attached example somewhere findable
 2) Home -> Tools -> Stage MARC records for import
 3) Click Browse and select the example MARC file
 4) Click Upload file
 5) Tweak as desired then click Stage for import
 6) Click Manage staged records
 7) Click Import this batch into the catalog
 8) Home -> Cataloging
 9) In the Cataloging search text box type 978-0-691-14289-0 and
     click Submit
    -- ISBN13 with hypens not found in reservoir
10) In the Cataloging search text box type 9780691142890 and
     click Submit
    -- ISBN13 without hypens not found in reservoir
11) In the Cataloging search text box type 0-691-14289-0 and
     click Submit
    -- ISBN10 with hypens not found in reservoir
12) In the Cataloging search text box type 0691142890 and
     click Submit
    -- ISBN10 without hypens found in reservoir
13) Apply patch
14) Repeat steps 9-12, this time it is always found! :)

Signed-off-by: Mark Tompsett <mtompset@hotmail.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2014-04-19 21:44:30 +00:00
183d4a55ca Bug 11923: fix option to sort contents of MARC record batches by citation descending
When the ability to stage authority records was added to Koha, sorting
record batches by citation ( i.e. title ) caused the addition of
"authorized_heading" to be added to the sort. When sorting by title
descending, this causes the order by clause to be "title,
authorized_heading DESC" which means sort by title ASC, then
authorized_heading DESC. This is incorrect and causes regular biblio
batches to always be sorted ascending.

Test plan:
1) Stage a batch of biblio records from a file
2) View the staged batch
3) Attempt to sort by title descending
4) Note it is still sorted by title ascending
5) Apply this patch
6) Note the sorting now works correctly

Signed-off-by: Owen Leonard <oleonard@myacpl.org>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Works as advertised. The code pertaining to sorting in routine
GetImportRecordsRange will probably not win beauty prizes.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2014-03-13 15:10:56 +00:00
0713f3bf03 Bug 10558 [QA Follow-up]
This patch addresses a number of issues with the main patch:

- regression on bug 2060 (i.e., displaying authority import batches
  correctly)
- regression on bug 10170 (translation of import record states)A
- use of datatables.inc
- lack of clarity as to the licensing of tools/batch_records_ajax.pl
- insufficent sanitizing of input used to generate an SQL statement

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2014-03-10 15:48:32 +00:00
4d32421634 Bug 10558 - Convert records table in manage-marc-import.pl to Ajax DataTable
Some libraries would like to sort by columns for the records of an
import batch. This seems like a good use of Ajax DataTables.

Test plan:
1) Apply this patch
2) Import a record batch into Koha
   a) Use some form of matching
   b) Have some records that will match and some that won't
   c) Have at least 30 records so you can test the pager
3) Verify the new table is functionally equivalent to the old static one

Signed-off-by: Owen Leonard <oleonard@myacpl.org>

Tests fine and looks good with the exception of the corrections I put in
a follow-up.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2014-03-10 15:46:49 +00:00
Jonathan Druart
0a176d4648 Bug 8015: (follow-up) trap exceptions thrown by SetUTF8Flag()
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Leila <koha.aixmarseille@gmail.com>

Bug 8015: Catch error in the SetUTF8Flag routine

The eval avoids the interface to run endlessly if an error occurred.

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Leila <koha.aixmarseille@gmail.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-10-31 22:48:59 +00:00
622430cfb5 Bug 8015: Add MARC Modifications Templates
The MARC Modification Templates system gives Koha users
the power to make alterations to MARC records automatically
while staging MARC records for import.

This tool is useful for altering MARC records from
various venders work with your MARC framework.

The system essentially allows one to create a basic script
using actions to Copy, Move, Add, Update and Delete fields.

Each action can also have an optional condition to check
the value or existance of another field.

The Copy & Move actions also support Regular Expressions,
which can be used to automatically modify field values during the
copy/move. An example would be to strip out the '$' character
in field 020$c.

Furthermore, the value for an update can include variables
that change each time the template is used. Currently,
the system supports two variables, __BRANCHCODE__ which
is replaced with the branchcode of the library currently
using the template, and __CURRENTDATE__ which is replaced
with the current date in ISO format ( YYYY-MM-DD ).

At its simplist, it can perform functions such as:
Copy field 092$a to 952$c
At its most complex it can run actions like:
Copy field 020$c to 020$c using RegEx s/\$// if 020$c equals RegEx m/^\$/

Signed-off-by: Leila <koha.aixmarseille@gmail.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-10-31 22:47:08 +00:00
d65266eb84 Bug 7131: (follow-up) initialized some variables properly
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-10-30 04:34:58 +00:00
805bec0a65 Bug 7131: (follow-up) allow overlaying by barcode
This patch adds the ability to overlay by either itemnumber,
or barcode. Itemnumbers take precendence over barcodes, which
allows us to batch update item barcodes with an overlay.

Test Plan:
1) Create a new record with 2 items, make sure to give it a unique ISBN
2) Download the record as MARCXML
3) Edit the MARC XML
   a) Delete one of the two items
   b) Change the barcode in the barcode field to something unused
4) Transform the xml file into marc with xml2marc
5) Browse to 'Stage MARC records for import'
6) Upload the binary marc file
7) Choose the following options:
    Record matching rule: ISBN
    Action if matching record found: Ignore incoming record
    Action if no match is found: Ignore incoming record
    Check for embedded item record data: Yes
    How to process items: Replace items if matching bib was found
8) Click 'Stage for import' button
9) Verify a matching record was found, then click 'Manage staged records' link
10) Verify the rules are still set correctly
11) Click 'Import this batch into the catalog'
12) The import should tell you:
    1 record was ignored
    1 item was replaced
13) View the record details and verify the item's barcode was replaced
    with your updated barcode value
14) Download the record as MARCXML
15) Edit the MARC XML
    a) Delete one of the two items
    b) Delete the itemnumber field for the remaining item
    c) Alter the item's callnumber to a new value
16) Repeat steps 4 through 12
17) View the record details and verify the item's callnumber was replace
    with your updated callnumber value

Signed-off-by: Henry Bankhead <hbankhead@losgatosca.gov>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-10-30 04:34:04 +00:00
b527baba52 Bug 7131: (follow-up) perltidy BatchCommitItems
Signed-off-by: Henry Bankhead <hbankhead@losgatosca.gov>
Signed-off-by: Melia Meggs <melia@bywatersolutions.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-10-30 04:33:32 +00:00
Elliott Davis
1dba9c6409 Bug 7131: teach MARC import how to overlay items
When staging biblios with items attached you previously had only two
options, add or don't add.

This patch adds a third option to replace an item record if a match is
found on itemnumber or barcode, else it adds the item.

Test Plan:
1) Stage a file of biblios with items attached.
2) Import the batch into the catalog.
3) Run the indexer so the matcher will match
4) Modify the item data for at least one bib in the file
5) Re-stage the file with the item matching option set to "Replace
   items if matching bib was found"
6) Let the indexer run again
7) You should see updated item information after the overlay

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Henry Bankhead <hbankhead@losgatosca.gov>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-10-30 04:31:07 +00:00
Jared Camins-Esakov
4f9174349c Bug 10230: Don't limit valid matches to bibs
The patch for bug 9523 added a JOIN to the biblio table when identifying
the best match so that if a matched record had been deleted it would
not hold up the import process. Unfortunately, this broke all authority
matching, since of course authorities don't appear in the biblio table.
This patch adds a join to auth_header as well, and decides which to
check based on the record type.

Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Comment on third patch of this series.
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
2013-05-19 08:40:13 -04:00
ea23dbeab2 Bug 9523: importing staged bib records hangs if a matched bib has been deleted
Test Plan:
1) Stage a MARC record file that will have matches with existing records
2) Delete the bib from Koha that was matched on
3) Attempt to import the records into Koha, the import will hang
4) Apply the patch
5) Reload manage-marc-import.pl and attempt to import again, this time it should succeed.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
2013-04-21 21:06:54 -04:00
a986c92c4d Bug 9839 - when importing MARC records, item record not created even when set to "always add"
Test Plan:
1) Catalog a new record with an ISBN
2) Add some items to the record
3) Download the record as MARCXML
4) Delete the itemnumbers from the 952 fields in the record,
   Change the barcode fields to unused barcodes
5) Use xml2marc to save as a standard MARC file
6) Import the record using the 'Stage MARC for import' tool
   Use the settings:
       Record matching rule: ISBN
       Action if matching record found: ignore
       Action if no match found: ignore
       Item processing: always_add
       Check for embedded item record data?: Yes
       How to process items: Always add items
7) Import, note the bib is ignored, and the items are not processed
8) Undo import into catalog
8) Apply this patch
9) Import this batch into the catalog
10) Note the items were processed and are now added to the matching
record

Signed-off-by: Mathieu Saby <mathieu.saby@univ-rennes2.fr>
Tested with UNIMARC record. I followed the test plan, just changing 952 by 995
Signed-off-by: Mason James <mtj@kohaaloha.com>
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
2013-04-21 09:57:08 -04:00
86fa020ef8 Bug 7368: Update GetXmlBiblio documentation
Only changing some documentation about GetXmlBiblio

Signed-off-by: Mirko Tietgen <mirko@abunchofthings.net>
Added the word 'contain'

Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
2012-12-27 10:14:19 -05:00
Frédérick Capovilla
37340e3718 Normalize records imported from Z39.50 servers.
Some Z39.50 server may use the MARC-8 encoding, which uses separated
diacritics. By forcing a normalization, all imported records will have
combined diacritics.

Records with separated diacritics might not show up in Zebra searches if
the search terms use accented characters.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

http://bugs.koha-community.org/show_bug.cgi?id=8610
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
checked it still works after the patch with UNIMARC and BNF server (that
provide utf-8 records)
2012-10-08 18:46:56 +02:00
Jared Camins-Esakov
e883a0f065 Bug 8829: Fix authority importing
A subroutine was not being imported by C4::ImportBatch (ironic, no?)
so this patch makes the call fully-qualified. This patch also cleans
up two warnings in C4::Auth that are raised when logged in as the
database user.

Signed-off-by: Nicole C. Engard <nengard@bywatersolutions.com>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
2012-10-08 18:06:55 +02:00
Jared Camins-Esakov
65c0174e63 Bug 2060 follow-up: add support for UNIMARCAUTH
The staged MARC management script was not correctly informing
the decoder ring that we had UNIMARC authorities, and the decoder
ring was dutifully trying to turn the authority records into a
bibliographic box of cereal.
2012-09-19 17:16:23 +02:00
Jared Camins-Esakov
7ad5e203da Bug 2060: Update command line MARC import scripts
Expose authority import functionality to the command line import
scripts, and rename them from commit_biblios_file.pl and
stage_biblios_file.pl to commit_file.pl and stage_file.pl.

To test (note that these instructions assume you have a MARC21
installation and are using the provided sample file):
1. Find a file of authorities (a sample file with MARC21 authorities
   is attached to bug 7475) and download it to your server
2. Stage the file using the following command (replace <filename> with
   the name of the file you saved in step 1):
   > misc/stage_file.pl --file <filename> --authorities
3. Note the batch number the script assigns to your batch
4. Commit the records using the following command (replace <batchnumber>
   with the batch number you made note of in step 3):
   > misc/commit_file.pl --batch-number <batchnumber>
5. Index the authorities Zebraqueue (or wait)
6. Confirm that the new authorities appear.
7. Create a matching rule with the following settings:
   Code: AUTHTEST
   Description: Personal name main entry
   Match threshold: 999
   Record type: Authority record
   Search index: Heading-main
   Score: 1000
   Tag: 100
   Subfields: a
   Offset: 0
   Length: 0
   (note the ID of this matching rule)
8. Stage the authority file again, this time using the following
   command:
   > misc/stage_file.pl --file <filename> --authorities \
     --match <matchingrule>
7. Revert the import with the following command:
   > misc/commit_file.pl --batch-number <batchnumber> --revert
8. Index the authorities Zebraqueue (or wait)
9. Confirm that the records have been removed
10. Import an authority record with the Stage MARC/Manage staged MARC
    tools in exactly the way you would for a bibliographic record,
    but choose "Authority" instead of "Bibliographic" for the record
    type.

Signed-off-by: Elliott Davis <elliott@bywatersolutions.com>

Testing plan delivers as it should.

Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Rebased on latest master 11 September 2012
2012-09-19 17:16:18 +02:00
Jared Camins-Esakov
6e71b80ca3 Bug 7475: Teach matching rules to handle authorities
* Add the code necessary to handle authorities with matching rules and
  import batches.
* Update all the scripts that use the matcher and import batch code
  to use the new API.
* Add authority records to the matching rules interface in the staff
  client.

http://bugs.koha-community.org/show_bug.cgi?id=2060
Signed-off-by: Elliott Davis <elliott@bywatersolutions.com>

Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Rebased on latest master 11 September 2012
2012-09-19 17:15:56 +02:00
Jonathan Druart
82dc7b55a8 Bug 4321: clean C4::Biblio::GetBiblio and uses
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
2012-09-18 12:11:54 +02:00
Sophie Meynieux
533c4c0f46 Bug 7329 [Followup] : when reverting an import, keep matching biblionumber in batch table, if items are not deleted because they are issued.
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>

http://bugs.koha-community.org/show_bug.cgi?id=7318
2012-06-20 09:38:01 +02:00