Commit graph

29 commits

Author SHA1 Message Date
Jared Camins-Esakov
65c0174e63 Bug 2060 follow-up: add support for UNIMARCAUTH
The staged MARC management script was not correctly informing
the decoder ring that we had UNIMARC authorities, and the decoder
ring was dutifully trying to turn the authority records into a
bibliographic box of cereal.
2012-09-19 17:16:23 +02:00
Jared Camins-Esakov
7ad5e203da Bug 2060: Update command line MARC import scripts
Expose authority import functionality to the command line import
scripts, and rename them from commit_biblios_file.pl and
stage_biblios_file.pl to commit_file.pl and stage_file.pl.

To test (note that these instructions assume you have a MARC21
installation and are using the provided sample file):
1. Find a file of authorities (a sample file with MARC21 authorities
   is attached to bug 7475) and download it to your server
2. Stage the file using the following command (replace <filename> with
   the name of the file you saved in step 1):
   > misc/stage_file.pl --file <filename> --authorities
3. Note the batch number the script assigns to your batch
4. Commit the records using the following command (replace <batchnumber>
   with the batch number you made note of in step 3):
   > misc/commit_file.pl --batch-number <batchnumber>
5. Index the authorities Zebraqueue (or wait)
6. Confirm that the new authorities appear.
7. Create a matching rule with the following settings:
   Code: AUTHTEST
   Description: Personal name main entry
   Match threshold: 999
   Record type: Authority record
   Search index: Heading-main
   Score: 1000
   Tag: 100
   Subfields: a
   Offset: 0
   Length: 0
   (note the ID of this matching rule)
8. Stage the authority file again, this time using the following
   command:
   > misc/stage_file.pl --file <filename> --authorities \
     --match <matchingrule>
7. Revert the import with the following command:
   > misc/commit_file.pl --batch-number <batchnumber> --revert
8. Index the authorities Zebraqueue (or wait)
9. Confirm that the records have been removed
10. Import an authority record with the Stage MARC/Manage staged MARC
    tools in exactly the way you would for a bibliographic record,
    but choose "Authority" instead of "Bibliographic" for the record
    type.

Signed-off-by: Elliott Davis <elliott@bywatersolutions.com>

Testing plan delivers as it should.

Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Rebased on latest master 11 September 2012
2012-09-19 17:16:18 +02:00
Jared Camins-Esakov
6e71b80ca3 Bug 7475: Teach matching rules to handle authorities
* Add the code necessary to handle authorities with matching rules and
  import batches.
* Update all the scripts that use the matcher and import batch code
  to use the new API.
* Add authority records to the matching rules interface in the staff
  client.

http://bugs.koha-community.org/show_bug.cgi?id=2060
Signed-off-by: Elliott Davis <elliott@bywatersolutions.com>

Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Rebased on latest master 11 September 2012
2012-09-19 17:15:56 +02:00
Dobrica Pavlinusic
130e3d9c10 Bug 8315 - remove use C4::* version
This patch touches a lot of code, but basically it removes version
information from use C4::* in our code.

Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
All script still compiles after the patch (confirmed by pre-applypatch hook)
2012-07-13 14:17:20 +02:00
Srdjan
12ff7355bb bug_7613: OCLC Connexion gateway
svc/import_bib:
* takes POST request with parameters in url and MARC XML as DATA
* pushes MARC XML to an impoort bach queue of type 'webservice'
* returns status and imported record XML
* is a drop-in replacement for svc/new_bib

misc/cronjobs/import_webservice_batch.pl:
* a cron job for processing impoort bach queues of type 'webservice'
* batches can also be processed through the UI

misc/bin/connexion_import_daemon.pl:
* a daemon that listens for OCLC Connexion requests and is compliant
  with OCLC Gateway spec
* takes request with MARC XML
* takes import batch params from a config file and forwards the lot to
  svc/import_bib
* returns status

ImportBatches:
* Added new import batch type of 'webservice'
* Changed interface to AddImportBatch() - now it takes a hashref
* Replaced batch_type = 'batch' with
  batch_type IN ( 'batch', 'webservice' ) in some SELECTs

Signed-off-by: MJ Ray <mjr@phonecoop.coop>
2012-04-06 17:26:20 +02:00
Dobrica Pavlinusic
8d7d8af618 Bug 7790 - tools/manage-marc-import.pl breaks with plack
transfer $template to add_matcher_list for correct scoping
and score $dbh and $runinbackground with our

Signed-off-by: Matthias Meusburger <matthias.meusburger@biblibre.com>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
2012-03-26 18:10:20 +02:00
Ian Walls
973371a68b Bug 6733: change branchcode call in Labels to use userenv instead of LoginBranchname
In the 5 places in the label editor, the user's current branch is determined by taking
the LoginBranchname and doing a reverse look up for the branchcode.  This is an unnecessary
query, since C4::Context->userenv has the immediate access to the branchcode; this is how
it's summoned in all other points in the Koha code.

This addresses bug 6673 because, since the move to T:T, the call in tools/manage-marc-import.pl
has used in correct syntax:  $template->param('LoginBranchname') instead of $template->{VARS}->param('LoginBranchname')

Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Ian Walls <ian.walls@bywatersolutions.com>
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
2011-09-28 20:29:39 +13:00
Chris Cormack
9573f444fc Merge remote-tracking branch 'kc/new/bug_5449' into kcmaster 2011-05-09 08:25:36 +12:00
b9cee99da4 Bug 5065 Add ability to choose framework on import
Signed-off-by: Nicole C. Engard <nengard@bywatersolutions.com>
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
2011-03-21 20:36:35 +13:00
Julian Maurice
6e85ee6db0 Bug 5449: JSON malformed in Koha - Blocker with jQuery 1.4.x
Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
2011-03-12 08:53:41 +13:00
Lars Wirzenius
49e0a8fc78 Fix FSF address in directory tools/
Signed-off-by: Galen Charlton <gmcharlt@gmail.com>
2010-03-16 20:17:51 -04:00
0bbcfeda0e replace syslog with warns
As discussed with Chris Nighswonger on #koha, this patch
removes the calls to syslog and replaces them with warns
so that error messages generated by the labels code
are sent to the Apache error log.  This avoids splitting
this sort of logging across multiple files and is consistent
with current practice in most of the rest of Koha.

Signed-off-by: Galen Charlton <gmcharlt@gmail.com>
2009-09-05 21:43:12 -04:00
Chris Nighswonger
02822ac805 [26/40] Refactoring auto label batch generation feature in bulk import tool and adding error trapping/handling. 2009-09-01 15:52:22 -04:00
ae5b76c89b bug 2157: add ability to 'clean' staged record batches
batches, it is now possible to 'clean' a batch by
removing all bib and item records staged in the batch.  This
has the effect of helping to reduce database space used
by old import batches as well as removing staged records
from the cataloging reservoir search.  Note that 'cleaning'
a batch affects only the copies of the records that were staged;
if the batch was committed, cleaning the batch does not
affect any bibs and items that were committed into the catalog.

Also note that once you clean a committed batch of records, it is
impossible to undo the previous commit operation.

Signed-off-by: Galen Charlton <gmcharlt@gmail.com>
2009-08-10 20:34:55 -04:00
f202faba18 bug 2505: enable warnings in tools/manage-marc-import.pl
Signed-off-by: Galen Charlton <gmcharlt@gmail.com>
2009-08-10 20:34:50 -04:00
Galen Charlton
7ce6822973 removed incorrect module import
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Signed-off-by: Galen Charlton <galen.charlton@liblime.com>
2009-06-07 20:09:22 -05:00
Mason James
31adf4ef83 patch for feature 2952 - v3
Signed-off-by: Galen Charlton <galen.charlton@liblime.com>
2009-02-12 16:39:33 -06:00
Galen Charlton
57fb7e45b7 kohabug 2186 - improve display of empty import batches
[1] Display the "No records have been staged" message only
    if no import batches exist.
[2] When displaying a batch that has no bibs in it, display
    a "There are no records in this batch to import".  Also,
    do not display the "import record into catalogue" button
    for an empty batch.
[3] Add CSS class "problem" to both messages.

Documentation changes: minor; only required if screenshots
exist of the manage staged MARC records page that has no
import batches on it.

Signed-off-by: Joshua Ferraro <jmf@liblime.com>
2008-05-31 21:59:03 -05:00
Galen Charlton
438ed23337 staging import - enhance record overlay behavior
Enhanced the ability of catalogers to specify how
bib and item records should be added, replaced, or
ignored during a staging import.

When an import batch of bib records is staged and commit,
the user can now explicitly specify what should occur
when an incoming bib record has, or does not have, a match
with a record already in the database.  The options are:

if match found (overlay_action):
  create_new (just add the incoming record)
  replace (replace the matched record with the incoming one)
  use_template (option not implemented)
  ignore (do nothing with the incoming bib; however, the
          items attached to it may still be processed
           based on the item action)

if no match is found (nomatch_action):
  create_new (just add the incoming record)
  ignore (do nothing with the incoming bib; in this
          case, any items attached to it will be
          ignored since there will be nothing to
          attach them to)

The following options for handling items embedded in the
bib record are now available:

  always_add (add the items to the new or replaced bib)
  add_only_if_match (add the items only if the incoming bib
                     matches an existing bib)
  add_only_if_add (add the items only if the incoming bib
                   does *not* match an existing bib)
  ignore (ignore the items entirely)

With these changes, it is now possible to support the following use cases:

[1] A library joining an existing Koha database wishes to add their
    items to existing bib records if they match, but does not want
    to overlay the bib records themselves.
[2] A library wants to load a file of records, but only handle
    the new ones, not ones that are already in the database.
[3] A library wants to load a file of records, but only
    handle the ones that match existing records (e.g., if
    the records are coming back from an authority control vendor).

Documentation changes:

* See description above; also, screenshots of the 'stage MARC records
for import' and 'manage staged MARC records' should be updated.

Test cases:

* Added test cases to exercise staging and committing import batches.

UI changes:

* The pages for staging and managing import batches now have
  controls for setting the overlay action, action if no match,
  and item action separately.
* in the manage import batch tool, user is notified when they
  change overlay action, no-match action, and item action
* HTML for manage import batch tool now uses fieldsets

Database changes (DB rev 076):

* added import_batches.item_action
* added import_batches.nomatch_action
* added 'ignore' as a valid value for import_batches.overlay_action
* added 'ignored' as a valid value for import_records.status
* added 'status' as a valid value for import_items.status

API changes:

* new accessor routines for C4::ImportBatch

    GetImportBatchNoMatchAction
    SetImportBatchNoMatchAction
    GetImportBatchItemAction
    SetImportBatchItemAction

* new internal functions for C4::ImportBatch to
  determine how a given bib and item are to be
  processed, based on overlay_action, nomatch_action,
  and item_action:

    _get_commit_action
    _get_revert_action

Signed-off-by: Joshua Ferraro <jmf@liblime.com>
2008-04-30 21:58:46 -05:00
Galen Charlton
fc6ccb1a61 granular permissions - updated Tools
Updated all scripts appearing on the tools page
to respect a granular permission defined for
each of them.

The tools menu and home page have been changed so
that only the specific tools that a user has
access are displayed.  This is simple, but depending on
the module and circumstance, it may be better to
display functions that the user has does not have
access to, but disable the links and do some sort
of visual styling to indicate that a function exists
but requires additional privileges to access.

Signed-off-by: Joshua Ferraro <jmf@liblime.com>
2008-04-10 15:53:30 -05:00
Ryan Higgins
46826aaee0 Mod C4::Labels::add_batch to accept an array of items.
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
2008-03-17 21:32:31 -05:00
Ryan Higgins
760ce0b18c Partial commit to add Create-Label-Batch-from-Import-Batch
Import batch portion herein. Still need to touch Labels.pm .

Signed-off-by: Joshua Ferraro <jmf@liblime.com>
2008-03-17 21:32:30 -05:00
Ryan Higgins
58efea0215 Allow stage_biblios_file to take matcher id as parameter.
Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
2008-02-08 06:07:49 -06:00
Galen Charlton
8f854c0867 speed boost: tools stage and commit bib records
Turned off autocommit; commit every 50 records.

Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
2008-01-03 16:26:26 -06:00
Galen Charlton
ad4e02f91d warn on attempts to add duplicate item barcodes during batch import
Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
2007-12-02 15:06:24 -06:00
Galen Charlton
da1f10a7c1 MARC import: part 6 of large file support
Added background job and AJAX progress updates
to MARC commit job.

Signed-off-by: Joshua Ferraro <jmf@liblime.com>
2007-11-26 20:05:35 -06:00
Galen Charlton
b69facedc4 matching enhancements -- allow matching rule to be changed on the fly
Enhancement to store the matching rule associated with an
import batch and to allow the current matching rule in
effect to be changed and the duplicate detection redone.

Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
2007-11-16 07:37:24 -06:00
Galen Charlton
01c19a265a batch import rework: added suport for loading and dropping items
Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
2007-10-31 05:56:51 -05:00
Galen Charlton
853aa657ba batch import rework -- implement stage-commit-undo for batch import
Revamps the import options on the tools menu to have two parts:

[1] Staging (load file into reservoir)
[2] Managing (review the list of staged batches, then
    choose to commit or undo a given batch.

Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
2007-10-29 16:47:59 -05:00