Commit graph

12 commits

Author SHA1 Message Date
Galen Charlton
438ed23337 staging import - enhance record overlay behavior
Enhanced the ability of catalogers to specify how
bib and item records should be added, replaced, or
ignored during a staging import.

When an import batch of bib records is staged and commit,
the user can now explicitly specify what should occur
when an incoming bib record has, or does not have, a match
with a record already in the database.  The options are:

if match found (overlay_action):
  create_new (just add the incoming record)
  replace (replace the matched record with the incoming one)
  use_template (option not implemented)
  ignore (do nothing with the incoming bib; however, the
          items attached to it may still be processed
           based on the item action)

if no match is found (nomatch_action):
  create_new (just add the incoming record)
  ignore (do nothing with the incoming bib; in this
          case, any items attached to it will be
          ignored since there will be nothing to
          attach them to)

The following options for handling items embedded in the
bib record are now available:

  always_add (add the items to the new or replaced bib)
  add_only_if_match (add the items only if the incoming bib
                     matches an existing bib)
  add_only_if_add (add the items only if the incoming bib
                   does *not* match an existing bib)
  ignore (ignore the items entirely)

With these changes, it is now possible to support the following use cases:

[1] A library joining an existing Koha database wishes to add their
    items to existing bib records if they match, but does not want
    to overlay the bib records themselves.
[2] A library wants to load a file of records, but only handle
    the new ones, not ones that are already in the database.
[3] A library wants to load a file of records, but only
    handle the ones that match existing records (e.g., if
    the records are coming back from an authority control vendor).

Documentation changes:

* See description above; also, screenshots of the 'stage MARC records
for import' and 'manage staged MARC records' should be updated.

Test cases:

* Added test cases to exercise staging and committing import batches.

UI changes:

* The pages for staging and managing import batches now have
  controls for setting the overlay action, action if no match,
  and item action separately.
* in the manage import batch tool, user is notified when they
  change overlay action, no-match action, and item action
* HTML for manage import batch tool now uses fieldsets

Database changes (DB rev 076):

* added import_batches.item_action
* added import_batches.nomatch_action
* added 'ignore' as a valid value for import_batches.overlay_action
* added 'ignored' as a valid value for import_records.status
* added 'status' as a valid value for import_items.status

API changes:

* new accessor routines for C4::ImportBatch

    GetImportBatchNoMatchAction
    SetImportBatchNoMatchAction
    GetImportBatchItemAction
    SetImportBatchItemAction

* new internal functions for C4::ImportBatch to
  determine how a given bib and item are to be
  processed, based on overlay_action, nomatch_action,
  and item_action:

    _get_commit_action
    _get_revert_action

Signed-off-by: Joshua Ferraro <jmf@liblime.com>
2008-04-30 21:58:46 -05:00
Galen Charlton
fc6ccb1a61 granular permissions - updated Tools
Updated all scripts appearing on the tools page
to respect a granular permission defined for
each of them.

The tools menu and home page have been changed so
that only the specific tools that a user has
access are displayed.  This is simple, but depending on
the module and circumstance, it may be better to
display functions that the user has does not have
access to, but disable the links and do some sort
of visual styling to indicate that a function exists
but requires additional privileges to access.

Signed-off-by: Joshua Ferraro <jmf@liblime.com>
2008-04-10 15:53:30 -05:00
Galen Charlton
b88682f1bb bug 1372: count MARC records correctly
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
2008-02-08 20:22:43 -06:00
Galen Charlton
8f854c0867 speed boost: tools stage and commit bib records
Turned off autocommit; commit every 50 records.

Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
2008-01-03 16:26:26 -06:00
Galen Charlton
74e51161d8 removed extraneous warns
Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
2007-11-28 21:44:27 -06:00
Galen Charlton
a869ed0435 MARC import: part 5 of large file support
Staging job now gets moved to background so that
it does not get killed if it takes longer
than the Apache timeout.

Added AJAX monitoring of job status.

Signed-off-by: Joshua Ferraro <jmf@liblime.com>
2007-11-26 20:05:34 -06:00
Galen Charlton
cabb06c56c MARC import: part 4 of large file support
* Added C4::BackgroundJob to monitor tracking of
  background jobs
* start modifying stage-marc-import to use it

Signed-off-by: Joshua Ferraro <jmf@liblime.com>
2007-11-26 20:05:30 -06:00
Galen Charlton
e9f36cc227 MARC import: part 3 of large file support
* Introduced C4::UploadedFile to handle management and
  progress tracking of uploaded files.
* Modified stage-marc-import.pl to handle new upload
  mechanism

Signed-off-by: Joshua Ferraro <jmf@liblime.com>
2007-11-25 16:45:53 -06:00
Galen Charlton
b69facedc4 matching enhancements -- allow matching rule to be changed on the fly
Enhancement to store the matching rule associated with an
import batch and to allow the current matching rule in
effect to be changed and the duplicate detection redone.

Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
2007-11-16 07:37:24 -06:00
Galen Charlton
75bd4b2dac bibliographic matching enhancements
* serialization of matching rules to database
* added two default matching rules for MARC21
* update import staging CGI to choose matching rule

Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
2007-11-16 07:37:21 -06:00
Galen Charlton
01c19a265a batch import rework: added suport for loading and dropping items
Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
2007-10-31 05:56:51 -05:00
Galen Charlton
853aa657ba batch import rework -- implement stage-commit-undo for batch import
Revamps the import options on the tools menu to have two parts:

[1] Staging (load file into reservoir)
[2] Managing (review the list of staged batches, then
    choose to commit or undo a given batch.

Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
2007-10-29 16:47:59 -05:00
Renamed from tools/import.pl (Browse further)