Commit graph

25 commits

Author SHA1 Message Date
Galen Charlton
9d4d8897b2 item rework: various changes
* Move CheckItemPreSave to C4::Items (from C4::Biblio)
* Modified C4::Biblio::AddBiblioAndItems to use appropriate
   internal routines from C4::Items
* Moved GetItemnumberFromBarcode to C4::Items
* Removed duplicate C4::Biblio::_koha_new_items
* Removed disused C4::Biblio::MARCitemchange

Currently AddBiblioAndItems is a special routine that
uses private subs from both C4::Biblio and C4::Items.
This needs to be refactored.

Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
2008-01-03 16:25:42 -06:00
Joshua Ferraro
5c23369af2 Fixing Database Definitions for Statuses *PARTIAL*
Prior to this fix, the status fields had three 'off' values, NULL, "",
and 0. I've reduced it to two in the db, removing the option for NULL, and
setting the default value to 0, however, we need to verify that we don't ever
write out as "" as this needlessly complicates the indexing process,
critical for searching or limiting by status (e.g., availability). Also,
queries that attempt to write a NULL value to one of these fields will fail
(based on my tests).

This patch includes the following changes:

* Updated the database definition for notforloan, damaged, itemlost, and
wthdrawn in kohastructure.sql to forbid NULL and default to 0; MySQL
can't forbid other values (such as empty ""), so this has to be handled
at the application layer and REQUIRES further patching.

* Fixed the 'limit by availability' query node in Search.pm to use a
much less confusing definition of 'available'

* Added code to set values to 0 where they are NULL or empty ( "" ) for
notforloan, damaged, itemlost or wthdrawn in both the MARC and the items
table:

  * Biblio.pm -> AddBiblioAndItems
  * catalogue/updateitem.pl
  * SEE NOTE BELOW, REQUIRES UPDATE TO THE REST OF KOHA'S ITEM MGT!

* Removed code in bulkmarcimport.pl that sets notforloan status depending
  on item-level or bib-level itemtype -- that flag is designed to be set
  only to override the notforloan setting for the item's (or bib's,
  depending on the syspref) assigned itemtype (it doesn't need to override
  to 'for loan', only to 'not for loan').

  added $dbh->do("truncate zebraqueue"); when operation is 'delete'

* I updated some notes in catalogue/updateitem.pl as to why ModItem can't be
used -- we don't have _a_ place where we can change the item and marc :/

  I've tested the following:

  bulkmarcimport.pl..........................MARC/items OK
  Staged Records Import......................NOT OK
  updateitem.pl (via moredetail.pl)..........MARC/items OK
  circulation.pl.............................NOT OK
  returns.pl.................................NOT OK
  addbiblio.pl...............................NOT OK
  additem.pl.................................NOT OK

Basically, there isn't a single place to apply this patch that will
update both item data and MARC data in one place ... a future patch
needs to address this issue.

Signed-off-by: Galen Charlton <galen.charlton@liblime.com>
Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
2008-01-03 16:23:04 -06:00
Galen Charlton
3508933c66 bulkmarcimport: enable MARC-8 to UTF-8 conversion
Enabled automatic conversion of MARC-8 records to
UTF-8.  Record is converted if its Leader/09 contains
a blank and the -s (skip) option hasn't been supplied
on the command-line.  Any record that cannot be converted
to UTF-8 is skipped.

Also now use Unicode Normalization Form C (NFC) for
records converted from MARC-8.

Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
2007-12-25 09:08:38 -06:00
Galen Charlton
d426a91d0e removed extraneous comments
Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
2007-12-25 09:08:35 -06:00
Galen Charlton
cb6cf680bc improved error detection in AddBiblioAndItems
Introduced new C4::Biblio function CheckItemPreSave,
which checks for duplicate barcodes and invalid
branch codes.  Not yet sure whether this function
needs to be exported or whether it will just be
used internally to C4::Bibli.

Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
2007-12-25 09:08:34 -06:00
Galen Charlton
6b49df4c3f removed superfluous comments
Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
2007-12-25 09:08:31 -06:00
Galen Charlton
7d47666f7e bulk MARC record import - speed improved
Changes to improve speed of MARC bib and item
imports:

[1] Turn off autocommit and commit database
    transactions in larger batches.
[2] Introduce a new C4::Biblio function (AddBiblioAndItems)
    to combine AddBiblio and AddItems -- this is faster
    because we are not parsing the MARC XML of the biblio
    every time we add an item.
[3] Introduce FasterTransformMarcToKoha, which is much
    faster than TransformMarcToKoha.  The new version,
    which will replace the old one once it has been
    fully tested, scans through each field in the
    MARC record just once, instead of potentially
    dozens of times.
[4] Remove code in bulkmarcexport that moved the
    item tags to separate MARC::Record objects.

Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
2007-12-25 09:08:28 -06:00
Galen Charlton
b8a58c4934 installer: command-line scripts improve finding C4 modules
Command-line scripts now use a new SCRIPT_DIR/kohalib.pl
to put installed location of Koha's Perl modules
into @INC.
2007-12-17 09:13:54 -06:00
Galen Charlton
ad4e02f91d warn on attempts to add duplicate item barcodes during batch import
Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
2007-12-02 15:06:24 -06:00
Mason James
a51118833c wrapping AddBiblio(), and AddItem() in evals{} to protect import from failure due to bad records.
Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
2007-11-11 18:44:13 -06:00
Paul POULAIN
1ac38782a1 #1474 : Bulkmarcimport croaks when Log is ON
set to 0 and restore at the end of the import

Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
2007-10-11 14:53:59 -05:00
Paul POULAIN
375d2f1158 (minor) updating doc & removing warn
Signed-off-by: Chris Cormack <crc@liblime.com>
2007-10-03 14:57:12 -05:00
Joshua Ferraro
b87d4924b9 commenting out set_service_options, but also removes commit op
Signed-off-by: Chris Cormack <crc@liblime.com>
2007-10-01 17:40:31 -05:00
Ryan Higgins
c44efe7b84 fix bad call to GetMarcFromKohaField in bulkmarcimport, and add -fk param, allowing disabling of fk constraints during import.
Signed-off-by: Chris Cormack <crc@liblime.com>
2007-09-30 21:16:50 -05:00
toins
4728830e34 it's faster to 'truncate' instead of using 'delete from'... 2007-06-08 09:41:14 +00:00
tipaul
a481fad4b7 Code cleaning :
== Biblio.pm cleaning (useless) ==
* some sub declaration dropped
* removed modbiblio sub
* removed moditem sub
* removed newitems. It was used only in finishrecieve. Replaced by a Koha2Marc+AddItem, that is better.
* removed MARCkoha2marcItem
* removed MARCdelsubfield declaration
* removed MARCkoha2marcBiblio

== Biblio.pm cleaning (naming conventions) ==
* MARCgettagslib renamed to GetMarcStructure
* MARCgetitems renamed to GetMarcItem
* MARCfind_frameworkcode renamed to GetFrameworkCode
* MARCmarc2koha renamed to TransformMarcToKoha
* MARChtml2marc renamed to TransformHtmlToMarc
* MARChtml2xml renamed to TranformeHtmlToXml
* zebraop renamed to ModZebra

== MARC=OFF ==
* removing MARC=OFF related scripts (in cataloguing directory)
* removed checkitems (function related to MARC=off feature, that is completly broken in head. If someone want to reintroduce it, hard work coming...)
* removed getitemsbybiblioitem (used only by MARC=OFF scripts, that is removed as well)
2007-03-29 13:30:31 +00:00
tipaul
a3999812e6 rel_3_0 moved to HEAD 2007-03-09 14:52:58 +00:00
thd
ad657e71eb For MARC 21, instead of deleting the whole subfield when a character does not
translate properly from MARC8 into UTF-8, only the problem characters are
deleted.
2006-09-01 17:11:53 +00:00
toins
eac83ccd45 Head & rel_2_2 merged 2006-07-04 15:02:42 +00:00
rangi
10b2315eb3 Fixing the problem that all items were getting biblioitem=1 set 2006-04-01 22:10:50 +00:00
kados
44b4d37b54 removed Zconns, no need for them anymore with new Context.pm setup 2006-02-27 01:06:30 +00:00
kados
fafe0896d6 minor bugfix with 'commit' option 2006-02-25 23:40:59 +00:00
kados
77abbe2caf A bulkmarcimport.pl that is based on the new Biblio.pm Zebra routines.
It now responds to:

-n : the number of records to import.
-commit : the number of records to wait before performing a 'commit' operation

ALSO: IMPORTANT: I took out the char_encoding as this should be handled by
MARC::File::XML now, unless I'm mistaken.
2006-02-25 21:53:48 +00:00
tipaul
c52e5b61dd synch'ing 2.2 and head 2005-08-04 14:10:52 +00:00
tipaul
51e204fa23 moving bulkmarcimport script to migration_tools directory 2005-01-03 15:25:50 +00:00
Renamed from misc/bulkmarcimport.pl (Browse further)