Changes to improve speed of MARC bib and item
imports:
[1] Turn off autocommit and commit database
transactions in larger batches.
[2] Introduce a new C4::Biblio function (AddBiblioAndItems)
to combine AddBiblio and AddItems -- this is faster
because we are not parsing the MARC XML of the biblio
every time we add an item.
[3] Introduce FasterTransformMarcToKoha, which is much
faster than TransformMarcToKoha. The new version,
which will replace the old one once it has been
fully tested, scans through each field in the
MARC record just once, instead of potentially
dozens of times.
[4] Remove code in bulkmarcexport that moved the
item tags to separate MARC::Record objects.
Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
The way this API is used by additem.pl, the $item hasref
already contains a 'items.cn_sort' key whose value is
undef. Setting $item->{'cn_sort'} instead of $item->{'items.cn_sort'}
ends putting with items.cn_sort and cn_sort in the UPDATE
items statement, making the final value of cn_sort in the DB
dependent on the order produced by Perl's keys function.
Also added a comment explaining a subtle (perhaps too subtle)
point in the code.
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
When an item was created, the renewals, issues, and reserves columns
were not set from the MARC data. This was noticed particularly
in the context of batch loads, where a user might supply
data for those fields.
Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
This should help make the items table more authoritative, and ensures
that the items editor does not blank circulation fields such as
issues.
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
Note: C4::Biblio::GetMarcFromKohaField returns (0, 0) if
the kohafield is not defined in the MARC framework, but
in may places the return value is not checked correctly
and does not recognize a subfield $0 (zero). It
would be better if GetMarcFromKohaField returned (undef, undef)
in that circumstance, but because of the number of places
where that function is used, the changed is deferred for
post 3.0.
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
Fixed so that when using updateitem.pl to change an
item's lost, damaged, or withdrawn status, the
bib's MARC XML is updated and an entry is added
to zebraqueue.
As part of the fix, C4::Biblio::ModItemInMarconefield
now works correctly.
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
If you have a $9 as authority, then you can directly search on an=<koha-auth-number>
For example, if you have :
600$912045$aEurope$xFrance$xMarseille (Europe--France--Marseille as subject of the biblio)
The link must be "an=12045"
The link is not changed in case there is no $9 (still apply on each term, one by one)
Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
* now properly deletes deleted authorities from index
* now actually ignores duplicate work
* standardized on recordDelete instead of delete_record
for bibs and recordDelete for authorities
Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
This reverts commit 991ae79bb6.
Note: the MARC21 (and UNIMARC) leader length is *24* characters,
not 25.
Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
The test was wrong, thus any leader defined by the library was discarded
(chris/kados : check for MARC21 that size should be 25 as in UNIMARC)
Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
* biblio.biblionumber
* biblioitems.biblioitemnumber
* items.itemnumber
These IDs are no longer increment by the Perl code, thus
allowing better concurrency during bib/item editing.
Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
Still some work to do, but people can see the idea and decide if its good
Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
- uses new C4::Dates instead of C4::Date
- start work on TransformMarcToKoha
* handle ambiguities in certain column names
* part of patch inspired by Chris Cormack
- fix bugs preventing biblioitems and items
from being properly updated
- ensure that biblionumber and biblioitemnumber
always get mapped to MARC
Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
- bugfix: update items.cn_sort upon edit of item
- bugfix: make failure of a value_builder plugin non-fatal
- bugfix: remove references to non-existent plugins
from optional MARC21 frameworks
bookseller.pl
marc21_callnumber.pl
marc21_classcodes.pl
marc21_locationqualifier.pl
- bugfix: corrected trying to enter 'u*' fields twice in the 'CF'
format in the optional MARC21 frameworks; resulted in
most of the framework not being loaded
Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
when Logguing was ON, this resulted in an internal server error, thus the discovery of this bug.
Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
the bug occur only if we have a malformed biblio, so it should "never" happend, but i've seen it once ;-)
Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
C4/Biblio.pm - removed following functions:
_biblioitem_cn_sort
_items_cn_sort
TODO: add a new call number filing routine to emulate logic of
_biblioitem_cn_sort for compatibility purposes
Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
Most of this commit by Joshua Ferraro.
updatedatabase changes by Galen Charlton.
Major changes:
This commit includes a lot of fairly major changes
to Koha's Biblio handling, largest is the addition
and deletion of several columns in the biblioitems,
items tables, as well as cleanup of deletedbiblioitems,
deleteditems tables. Some of the changes are simple
cleanup, but most have to do with improvements to
storage of call numbers in Koha.
Also, I had to clean up the _koha_* routines quite a
lot to make them work -- there was lots of data just
being lost because columns weren't being updated.
I'm still not completely convinced that the items
table is being treated as authoritative for items
data, investigating further.
DB Changes (updated in kohastructure.sql and in
updatedatabases):
ADDED:
biblioitems.cn_source ( auth value, CN_SOURCE, stores the source of the
call number: DDC, LCC, NLM, etc.)
biblioitems.cn_class ( plugin, marc21_callnumber.pl, helps fill in
the rest of the biblio-level fields)
biblioitems.cn_item
biblioitems.cn_suffix
biblioitems.cn_sort ( for zebra sorting, stored as a decimal number)
biblioitems.totalissues ( for counting the total times issued )
items.cn_source ( auth value, CN_SOURCE, stores DDC, LCC, NLM, etc.)
items.itemcallnumber ( plugin, marc21_itemcallnumber.pl, helps fill in
the itemcallnumber based on the record data )
items.cn_sort ( for zebra sorting, stored as a decimal number)
items.ccode ( auth value, CCODE, stores the Collection Code
of the item, can be used as call number prefix
by some libraries )
items.uri
items.materials
items.damaged
DELETED:
items.itype
items.cutterextra
biblioitems.classification
biblioitems.subclass
biblioitems.dewey
biblioitems.lcsort
biblioitems.lccn
biblioitems.ccode
DB version now 3.00.00.009.
Minor changes:
* Drop revision history from C4/Biblio.pm
* GetMarcAuthors now returns additional authors (7XX), not
main authors (1XX)
* Debug warnings in C4/Search.pm commented out
Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
- the availability status was not available on result list. This patch reintroduces that
- notforloan as itemtype was not properly managed : an itemtype that was notforloan resulted in nothing in detail. Not, the user can't place a reserve anymore, and the status is correctly displayed
the fix is for OPAC as well as staff
(owen, pls, validate cat-toolbar.inc & catalogue/detail.tmpl)
Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
It will only delete biblios that have no items attached.
You must delete the items first then delete the biblio
Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
The top issue was based on a timestamp field, that was updated everytime a biblio is modified
This commit add a datecreated field, that is filled only when a biblio is ADDED
it is used by opac-topissues & can be useful at other places i'm sure
Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
