Adding encoding for z3950 server information.
Uses Text::Iconv for conversion (ISO6937 and ISO_5428 and ISO5427)
For ISO 5426 (ANSEL or MARC-8) new char_decode5426 based on marc4j tool.
Not Tested on LOC or any USMARC z3950 source. But tested OK on BNF and SUDOC.
Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
This commit makes db changes to subscription, serials tables:
allow subscriptions larger than 127 issues
link serial to item uniquely.
MARC Framework:
Adding MARC tag 942$s (990$s unimarc) to map biblio.serial
Udate biblio.serial on new-subscription.
Use biblio.serial to check if Items.pm should return
pubdate and enumeration text string from serials table with items.
Update opac-detail so serial items are sorted most recent first.
Adding link from items table to serial table.
Allows sorting of serials in details pages, and removes
enum data from itemnotes field.
Also fixes visibility in serials item editor
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
Replace C4::Biblio::AddBiblioAndItems with two
things:
* An option to C4::Biblio::AddBiblio to defer writing
biblioitems.marc and biblioitems.marcxml. This
option was created to give a significant
speed boost to bulkmarcimport.pl, but is *not*
recommended for general use.
* C4::Items::AddItemBatchFromMarc
This refactoring removes the need to have functions
in C4::Biblio and C4::Items that call each other's
private functions.
Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
* Move CheckItemPreSave to C4::Items (from C4::Biblio)
* Modified C4::Biblio::AddBiblioAndItems to use appropriate
internal routines from C4::Items
* Moved GetItemnumberFromBarcode to C4::Items
* Removed duplicate C4::Biblio::_koha_new_items
* Removed disused C4::Biblio::MARCitemchange
Currently AddBiblioAndItems is a special routine that
uses private subs from both C4::Biblio and C4::Items.
This needs to be refactored.
Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
Moved following functions from C4::Biblio to
C4::Items:
GetItemStatus
GetItemLocation
GetLostItems
GetItemsForInventory
GetItemsCount
GetItemInfosOf
GetItemsByBiblioitemnumber
GetItemsInfo
get_itemnumbers_of
Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
Moved this function from an exported function in
C4::Biblio to a public but unexported function
in C4::Items. Added comment noting that this
is used only by additem.pl.
Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
* Moved exported ModItemInMarc from C4::Biblio to
C4::Items and renamed to _replace_item_field_in_biblio.
Function is now private and is not exported, as
ModItem is now the sole entry point for updating
an item record.
* Replaced calls to ModItemInMarc in C4::Circulation
with appropriate ModItem calls.
Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
Introduced C4::Items module to separate items API
from biblio API. Details on changes will be
put in later commit messages.
Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
Prior to this fix, the status fields had three 'off' values, NULL, "",
and 0. I've reduced it to two in the db, removing the option for NULL, and
setting the default value to 0, however, we need to verify that we don't ever
write out as "" as this needlessly complicates the indexing process,
critical for searching or limiting by status (e.g., availability). Also,
queries that attempt to write a NULL value to one of these fields will fail
(based on my tests).
This patch includes the following changes:
* Updated the database definition for notforloan, damaged, itemlost, and
wthdrawn in kohastructure.sql to forbid NULL and default to 0; MySQL
can't forbid other values (such as empty ""), so this has to be handled
at the application layer and REQUIRES further patching.
* Fixed the 'limit by availability' query node in Search.pm to use a
much less confusing definition of 'available'
* Added code to set values to 0 where they are NULL or empty ( "" ) for
notforloan, damaged, itemlost or wthdrawn in both the MARC and the items
table:
* Biblio.pm -> AddBiblioAndItems
* catalogue/updateitem.pl
* SEE NOTE BELOW, REQUIRES UPDATE TO THE REST OF KOHA'S ITEM MGT!
* Removed code in bulkmarcimport.pl that sets notforloan status depending
on item-level or bib-level itemtype -- that flag is designed to be set
only to override the notforloan setting for the item's (or bib's,
depending on the syspref) assigned itemtype (it doesn't need to override
to 'for loan', only to 'not for loan').
added $dbh->do("truncate zebraqueue"); when operation is 'delete'
* I updated some notes in catalogue/updateitem.pl as to why ModItem can't be
used -- we don't have _a_ place where we can change the item and marc :/
I've tested the following:
bulkmarcimport.pl..........................MARC/items OK
Staged Records Import......................NOT OK
updateitem.pl (via moredetail.pl)..........MARC/items OK
circulation.pl.............................NOT OK
returns.pl.................................NOT OK
addbiblio.pl...............................NOT OK
additem.pl.................................NOT OK
Basically, there isn't a single place to apply this patch that will
update both item data and MARC data in one place ... a future patch
needs to address this issue.
Signed-off-by: Galen Charlton <galen.charlton@liblime.com>
Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
Prior to this fix, those two keys were always
created by TransformMarcToKoha even when
not appropriate (e.g., for items).
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
Replaced with FasterTransformMarcToKoha
from a previous commit. Main differences are:
[1] At least twice as fast because of improvement
in algorithm for processing the MARC record --
each tag is processed only once.
[2] Fixed bug where biblio.notes would end up with
an extra ' | ' at the end.
Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
Introduced new C4::Biblio function CheckItemPreSave,
which checks for duplicate barcodes and invalid
branch codes. Not yet sure whether this function
needs to be exported or whether it will just be
used internally to C4::Bibli.
Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
Changes to improve speed of MARC bib and item
imports:
[1] Turn off autocommit and commit database
transactions in larger batches.
[2] Introduce a new C4::Biblio function (AddBiblioAndItems)
to combine AddBiblio and AddItems -- this is faster
because we are not parsing the MARC XML of the biblio
every time we add an item.
[3] Introduce FasterTransformMarcToKoha, which is much
faster than TransformMarcToKoha. The new version,
which will replace the old one once it has been
fully tested, scans through each field in the
MARC record just once, instead of potentially
dozens of times.
[4] Remove code in bulkmarcexport that moved the
item tags to separate MARC::Record objects.
Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
The way this API is used by additem.pl, the $item hasref
already contains a 'items.cn_sort' key whose value is
undef. Setting $item->{'cn_sort'} instead of $item->{'items.cn_sort'}
ends putting with items.cn_sort and cn_sort in the UPDATE
items statement, making the final value of cn_sort in the DB
dependent on the order produced by Perl's keys function.
Also added a comment explaining a subtle (perhaps too subtle)
point in the code.
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
When an item was created, the renewals, issues, and reserves columns
were not set from the MARC data. This was noticed particularly
in the context of batch loads, where a user might supply
data for those fields.
Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
This should help make the items table more authoritative, and ensures
that the items editor does not blank circulation fields such as
issues.
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
Note: C4::Biblio::GetMarcFromKohaField returns (0, 0) if
the kohafield is not defined in the MARC framework, but
in may places the return value is not checked correctly
and does not recognize a subfield $0 (zero). It
would be better if GetMarcFromKohaField returned (undef, undef)
in that circumstance, but because of the number of places
where that function is used, the changed is deferred for
post 3.0.
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
Fixed so that when using updateitem.pl to change an
item's lost, damaged, or withdrawn status, the
bib's MARC XML is updated and an entry is added
to zebraqueue.
As part of the fix, C4::Biblio::ModItemInMarconefield
now works correctly.
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
If you have a $9 as authority, then you can directly search on an=<koha-auth-number>
For example, if you have :
600$912045$aEurope$xFrance$xMarseille (Europe--France--Marseille as subject of the biblio)
The link must be "an=12045"
The link is not changed in case there is no $9 (still apply on each term, one by one)
Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
* now properly deletes deleted authorities from index
* now actually ignores duplicate work
* standardized on recordDelete instead of delete_record
for bibs and recordDelete for authorities
Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
This reverts commit 991ae79bb6.
Note: the MARC21 (and UNIMARC) leader length is *24* characters,
not 25.
Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
The test was wrong, thus any leader defined by the library was discarded
(chris/kados : check for MARC21 that size should be 25 as in UNIMARC)
Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
* biblio.biblionumber
* biblioitems.biblioitemnumber
* items.itemnumber
These IDs are no longer increment by the Perl code, thus
allowing better concurrency during bib/item editing.
Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
Still some work to do, but people can see the idea and decide if its good
Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
- uses new C4::Dates instead of C4::Date
- start work on TransformMarcToKoha
* handle ambiguities in certain column names
* part of patch inspired by Chris Cormack
- fix bugs preventing biblioitems and items
from being properly updated
- ensure that biblionumber and biblioitemnumber
always get mapped to MARC
Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
- bugfix: update items.cn_sort upon edit of item
- bugfix: make failure of a value_builder plugin non-fatal
- bugfix: remove references to non-existent plugins
from optional MARC21 frameworks
bookseller.pl
marc21_callnumber.pl
marc21_classcodes.pl
marc21_locationqualifier.pl
- bugfix: corrected trying to enter 'u*' fields twice in the 'CF'
format in the optional MARC21 frameworks; resulted in
most of the framework not being loaded
Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
when Logguing was ON, this resulted in an internal server error, thus the discovery of this bug.
Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
the bug occur only if we have a malformed biblio, so it should "never" happend, but i've seen it once ;-)
Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
C4/Biblio.pm - removed following functions:
_biblioitem_cn_sort
_items_cn_sort
TODO: add a new call number filing routine to emulate logic of
_biblioitem_cn_sort for compatibility purposes
Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
Most of this commit by Joshua Ferraro.
updatedatabase changes by Galen Charlton.
Major changes:
This commit includes a lot of fairly major changes
to Koha's Biblio handling, largest is the addition
and deletion of several columns in the biblioitems,
items tables, as well as cleanup of deletedbiblioitems,
deleteditems tables. Some of the changes are simple
cleanup, but most have to do with improvements to
storage of call numbers in Koha.
Also, I had to clean up the _koha_* routines quite a
lot to make them work -- there was lots of data just
being lost because columns weren't being updated.
I'm still not completely convinced that the items
table is being treated as authoritative for items
data, investigating further.
DB Changes (updated in kohastructure.sql and in
updatedatabases):
ADDED:
biblioitems.cn_source ( auth value, CN_SOURCE, stores the source of the
call number: DDC, LCC, NLM, etc.)
biblioitems.cn_class ( plugin, marc21_callnumber.pl, helps fill in
the rest of the biblio-level fields)
biblioitems.cn_item
biblioitems.cn_suffix
biblioitems.cn_sort ( for zebra sorting, stored as a decimal number)
biblioitems.totalissues ( for counting the total times issued )
items.cn_source ( auth value, CN_SOURCE, stores DDC, LCC, NLM, etc.)
items.itemcallnumber ( plugin, marc21_itemcallnumber.pl, helps fill in
the itemcallnumber based on the record data )
items.cn_sort ( for zebra sorting, stored as a decimal number)
items.ccode ( auth value, CCODE, stores the Collection Code
of the item, can be used as call number prefix
by some libraries )
items.uri
items.materials
items.damaged
DELETED:
items.itype
items.cutterextra
biblioitems.classification
biblioitems.subclass
biblioitems.dewey
biblioitems.lcsort
biblioitems.lccn
biblioitems.ccode
DB version now 3.00.00.009.
Minor changes:
* Drop revision history from C4/Biblio.pm
* GetMarcAuthors now returns additional authors (7XX), not
main authors (1XX)
* Debug warnings in C4/Search.pm commented out
Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
- the availability status was not available on result list. This patch reintroduces that
- notforloan as itemtype was not properly managed : an itemtype that was notforloan resulted in nothing in detail. Not, the user can't place a reserve anymore, and the status is correctly displayed
the fix is for OPAC as well as staff
(owen, pls, validate cat-toolbar.inc & catalogue/detail.tmpl)
Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
It will only delete biblios that have no items attached.
You must delete the items first then delete the biblio
Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
The top issue was based on a timestamp field, that was updated everytime a biblio is modified
This commit add a datecreated field, that is filled only when a biblio is ADDED
it is used by opac-topissues & can be useful at other places i'm sure
Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
When updating a biblio, all repeated fields had the same input name.
Thus, when retrieving them through cgi->param resulted in a single line
Thus all subfields where merged in a single MARC field.
Adding a random() part to the name solves the problem
+ removing some warn lines
Signed-off-by: Chris Cormack <crc@liblime.com>
hdl defined 5 new fields in biblioitems, to handle collections.
The _koha_modify_biblioitem didn't deal with those fields. it's fixed now.
GetMarcAuthor in UNIMARC have a $4 field that is related to the responsability the author has
it is connected to an authorised value.
The improvement expand the numeric value to the expanded one.
Signed-off-by: Chris Cormack <crc@liblime.com>
In Koha 2.2 the biblionumber was an hidden field in the form,
exactly as any other subfield.
As we also stored biblionumber as specific field, it was useless to have it
twice.
with this commit, we restore the biblionumber in the MARC:Record in TransformHtml2Marc
Signed-off-by: Chris Cormack <crc@liblime.com>
So, deal carefully with this commit pls, and check it for your setups, because the patch works for me, but I'm not sure to understand well why :\
Signed-off-by: Chris Cormack <crc@liblime.com>
- updating templates to have tmpl_process3.pl running without any errors
- adding a drupal-like css for prog templates (with 3 small images)
- fixing some bugs in circulation & other scripts
- updating french translation
- fixing some typos in templates
- support for authorities
- some bugfixes in ordering and "CCL" parsing
- support for authorities <=> biblios walking
Seems I can do what I want now, so I consider its done, except for bugfixes that will be needed i m sure !
- NoZebra features : seems they work fine now (adding, modifying, deleting)
- Biblio edition major bugfix : before this commit editing a biblio resulted in an item removal in marcxml field
* adding 3 subs in Biblio.pm
- GetNoZebraIndexes, that get the index structure in a new systempreference (added with this commit)
- _DelBiblioNoZebra, that retrieve all index entries for a biblio and remove in a variable the biblio reference
- _AddBiblioNoZebra, that add index entries for a biblio.
Note that the 2 _Add and _Del subs work only in a hash variable, to speed up things in case of a modif (ie : delete+add). The effective SQL update is done in the ModZebra sub (that existed before, and dealed with zebra index).
I think the code has to be more deeply tested, but it works at least partially.
- add nozebra table management on biblio editing
- the index table content is hardcoded. I still have to add some specific systempref to let the library update it
- manage pagination (next/previous)
- manage facets
WHAT works :
- NZgetRecords : has exactly the same API & returns as zebra getQuery, except that some parameters are unused
- search & sort works quite good
- CQL parser is better that what I thought I could do : title="harry and sally" and publicationyear>2000 not itemtype=LIVR should work fine
All subs have be cleaned :
- removed useless
- merged some
- reordering Biblio.pm completly
- using only naming conventions
Seems to have broken nothing, but it still has to be heavily tested.
Note that Biblio.pm is now much more efficient than previously & probably more reliable as well.
== Biblio.pm cleaning (useless) ==
* some sub declaration dropped
* removed modbiblio sub
* removed moditem sub
* removed newitems. It was used only in finishrecieve. Replaced by a Koha2Marc+AddItem, that is better.
* removed MARCkoha2marcItem
* removed MARCdelsubfield declaration
* removed MARCkoha2marcBiblio
== Biblio.pm cleaning (naming conventions) ==
* MARCgettagslib renamed to GetMarcStructure
* MARCgetitems renamed to GetMarcItem
* MARCfind_frameworkcode renamed to GetFrameworkCode
* MARCmarc2koha renamed to TransformMarcToKoha
* MARChtml2marc renamed to TransformHtmlToMarc
* MARChtml2xml renamed to TranformeHtmlToXml
* zebraop renamed to ModZebra
== MARC=OFF ==
* removing MARC=OFF related scripts (in cataloguing directory)
* removed checkitems (function related to MARC=off feature, that is completly broken in head. If someone want to reintroduce it, hard work coming...)
* removed getitemsbybiblioitem (used only by MARC=OFF scripts, that is removed as well)
Uses a complete new ZEBRA Indexing.
ZEBRA is now XML and comprises of a KOHA meta record. Explanatory notes will be on koha-devel
Fixes UTF8 problems
Fixes bug with authorities
SQL database major changes.
Separate biblioograaphic and holdings records. Biblioitems table depreceated
etc. etc.
Wait for explanatory document on koha-devel
the intranet. The development was made on branch 2.2 by Arnaud Laurin from
Ouest Provence and integrated on HEAD by Pierrick Le Gall from INEO media
system.
New page reserve/request.pl taking a biblionumber as entry point.
New functions:
- C4::Biblio::get_iteminfos_of retrieves item informations for a list of
itemnumbers
- C4::Biblio::get_biblioiteminfos_of retrieves biblioitem informations for a
list of biblioitemnumbers
- C4::Biblio::get_itemnumbers_of retrieve the list of itemnumbers related to
each biblionumber given in argument.
- C4::Circulation::Circ2::get_return_date_of retrieves return date for a
list of itemnumbers.
- C4::Koha::get_itemtypeinfos_of retrieves the informations related to a
list of itemtypes.
- C4::Koha::get_branchinfos_of retrieves the informations related to a list
of branchcodes.
- C4::Koha::get_notforloan_label_of retrives the list of status/label for
the authorised_values related to notforloan.
- C4::Koha::get_infos_of is the generic function used by all get_*infos_of.
- C4::Reserves2::GetNumberReservesFromBorrower
- C4::Reserves2::GetFirstReserveDateFromItem
Modified functions:
- C4::Reserves2::FindReserves was simplified to be more readable.
The reservation page is reserve/request.pl and is linked from nowhere as
long as zebra is not stable yet on HEAD.
