* Defined local field 942$a to store the authority type
for MARC21 instead of 152$b
* Added 942$b to MARC21 authority framework.
* Added auth_header.authid and auth_header.authtypecode
to appropriate subfields in MARC21 authority framework.
* Started work on two new modules:
C4::AuthoritiesMarc::MARC21
C4::AuthoritiesMarc::UNIMARC
These modules will be used to extract MARC-format-specific
behavior out of C4::AuthoritiesMarc
* Updated Zebra config for MARC21 to use only the 942$a
for the authority type.
* For MARC21, added logic to move 152$b to 942$a for
existing authority records. Specifically, AddAuthority
now does this move when a record is saved, while
GetAuthority and GetAuthorityXML do this when
extracting a record for other use. This logic
is temporary, and can hopefully be removed later, once
use of 152$b in MARC21 authorities is confirmed to be
absent for Koha users. I will also create a batch
job to do this update in one fell swoop.
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
For some reason currently unclear to me, a use 'C4::XXX' does
not seem to guarantee that the contents of C4::XXX's
@EXPORT gets imported into C4::Circulation's symbol table.
Added an INIT block to ensure that this gets done as
part of runtime initialization.
Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
Batch import was not removing item fields (e.g., 952
or 995) from MARC records.
Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
Replace C4::Biblio::AddBiblioAndItems with two
things:
* An option to C4::Biblio::AddBiblio to defer writing
biblioitems.marc and biblioitems.marcxml. This
option was created to give a significant
speed boost to bulkmarcimport.pl, but is *not*
recommended for general use.
* C4::Items::AddItemBatchFromMarc
This refactoring removes the need to have functions
in C4::Biblio and C4::Items that call each other's
private functions.
Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
* Move CheckItemPreSave to C4::Items (from C4::Biblio)
* Modified C4::Biblio::AddBiblioAndItems to use appropriate
internal routines from C4::Items
* Moved GetItemnumberFromBarcode to C4::Items
* Removed duplicate C4::Biblio::_koha_new_items
* Removed disused C4::Biblio::MARCitemchange
Currently AddBiblioAndItems is a special routine that
uses private subs from both C4::Biblio and C4::Items.
This needs to be refactored.
Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
Moved following functions from C4::Biblio to
C4::Items:
GetItemStatus
GetItemLocation
GetLostItems
GetItemsForInventory
GetItemsCount
GetItemInfosOf
GetItemsByBiblioitemnumber
GetItemsInfo
get_itemnumbers_of
Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
If an item field with the appropriate itemnumber does not
exist in the MARC bib, the incoming item field is now
just added instead of being ignored.
Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
Moved this function from an exported function in
C4::Biblio to a public but unexported function
in C4::Items. Added comment noting that this
is used only by additem.pl.
Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
This patch is meant to guarantee that a bulkedit
does not try to edit an item tag embedded in a MARC
biblio without updating the items feature. It is
not a comprehensive fix of the bulkedit feature, which
currently does not appear to be functional and
needs some thought:
* The general search results is probably not the
best place to put this feature -- it should
probably be in tools.
* A bulk edit of something like items is desireable,
but needs to be designed so that it respects
business logic for circulation and acquisitions.
Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
* Moved exported ModItemInMarc from C4::Biblio to
C4::Items and renamed to _replace_item_field_in_biblio.
Function is now private and is not exported, as
ModItem is now the sole entry point for updating
an item record.
* Replaced calls to ModItemInMarc in C4::Circulation
with appropriate ModItem calls.
Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
Introduced C4::Items module to separate items API
from biblio API. Details on changes will be
put in later commit messages.
Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
Prior to this fix, the status fields had three 'off' values, NULL, "",
and 0. I've reduced it to two in the db, removing the option for NULL, and
setting the default value to 0, however, we need to verify that we don't ever
write out as "" as this needlessly complicates the indexing process,
critical for searching or limiting by status (e.g., availability). Also,
queries that attempt to write a NULL value to one of these fields will fail
(based on my tests).
This patch includes the following changes:
* Updated the database definition for notforloan, damaged, itemlost, and
wthdrawn in kohastructure.sql to forbid NULL and default to 0; MySQL
can't forbid other values (such as empty ""), so this has to be handled
at the application layer and REQUIRES further patching.
* Fixed the 'limit by availability' query node in Search.pm to use a
much less confusing definition of 'available'
* Added code to set values to 0 where they are NULL or empty ( "" ) for
notforloan, damaged, itemlost or wthdrawn in both the MARC and the items
table:
* Biblio.pm -> AddBiblioAndItems
* catalogue/updateitem.pl
* SEE NOTE BELOW, REQUIRES UPDATE TO THE REST OF KOHA'S ITEM MGT!
* Removed code in bulkmarcimport.pl that sets notforloan status depending
on item-level or bib-level itemtype -- that flag is designed to be set
only to override the notforloan setting for the item's (or bib's,
depending on the syspref) assigned itemtype (it doesn't need to override
to 'for loan', only to 'not for loan').
added $dbh->do("truncate zebraqueue"); when operation is 'delete'
* I updated some notes in catalogue/updateitem.pl as to why ModItem can't be
used -- we don't have _a_ place where we can change the item and marc :/
I've tested the following:
bulkmarcimport.pl..........................MARC/items OK
Staged Records Import......................NOT OK
updateitem.pl (via moredetail.pl)..........MARC/items OK
circulation.pl.............................NOT OK
returns.pl.................................NOT OK
addbiblio.pl...............................NOT OK
additem.pl.................................NOT OK
Basically, there isn't a single place to apply this patch that will
update both item data and MARC data in one place ... a future patch
needs to address this issue.
Signed-off-by: Galen Charlton <galen.charlton@liblime.com>
Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
1. doesn't remove author_nospan, and <span>s around bib fields for
display, perhaps they should just get a separate style?
2. using subfields now
3. improved ability to highlight small words (3 chars and less)
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
Adding NZoperatorAND NZoperatorOR NZoperatorNOT
Using NZOperatorAND NOT in NZanalyse
Some problem solved.
Please test.
Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
This commit is a partial fix to authority control for MARC21,
and better authority handling in general (for UNIMARC too).
Before this patch, authority searching, editing, saving, was
not functioning, or was extremely buggy.
WARNING: You will need to re-index your authority database after
applying this commit.
The following changes have been made:
* Normalizing record.abs index names (in both MARC21 and UNIMARC)
* Synching authorities/bib1.att, ccl.properties, AuthoritiesMarc.pm
with new indexes (UNIMARC too)
* Clean up biblios/bib1.att (remove duplicate att defs)
* Clean up authorities-* templates to conform to new styles
* Fixed search failure when using Default framework (now searches
All)
Also included are several fixes to the built-in SRU server for
Authority and Biblio, it's recommended that you update your
koha-conf.xml file:
* adding explain-authorities.xml and explain-biblios.xml
* adding necessary info to koha-conf.xml to enable SRU/W
* adding several example XSLT stylesheets, that can be used
for SRU on-the-fly transformations (to MODS, DC, RDF, etc.)
Still remaining for 3.0 are the following tasks:
* update MARC21 frameworks (authority and cross-reference bib)
* update display code/templates in authority results list
* update search code/templates to utilize index points
* implement 'grouping' of authtypes for searching (Name, Title, Subject)
* repair utility to import auths and perform matching
* repair bibliographic import to match auths and warn if no match
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
had been losing datereceived and replacementprice on modorder.
Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
bug fixes to display and save publishercode and purchase order numbers.
use invoice number in place of parcel code
fix template apparently allowing user to modify biblio details on add/mod order
removing CGI scrolling_list
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
Prior to this fix, those two keys were always
created by TransformMarcToKoha even when
not appropriate (e.g., for items).
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
Getting Search.pm air tight ... cleaned up some local variables
that were declared global
fix to asynchronous federated searching lost since dev_week, immediate
use is authority search
template fixes to item-level itemtypes and bib-level itemtypes
temp workaround to javascript problems preventing item edits
fix to installer
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
When using mysql and a sufficiently recent version
of DBD::mysql that actually respects the mysql_enable_utf8
option, setting the current branch to a name that
includes a diacritic causes the session to fail
and forces the user to the login screen upon
clicking another link.
This turned out to be due to the default (Data::Dumper)
serialization used by CGI::Session, which truncated the
session object at the diacritic. By enabling
that module's YAML serialization option, the problem
was resolved.
This patch introduces a dependency on the YAML::Syck
module.
Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
Doc for DBD::mysql states that if not setting
mysql_enable_utf8 as part of the database
connect(), must do it before doing "set names utf8"
Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
Replaced with FasterTransformMarcToKoha
from a previous commit. Main differences are:
[1] At least twice as fast because of improvement
in algorithm for processing the MARC record --
each tag is processed only once.
[2] Fixed bug where biblio.notes would end up with
an extra ' | ' at the end.
Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
Introduced new C4::Biblio function CheckItemPreSave,
which checks for duplicate barcodes and invalid
branch codes. Not yet sure whether this function
needs to be exported or whether it will just be
used internally to C4::Bibli.
Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
Changes to improve speed of MARC bib and item
imports:
[1] Turn off autocommit and commit database
transactions in larger batches.
[2] Introduce a new C4::Biblio function (AddBiblioAndItems)
to combine AddBiblio and AddItems -- this is faster
because we are not parsing the MARC XML of the biblio
every time we add an item.
[3] Introduce FasterTransformMarcToKoha, which is much
faster than TransformMarcToKoha. The new version,
which will replace the old one once it has been
fully tested, scans through each field in the
MARC record just once, instead of potentially
dozens of times.
[4] Remove code in bulkmarcexport that moved the
item tags to separate MARC::Record objects.
Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
Major bugfixes:
* Circulation status in moredetail.pl now correctly displays On loan
status and Date due
* Bugfixes on Status column in detail.pl
* Removed hard-coded English strings from status column in detail.pl
* Added missing auth values for lost and damaged statuses, sane
defaults if those auth
values don't exist
* Added missing 'On hold' status and 'Hold waiting' status
display
Medium bugfixes:
* Now correctly shows itemtype at level specified in systempreference
* Uses itemtypeimage for display, image title is full itemtype
description
* Fixed Location column concatenation between branchname location and
itemcallnumber
* broke out Location (branch), Collection (items.location)
Call Number (itemcallnumber) into their own columns
* Added auth value display for items.location
Minor fixes:
* detail.pl didn't show dates correctly
* nomenclature s/Checked out/On loan/ in opac-detail.tmpl
* s/Book/Item/ in my $desc="Item Returned
".$iteminfo->{'barcode'}; in Circulation.pm
* request.tmpl: s/date_due/onloan/ to display date due
* Fixed SimilarItems, XISBNSimilarItems
NOTE: Needs to be ported to OPAC
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
Major Fix:
Fixing bug where 'expected' label existed for items that
were 'waiting'
Minor Cleanup:
Show correct statuses
Show itemtypes at correct levels depending on
global system preference
Display branch names, not branch codes
Adding in visibility option for 'canreservefromotherbranches' syspref
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
This reverts commit 51ec728f7b.
That commit basically reverted 95% of the status handling I'd done
last week ...
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
When attempting to return an item that was not actually
on loan, would crash because GetItemIssue did not return
biblionumber in hash. Add call to get biblionumber
directly.
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
* adding zebra daemons for managing server and queue processes
* improvements to the README.debian file
* Fixes to Search.pm since last series of commits broke zebra-based
searching (again)
* moving some files to new misc/bin and misc/cronjobs
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
When creating a new C4::Dates object, now carps
if the supplied date and/or time is semantically
invalid, i.e., 2007-01-32 will pass the regex
check but is not a valid date (there being no
32nd of January).
As the C4::Dates API is nailed down, these may
be converted to croaks in the future.
Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
Bug fixing search on alphanumeral string such as EN0 couldnot work.
Bug fixing AND would be too much restrictive on results (required count of occurence of search criteria to be the same).
Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
my $biblionumbers was badly placed
renaming operand into operator
Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
GetNextSeq calls to Date::Calc is now caught in evals so that invalid dates would not make it die.
Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
Also, adds google-style search query highlighting anywhere
in the record
Need to check if this fix is in the OPAC
Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
Previous patch had referred to column as category_id,
which is incorrect.
Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
Summary of Koha 3.0 date indexing for MARC21:
Index Expected format Notes
-----------------------------------------------------
date-entered-on-file [yymmdd] (008/0-5, indexed in word and sort indexes)
copydate [yyyy] (260$c, indexed in word and sort indexes)
acqdate [yyyy-mm-dd] (952$d, indexed in date,word,sort indexes)
pubdate [yyyy] (008/7-10, indexed in year,word,sort indexes)
Template Search Parameters Tested:
limit-yr (either yyyy or yyyy-yyyy) (added processing for ge le, structure attribute st-numeric, etc.)
yr pubdate (yyyy)
acqdate,st-date-normalized (yyyy-mm-dd)
Template Sort Parameters Tested:
pubdate_dsc
pubdate_asc
acqdate_dsc
acqdate_asc
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
* rewrite-config.PL now puts in installed location
of koha-conf.xml in C4/Context.pm so that
correct config can be found even when
KOHA_CONF is not set. Note that setting KOHA_CONF
will still override path set by installer.
* changed references from koha.xml to koha-conf.xml
the reported tag contains all subfields to report from the authority into the biblio.
NOTE : It is a feature usefull only for OpenCataloger, but OpenCataloger needs it.
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
- the guarantor is no more readonly: it can be typed manually
- useless fields in case of an institution are removed (like DoB, Firstname, gender)
- reintroducing the relashionship select, that was missing in the template (but was handled by the pl)
- moving B_phone to a more logical place (with alternate coordinate)
- reintrocing contactnote in "alt contact"
- changing the select_city behaviour : when a city is choosen, the zipcode & city are automatically filled.
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
finalresult contained the whole list and not only the useful results.
resultlist contained only 19 elements. adding one
parameters passed through pages contained also empty parameters deleting them.
Conflicts:
C4/AuthoritiesMarc.pm
Signed-off-by: Galen Charlton <galen.charlton@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
Apparently as an inadvertant result of the patches to
move the SQL files to load the MARC frameworks, the
authorised_value of 'cn_source' was dropped for
kohafield = 'items.cn_source' and 'biblioitems.cn_source'.
This patch restores those values, thus making the
classification source set by a dropdown in the bib
and item editors.
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
The way this API is used by additem.pl, the $item hasref
already contains a 'items.cn_sort' key whose value is
undef. Setting $item->{'cn_sort'} instead of $item->{'items.cn_sort'}
ends putting with items.cn_sort and cn_sort in the UPDATE
items statement, making the final value of cn_sort in the DB
dependent on the order produced by Perl's keys function.
Also added a comment explaining a subtle (perhaps too subtle)
point in the code.
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
Now warns and returns error value, instead of aborting
the calling CGI.
Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
When an item was created, the renewals, issues, and reserves columns
were not set from the MARC data. This was noticed particularly
in the context of batch loads, where a user might supply
data for those fields.
Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
limits would provide with unnecessary repeated "and"
Thus causing wild loop
So removing them.
aud was transformed into author when it qualified audience.
Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
parenthesis introduce a priority.
yr:XXXX-YYYY is now processed correctly.
Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
this fix add a author_nospan, that contains the author in it's native form
Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
If a MARC batch was imported, then reverted, the 952s
from the saved copied of the bib were added back
when the bib was replaced, leading to duplication
Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
This should help make the items table more authoritative, and ensures
that the items editor does not blank circulation fields such as
issues.
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
Note: C4::Biblio::GetMarcFromKohaField returns (0, 0) if
the kohafield is not defined in the MARC framework, but
in may places the return value is not checked correctly
and does not recognize a subfield $0 (zero). It
would be better if GetMarcFromKohaField returned (undef, undef)
in that circumstance, but because of the number of places
where that function is used, the changed is deferred for
post 3.0.
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
The correct field is 'items.wthdrawn', not 'items.withdrawn'.
Spelling may be corrected in post-3.0 version.
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
Fixed so that when using updateitem.pl to change an
item's lost, damaged, or withdrawn status, the
bib's MARC XML is updated and an entry is added
to zebraqueue.
As part of the fix, C4::Biblio::ModItemInMarconefield
now works correctly.
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
Staging job now gets moved to background so that
it does not get killed if it takes longer
than the Apache timeout.
Added AJAX monitoring of job status.
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
* Added C4::BackgroundJob to monitor tracking of
background jobs
* start modifying stage-marc-import to use it
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
Also cleaning up some missing params and language.
Add another js library: greyox (jquery derivative iiuc).
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
There were 2 bugs :
- one in moremember.pl that supposed the date value to be in preffered order, which is wrong, it's in iso at this time
- the output does not work as expected, as it always return iso.
Pls atz, confirm Dates.pm was bugged.
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
the modules/ directory is now in opac as well as in intranet.
non english languages were unreachable without this patch.
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
* Introduced C4::UploadedFile to handle management and
progress tracking of uploaded files.
* Modified stage-marc-import.pl to handle new upload
mechanism
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
Refactoring to provide single place to get CGI::Session ojbject;
fixes bug for DB storage method other than 'mysql'.
This refactoring is also part of the patch series for
handling large input files for staging and processing
MARC records.
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
After experiencing this for myself I've decided to just
add them by default, if we want to wrap in a syspref at some
point we can.
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
sort {$a cmp $b} @array is the same as sort @array
But sort {$b cmp $a} @array reverse order.
+ Adapting GetMarcFromKohaField to new signature.
Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
If you have a $9 as authority, then you can directly search on an=<koha-auth-number>
For example, if you have :
600$912045$aEurope$xFrance$xMarseille (Europe--France--Marseille as subject of the biblio)
The link must be "an=12045"
The link is not changed in case there is no $9 (still apply on each term, one by one)
Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
