Intermittently problems in the calling environment
cause a C4::Biblio routine to be called with an undefined
MARC::Record object. This results in the process
dying and returning to the end user a low level
message such as 'cannot call method x on an undefined
object'.
For exported subroutines taking a MARC::Record object,
check that object is defined otherwise return a logical
return value and log a stack trace to the error log.
A couple of cases were checking but dying, this may have
unwelcome results in a persistent environment so croak has
been downgraded to carp
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Adds lots of checks for $record in various places, should
not affect behaviour.
Passed all tests and QA script, including new unit tests.
Tested adding and saving a new record.
Also tested detail and result pages without XSLT.
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
No code implements the routines Get and TransformHtmlToMarc2,
so don't export them into users' namespace
Signed-off-by: Mark Tompsett <mtompset@hotmail.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
The patch corrects the issue -- the content of the field 225 shall be
displayes under Series now.
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Comment: Work as described, no koha-qa errors
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Comparing the XSLT with the normal view the patch seems to
work correctly. Passes all tests and QA script.
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
This patch switches from using a combination of
biblionumber/borrowernumber to using reserve_id where possible.
Test Plan:
1) Apply patch
2) Run t/db_dependent/Holds.t
Signed-off-by: Maxime Pelletier <maxime.pelletier@libeo.com>
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Removed NoZebra vestiges. This comprises several code blocks that depend on the NoZebra syspref and NZ related functions/methods.
C4::Biblio->
GetNoZebraIndexes
_DelBiblioNoZebra
_AddBiblioNoZebra
C4::Search->
NZgetRecords
NZanalyse
NZoperatorAND
NZoperatorOR
NZoperatorNOT
NZorder
C4::Installer->
set_indexing_engine
Sponsored-by: Universidad Nacional de Córdoba
Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Remove a line of debug code from EG, provide better error handling
when presented with weird data in the authority linker, and correct
queryparser configuration to reflect the correct configuration for
Zebra.
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
This patch rewrites the GetReserveStatus routine in order to take in
parameter the itemnumber and/or the biblionumber.
In some places, the C4::Reserves::CheckReserves routine is called when
we just want to get the status of the reserve. In these cases, the
C4::Reserves::GetReserveStatus is now called.
This routine executes 1 sql query (or 2 max).
Test plan:
Check that there is no regression on the different pages where reserves
are used. The different status will be the same than before applying
this patch.
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Added a new system preference to set the UNIMARC field 100 default language.
The default value for that system preference is 'fre'.
Changed Biblio.pm to use the system preference:
- if the language is bad filled in the preferences it uses 'fre' as default value
- only replaces the language when the field 100 is empty
- if the language is filled with the plugin only replaces the positions 25-28 to 'y50'
Signed-off-by: Rolando Isodoro <rolando.isidoro@gmail.com>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
When in framework a subfield has a number > 0 in hidden, it his hidden in MARCview.
It should be hidden also in ISBD view.
Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
Does what it says, hides hidden fields on the OPAC
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
All tests and QA script pass.
To test:
1) Hide 245$b or another field from ISBD view in your MARC
framework by assigning a hidden value greater 0
2) Check the different views in OPAC, the field should be hidden now from
- Labelled MARC view (as it was before this patch)
- ISBD view
It will still show up for plain MARC and XSLT views.
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Only changing some documentation about GetXmlBiblio
Signed-off-by: Mirko Tietgen <mirko@abunchofthings.net>
Added the word 'contain'
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Added a new system preference to control the fields to not appear in the separator.
Change GetMarcNotes to use the system preference created to only appear the fields that aren't in the list,
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
FIX some indentation in C4/Biblio.pm
+FIX 1 end of parentheses in sysprefs.sql
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
- Possibility to select for line and column: items.homebranch and
items.ccode
- Possibility to filter on these fields
- Possibility to count unique biblios (count(distinct biblionumber)),
ordered amount and spent amount (based on aqorders.datereceived)
Filtering on item homebranch and ccode works only on items that were
created at ordering or receiving (ie items are linked to an order)
Some refactoring is done, mainly replacing switch-like if statements by
given/when
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
If the itemnumbers parameter is undef, perl raises an error :
"Can't use an undefined value as an ARRAY reference"
Signed-off-by: Mason James <mtj@kohaaloha.com>
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
EmbedItemsInMarcBiblio does not embed items when the itemnumbers param
is given. That breaks the export tools (used from commandline).
Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Passed-QA-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
On 3.8.x, it was possible for multiple automatically generated
authorities to be linked to a single heading. This patch deletes
previous links from headings prior to linking them to
automatically-generated headings. This patch also corrects a
potential problem wherein multiple authorities might be generated if
a record is edited repeatedly in quick succession. The latter problem
exists on Master and 3.6.x as well, and the code that corrects the
multiple linkages is equally applicable if seemingly unnecessary.
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
An eval { eval "require $module;" }; was replaced with
eval { eval { require $module; }; }; which is a no-op, meaning that
the linker was not getting loaded, and the catalog module was throwing
up a big nasty error every time someone tried to save a record with a
heading. This patch replaces the require with can_load from
Module::Load::Conditional, which is PBP-friendly, and offers equivalent
functionality.
Signed-off-by: Nicole C. Engard <nengard@bywatersolutions.com>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
- Expression form of "eval" at line 492, column 12. See page 161 of PBP. (Severity: 5)
- "return" statement with explicit "undef" at line 891, column 5. See page 199 of PBP. (Severity: 5)
- Subroutine prototypes used at line 1148, column 1. See page 194 of PBP. (Severity: 5)
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
Cherry-picked from BibLibre's work on bug 5888:
opac-detail subject/author links improvements
Added a link to opac-authoritiesdetail.pl when possible.
Only affects 'Normal view'. Does not affect XSLT display.
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
Do not automatically populate $9 in bibliographic headings when the
$9 is set in the authorized heading field of the authority record.
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
In the circulation page, you can now export (as csv or iso2709) a list
of items which are currently checked out by a borrower.
3 export types:
- iso2709 with items: Export the items list in iso2709 format with item
informations.
- iso2709 without items: Export the items list in iso2709 format without
item informations.
- CSV: Export the items list based on a csv profil.
2 new system preferences:
- DontExportFields: a list of fields not to be export
- CsvProfileForExport: The Csv profile name used for the csv export
Test plan:
- Fill the CsvProfileForExport syspref
- go on the borrower circulation page containing checkouts
- Select one or more items and export them to the 3 different formats.
- check if the result file is what you expected
- Test there is no regression with the export authority
- Test there is no regression using tools/export.pl with the command
line interface
Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
This reverts commit 215abc8024.
The 3 patches for bug 8089 have been reverted, because they break
jenkins & Koha.
A follow-up has been provided, but it does not solve the problem on my
test server, it just changes the error message.
After a discussion with jared, Dobrica should work on another patch, so
the best option is to revert.
1. Replace all instances of memoize_memcached with appropriate calls
into Koha::Cache:
* reports/guided_reports.pl
* C4::Biblio::GetMarcStructure
* C4::Languages::getFrameworkLanguages
* C4::Languages::getAllLanguages
* C4::SQLHelper::GetPrimaryKeys
* C4::SQLHelper::_get_columns
2. Replace all references to memcached with the appropriate calls into
Koha::Cache in C4::Context.
Test plan :
* have DEBUG env set to 1
* reach addbiblio page to test the patch in Biblio.pm, or setup more than 1
language
* you should see in the logs that you're reading and writing from cache
* run the test suite twice both with and without the following environment
variables set:
export MEMCACHED_SERVERS=127.0.0.1:11211
export MEMCACHED_NAMESPACE=KOHA
export CACHING_SYSTEM=memcached
Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
I'm unsure about some of the caching times 10000 is a long long time,
but other than that, works fine.
Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
New version implementing Paul's advice.
See Wiki http://wiki.koha-community.org/wiki/Age_restrictiotion
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
fix updatedatabase.pl
New fix updatedatabase.pl to apply to current master by Marc Veron veron@veron.ch
...and fixed missing curly bracket after merging updatedatabase.pl
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
export.pl [--format=format] [--date=date] [--dont_export_items]
[--deleted_barcodes] [--clean] --filename=outputfile
* format is either 'xml' or 'marc' (default)
* date should be entered as the 'dateformat' syspref is set
(dd/mm/yyyy for metric, yyyy-mm-dd for iso, mm/dd/yyyy for us)
* records exported are the ones that have been modified since 'date'
* if --deleted_barcodes is used, a list of barcodes of items deleted
since 'date' is produced (or from all deleted items if no date is
specified)
* --clean removes NSE/NSB
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
Because updating the total issues count associated with a bibliographic
record on issue could cause a significant load on the server, this
commit adds the syspref UpdateTotalIssuesOnCirc (which defaults to OFF
to match existing behavior). The syspref has the following description:
Do/Do not update a bibliographic record's total issues count whenever
an item is issued (WARNING! This increases server load significantly;
if performance is a concern, use the update_totalissues.pl cron job
to update the total issues count).
Bug 6557: automatically increment totalissues
Adds the ability to automatically increment biblioitems.totalissues
whenever an item is issued.
To test:
1) Choose a record with at least one item that can circulate
2) Check the value of 942$0 (you may need to look at the plain MARC view
on the OPAC). Most likely there won't be any 942$0 at all
3) Enable UpdateTotalIssuesOnCirc
4) Check out the item you selected
5) Check the value of 942$0 (you may need to look at the plain MARC view
on the OPAC). That value should now be one greater than before
6) Discharge the item
7) Disable UpdateTotalIssuesOnCirc
8) Check out the item you selected again
9) Check the value of 942$0 (you may need to look at the plain MARC view
on the OPAC). That value should not have changed
Bug 6557: add script to update totalissues from stats
NAME
update_totalissues.pl
SYNOPSIS
update_totalissues.pl --use-stats
update_totalissues.pl --use-items
update_totalissues.pl --commit=1000
update_totalissues.pl --since='2012-01-01'
update_totalissues.pl --interval=30d
DESCRIPTION
This batch job populates bibliographic records' total issues count
based on historical issue statistics.
--help Prints this help
-v|--verbose
Provide verbose log information (list every bib modified).
--use-stats
Use the data in the statistics table for populating total
issues.
--use-items
Use items.issues data for populating total issues. Note that
issues data from the items table does not respect the --since
or --interval options, by definition. Also note that if both
--use-stats and --use-items are specified, the count of biblios
processed will be misleading.
-s|--since=DATE
Only process issues recorded in the statistics table since
DATE.
-i|--interval=S
Only process issues recorded in the statistics table in the
last N units of time. The interval should consist of a number
with a one-letter unit suffix. The valid suffixes are h
(hours), d (days), w (weeks), m (months), and y (years). The
default unit is days.
--incremental
Add the number of issues found in the statistics table to the
existing total issues count. Intended so that this script can
be used as a cron job to update popularity information during
low-usage periods. If neither --since or --interval are
specified, incremental mode will default to processing the
last twenty-four hours.
--commit=N
Commit the results to the database after every N records are
processed.
--test Only test the popularity population script.
WARNING
If the time on your database server does not match the time on your Koha
server you will need to take that into account, and probably use the
--since argument instead of the --interval argument for incremental
updating.
=== TESTING PLAN ===
NOTE: in order to test this script, you will need to have some sort of
circulation data already existing in your Koha installation.
1) Disable UpdateTotalIssuesOnCirc
2) Run: misc/cronjobs/update_totalissues.pl --use-items -t -v
3) If you have total checkout data in your item records (i.e. anything
in 952$l), you should see messages like "Processing bib 43 (1 issues)"
4) Choose one of the lines that shows more than 0 issues, and view the
record with that biblionumber in the staff client, choosing the "Items"
tab (moredetail.pl). Add up the "Total checkouts" listed for each item,
and confirm it matches what the script reported
5) Run: misc/cronjobs/update_totalissues.pl --use-stats -t -v
6) If you have any circulation statistics in your database (i.e. any
'issue' entries in your statistics table), you should see messages
like "Processing bib 43 (1 issues)";
7) Choose one of the lines and view the record with that biblionumber in
the staff client, choosing the "Items" tab (moredetail.pl). If you
count the number of checkouts listed in each item's checkout history,
the total should match what the script reported.
8) Check out an item
9) Run: misc/cronjobs/update_totalissues.pl --use-stats
--incremental --interval=1h -t -v
10) You should see one line reporting a single circ for the bib record
associated with the item you just checked out (there may be more if
you checked out any books in the hour prior to running these tests
11) If the results in steps 4, 7, and 10 match the predictions, the
script worked
This patch to Koha was sponsored by the Arcadia Public Library and the
Arcadia Public Library Foundation in honor of Jackie Faust-Moreno, late
director of the Arcadia Public Library.
Signed-off-by: Liz Rea <wizzyrea@gmail.com>
Tested this with my test data - numbers are correct and updated appropriately.
More importantly - if I do a popularity search, the most popular items *come up first*. Amazing.
- in various acquisition pages and serials home
- in database : biblioitems.ean
- adds ean and its mapping in default english bibliographic framework
- adds ean mapping in default french bibliographic framework
- ean search is not enabled for MARC21
The required mapping between the ean marc field and the biblioitems.ean
database field will be automatically added on an existing unimarc installation.
However, if you already have records with ean, you will have to
run misc/batchRebuildBiblioTables.pl to populate biblioitems.ean
Signed-off-by: jmbroust <jean-manuel.broust@univ-lyon2.fr>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Passed QA at second run. Removed a merge marker only.
The root problem here is that the price is being pulled from the MARC record
and is then run through Number::Format::unformat_number. This routine is
really being misused, and should only be used to reverse the effects of
Number::Format on a number string. We are apparently using it to strip
out currency characters and the like.
Number::Format::unformat_number will choke if there is more than one period (.)
in the price field. MARC standards do not limit this field to a single period,
so unless there is only one period, we should skip number unformatting.
Examples of that break unformat_number include '18.95 (U.S.)', and
'$5.99 ($7.75 CAN)', both of which are perfectly valid.
This commit adds the function MungeMarcPrice that will better handle
find a real price value in a given price field. It does a very good
job at finding a price in any currency format, and attempts to find
a price in whichever currency is active before falling back to
the first valid price it can find.
The variable $price may fail to have an actual price, in which case
the price then defaults to '0.00', which would be rarely if ever the
correct price. To combat this, I have added highlighting to any
price in the Order Details table that begins with 0 ( i.e. '0.00' ).
Also, fixed the incomplete table footer, adding a new td with a
span of 3 to fill in the nonexistant cells.
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
I cannot find the root cause of this issue, but multiple libraries that I am aware of have problems searching on particular search terms ( and never the same terms at the same library ). The error they get when they trigger this problem is:
Tag "" is not a valid tag. at /home/koha/kohaclone/C4/Biblio.pm line 1849
Something somewhere is adding empty keys to C4::Context->marcfromkohafield, I think it may have something to do with the analytics feature that was added.
In the while loop for TransformKohaToMarc, there is a line
next unless my $dtm = $db_to_marc->{''}->{$name};
I don't think it's working.
If I dump $dtm, for each search, I see the dump twice.
It looks like this:
$VAR1 = [
'952',
'w'
];
$VAR1 = [];
I think the second time, when it is empty is what's breaking this.
The next never fails because even though it is empty, it is still a valid arrayref.
The solution I have some up with is to skip over the elements where the arrayref is empty.
Signed-off-by: Ian Walls <koha.sekjal@gmail.com>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
This adds the -dedupbarcode option that allows bulkmarkimport to erase
a barcode but keep the item of any items it finds with duplicate
barcodes.
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
We already remove $9 with Koha's authority number from output
of GetMarcSubjects and GetMarcAuthors.
Patch additionally removes $0 subfields with identity numbers.
Patch also effects detail pages with normal (non-XSLT) views.
Revised to always remove $0 subfields, they are not used in UNIMARC.
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
Optionally delete bibliographic record when batch deleting items, if no items remain on the record.
Adds deleting of reserves to DelBiblio. Since subscriptions are deleted automatically,
it made sense for deletion of reserves to maintain the same behavior.
Signed-off-by: Liz Rea <wizzyrea@gmail.com>
I like the way this works, and it does. Passes tests.
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
1) Removes unused subroutine get_host_control_num
2) Fixes small mistake, correct subfield for ISBN 020 is z
3) Checks system preference for correct marcflavour instead of
assuming MARC21
4) Fixes MARC21 to not use author(), because it would also add
fields like $w and $0 to 773$a
5) Fixes MARC21 to not use title(), but 245$a, because it would
also add too many subfields.
6) Adds definitions for UNIMARC and includes NORMARC
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Works properly with all supported MARC flavours.
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
Simplifies the adding of analytical records and ensures that
the data populating the 773 tag is correct. From the host record
add child record is selected and create bib is entered to generate
a new record with host item tag populated from the parent
Caveat: currently prepare_host_field only returns a field for
MARC21. Values for UNIMARC and NORMARC can easily be added but
should be done by someone familar with those formats
and conventions
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
To test:
- create a new record
--> if you enter a value in 001 anaytics will use that in $w for linking later
--> if you set 000/LDR 19 - Multipart resource record level to 'a' there will
be a link from the parent record to the child record later
- save your record and go to the staff detail page
- in toolbar select 'New' > 'New child record'
- check field 773, 245 and 001 from the parent record should have been copied there
- check links between child and parent in staff
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Fixed conflicts in all 3 files.
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Works properly for MARC21, and follow-up adds support for NORMARC and UNIMARC.
New sql tables:
- oai_sets: contains the list of sets, described by a spec and a name
- oai_sets_descriptions: contains a list of descriptions for each set
- oai_sets_mappings: conditions on marc fields to match for biblio to be
in a set
- oai_sets_biblios: list of biblionumbers for each set
New admin page: allow to configure sets:
- Creation, deletion, modification of spec, name and descriptions
- Define mappings which will be used for building oai sets
Implements OAI Sets in opac/oai.pl:
- ListSets, ListIdentifiers, ListRecords, GetRecord
New script misc/migration_tools/build_oai_sets.pl:
- Retrieve marcxml from all biblios and test if they belong to defined
sets. The oai_sets_biblios table is then updated accordingly
New system preference OAI-PMH:AutoUpdateSets. If on, update sets
automatically when a biblio is created or updated.
Use OPACBaseURL in oai_dc xslt
Squashed patch incorporating all previous patches (there is no functional
change compared to the previous version of this patch, this patch merely
squashes the original patch and follow-up, and rebases on latest master).
=== TL;DR VERSION ===
*** Installation ***
1. Run installer/data/mysql/atomicupdate/bug_7284_authority_linking_pt1
and installer/data/mysql/atomicupdate/bug_7284_authority_linking_pt2
2. Make sure you copy the following files from kohaclone to koha-dev:
etc/zeradb/authorities/etc/bib1.att,
etc/zebradb/marc_defs/marc21/authorities/authority-koha-indexdefs.xml,
etc/zebradb/marc_defs/marc21/authorities/authority-zebra-indexdefs.xsl,
etc/zebradb/marc_defs/marc21/authorities/koha-indexdefs-to-zebra.xsl, and
etc/zebradb/marc_defs/unimarc/authorities/record.abs
3. Run misc/migration_tools/rebuild_zebra.pl -a -r
*** New sysprefs ***
* AutoCreateAuthorities
* CatalogModuleRelink
* LinkerModule
* LinkerOptions
* LinkerRelink
* LinkerKeepStale
*** Important notes ***
You must have rebuild_zebra processing the zebraqueue for bibs when testing this
patch.
=== DESCRIPTION ===
*** Cataloging module ***
* Added an additional box to the authority finder plugin for "Heading match,"
which consults not just the main entry but also See-from and See-also-from
headings.
* With this patch, the automatic authority linking will actually work properly
in the cataloging module. As Owen pointed out while testing the patch,
though, longtime users of Koha will not be expecting that. In keeping with
the principles of least surprise and maximum configurability, a new syspref,
CatalogModuleRelink makes it possible to disable authority relinking in the
cataloging module only (i.e. leaving it enabled for future runs of
link_bibs_to_authorities.pl). Note that though the default behavior matches
the current behavior of Koha, it does not match the intended behavior.
Libraries that want the intended behavior rather than the current behavior
will need to adjust the CatalogModuleRelink syspref.
*** misc/link_bibs_to_authorities.pl ***
Added the following options to the misc/link_bibs_to_authorities.pl script:
--auth-limit Only process those headings that match the authorities
matching the user-specified WHERE clause.
--bib-limit Only process those bib records that match the
user-specified WHERE clause.
--commit Commit the results to the database after every N records
are processed.
--link-report Display a report of all the headings that were processed.
Converted misc/link_bibs_to_authorities.pl to use POD.
Added a detailed report of headings that linked, did not link, and linked
in a "fuzzy" fashion (the exact semantics of fuzzy are up to the individual
linker modules) during the run.
*** C4::Linker ***
Implemented new C4::Linker functionality to make it possible to easily add
custom authority linker algorithms. Currently available linker options are:
* Default: retains the current behavior of only creating links when there is
an exact match to one and only one authority record; if the 'broader_headings'
option is enabled, it will try to link to headings to authority records for
broader headings by removing subfields from the end of the heading (NOTE:
test the results before enabling broader_headings in a production system
because its usefulness is very much dependent on individual sites' authority
files)
* First Match: based on Default, creates a link to the *first* authority
record that matches a given heading, even if there is more than one
authority record that matches
* Last Match: based on Default, creates a link to the *last* authority
record that matches a given heading, even if there is more than one record
that matches
The API for linker modules is very simple. All modules should implement the
following two functions:
<get_link ($field)> - return the authid for the authority that should be
linked to the provided MARC::Field object, and a boolean to indicate whether
the match is "fuzzy" (the semantics of "fuzzy" are up to the individual plugin).
In order to handle authority limits, get_link should always end with:
return $self->SUPER::_handle_auth_limit($authid), $fuzzy;
<flip_heading ($field)> - return a MARC::Field object with the heading flipped
to the preferred form. At present this routine is not used, and can be a stub.
Made the linking functionality use the SearchAuthorities in C4::AuthoritiesMarc
rather than SimpleSearch in C4::Search. Once C4::Search has been refactored,
SearchAuthorities should be rewritten to simply call into C4::Search. However,
at this time C4::Search cannot handle authority searching. Also fixed numerous
performance issues in SearchAuthorities and the Linker script:
* Correctly destroy ZOOM recordsets in SearchAuthorities when finished. If left
undestroyed, efficiency appears to approach O(log n^n)
* Add an optional $skipmetadata flag to SearchAuthorities that can be used to
avoid additional calls into Zebra when all that is wanted are authority
records and not statistics about their use
*** New sysprefs ***
* AutoCreateAuthorities - When this and BiblioAddsAuthorities are both turned
on, automatically create authority records for headings that don't have
any authority link when cataloging. When BiblioAddsAuthorities is on and
AutoCreateAuthorities is turned off, do not automatically generate authority
records, but allow the user to enter headings that don't match an existing
authority. When BiblioAddsAuthorities is off, this has no effect.
* CatalogModuleRelink - when turned on, the automatic linker will relink
headings when a record is saved in the cataloging module when LinkerRelink
is turned on, even if the headings were manually linked to a different
authority by the cataloger. When turned off (the default), the automatic
linker will not relink any headings that have already been linked when a
record is saved.
* LinkerModule - Chooses which linker module to use for matching headings
(current options are as described above in the section on linker options:
"Default," "FirstMatch," and "LastMatch")
* LinkerOptions - A pipe-separated list of options to set for the authority
linker (at the moment, the only option available is "broader_headings," which
is described below)
* LinkerRelink - When turned on, the linker will confirm the links for headings
that have previously been linked to an authority record when it runs. When
turned off, any heading with an existing link will be ignored.
* LinkerKeepStale - When turned on, the linker will never *delete* a link to an
authority record, though, depending on the value of LinkerRelink, it may
change the link.
*** Other changes ***
* Cleaned up authorities code by removing unused functions and adding
unimplemented functions and added some unit tests.
* This patch also modifies the authority indexing to remove trailing punctuation
from Match indexes.
* Replace the old BiblioAddAuthorities subroutines with calls into the new
C4::Linker routines.
* Add a simple implementation for C4::Heading::UNIMARC. (With thanks to F.
Demians, 2011.01.09) Correct C4::Heading::UNIMARC class loading. Create
biblio tag to authority types data structure at initialization rather than
querying DB.
* Ran perltidy on all changed code.
*** Linker Options ***
Enter "broader_headings" in LinkerOptions. With this option, the linker will
try to match the following heading as follows:
=600 10$aCamins-Esakov, Jared$xCoin collections$vCatalogs$vEarly works to
1800.
First: Camins-Esakov, Jared--Coin collections--Catalogs--Early works to 1800
Next: Camins-Esakov, Jared--Coin collections--Catalogs
Next: Camins-Esakov, Jared--Coin collections
Next: Camins-Esakov, Jared (matches! if a previous attempt had matched, it
would not have tried this)
This is probably relevant only to MARC21 and LCSH, but could potentially be of
great use to libraries that make heavy use of floating subdivisions.
=== TESTING PLAN ===
Note: all of these tests require that you have some authority records,
preferably for headings that actually appear in your bibliographic data. At
least one authority record must contain a "see from" reference (remember which
one contains this, as you'll need it for some of the tests). The number shown
in the "Used in" column in the authority module is populated using Zebra
searches of the bibliographic database, so you *must* have
rebuild_zebra.pl -b -z [-x] running in cron, or manually run it after running
the linker.
*** Testing the Heading match in the cataloging plugin ***
1. Create a new record, and open the cataloging plugin for an
authority-controlled field.
2. Search for an authority by entering the "see from" term in the Heading Match
box
3. Confirm that the appropriate heading shows up
4. Search for an authority by entering the preferred heading into the Main
entry or Main entry ($a only) box (i.e., repeat the procedure you usually
use for cataloging, whatever that may be)
5. Confirm that the appropriate heading shows up
*** Testing the cataloging interface ***
6. Turn off BiblioAddsAuthorities
7. Confirm that you cannot enter text directly in an authority-controlled field
8. Confirm that if you search for a heading using the authority control plugin
the heading is inserted (note, however, that this patch does not AND IS NOT
INTENDED TO fix the bugs in the authority plugin with duplicate subfields;
those are wholly out of scope- this check is for regressions)
9. Turn on BiblioAddsAuthorities and AutoCreateAuthorities
10. Confirm that you can enter text directly into an authority-controlled field,
and if you enter a heading that doesn't currently have an authority record,
an authority record stub is automatically created, and the heading you
entered linked
11. Confirm that if you enter a heading with only a subfield $a that fully
*matches* an existing heading (i.e. the existing heading has only
subfield $a populated), the authid for that heading is inserted into
subfield $9
12. Confirm that if you enter a heading with multiple subfields that *matches*
an existing heading, the authid for that heading is inserted into
subfield $9
13. Turn on BiblioAddsAuthorities and turn off AutoCreateAuthorities
14. Confirm that you can enter text directly into an authority-controlled field,
and if you enter a heading that doesn't currently have an authority record,
an authority record stub is *not* created
15. Confirm that if you enter a heading with only a subfield $a that *matches*
an existing heading, the authid for that heading is inserted into
subfield $9
16. Confirm that if you enter a heading with multiple subfields that *matches*
an existing heading, the authid for that heading is inserted into
subfield $9
17. Create a record and link an authority record to an authorized field using
the authority plugin.
18. Save the record. Ensure that the heading is linked to the appropriate
authority.
19. Open the record. Change the heading manually to something else, leaving
the link. Save the record.
20. Ensure that the heading remains linked to that same authority.
21. Change CatalogModuleRelink to "on."
22. Open the record. Use the authority plugin to link that heading to the
same authority record you did earlier.
23. Save the record. Ensure that the heading is linked to the appropriate
authority.
24. Open the record. Change the heading manually to something else, leaving
the link. Save the record.
25. Ensure that the heading is no longer linked to the old authority record.
*** Testing link_bibs_to_authorities.pl ***
26. Set LinkerModule to "Default," turn on LinkerRelink and
BiblioAddsAuthorities, and turn AutoCreateAuthorities and
LinkerKeepStale off
27. Edit one bib record so that an authority controlled field that has already
been linked (i.e. has data in $9) has a heading that does not match any
authority record in your database
28. Run misc/link_bibs_to_authorities.pl --link-report --verbose --test (you may
want to pipe the output into less or a file, as the result is quite a lot of
information)
29. Look over the report to see if the headings that you have authority records
for report being matched, that the heading you modified in step 2 is
reported as "unlinked," and confirm that no changes were actually made to
the database (to check this, look at the bib record you edited earlier, and
check that the authid in the field you edited hasn't changed)
30. Run misc/link_bibs_to_authorities.pl --link-report --verbose (you may want
to pipe the output into less or a file, as the result is quite a lot of
information)
31. Check that the heading you modified has been unlinked
32. Change the modified heading back to whatever it was, but don't use the
authority control plugin to populate $9
33. Run misc/link_bibs_to_authorities.pl --link-report --verbose
--bib-limit="biblionumber=${BIB}" (replacing ${BIB} with the biblionumber
of the record you've been editing)
34. Confirm that the heading has been linked to the correct authority record
35. Turn LinkerKeepStale on
36. Change that heading to something else
37. Run misc/link_bibs_to_authorities.pl --link-report --verbose
--bib-limit="biblionumber=${BIB}" (replacing ${BIB} with the biblionumber
of the record you've been editing)
38. Confirm that the $9 has not changed
39. Turn LinkerKeepStale off
40. Create two authorities with the same heading
41. Run misc/migration_tools/rebuild_zebra.pl -a -z
42. Enter that heading into the bibliographic record you are working with
43. Run misc/link_bibs_to_authorities.pl --link-report --verbose
--bib-limit="biblionumber=${BIB}" (replacing ${BIB} with the biblionumber
of the record you've been editing)
44. Confirm that the heading has not been linked
45. Change LinkerModule to "FirstMatch"
46. Run misc/link_bibs_to_authorities.pl --link-report --verbose
--bib-limit="biblionumber=${BIB}" (replacing ${BIB} with the biblionumber
of the record you've been editing)
47. Confirm that the heading has been linked to the first authority record it
matches
48. Change LinkerModule to "LastMatch"
49. Run misc/link_bibs_to_authorities.pl --link-report --verbose
--bib-limit="biblionumber=${BIB}" (replacing ${BIB} with the biblionumber
of the record you've been editing)
50. Confirm that the heading has been linked to the second authority record it
matches
51. Run misc/link_bibs_to_authorities.pl --link-report --verbose
--auth-limit="authid=${AUTH}" (replacing ${AUTH} with an authid)
52. Confirm that only that heading is displayed in the report, and only those
bibs with that heading have been changed
If all those things worked, good news! You're ready to sign off on the patch
for bug 7284.
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Rebased on latest master and squashed follow-up, 16 February 2012
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Rebased on latest master, 21 February 2012
Signed-off-by: schuster <dschust1@gmail.com>
Adds a new placeholder {ISSN} to the system preference SearchForTitleIn.
For a record with multiple ISSNs only the first ISSN will be used.
Addition: Makes a small change to GetMarcControlnumber so that it checks for
NORMARC too. If you set your system preference to NORMARC, it should output
{CONTROLNUMBER} correctly now.
For testing add following code to the system preference and check output
of SearchForTitleIn for different records in your OPAC and all 3 available
views (normal, MARC and ISBD):
<li>ISSN: {ISSN}</li>
<li>ISBN: {ISBN}</li>
<li>001: {CONTROLNUMBER}</li>
Patch also includes some unit tests:
perl t/db_dependent/Biblio.t
Signed-off-by: Magnus Enger <magnus@enger.priv.no>
Tested with marcflavour = NORMARC, on one book and one periodical record.
* Book
- Before the patch:
ISSN: {ISSN}
ISBN: 0375726446
001:
- After the patch:
ISSN:
ISBN: 0375726446
001: 022976914
* Journal
- Before the patch:
ISSN: {ISSN}
ISBN:
001:
- After the patch:
ISSN: 1890-6931
ISBN:
001: 080721370
Looks good in all 3 views! Thanks for fixing the 001 thing for NORMARC!
Also tested with marcflavour = MARC21, on the same records with the same good
results. Signing off!
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
Tested marcflavour= UNIMARC, works fine too
the sub _find_value is used only in PrepareItemRecord sub, that has been moved to Items package
This patch moves the _find_value in Items as well.
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Editing an already arrived serial issue with an attached item
resulted in an error. After applying the patch it's fixed.
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Confirmed that memcached is still being used after the memcached configuration
in koha-conf.xml was removed, and the following two lines were added to
both virtual hosts in koha-httpd.conf:
SetEnv MEMCACHED_SERVERS "127.0.0.1:11211"
SetEnv MEMCACHED_NAMESPACE "KOHA"
C4::Biblio is used in many many places. The goal of this cleaning is to do from C4::Biblio a package with as many dependancies as possible.
* C4::Heading is called only in 1 place, highly rarely used (only in 1 misc/link_bibs_to_authorities.pl), moving to require
* PrepareItemrecordDisplay is a sub that is more related to Items, moving it here. It means some scripts that used this sub must be checked against use C4::Items
* C4::Items is needed in EmbedItemsInMarcBiblio, moving it only in this sub, and switching to require
* 2 subs are totally useless z3950_extended_services and set_service_options, removing them
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
No test plan included, but tested some opac and cataloging functions.
Code looks good. Marked as Passed QA.
All subfields following the removed subfield were not saved.
Problem is in C4/Biblio routine TransformHtmlToMarc.
If the field is emptied, the param list contains a code param but no subfield
param. The while loop handling the subfields could not handle that. Also added
a FIXME because the whole routine depends on an assumption about the order of
cgi parameters that is not strictly guaranteed.
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
I was unable to replicate the problem, but can confirm that the patch does
not break anything under any of the following platforms/browsers:
Mac OS X 10.6.8:
Chrome 16.0.912.77
Firefox 9.0.1
Windows 7:
Firefox 3.6.3
Firefox 9.0.1
IE 8.0.7600.16385
Ubuntu 11.10
Firefox 8.0
Chromium 15.0.874.106 (Developer Build 107270 Linux)
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
