Introduce the new preference and remove dontmerge.
If dontmerge was enabled, the new pref will be set to zero and we will
print a message about raising this value now.
Test plan:
Run the database revision or a new install.
In case of an upgrade, check the new pref value. Should be 0 when
dontmerge was enabled, otherwise 50.
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Also tested sysprefs.sql on a new database.
Signed-off-by: Marc Véron <veron@veron.ch>
Signed-off-by: Jacek Ablewicz <abl@biblos.pk.edu.pl>
Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Original patch from bug 5572, dating back to 2011!
Amended by Marcel de Rooy January 2017. Renamed the pref.
The fix on this report is based on this preference.
Depending on the pref, subfields will be deleted or kept.
Test plan:
Run the dbrev.
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
This patch rephrases the description or examples of 5 sysprefs:
1/ MARCAuthorityControlField008: "MARC" -> "MARC21"
2/ itemcallnumber: "Examples" -> "Examples (for MARC21 records)"
3/ DefaultLanguageField008: "Range 35-37" -> "Range 35-37 of MARC21 records"
4/ MARCOrgCode: "new MARC records" -> "new MARC21 records"
5/ UNIMARCAuthorityField100 description: "Do NOT include the date
(position 00-05)." -> "position 08-35. Do NOT include the date
(position 00-07)."
It also adds description in SQL systempreferences table for
UNIMARCAuthorityField100, MARCAuthorityControlField008 and MARCOrgCode.
Test plan:
- Apply and run updatedatabase.pl
- Check the changes are taken into account in syspref administration
page
- Check the changes are taken into account in systempreferences table
(for UNIMARCAuthorityField100, MARCAuthorityControlField008 and
MARCOrgCode)
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
New Authority system preference created UNIMARCAuthorityField100 with default value "afrey50 ba0".
AuthoritiesMarc.pm uses the system preference instead of label "afrey50 ba0".
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Works as described, all tests and QA script pass.
1) Check system preference is created correctly.
2) Change pref.
3) Catalog a new authority.
4) Change framework to make 100 show in frameworks or check in the
the database, that now 100 has the new defined value.
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
This commit adds support for displaying authority hierarchies for all
flavours of MARC, not just UNIMARC. Display now uses the jQuery
jstree plugin, selected with the help of Owen Leonard, resulting in a
much faster experience for users.
Be aware that the jstree file uses tabs rather than 4-space indentation,
which I left as-is so as to make it easier to integrate upstream
releases in the future.
To test:
1) Enable the AuthDisplayHierarchy syspref
2) Create authority records with a hierarchy of see also fields
(in MARC21/NORMARC, you'll be using 5xx fields for this, with a
subfield $w=g for broader terms and subfield $w=h for narrower
terms)
3) View the authorities in the OPAC, noting the hierarchical view at
the top of the page.
This initial patch does not create bidirection linkages from
unidirectional links in MARC21 authorities. This means that when moving
up the authority hierarchy, lower levels will disappear. This is
intentional, as the first patch is intended merely to ensure that
AuthDisplayHierarchy functions the same for all marcflavours. A future
patch will add a cron job to generate the bidirectional linkages, once
we are sure that the hierarchy functionality for UNIMARC and
MARC21/NORMARC coexists peaceably.
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Resolved conflicts in updatedatabase.pl, sysprefs.sql and in one of
the CSS files.
Test plan:
1) Run t/AuthoritiesMarc.t
New tests complete without any errors.
2) Make sure updatedatabase works correctly.
Update works nicely, new system preference is also added to syspref.sql
3) Make sure new terms are translatable.
Created new po files for de-DE and checked for new terms.
All translations appear correctly.
4) Make sure everything works with AuthDisplayHieararchy OFF
- Add authority
- Edit authority
- Delete authority
5) Test feature with AuthDisplayHieararchy ON
- Add authority
- Edit authority
- Delete authority
6) Add a couple of hierarchically linked authorities
Note: links have to be created in both directions
Example:
151 $aGermany
551 $a Baden-Württemberg $w h
151 $aBaden-Württemberg
551 $a Konstanz $w h
551 $a Germany $w g
151 $aKonstanz
551 $a Baden-Württemberg $w g
551 $a Fürstenberg $w h
551 $a Paradies $w h
151 $a Fürstenberg
551 $a Konstanz $w g
151 $a Paradies
551 $a Konstanz $w g
Tree shows up nicely above the authority record
- in staff
- in OPAC
- on the normal view tab
- on the MARC view tab
7) Checking the logs for warnings
- no Javascript errors or warnings
- no warnings or errors in log files
Squashed patch incorporating all previous patches (there is no functional
change compared to the previous version of this patch, this patch merely
squashes the original patch and follow-up, and rebases on latest master).
=== TL;DR VERSION ===
*** Installation ***
1. Run installer/data/mysql/atomicupdate/bug_7284_authority_linking_pt1
and installer/data/mysql/atomicupdate/bug_7284_authority_linking_pt2
2. Make sure you copy the following files from kohaclone to koha-dev:
etc/zeradb/authorities/etc/bib1.att,
etc/zebradb/marc_defs/marc21/authorities/authority-koha-indexdefs.xml,
etc/zebradb/marc_defs/marc21/authorities/authority-zebra-indexdefs.xsl,
etc/zebradb/marc_defs/marc21/authorities/koha-indexdefs-to-zebra.xsl, and
etc/zebradb/marc_defs/unimarc/authorities/record.abs
3. Run misc/migration_tools/rebuild_zebra.pl -a -r
*** New sysprefs ***
* AutoCreateAuthorities
* CatalogModuleRelink
* LinkerModule
* LinkerOptions
* LinkerRelink
* LinkerKeepStale
*** Important notes ***
You must have rebuild_zebra processing the zebraqueue for bibs when testing this
patch.
=== DESCRIPTION ===
*** Cataloging module ***
* Added an additional box to the authority finder plugin for "Heading match,"
which consults not just the main entry but also See-from and See-also-from
headings.
* With this patch, the automatic authority linking will actually work properly
in the cataloging module. As Owen pointed out while testing the patch,
though, longtime users of Koha will not be expecting that. In keeping with
the principles of least surprise and maximum configurability, a new syspref,
CatalogModuleRelink makes it possible to disable authority relinking in the
cataloging module only (i.e. leaving it enabled for future runs of
link_bibs_to_authorities.pl). Note that though the default behavior matches
the current behavior of Koha, it does not match the intended behavior.
Libraries that want the intended behavior rather than the current behavior
will need to adjust the CatalogModuleRelink syspref.
*** misc/link_bibs_to_authorities.pl ***
Added the following options to the misc/link_bibs_to_authorities.pl script:
--auth-limit Only process those headings that match the authorities
matching the user-specified WHERE clause.
--bib-limit Only process those bib records that match the
user-specified WHERE clause.
--commit Commit the results to the database after every N records
are processed.
--link-report Display a report of all the headings that were processed.
Converted misc/link_bibs_to_authorities.pl to use POD.
Added a detailed report of headings that linked, did not link, and linked
in a "fuzzy" fashion (the exact semantics of fuzzy are up to the individual
linker modules) during the run.
*** C4::Linker ***
Implemented new C4::Linker functionality to make it possible to easily add
custom authority linker algorithms. Currently available linker options are:
* Default: retains the current behavior of only creating links when there is
an exact match to one and only one authority record; if the 'broader_headings'
option is enabled, it will try to link to headings to authority records for
broader headings by removing subfields from the end of the heading (NOTE:
test the results before enabling broader_headings in a production system
because its usefulness is very much dependent on individual sites' authority
files)
* First Match: based on Default, creates a link to the *first* authority
record that matches a given heading, even if there is more than one
authority record that matches
* Last Match: based on Default, creates a link to the *last* authority
record that matches a given heading, even if there is more than one record
that matches
The API for linker modules is very simple. All modules should implement the
following two functions:
<get_link ($field)> - return the authid for the authority that should be
linked to the provided MARC::Field object, and a boolean to indicate whether
the match is "fuzzy" (the semantics of "fuzzy" are up to the individual plugin).
In order to handle authority limits, get_link should always end with:
return $self->SUPER::_handle_auth_limit($authid), $fuzzy;
<flip_heading ($field)> - return a MARC::Field object with the heading flipped
to the preferred form. At present this routine is not used, and can be a stub.
Made the linking functionality use the SearchAuthorities in C4::AuthoritiesMarc
rather than SimpleSearch in C4::Search. Once C4::Search has been refactored,
SearchAuthorities should be rewritten to simply call into C4::Search. However,
at this time C4::Search cannot handle authority searching. Also fixed numerous
performance issues in SearchAuthorities and the Linker script:
* Correctly destroy ZOOM recordsets in SearchAuthorities when finished. If left
undestroyed, efficiency appears to approach O(log n^n)
* Add an optional $skipmetadata flag to SearchAuthorities that can be used to
avoid additional calls into Zebra when all that is wanted are authority
records and not statistics about their use
*** New sysprefs ***
* AutoCreateAuthorities - When this and BiblioAddsAuthorities are both turned
on, automatically create authority records for headings that don't have
any authority link when cataloging. When BiblioAddsAuthorities is on and
AutoCreateAuthorities is turned off, do not automatically generate authority
records, but allow the user to enter headings that don't match an existing
authority. When BiblioAddsAuthorities is off, this has no effect.
* CatalogModuleRelink - when turned on, the automatic linker will relink
headings when a record is saved in the cataloging module when LinkerRelink
is turned on, even if the headings were manually linked to a different
authority by the cataloger. When turned off (the default), the automatic
linker will not relink any headings that have already been linked when a
record is saved.
* LinkerModule - Chooses which linker module to use for matching headings
(current options are as described above in the section on linker options:
"Default," "FirstMatch," and "LastMatch")
* LinkerOptions - A pipe-separated list of options to set for the authority
linker (at the moment, the only option available is "broader_headings," which
is described below)
* LinkerRelink - When turned on, the linker will confirm the links for headings
that have previously been linked to an authority record when it runs. When
turned off, any heading with an existing link will be ignored.
* LinkerKeepStale - When turned on, the linker will never *delete* a link to an
authority record, though, depending on the value of LinkerRelink, it may
change the link.
*** Other changes ***
* Cleaned up authorities code by removing unused functions and adding
unimplemented functions and added some unit tests.
* This patch also modifies the authority indexing to remove trailing punctuation
from Match indexes.
* Replace the old BiblioAddAuthorities subroutines with calls into the new
C4::Linker routines.
* Add a simple implementation for C4::Heading::UNIMARC. (With thanks to F.
Demians, 2011.01.09) Correct C4::Heading::UNIMARC class loading. Create
biblio tag to authority types data structure at initialization rather than
querying DB.
* Ran perltidy on all changed code.
*** Linker Options ***
Enter "broader_headings" in LinkerOptions. With this option, the linker will
try to match the following heading as follows:
=600 10$aCamins-Esakov, Jared$xCoin collections$vCatalogs$vEarly works to
1800.
First: Camins-Esakov, Jared--Coin collections--Catalogs--Early works to 1800
Next: Camins-Esakov, Jared--Coin collections--Catalogs
Next: Camins-Esakov, Jared--Coin collections
Next: Camins-Esakov, Jared (matches! if a previous attempt had matched, it
would not have tried this)
This is probably relevant only to MARC21 and LCSH, but could potentially be of
great use to libraries that make heavy use of floating subdivisions.
=== TESTING PLAN ===
Note: all of these tests require that you have some authority records,
preferably for headings that actually appear in your bibliographic data. At
least one authority record must contain a "see from" reference (remember which
one contains this, as you'll need it for some of the tests). The number shown
in the "Used in" column in the authority module is populated using Zebra
searches of the bibliographic database, so you *must* have
rebuild_zebra.pl -b -z [-x] running in cron, or manually run it after running
the linker.
*** Testing the Heading match in the cataloging plugin ***
1. Create a new record, and open the cataloging plugin for an
authority-controlled field.
2. Search for an authority by entering the "see from" term in the Heading Match
box
3. Confirm that the appropriate heading shows up
4. Search for an authority by entering the preferred heading into the Main
entry or Main entry ($a only) box (i.e., repeat the procedure you usually
use for cataloging, whatever that may be)
5. Confirm that the appropriate heading shows up
*** Testing the cataloging interface ***
6. Turn off BiblioAddsAuthorities
7. Confirm that you cannot enter text directly in an authority-controlled field
8. Confirm that if you search for a heading using the authority control plugin
the heading is inserted (note, however, that this patch does not AND IS NOT
INTENDED TO fix the bugs in the authority plugin with duplicate subfields;
those are wholly out of scope- this check is for regressions)
9. Turn on BiblioAddsAuthorities and AutoCreateAuthorities
10. Confirm that you can enter text directly into an authority-controlled field,
and if you enter a heading that doesn't currently have an authority record,
an authority record stub is automatically created, and the heading you
entered linked
11. Confirm that if you enter a heading with only a subfield $a that fully
*matches* an existing heading (i.e. the existing heading has only
subfield $a populated), the authid for that heading is inserted into
subfield $9
12. Confirm that if you enter a heading with multiple subfields that *matches*
an existing heading, the authid for that heading is inserted into
subfield $9
13. Turn on BiblioAddsAuthorities and turn off AutoCreateAuthorities
14. Confirm that you can enter text directly into an authority-controlled field,
and if you enter a heading that doesn't currently have an authority record,
an authority record stub is *not* created
15. Confirm that if you enter a heading with only a subfield $a that *matches*
an existing heading, the authid for that heading is inserted into
subfield $9
16. Confirm that if you enter a heading with multiple subfields that *matches*
an existing heading, the authid for that heading is inserted into
subfield $9
17. Create a record and link an authority record to an authorized field using
the authority plugin.
18. Save the record. Ensure that the heading is linked to the appropriate
authority.
19. Open the record. Change the heading manually to something else, leaving
the link. Save the record.
20. Ensure that the heading remains linked to that same authority.
21. Change CatalogModuleRelink to "on."
22. Open the record. Use the authority plugin to link that heading to the
same authority record you did earlier.
23. Save the record. Ensure that the heading is linked to the appropriate
authority.
24. Open the record. Change the heading manually to something else, leaving
the link. Save the record.
25. Ensure that the heading is no longer linked to the old authority record.
*** Testing link_bibs_to_authorities.pl ***
26. Set LinkerModule to "Default," turn on LinkerRelink and
BiblioAddsAuthorities, and turn AutoCreateAuthorities and
LinkerKeepStale off
27. Edit one bib record so that an authority controlled field that has already
been linked (i.e. has data in $9) has a heading that does not match any
authority record in your database
28. Run misc/link_bibs_to_authorities.pl --link-report --verbose --test (you may
want to pipe the output into less or a file, as the result is quite a lot of
information)
29. Look over the report to see if the headings that you have authority records
for report being matched, that the heading you modified in step 2 is
reported as "unlinked," and confirm that no changes were actually made to
the database (to check this, look at the bib record you edited earlier, and
check that the authid in the field you edited hasn't changed)
30. Run misc/link_bibs_to_authorities.pl --link-report --verbose (you may want
to pipe the output into less or a file, as the result is quite a lot of
information)
31. Check that the heading you modified has been unlinked
32. Change the modified heading back to whatever it was, but don't use the
authority control plugin to populate $9
33. Run misc/link_bibs_to_authorities.pl --link-report --verbose
--bib-limit="biblionumber=${BIB}" (replacing ${BIB} with the biblionumber
of the record you've been editing)
34. Confirm that the heading has been linked to the correct authority record
35. Turn LinkerKeepStale on
36. Change that heading to something else
37. Run misc/link_bibs_to_authorities.pl --link-report --verbose
--bib-limit="biblionumber=${BIB}" (replacing ${BIB} with the biblionumber
of the record you've been editing)
38. Confirm that the $9 has not changed
39. Turn LinkerKeepStale off
40. Create two authorities with the same heading
41. Run misc/migration_tools/rebuild_zebra.pl -a -z
42. Enter that heading into the bibliographic record you are working with
43. Run misc/link_bibs_to_authorities.pl --link-report --verbose
--bib-limit="biblionumber=${BIB}" (replacing ${BIB} with the biblionumber
of the record you've been editing)
44. Confirm that the heading has not been linked
45. Change LinkerModule to "FirstMatch"
46. Run misc/link_bibs_to_authorities.pl --link-report --verbose
--bib-limit="biblionumber=${BIB}" (replacing ${BIB} with the biblionumber
of the record you've been editing)
47. Confirm that the heading has been linked to the first authority record it
matches
48. Change LinkerModule to "LastMatch"
49. Run misc/link_bibs_to_authorities.pl --link-report --verbose
--bib-limit="biblionumber=${BIB}" (replacing ${BIB} with the biblionumber
of the record you've been editing)
50. Confirm that the heading has been linked to the second authority record it
matches
51. Run misc/link_bibs_to_authorities.pl --link-report --verbose
--auth-limit="authid=${AUTH}" (replacing ${AUTH} with an authid)
52. Confirm that only that heading is displayed in the report, and only those
bibs with that heading have been changed
If all those things worked, good news! You're ready to sign off on the patch
for bug 7284.
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Rebased on latest master and squashed follow-up, 16 February 2012
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Rebased on latest master, 21 February 2012
Signed-off-by: schuster <dschust1@gmail.com>
Signed-off-by: Nicole C. Engard <nengard@bywatersolutions.com>
Signed-off-by: MJ Ray <mjr@phonecoop.coop>
Signed-off-by: Ian Walls <ian.walls@bywatersolutions.com>
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Currently, the default for this MARC control field is hardcoded in the plugin code of marc21_field_008_authorities.pl (two occurrences).
This patch moves it into a new syspref. If this pref should not exist, the old hardcoded value is used. Authorities preference file is updated too.
A warning is generated if the length of the field is not correct.
Furthermore, the default for position 14 (heading use: main or added entry)
should be dependent on the authority type. For CHRON_TERM, GENRE/FORM and
TOPIC_TERM, it should be b (not appropriate) instead of a (appropriate).
Rebased on March 17. Follow-up patch added for installation pref.
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Signed-off-by: Ian Walls <ian.walls@bywatersolutions.com>
This patch adds the syspref TraceCompleteSubfields. When TraceCompleteSubfields
is set to "force," clicking on links in non-authority controlled subject
tracings will only find other records where the entire subfields match. Leaving
it at "don't force" keeps the current behavior of doing a keyword search of the
subject indexes.
This patch implements complete-subfield subject tracings in MARC21 XSLTs (OPAC
and Staff) and all Normal mode interfaces. UNIMARC XSLTs have not been updated.
This patch also adds the syspref UseAuthoritiesForTracings. When set to "Do not"
tracings in the XSLT detail displays will never generate links using authority
numbers, but rather use the heading strings (NB: the tracings currently use only
subfield 'a' for generating links).
Signed-off-by: Nicole C. Engard <nengard@bywatersolutions.com>
Signed-off-by: Jane Wagner <jwagner@ptfs.com>
Signed-off-by: Jared Camins-Esakov <jcamins@bywatersolutions.com>
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Mathieu Saby
Vitor FERNANDES
Jared Camins-Esakov
Nicole C. Engard
Jared Camins-Esakov
Jesse Weaver
