This patch (which is incidental to the main patches for bug 9282),
improves the AJAX authority search code by fixing incorrect contruction
of the parameters to SearchAuthorities() that led to errors like this
in the Apache log when doing auto-completion searches on "main entry"
and "anywhere" in the authority finder:
ysearch.pl: Use of uninitialized value $i in string eq ...
In the process, this patch also removes a use of the now-deprecated
Perl smartmatch operator.
To test:
[1] Verify that the main test plan for bug 9282 still works.
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
This patch adjusts the auto-completion on the authority record
finder (accessed from the bib editor) so that if you do
start typing in the "Main entry ($a only)" input field, it will
return only the $a of the main heading for matching authority
records.
This fixes a problem where typing "shakes", choosing
"Shakespeare, William, 1564-1616" from the auto-completion
result list, then hitting the search button fails to bring
up results, as the dates come from the $d of the 100 field
(in MARC21).
Signed-off-by: Mathieu Saby <mathieu.saby@univ-rennes2.fr>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Works as advertised.
Tested with an authority where I added my search term in $b.
The modified authority came up in main entry, not in mainmainentry.
That was the desired result.
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
The link on the merge reference had been wrong, and framework types
were not indicated when you were merging records of two types. This
patch fixes those problems.
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Passes all tests and QA script.
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
$mergereference is not always defined, so we should check it is
defined before checking its value.
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
When rebasing the patch for this, I accidentally moved the line
setting the mergereference when merging from breeding to *before*
I checked whether we were merging from breeding. Needless to say,
this caused problems. This patch fixes that, and:
* Makes the error translatable.
* Adds a missing authtypesloop argument for the template.
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Due to massively incorrect data in the default authority frameworks,
we were getting warnings about undefined values and spaces from checking
which tab subfields/fields were displayed in. This patch eliminates those
warnings.
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
This patch gives Koha the ability to merge authority records using the
same interface used by bibliographic records, though slightly different
methods for selecting which records to merge. The two ways to select
records are as follows:
1) Records can be selected from authority search results by clicking
the "Merge" link for two records.
2) Authority records can be merged from the reservoir by clicking the
merge-related links in the Manage staged MARC batch screen.
To test:
1) Apply patch.
2) Do a search for an authority record that will turn up multiple
identical records (or at least two records that you don't mind
merging).
3) Click the "Merge" link for the first record.
4) Click the "Merge" link for the second record.
5) Choose which fields from which record you want to appear in the
resulting record.
6) Confirm that those are the fields that exist in the resulting record.
7) Stage an authority record (for example, an authority record you
saved from your catalog.
8) Search for a record to merge with it using the "Search for a record
to merge in a new window" link.
9) Merge these records, confirming that the resulting record (after
going through the entire merging process) matches your expectations.
10) Set up a matching rule for authorities, and export an authority from
your catalog that will match based on that rule. For MARC21, the
following is a good choice for a rule:
Matching rule code: AUTHPER
Description: Personal name main entry
Match threshold: 999
Record type: Authority record
[Match point 1:]
Search index: mainmainentry
Score: 1000
Tag: 100
Subfields: a
11) Stage the record you just exported, choosing the matching rule you
just created.
12) Merge the record using the "Merge" link, confirming that the
resulting record (after going through the entire merging process)
matches your expectations.
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Testing notes on last patch in series.
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
This patch introduces a new Z39.50 interface for searching Z39.50
compliant databases for MARC authority records.
These databases aren't as common as their bibliographic equivalents,
but they're out there and very useful. I have included info at the
bottom of this messsage for sample authority databases you can try.
To test this patch:
1) Set up Z39.50 client targets for authority databases. (I've included
information at the bottom of this message for LibrariesAustralia's
test server for authorities as well as instructions on how to use
your Koha's z39.50 authority server as well. The Library of Congress
also has authority databases available (unsure if these are test or
prod), and you might have access to others through OCLC or RLIN. OCLC
provides login credentials for their test databases.
2) Go to the Authorities module
3) Click on the new "Z39.50 search button"
4) Select your authority search targets from the list.
5) Do a search for an authority you would like using either the "Raw"
input box or the more specific input boxes for names, subjects, subject
sub divisions, or titles. (I like searching Name (personal): Eric on
the LibrariesAustralia test DB.)
6) You should see a table listing the server, heading, authority type,
and two other columns (MARC and a nameless column). "Authority type"
is the type of authority it will become when imported in to Koha. In
the Eric example, "PERSO_NAME".
7) Click on "MARC" next to the results of interest to review the MARC
authority record.
8) When you're satisfied with a record, click on "Import".
9) The pop-up window will close and your original Koha window will
change to the "Adding authority Personal Name" screen (in the Eric
example).
10) All the relevant fields should be filled out for the record. Review
them and make any changes as necessary. (N.B. The 001 will be cleared
when saved, so if you have a use for the imported control number, move
it to the 010, 016, or 035 as appropriate. If you have a default value
for the 003, this will also likely be overwritten. Move it if necessary.
The 005 will also be updated when saved, so do not worry about that.)
11) When you're satisfied, click save.
12) Presto! You've imported your first authority record via Z39.50!
--
Here is the info for the LibrariesAustralia test Z39.50 authority
database:
Z39.50 server: LibrariesAustralia Authorities
Hostname: z3950-test.librariesaustralia.nla.gov.au
Port: 210
Database: AuthTraining
Userid: ANLEZ
Password: z39.50
Syntax: MARC21/USMARC
Encoding: utf8
-
The U.S.A. Library of Congress also provides Z39.50 access to its Name
and Subject Authorities (http://www.loc.gov/z3950/lcserver.html).
Name Authority:
Z39.50 server: Library of Congress Name Authority File
Hostname: lx2.loc.gov
Port: 210
Database: NAF
Syntax: MARC21/USMARC
Encoding: utf8
Subject Authority:
Z39.50 server: Library of Congress Subject Authority File
Hostname: lx2.loc.gov
Port: 210
Database: SAF
Syntax: MARC21/USMARC
Encoding: utf8
(N.B. Both of these databases also include title authorities.)
-
For testing purposes, you can also set up a Z39.50 client target,
which points at your own Koha instance's Z39.50 authority server.
To find the hostname, go to /etc/koha-conf.xml and find the value for
the <listen id="authorityserver"> element. Depending on your
configuration, this could be something like the following:
unix:/zebra/koha/var/run/zebradb/authoritysocket
(N.B. You might be using a different scheme than unix sockets...)
To find the database, scroll down to the bottom of koha-conf.xml until
you reach the <config> element. Within this, look for the value of the
element <authorityserver>. It should probably be "authorities".
To set up this Z39.50 client target in Koha...
Z39.50 server: my koha authorities
Hostname: unix:/zebra/koha/var/run/zebradb/authoritysocket
Port:
Database: authorities
Userid:
Password:
Syntax: MARC21/USMARC (or whichever flavour you need)
Encoding: utf8
Signed-off-by: Mason James <mtj@kohaaloha.com>
Bug 10096 [FOLLOW-UP] - Add a z39.50 interface for authority searching
This patch adds the "recordtype" column to the "z3950servers" table.
The value in this column (biblio or authority) then controls whether
the z3950 server shows up in a bibliographic search (through the
Acq and Cataloguing modules) or in an authority search (through
the Authorities module).
I also edited the z3950 management console to show this value
and allow users to edit it. The default value is "biblio", since
the vast majority of z3950 targets will be bibliographic. However,
there is an option to add/edit a z3950 target as a source of
authority records.
Test Plan:
1) Apply both patches
2) Run updatedatabase.pl (after setting your KOHA_CONF and PERL5
environmental variables)
3) Use the test plan from the 1st patch
N.B. Make sure that your Z39.50 client target has a Record Type
of Authority, otherwise it won't display when you're doing a
Z3950 search for authorities.
Signed-off-by: Mason James <mtj@kohaaloha.com>
Bug 10096 [FOLLOW-UP] - fix tabs/whitespace errors to pass QA
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
It should be possible for staff client users to search and view
authority records even if they don't have permission to edit them.
To test, apply the patch and view the staff client as a user with and
without "editauthorities" permission.
With "editauthorities" permission, authority search results should show
both an edit and delete link. Viewing the details of an authority
record, one should see a toolbar with edit/delete/new options.
The detail view has been altered to use the term "record" in place of
"biblio" ("Used in X record(s)).
Without "editauthorities" permission, authority search results should
show no edit or delete link. Viewing the details of an authority, the
only option shown in the toolbar should be "Save."
On the staff client home page and in the header's "More" menu the link
to the authorities module should now appear with and without permission
to edit authorities.
This patch also corrects the permissions in the authority export script
to allow saving of authority records by users who do not have permission
to edit.
Signed-off-by: Paola Rossi <paola.rossi@cineca.it>
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
With this patch Koha should correctly copy indicators
(and create $2 subfield in MARC 21 if need) from the chosen authority
record to the edited bibliographic record (according to discussion in
bugzilla). UNIMARC and MARC 21 flavors are covered.
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Comment: work as described, testing in comments 9 and 12.
No errors.
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Passes QA script and tests. Tested functionality repeating
some of the tests noted by Bernardo - checking mostly 1xx, 490,
and 7xx.
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
In current implementation (mostly commented out in this patch)
uses heuristic to guess which strings need decoding from utf-8
to binary representation and doesn't support utf-8 characters
in templates and has problems with utf-8 data from database.
With this changes, Koha perl code always uses utf-8 encoding
correctly. All incomming data from database is allready
correctly marked as utf-8, and decoding of utf8 is required
only from Zebra and XSLT transfers which don't set utf-8 flag
correctly.
For output, standard perl :encoding(utf8) handler is used
so it also removes various "wide character" warnings as side-effect.
Test scenario:
1. make sure that you have utf-8 characters in your biblio
records, patrons, categories etc.
2. try to search records on intranet and opac which contain
utf-8 characters
3. install language which has utf-8 characters, e.g. uk-UA
dpavlin@koha-dev:/srv/koha/misc/translator(bug_6554) $
PERL5LIB=/srv/koha/ perl translate install uk-UA
4. switch language to uk-UA and verify that templates
display correctly
5. test search and Z39.50 search and verify that caracters
are correct
Signed-off-by: Owen Leonard <oleonard@myacpl.org>
I followed the test plan, adding utf-8 characters to library names,
patron categories, titles, and authorized values. I tried the uk-UA
translation and everything looked good.
When performing Z39.50 searches for titles containing utf-8 characters I
got results which were still occasionally contaminated with dummy
characters [?] but I assume this is Z39.50's fault not the patch's.
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Already signed, add mine.
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
New Authority system preference created UNIMARCAuthorityField100 with default value "afrey50 ba0".
AuthoritiesMarc.pm uses the system preference instead of label "afrey50 ba0".
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Works as described, all tests and QA script pass.
1) Check system preference is created correctly.
2) Change pref.
3) Catalog a new authority.
4) Change framework to make 100 show in frameworks or check in the
the database, that now 100 has the new defined value.
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Passed-QA-by: Mason James <mtj@kohaaloha.com>
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Passed-QA-by: Mason James <mtj@kohaaloha.com>
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Although fields in the authority editor should support thesaurus
control, they should never be locked, as authority records commonly
refer to other records that may not be in use locally (and therefore
might not be in the authority file).
Test plan:
1) Make sure a 5xx field in one of your authority frameworks is
thesaurus-controlled.
2) Note that you cannot edit the field directly.
3) Apply patch.
4) Note that you can now edit the field irectly.
Signed-off-by: wajasu <matted-34813@mypacks.net>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
This commit adds support for displaying authority hierarchies for all
flavours of MARC, not just UNIMARC. Display now uses the jQuery
jstree plugin, selected with the help of Owen Leonard, resulting in a
much faster experience for users.
Be aware that the jstree file uses tabs rather than 4-space indentation,
which I left as-is so as to make it easier to integrate upstream
releases in the future.
To test:
1) Enable the AuthDisplayHierarchy syspref
2) Create authority records with a hierarchy of see also fields
(in MARC21/NORMARC, you'll be using 5xx fields for this, with a
subfield $w=g for broader terms and subfield $w=h for narrower
terms)
3) View the authorities in the OPAC, noting the hierarchical view at
the top of the page.
This initial patch does not create bidirection linkages from
unidirectional links in MARC21 authorities. This means that when moving
up the authority hierarchy, lower levels will disappear. This is
intentional, as the first patch is intended merely to ensure that
AuthDisplayHierarchy functions the same for all marcflavours. A future
patch will add a cron job to generate the bidirectional linkages, once
we are sure that the hierarchy functionality for UNIMARC and
MARC21/NORMARC coexists peaceably.
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Resolved conflicts in updatedatabase.pl, sysprefs.sql and in one of
the CSS files.
Test plan:
1) Run t/AuthoritiesMarc.t
New tests complete without any errors.
2) Make sure updatedatabase works correctly.
Update works nicely, new system preference is also added to syspref.sql
3) Make sure new terms are translatable.
Created new po files for de-DE and checked for new terms.
All translations appear correctly.
4) Make sure everything works with AuthDisplayHieararchy OFF
- Add authority
- Edit authority
- Delete authority
5) Test feature with AuthDisplayHieararchy ON
- Add authority
- Edit authority
- Delete authority
6) Add a couple of hierarchically linked authorities
Note: links have to be created in both directions
Example:
151 $aGermany
551 $a Baden-Württemberg $w h
151 $aBaden-Württemberg
551 $a Konstanz $w h
551 $a Germany $w g
151 $aKonstanz
551 $a Baden-Württemberg $w g
551 $a Fürstenberg $w h
551 $a Paradies $w h
151 $a Fürstenberg
551 $a Konstanz $w g
151 $a Paradies
551 $a Konstanz $w g
Tree shows up nicely above the authority record
- in staff
- in OPAC
- on the normal view tab
- on the MARC view tab
7) Checking the logs for warnings
- no Javascript errors or warnings
- no warnings or errors in log files
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
The correct tab is now visible on load. The problem with authtypecode
showing up in the search box is specific to UNIMARC, so I could not
check that it was gone, but I am comfortable signing off on this.
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
We didn't notice this one with jcamins when we fixed this bug.
Sponsored-by: Universidad Nacional de Córdoba
Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
Do not automatically populate $9 in bibliographic headings when the
$9 is set in the authorized heading field of the authority record.
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
In addition to the work in bug 8207 that enables auth_finder use, it
would be very useful when creating authorities to have the auth_finder
plugin automatically fill out the relationship information in $w (in
MARC21).
To test (note that you must either apply the patch for bug 8207 or
manually add a thesaurus to a 5xx linking field in one of the authority
type frameworks):
1. Create a new authority record.
2. Go to the 5xx tab.
3. Click the authority control plugin ellipsis.
4. Do a search for an authority.
5. Select a relationship type.
6. Note that after you choose an authority, that the $w control field
is automatically populated with the relationship type.
Signed-off-by: Frédéric Demians <f.demians@tamil.fr>
After applying patch for bug 8207, I had been able to follow the test plan step
by step, and get the expected result.
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Rebased on master 1 August 2012
Makes clicking a Delete link refresh the search that
was performed so that the results are still on the screen.
Signed-off-by: Frédéric Demians <f.demians@tamil.fr>
Works as advertised. Very usefull. Still the issue that indexing being
not in real time, the search result displays the just deleted authority.
But there is no way to do better.
When performing subject analysis, it is not uncommon to find that an
authority record you need does not exist. This commit adds a link in
the search results to enable the cataloger to create an authority on
the fly and have the heading populated by the new authority.
To test:
1. Use the auth finder plugin the cataloguing module to search for a
heading you do not have an authority record for.
2. Click the "Create new" link.
3. Fill in your authority and save it.
4. Note that the heading field was populated with the data you entered
in the authority record.
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
1) Added a new authority using the plugin
2) Used an existing authority using the plugin
3) Created a new authority in the authority module
All worked nicely.
If you create a new authority record and the name already exists in
your database, you get a duplicate warning. This is good, but there is
a small error in the message:
Duplicate record suspected
Is this a duplicate of HASH(0x47fa670)?
This patch corrects the error message to show the title.
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Error message correctly shows the name after applying this patch.
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
Adds the ability to save individual authority records in MADS, MARCXML, or
binary MARC format to the staff client.
To test:
1. Apply patch
2. View authority record in staff client
3. Try saving record as MADS, MARCXML, and MARC, and confirm that the
resulting files are what you expect
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Works nicely, tested different export options on different records.
UNIMARC is the only marcflavour that does not already have an option
for searching all authority types, so check that the marcflavour is
UNIMARC before displaying the additional "All authority types" option.
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Authority type on detail page in staff now shows correctly.
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
The HTML for authority search results was previously generated in
C4::AuthoritiesMarc::BuildSummary, which meant that it couldn't be
translated. This patch moves the HTML generation into the templates
by introducing a new authorities-search-results.inc include file for
both the OPAC and the Intranet which contains a Template::Toolkit BLOCK
for rendering the authority results. Fixes the authority autocomplete
by removing the untranslatable strings, and returning only data from
the database.
To test:
1. Apply patch.
2. Test authority searching in the authority module in the staff client
3. Test authority searching in the authority control plugin in the
cataloguing module (and the plugin for UNIMARC field 210$c, if you
can figure out how)
4. Test authority searching in the OPAC
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Tested with MARC21 data.
1) Applies cleanly on current HEAD.
2) Authority search in staff
Patch works wonderfully, only some small notes found while testing that have
not been changed by this patch:
ENH note: Search terms show up nowhere. So if I want to change the sorting, I have to
repeat typing in my search term. Even if the form does not keep the term, it
should be visible somewhere on the screen what I searched for.
ENH note: The pagination on top and at the bottom of the result list are formatted
differently. Maybe some missing CSS?
ENH note: Also the authority type is not shown at all in the result list.
3) Cataloguing and authority plugins
The autocomplete function works nicely.
ENH note: There is only one small enhancement
I could imagine. If I start my search from 100 it will limit the search to
'persons' but the autocomplete will also suggest other authorities. It would
be a bit cleaner, if the autocomplete could limit by the appropriate authority
type too. Very nice feature.
Plugins overall work nicely. Created links include the authority numbers and work
correctly.
4) Authority search in OPAC
Works nicely. Display is consistent, but translatability greatly improved.
ENH note: In staff we use 'Details' in OPAC we use 'View full heading' - I wonder
if maybe 'details' would be better understandable for users?
Note: Code reveals a system preference 'AuthDisplayHierarchy' that is
not available in the system preference editor. I talked to Jared and he
will work on this feature later on. For now it's no regression, as the
preference has never been visible.
In order to facilitate a more painless process for converting
to jQueryUI I will submit separate patches for various "widgets,"
starting with Autocomplete.
This patch replaces all instances of YUI autocomplete with
a jQueryUI version. The patch includes an up-to-date version
of jQuery and jQueryUI libraries.
The patch also moves some markup in instances where it should
have been removed in favor of a different include.
To test, find the various autocomplete instances and confirm
that they are working:
- Circulation search header autocomplete
- Overdues patron attribute authorized value filter (must
have patron attributes enabled, and a patron attribute
defined which uses authorized values.
- Authorities search plugin. Edit a MARC record and use
an authorities plugin link to do a search for authority
records.
Incomplete: There is a YUI autocomplete instance in a UNIMARC
plugin (unimarc_field_210c_bis.tt) which I couldn't figure out
how to test, even on a sandbox set up with UNIMARC. I could use
help with a follow-up.
http://bugs.koha-community.org/show_bug.cgi?id=7447
Signed-off-by: Liz Rea <wizzyrea@gmail.com>
Passes all tests outlined, is quite pretty.
Passes t xt
Signed-off-by: Ian Walls <koha.sekjal@gmail.com>
Intention of code is that data is output in utf-8 requires an
encoding layer to ensure that. ':utf-8' flags the stream as utf-8
but does not ensure output characters are correctly encoded
signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
While typing an authority, will automatically propose authorities (similar to
autocompletion for patron search if activated)
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Tested searching for authorities with and without autocomplete. Note that
this is most useful when used in the "Main entry" box instead of the
"Main entry ($a only)" box.
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Corrected tabs to spaces in auth-finder-search.inc while resolving merge
conflict.
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
Squashed patch incorporating all previous patches (there is no functional
change compared to the previous version of this patch, this patch merely
squashes the original patch and follow-up, and rebases on latest master).
=== TL;DR VERSION ===
*** Installation ***
1. Run installer/data/mysql/atomicupdate/bug_7284_authority_linking_pt1
and installer/data/mysql/atomicupdate/bug_7284_authority_linking_pt2
2. Make sure you copy the following files from kohaclone to koha-dev:
etc/zeradb/authorities/etc/bib1.att,
etc/zebradb/marc_defs/marc21/authorities/authority-koha-indexdefs.xml,
etc/zebradb/marc_defs/marc21/authorities/authority-zebra-indexdefs.xsl,
etc/zebradb/marc_defs/marc21/authorities/koha-indexdefs-to-zebra.xsl, and
etc/zebradb/marc_defs/unimarc/authorities/record.abs
3. Run misc/migration_tools/rebuild_zebra.pl -a -r
*** New sysprefs ***
* AutoCreateAuthorities
* CatalogModuleRelink
* LinkerModule
* LinkerOptions
* LinkerRelink
* LinkerKeepStale
*** Important notes ***
You must have rebuild_zebra processing the zebraqueue for bibs when testing this
patch.
=== DESCRIPTION ===
*** Cataloging module ***
* Added an additional box to the authority finder plugin for "Heading match,"
which consults not just the main entry but also See-from and See-also-from
headings.
* With this patch, the automatic authority linking will actually work properly
in the cataloging module. As Owen pointed out while testing the patch,
though, longtime users of Koha will not be expecting that. In keeping with
the principles of least surprise and maximum configurability, a new syspref,
CatalogModuleRelink makes it possible to disable authority relinking in the
cataloging module only (i.e. leaving it enabled for future runs of
link_bibs_to_authorities.pl). Note that though the default behavior matches
the current behavior of Koha, it does not match the intended behavior.
Libraries that want the intended behavior rather than the current behavior
will need to adjust the CatalogModuleRelink syspref.
*** misc/link_bibs_to_authorities.pl ***
Added the following options to the misc/link_bibs_to_authorities.pl script:
--auth-limit Only process those headings that match the authorities
matching the user-specified WHERE clause.
--bib-limit Only process those bib records that match the
user-specified WHERE clause.
--commit Commit the results to the database after every N records
are processed.
--link-report Display a report of all the headings that were processed.
Converted misc/link_bibs_to_authorities.pl to use POD.
Added a detailed report of headings that linked, did not link, and linked
in a "fuzzy" fashion (the exact semantics of fuzzy are up to the individual
linker modules) during the run.
*** C4::Linker ***
Implemented new C4::Linker functionality to make it possible to easily add
custom authority linker algorithms. Currently available linker options are:
* Default: retains the current behavior of only creating links when there is
an exact match to one and only one authority record; if the 'broader_headings'
option is enabled, it will try to link to headings to authority records for
broader headings by removing subfields from the end of the heading (NOTE:
test the results before enabling broader_headings in a production system
because its usefulness is very much dependent on individual sites' authority
files)
* First Match: based on Default, creates a link to the *first* authority
record that matches a given heading, even if there is more than one
authority record that matches
* Last Match: based on Default, creates a link to the *last* authority
record that matches a given heading, even if there is more than one record
that matches
The API for linker modules is very simple. All modules should implement the
following two functions:
<get_link ($field)> - return the authid for the authority that should be
linked to the provided MARC::Field object, and a boolean to indicate whether
the match is "fuzzy" (the semantics of "fuzzy" are up to the individual plugin).
In order to handle authority limits, get_link should always end with:
return $self->SUPER::_handle_auth_limit($authid), $fuzzy;
<flip_heading ($field)> - return a MARC::Field object with the heading flipped
to the preferred form. At present this routine is not used, and can be a stub.
Made the linking functionality use the SearchAuthorities in C4::AuthoritiesMarc
rather than SimpleSearch in C4::Search. Once C4::Search has been refactored,
SearchAuthorities should be rewritten to simply call into C4::Search. However,
at this time C4::Search cannot handle authority searching. Also fixed numerous
performance issues in SearchAuthorities and the Linker script:
* Correctly destroy ZOOM recordsets in SearchAuthorities when finished. If left
undestroyed, efficiency appears to approach O(log n^n)
* Add an optional $skipmetadata flag to SearchAuthorities that can be used to
avoid additional calls into Zebra when all that is wanted are authority
records and not statistics about their use
*** New sysprefs ***
* AutoCreateAuthorities - When this and BiblioAddsAuthorities are both turned
on, automatically create authority records for headings that don't have
any authority link when cataloging. When BiblioAddsAuthorities is on and
AutoCreateAuthorities is turned off, do not automatically generate authority
records, but allow the user to enter headings that don't match an existing
authority. When BiblioAddsAuthorities is off, this has no effect.
* CatalogModuleRelink - when turned on, the automatic linker will relink
headings when a record is saved in the cataloging module when LinkerRelink
is turned on, even if the headings were manually linked to a different
authority by the cataloger. When turned off (the default), the automatic
linker will not relink any headings that have already been linked when a
record is saved.
* LinkerModule - Chooses which linker module to use for matching headings
(current options are as described above in the section on linker options:
"Default," "FirstMatch," and "LastMatch")
* LinkerOptions - A pipe-separated list of options to set for the authority
linker (at the moment, the only option available is "broader_headings," which
is described below)
* LinkerRelink - When turned on, the linker will confirm the links for headings
that have previously been linked to an authority record when it runs. When
turned off, any heading with an existing link will be ignored.
* LinkerKeepStale - When turned on, the linker will never *delete* a link to an
authority record, though, depending on the value of LinkerRelink, it may
change the link.
*** Other changes ***
* Cleaned up authorities code by removing unused functions and adding
unimplemented functions and added some unit tests.
* This patch also modifies the authority indexing to remove trailing punctuation
from Match indexes.
* Replace the old BiblioAddAuthorities subroutines with calls into the new
C4::Linker routines.
* Add a simple implementation for C4::Heading::UNIMARC. (With thanks to F.
Demians, 2011.01.09) Correct C4::Heading::UNIMARC class loading. Create
biblio tag to authority types data structure at initialization rather than
querying DB.
* Ran perltidy on all changed code.
*** Linker Options ***
Enter "broader_headings" in LinkerOptions. With this option, the linker will
try to match the following heading as follows:
=600 10$aCamins-Esakov, Jared$xCoin collections$vCatalogs$vEarly works to
1800.
First: Camins-Esakov, Jared--Coin collections--Catalogs--Early works to 1800
Next: Camins-Esakov, Jared--Coin collections--Catalogs
Next: Camins-Esakov, Jared--Coin collections
Next: Camins-Esakov, Jared (matches! if a previous attempt had matched, it
would not have tried this)
This is probably relevant only to MARC21 and LCSH, but could potentially be of
great use to libraries that make heavy use of floating subdivisions.
=== TESTING PLAN ===
Note: all of these tests require that you have some authority records,
preferably for headings that actually appear in your bibliographic data. At
least one authority record must contain a "see from" reference (remember which
one contains this, as you'll need it for some of the tests). The number shown
in the "Used in" column in the authority module is populated using Zebra
searches of the bibliographic database, so you *must* have
rebuild_zebra.pl -b -z [-x] running in cron, or manually run it after running
the linker.
*** Testing the Heading match in the cataloging plugin ***
1. Create a new record, and open the cataloging plugin for an
authority-controlled field.
2. Search for an authority by entering the "see from" term in the Heading Match
box
3. Confirm that the appropriate heading shows up
4. Search for an authority by entering the preferred heading into the Main
entry or Main entry ($a only) box (i.e., repeat the procedure you usually
use for cataloging, whatever that may be)
5. Confirm that the appropriate heading shows up
*** Testing the cataloging interface ***
6. Turn off BiblioAddsAuthorities
7. Confirm that you cannot enter text directly in an authority-controlled field
8. Confirm that if you search for a heading using the authority control plugin
the heading is inserted (note, however, that this patch does not AND IS NOT
INTENDED TO fix the bugs in the authority plugin with duplicate subfields;
those are wholly out of scope- this check is for regressions)
9. Turn on BiblioAddsAuthorities and AutoCreateAuthorities
10. Confirm that you can enter text directly into an authority-controlled field,
and if you enter a heading that doesn't currently have an authority record,
an authority record stub is automatically created, and the heading you
entered linked
11. Confirm that if you enter a heading with only a subfield $a that fully
*matches* an existing heading (i.e. the existing heading has only
subfield $a populated), the authid for that heading is inserted into
subfield $9
12. Confirm that if you enter a heading with multiple subfields that *matches*
an existing heading, the authid for that heading is inserted into
subfield $9
13. Turn on BiblioAddsAuthorities and turn off AutoCreateAuthorities
14. Confirm that you can enter text directly into an authority-controlled field,
and if you enter a heading that doesn't currently have an authority record,
an authority record stub is *not* created
15. Confirm that if you enter a heading with only a subfield $a that *matches*
an existing heading, the authid for that heading is inserted into
subfield $9
16. Confirm that if you enter a heading with multiple subfields that *matches*
an existing heading, the authid for that heading is inserted into
subfield $9
17. Create a record and link an authority record to an authorized field using
the authority plugin.
18. Save the record. Ensure that the heading is linked to the appropriate
authority.
19. Open the record. Change the heading manually to something else, leaving
the link. Save the record.
20. Ensure that the heading remains linked to that same authority.
21. Change CatalogModuleRelink to "on."
22. Open the record. Use the authority plugin to link that heading to the
same authority record you did earlier.
23. Save the record. Ensure that the heading is linked to the appropriate
authority.
24. Open the record. Change the heading manually to something else, leaving
the link. Save the record.
25. Ensure that the heading is no longer linked to the old authority record.
*** Testing link_bibs_to_authorities.pl ***
26. Set LinkerModule to "Default," turn on LinkerRelink and
BiblioAddsAuthorities, and turn AutoCreateAuthorities and
LinkerKeepStale off
27. Edit one bib record so that an authority controlled field that has already
been linked (i.e. has data in $9) has a heading that does not match any
authority record in your database
28. Run misc/link_bibs_to_authorities.pl --link-report --verbose --test (you may
want to pipe the output into less or a file, as the result is quite a lot of
information)
29. Look over the report to see if the headings that you have authority records
for report being matched, that the heading you modified in step 2 is
reported as "unlinked," and confirm that no changes were actually made to
the database (to check this, look at the bib record you edited earlier, and
check that the authid in the field you edited hasn't changed)
30. Run misc/link_bibs_to_authorities.pl --link-report --verbose (you may want
to pipe the output into less or a file, as the result is quite a lot of
information)
31. Check that the heading you modified has been unlinked
32. Change the modified heading back to whatever it was, but don't use the
authority control plugin to populate $9
33. Run misc/link_bibs_to_authorities.pl --link-report --verbose
--bib-limit="biblionumber=${BIB}" (replacing ${BIB} with the biblionumber
of the record you've been editing)
34. Confirm that the heading has been linked to the correct authority record
35. Turn LinkerKeepStale on
36. Change that heading to something else
37. Run misc/link_bibs_to_authorities.pl --link-report --verbose
--bib-limit="biblionumber=${BIB}" (replacing ${BIB} with the biblionumber
of the record you've been editing)
38. Confirm that the $9 has not changed
39. Turn LinkerKeepStale off
40. Create two authorities with the same heading
41. Run misc/migration_tools/rebuild_zebra.pl -a -z
42. Enter that heading into the bibliographic record you are working with
43. Run misc/link_bibs_to_authorities.pl --link-report --verbose
--bib-limit="biblionumber=${BIB}" (replacing ${BIB} with the biblionumber
of the record you've been editing)
44. Confirm that the heading has not been linked
45. Change LinkerModule to "FirstMatch"
46. Run misc/link_bibs_to_authorities.pl --link-report --verbose
--bib-limit="biblionumber=${BIB}" (replacing ${BIB} with the biblionumber
of the record you've been editing)
47. Confirm that the heading has been linked to the first authority record it
matches
48. Change LinkerModule to "LastMatch"
49. Run misc/link_bibs_to_authorities.pl --link-report --verbose
--bib-limit="biblionumber=${BIB}" (replacing ${BIB} with the biblionumber
of the record you've been editing)
50. Confirm that the heading has been linked to the second authority record it
matches
51. Run misc/link_bibs_to_authorities.pl --link-report --verbose
--auth-limit="authid=${AUTH}" (replacing ${AUTH} with an authid)
52. Confirm that only that heading is displayed in the report, and only those
bibs with that heading have been changed
If all those things worked, good news! You're ready to sign off on the patch
for bug 7284.
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Rebased on latest master and squashed follow-up, 16 February 2012
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Rebased on latest master, 21 February 2012
Signed-off-by: schuster <dschust1@gmail.com>
use encoding(UTF-8) rather than utf-8 for stricter
encoding
Marking output as ':utf8' only flags the data as utf8
using :encoding(UTF-8) also checks it as valid utf-8
see binmode in perlfunc for more details
In accordance with the robustness principle input
filehandles have not been changed as code may make
the undocumented assumption that invalid utf-8 is present
in the imput
Fixes errors reported by t/00-testcritic.t
Where feasable some filehandles have been made lexical rather than
reusing global filehandle vars
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
Before this patch, if we tried to import an authority with multiple $x
subfields into a bibliographic record, only the last value get added
to the form.
All repeated values should now be sent to the form.
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
Signed-off-by: Ian Walls <ian.walls@bywatersolutions.com>
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
TransformHtmlToMarc(): changed interface - no point passing params when
they can be accessed from $cgi
Signed-off-by: Liz Rea <lrea@nekls.org>
Signed-off-by: Ian Walls <ian.walls@bywatersolutions.com>
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Patch reworked for master using Template::Toolkit.
To+
Signed-off-by: Nicole C. Engard <nengard@bywatersolutions.com>
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Remove some unnecessary checks when check of error is
sufficient. Make the order in some cases more logical
Should remove some possibilities of runtime warning noise.
Although some calls belong to the 'Nothing could
ever go wrong' school have added some warnings
Signed-off-by: Christophe Croullebois <christophe.croullebois@biblibre.com>
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
This patch addresses both security issues mentioned in the summary of the report
submitted by Frère Sébastien Marie included below.
---------------------------
The problem is here: 'C4/AuthoritiesMarc.pm' in the function 'DelAuthority':
The argument $authid is included directly (not via statement) in the SQL.
For the exploit of this problem, you can use 'authorities/authorities-home.pl'
with authid on the URL and op=delete (something like
"authorities/authorities-home.pl?op=delete&authid=xxx").
This should successfully call DelAuthority, without authentification...
(DelAuthority is call BEFORE get_template_and_user, so before authentification
[This should be an issue also...]).
Please note that the problem isn't only that anyone can delete an authority of
this choose, it is more general: with "authid=1%20or%1=1" (after inclusion sql
will be like: "delete from auth_header where authid=1 or 1=1") you delete all
authorities ; with "authid=1;delete%20from%xxx" it is "delete from auth_header
where authid=1;delete from xxx" and so delete what you want...
SQL-INJECTION is very permissive: you can redirect the output in a file (with
some MySQL function), so write thea file of you choose in the server, in order
to create a backdoor, and compromise the server.
Signed-off-by: Frère Sébastien Marie <semarie-koha@latrappe.fr>
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
With this patch, in biblio record data entry form, when ... is clicked for an
authority controlled field, it's possible to select which heading repetion to
copy if the authority has repeated headings.
When there is just one authority repetition, the first one is displayed to
choose, as previously.
This patch is REQUIRED by French libraries following SUDOC UNIMARC format, and
cataloguing multilingual materials ie all Higher Educational and Research
libraries.
Signed-off-by: Henri-Damien LAURENT <henridamien.laurent@biblibre.com>
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Fix warnings for uninitialized authtypecode on lines 646, 653.
Signed-off-by: Colin Campbell <colin.campbell@ptfs-europe.com>
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
