Commit graph

273 commits

Author SHA1 Message Date
d49e8bd5e2 Bug 9988: Few subtle changes for postponed merge
The fails in the previous test showed that we need the first three
changes here. Some final polishing in points 4 to 6.

[1] Sub merge: Refine the condition for initializing $tags_new.
    A postponed 'modify'-merge (A to B) makes that $authtypefrom is not
    defined when running merge later. When crossing the type boundary, we
    need a new field too.
[2] Sub merge: Add condition for an empty @record_to array.
    This indicates also that a field should be removed, since we should
    otherwise only add a $9 subfield.
[3] Sub merge: Adjust initializing @record_from.
    This change is tested by verifying a cleared subfield in the test.
[4] DelAuthority: Adding a skipmerge parameter to allow the call from
    authorities/merge.pl to skip an unneeded merge.
    This also prevents that the 'delete-merge' would precede the
    'modify-merge' under a hypothetical race condition.
[5] DelAuthority: There is actually no need to call GetAuthority.
    The subfields of the old record are not relevant in this case.
[6] Added a few POD lines to merge.
[7] Removed a trailing space in a comment line in merge.

Test plan:
[1] Run t/db_dependent/Authorities/Merge.t.
    The last subtest should no longer fail now.
[2] See test plan of next patch.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Marc Véron <veron@veron.ch>
Signed-off-by: Jacek Ablewicz <abl@biblos.pk.edu.pl>
Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
2017-04-13 08:53:47 -04:00
04263912c7 Bug 9988: Check the merge limit in sub merge
At this point, we are replacing dontmerge functionality by the new
AuthorityMergeLimit logic. Instead of doing this check before calling
merge, we just call merge and check it there. In order to let the cron
job do the larger (postponed) merges, we add a parameter override_limit.

A subtest is added in Merge.t to test the 'postponed merge' feature. Since
merge now also calls get_usage_count, an additional mock is added. All
references to dontmerge are removed.

In merge two lines, initializing $dbh and $counteditbiblios, are moved.
The dontmerge test in DelAuthority and ModAuthority is removed. Since this
did not leave much in ModAuthority, I fixed the whitespace on the remaining
lines rightaway (yes, I know).
A minimal set of changes is applied to the cron script; it will get further
attention on a next patch.

Test plan:
[1] Run t/db_dependent/Authorities/Merge.t
[2] Set AuthorityMergeLimit to 2. Modify an authority with two linked
    biblios. Check that the merge was done immediately.
[3] Now modify an authority with more than 2 linked records.
    Verify that the merge was postponed; a record must be inserted in
    the need_merge_authorities table.
[4] Testing of the merge cron job is *postponed* to a next patch.
    Note: I tested a modification, but the script just needs more
    attention.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Marc Véron <veron@veron.ch>
Signed-off-by: Jacek Ablewicz <abl@biblos.pk.edu.pl>
Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
2017-04-13 08:53:46 -04:00
06647d29a7 Bug 9988: Remove the Zebra code from sub merge
Since we can now call linked_biblionumbers, we can now remove all Zebra
related code from merge. We also add a parameter biblionumbers; we use it
in the test now, but it may be handy too later in the maintenance script
when we want to trigger a merge for specific biblionumber(s). See bug
report 18071.

All mocks for ZOOM, Context::Zconn, Search::new_record_for_zebra in the
merge test can now be replaced by one mock for linked_biblionumbers. Note
that we test the biblionumbers parameter in the last subtest without that
mock.

Remove unused vars $countunmodifiedbiblio, $counterrors from merge.
Renamed zebrarecords to linkedrecords in the Merge test.

Test plan:
[1] Run t/db_dependent/Authorities/Merge.t
[2] Modify an authority record. Check the linked biblio records.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Marc Véron <veron@veron.ch>
Signed-off-by: Jacek Ablewicz <abl@biblos.pk.edu.pl>
Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
2017-04-13 08:53:46 -04:00
4c762ba69c Bug 9988: Merge should have a parameter hash
We will need a few additional parameters for merge later on. This patch
puts the original parameters in a parameter hash.
For the same reason DelAuthority gets a parameter hash here.

Note: We remove the second parameter from the DelAuthority call in
authorities/authorities-home.pl here. It was not used and could have
presented problems in the future.

Test plan:
[1] Run t/db_dependent/AuthoritiesMarc.t.
[2] Run t/db_dependent/Authorities/Merge.t.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Marc Véron <veron@veron.ch>
Signed-off-by: Jacek Ablewicz <abl@biblos.pk.edu.pl>
Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
2017-04-13 08:53:46 -04:00
fd8c2d4e35 Bug 18070: Correct error from podchecker on line 1376
Resolves:
 *** ERROR: Spurious =cut command at line 1376 in file C4/AuthoritiesMarc.pm

Test plan:
[1] Run podchecker on C4/AuthoritiesMarc.pm

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
2017-03-03 18:12:10 +00:00
17e84750d9 Bug 18070: [Follow-up] What should DelAuthority return?
Script tools/batch_delete_records.pl only checks the return value of
DelAuthority. The return value depends on DBI rows, which is not always
reliable. It may return -1 when it 'does not know'.

Testing $@ in tools/batch_delete_records.pl should actually be enough.
The return value was discarded in C4/ImportBatch.pm. Removing an unused
variable.

Test plan:
[1] Pick an authority record with a few linked biblios.
    Delete this authority record via tools/batch_delete_records.pl.
    Check if the linked biblio records are cleaned up.
[2] Bonus: Make a typo in the SQL statement of DelAuthority. Check
    if batch_delete_records shows you the error message.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
2017-03-03 18:12:09 +00:00
cb58b11f67 Bug 18070: Extend sub merge to remove fields for deleted authorities
In order to accomplish this, we need to add some additional checks in
the merge routine. The actual change to remove the field, is quite
small.

Furthermore, we need to add a merge call in DelAuthority and adjust
the merge cron job accordingly.

The change is well supported by additional tests, including a simulation
of postponed removal via cron, if dontmerge is enabled.

Note: Deleting an authority with linked biblios is tested on the next
patch.

Test plan:
[1] Run t/db_dependent/Authorities/Merge.t
[2] Delete an authority without linked biblios from the Authorities
    module. If the indexer is not fast enough, wait a bit and refresh to
    verify that the authority is gone on authorities-home.pl.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
2017-03-03 18:12:09 +00:00
ad38b24308 Bug 18014: AddAuthority should respect AUTO_INCREMENT
Instead of using the MAX(authid)+1 logic, AddAuthority should just save
the record and get the new id. The authid column is an autoincrement.

This eliminates problems where a newly assigned authid also refers to a
previously deleted record. (And it will not cause problems when refining
the dontmerge functionality on report 9988.)

Note: ModAuthority also calls AddAuthority to update an existing record; in
that case we should not create a new record even if the record should not
be found any more (which should be exceptional).

This patch also simplifies handling of 001 in the authority record: in all
cases this field is updated now; no need to check its contents.

Test plan:
[1] Run t/db_dependent/AuthoritiesMarc.t
[2] Add a new authority record via the interface

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Mark Tompsett <mtompset@hotmail.com>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
2017-02-14 14:01:11 +00:00
6ddd4b2f5f Bug 17913: [Follow-up] Fix duplicate $9s after merging in loose mode
We need to add $9 to the skip_subfields hash too. Formerly, it was
added to $exclude as well.
Thanks, Julian, for catching this one.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
2017-01-20 13:55:13 +00:00
dd1c63230f Bug 17913: Run perltidy on the inner foreach loop
Kept the same number of lines.
You could verify with diff -w.

Test plan:
[1] Run t/db_dependent/Authorities/Merge.t
[2] As the last patch in this series, also test the interface:
    Set AuthorityMergeMode to loose. Set dontmerge to Do.
    Modify an authority record attached to multiple biblios.
    Edit a subfield, clear a subfield and add a subfield.
    Save. Wait a bit for the merge and Zebra update.
    Verify that the changes are merged properly into biblio records.
[3] Repeat step 2 with AuthorityMergeMode to strict.
    Remember that this affects the extra subfields in biblio records.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
2017-01-20 13:55:12 +00:00
28b74224d3 Bug 17913: We always need some housekeeping
Remove some commented warnings
Remove the commented old code at the end of sub merge
Explicitly set merge mode in the first subtest
Move the return to loose mode from the second subtest to the third

Test plan:
Run t/db_dependent/Authorities/Merge.t

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
2017-01-20 13:55:12 +00:00
ede08d725e Bug 17913: Remove possible duplicates in strict merge mode
Since strict mode does not allow additional subfields that would make
identical fields linked to the same authority different, there is no
need to keep them while merging.

We achieve this goal by simply:
[1] Count the number of same fields linked to mergefrom in strict mode to
    eliminate duplicates.
[2] Replaces the if-statement on auth_number by a next. (Tidy follows.)

Test plan:
Run t/db_dependent/Authorities/Merge.t

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
2017-01-20 13:55:12 +00:00
681ae8430e Bug 17913: Do not keep a cleared subfield in loose merge mode
If you modify an authority and clear a specific subfield, you expect that
merge respects your edit and clears this subfield too in the biblio
records. It does in the new strict mode, but it does not yet in the
default loose mode.

This patch fixes that by adjusting the code around $exclude so that it
uses a new hash skip_subfields, built from the reporting tags from the old
and the new authority record.

This is supported again by some changes in the unit test.

Test plan:
Run t/db_dependent/Authorities/Merge.t

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
2017-01-20 13:55:11 +00:00
8cde85451e Bug 17913: Fix the new field tag in merge when changing type
Originally aimed for 9988, adjusted for this report.

Old behavior was: pick the first tag. This is definitely wrong.
If you (would) merge 610 to 611, you don't want to get a 111.

This patch resolves the problem by determining the new tag in a small
helper routine _merge_newtag, and corrects the position of the new field
in the MARC record with append_fields_ordered. Too bad that MARC::Record
does not have such a function; it looks like insert_fields_ordered, but
it is different in case of multiple fields with the same tag.

Note: These two small helper functions are not tested separately, since they
should not be called outside of merge. They are implicitly tested by the
adjusted tests in Merge.t.

Note: In adding tests for this fix, I chose to simplify compare_field_count
(no need for the pass parameter), and replace the pass parameter of sub
compare_field_order by an exclude parameter, a hash of fields to exclude in
counting fields.

Test plan:
Run t/db_dependent/Authorities/Merge.t

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
2017-01-20 13:55:11 +00:00
Frédérick
79c81920ab Bug 17913: Use AuthorityMergeMode pref in sub merge
Original fix from a patch on bug 11315.
Amended by Marcel de Rooy January 2017.

Test plan:
If you set mode to loose, the test will still pass.
If you set mode to strict, one test will fail. (Fixed later.)

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
2017-01-20 13:55:10 +00:00
ffe1f6555c Bug 17913: Use replace_with instead of insert_grouped_field
Original fix from a patch on bug 5572.
Amended by Marcel de Rooy January 2017.

Note: This does not yet resolve the field order when merging to another
auth type, but is a good start.

Test plan:
Run t/db_dependent/Authorities/Merge.t

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
2017-01-20 13:55:10 +00:00
172ebfe520 Bug 16431: Use Koha::Cache to cache marc subfield structure
The marc subfield structure is currently cached using a global variable
of C4::Context. The infos are retrieved every time a new context is
created.
This patch suggests to use Koha::Cache instead.

To achieve this goal, a new subroutine is created
C4::Biblio::GetMarcSubfieldStructure, it will be called from code which
needs to get the marc subfield structure. GetMarcFromKohaField,
GetMarcSubfieldStructureFromKohaField, TransformKohaToMarc and
_get_inverted_marc_field_map are modified accordingly and the cache is cleared
when the table is updated (from the 3 pl scripts modified by this patch).

The caching done in C4::Context (marcfromkohafield) is removed.

Test plan:
Play with the marc subfield structure (in the administration module),
then add and edit records and make sure everything went fine.

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Everything works as expected on my functional tests. I'm really happy to see the
patch introduces relevant tests for previously untested functions.

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
2016-07-08 12:44:58 +00:00
Jesse Weaver
37ea74669c Bug 12748: (QA followup) fix several tiny QA tools warnings
* admin/searchengine/elasticsearch/mappings.yaml:
    * Fix indentation so QA tools believe it's valid YAML
  * C4::AuthoritiesMarc: Remove tab character
  * C4::Search: Fix POD
  * Koha::BiblioUtils: remove "return undef"
  * Koha::Schema::Result::Search*: rename type to _type
  * Koha::SE::ES::Search:
    * Fix spelling/@_[0]
    * Add my to loop variable
    * Remove unused var $error
    * Add use Modern::Perl
    * Fix undefined variables
  * Koha::ES::Indexer: fix POD
  * Koha::SE::ES::QueryBuilder:
    * Don't define variables in conditionals
    * Fix POD
  * Koha::SE::Zebra::Search:
    * Fix debug warns/POD/barewords
    * Koha::SE::Zebra::Search: add use Modern::Perl
  * kohastructure.sql: remove old, unused elasticsearch_mappings table,
    add new tables
  * labels/label-item-search.pl: remove C4::Dates
  * opac/elasticsearch.pl: remove old file

Signed-off-by: Jesse Weaver <jweaver@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>

Signed-off-by: Brendan Gallagher <brendan@bywatersolutions.com>
2016-04-26 20:20:13 +00:00
Robin Sheat
b4392018bc Bug 12478: make things using SimpleSearch use the new version
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Jesse Weaver <jweaver@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>

Signed-off-by: Brendan Gallagher <brendan@bywatersolutions.com>
2016-04-26 20:20:07 +00:00
c564f30155 Bug 16169: Change prototype for C4::Biblio::TransformMarcToKoha
subroutines should not take $dbh in parameter.
C4::Biblio::TransformMarcToKoha has it and does not use it.

Test plan:
Look at the patch and confirm that all occurrences of
TransformMarcToKoha have been modified.

Signed-off-by: Jacek Ablewicz <abl@biblos.pk.edu.pl>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

Signed-off-by: Brendan Gallagher brendan@bywatersolutions.com
2016-04-07 00:04:21 +00:00
2d961b5ad4 Bug 16056: Do not crash when searching for an authority if zebra's index is not up-to-date
This patch fixes a regression introduced by
commit 7e70202d34
  Bug 15381: Remove GetAuthType and GetAuthTypeCode

This first version will reintroduce the same behavior as before bug
15381: the record will be displayed even if it's no in the DB

Test plan:
Search for authorities
delete one
The zebra's index is not updated yet and the results will contain the
record you have deleted.

Signed-off-by: Tomas Cohen Arazi <tomascohen@unc.edu.ar>

Signed-off-by: Brendan A Gallagher <brendan@bywatersolutions.com>
2016-04-01 19:04:38 +00:00
798d38e4c7 Bug 16011: $VERSION - Remove comments
perl -p -i -e 's/^.*set the version for version checking.*\n//' **/*.pm

+ manual adjustements

Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@unc.edu.ar>

Signed-off-by: Brendan A Gallagher <brendan@bywatersolutions.com>
2016-03-24 17:20:29 +00:00
017699c345 Bug 16011: $VERSION - Remove the $VERSION init
Mainly a
  perl -p -i -e 's/^.*3.07.00.049.*\n//' **/*.pm
Then some adjustements

Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@unc.edu.ar>

Signed-off-by: Brendan A Gallagher <brendan@bywatersolutions.com>
2016-03-24 17:20:28 +00:00
3830d78d46 Bug 16011: $VERSION - remove use vars $VERSION
perl -p -i -e 's/^(use vars .*)\$VERSION\s?(.*)/$1$2/' **/*.pm

Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@unc.edu.ar>

Signed-off-by: Brendan A Gallagher <brendan@bywatersolutions.com>
2016-03-24 17:20:26 +00:00
Julian Maurice
5ef5fb5617 Bug 15358: Fix authorities merge
Bug 8064 (Merge several biblio records) change some code used in both
biblios and authorities merge tool without updating the authorities
merge tool.
This patch fixes that.

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Signed-off-by: Brendan Gallagher brendan@bywatersolutions.com
2016-02-03 23:03:33 +00:00
72f96d3e3b Bug 15467: If authtypecode is not defined, do not explode
The test t/db_dependent/AuthoritiesMarc.t fails because one test expect
the BuildSummary subroutine to not crash if the authtypecode passed in
parameter does not exist in the DB.

Test plan:
prove t/db_dependent/AuthoritiesMarc.t
should return green.

Signed-off-by: Frédéric Demians <f.demians@tamil.fr>
  Test fails before patch, and doesn't anymore after.

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

Signed-off-by: Brendan A Gallagher <brendan@bywatersolutions.com>
2016-01-07 19:03:27 +00:00
4a34fd2fd3 Bug 15381: Fix error in SearchAuthorities when no authtypecode
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Brendan Gallagher brendan@bywatersolutions.com
2015-12-31 18:59:03 +00:00
7e70202d34 Bug 15381: Remove GetAuthType and GetAuthTypeCode
Test this patch with the previous one.

Signed-off-by: Frédéric Demians <f.demians@tamil.fr>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Brendan Gallagher brendan@bywatersolutions.com
2015-12-31 18:59:02 +00:00
7283069a9d Bug 15381: Remove getauthtypes and getauthtype
These 2 subroutines are now not necessary anymore, we can use
Koha::Authority::Types to retrieve the authority types.

This patch won't be easy to test. Indeed, a lot of file are updated.
To test it I would suggest to focus on the merge authority

You should also confirm that the authtypecode and the authtypetext
values are correctly displayed when navigating in the authority module.

Signed-off-by: Frédéric Demians <f.demians@tamil.fr>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Brendan Gallagher brendan@bywatersolutions.com
2015-12-31 18:59:02 +00:00
e28d54f7d9 Bug 15381: Use Koha::Authority::Types in some other places
There are 3 place where the auth_types table were requested directly
from the script or a subroutine.  These 3 occurrences are easy to
replace with the new module.

Test plan:
1/ Search for authorities and use the "did you mean" feature for
authorities.
Focus on the authtypecode
2/ Edit authority and biblio frameworks.
The "Thesaurus" dropdown list should be correctly populated (UNIMARC
only).

Signed-off-by: Frédéric Demians <f.demians@tamil.fr>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Brendan Gallagher brendan@bywatersolutions.com
2015-12-31 18:59:02 +00:00
03c89fa02e Bug 15380: Move Koha::Authority to Koha::MetadataRecord::Authority
The existing Koha::Authority does not use Koha::Objects and does not
exclusively deal with DB objects.
It makes much sense to move it to let the space free for a
Koha::Authority and Koha::Authorities modules based on Koha::Object.

Signed-off-by: Marc Véron <veron@veron.ch>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Brendan Gallagher brendan@bywatersolutions.com
2015-12-31 18:50:41 +00:00
Lyon3 Team
6c94dc7987 Bug 15130: useless unitialization warnings when updating authorities
Signed-off-by: Frederic Demians <f.demians@tamil.fr>
  Fixes a trivial coding error.

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2015-11-06 12:08:51 -03:00
Stefan Weil
64925f7522 Bug 14383: C4: Fix some typos (mostly in comments and documentation)
Most of them were found and fixed using codespell.
Fix also some related grammar issues.

In C4/Serials.pm a variable was renamed to make future codespelling
checks easier.

Signed-off-by: Stefan Weil <sw@weilnetz.de>

http://bugs.koha-community.org/show_bug.cgi?id=14383
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>

Signed-off-by: Jonathan Druart <jonathan.druart@koha-community.org>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2015-06-22 17:34:45 -03:00
Aleisha
bf6ded93e8 Bug 14075: Undefined value creates noisy warns in C4::AuthoritiesMarc
This match sets $sortby (previously undefined value) as an empty string to get rid of the warns.

To test:

1) Go to a URL such as http://localhost:8080/cgi-bin/koha/opac-authorities-home.pl?op=do_search&type=opac&operator=contains&value=a&marclist=any&and_or=and
2) Notice the warns in the error log
3) Apply patch
4) Reload URL
5) Notice page still works but no warns in error log

Signed-off-by: Mark Tompsett <mtompset@hotmail.com>

NOTE: I would have done $sortby //= '';
      But this works too. :)

Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
2015-05-06 15:26:04 -03:00
Jonathan Druart
927aedafa1 Bug 10985: [UNIMARC] Fix authority summary
The problem is the template in authority type summary is not respected
at all. It is only read to see which fields and subfields should appear
in the summary.
This patch fixes that.
It also fixes a bug in auth_finder.pl plugin when summary contains
fields other than 2XX.

Test plan:
0/ You must use a UNIMARC setup for those tests
1/ edit an authority type summary with:
     NP : [200a][, 200b][ 200d][-- 152b --][ ; 200c][ (200f)] [001*] [ppn: 009*]
2/ create a new authority with previous fields (it is possible some
   fields don't exist).
3/ search this authority and verify the summary is someting like:
     NP : Name, D.-- NP -- 23849 ppn: my_ppn
4/ Verify some summary for existing authorities and check they are
   correct.
5/ Edit a biblio record and use the plugin auth_finder.pl (for example
   in a 7XX field)
6/ Do a search and verify the summary is correct
7/ Click on 'choose' or one of the numbered links ('1', '2', ... ; you
should have multiple 2XX fields for the numbered links to show up)
8/ Verify that the biblio field is correctly filled.

/!\ For the ppn, it should be defined in the zebra indexes.

In MARC21 and NORMARC setups, this patch should change nothing, please
verify that too (you can check that the auth_finder.pl plugin is still
working and the auth type summary is correctly displayed in authorities
search and auth_finder.pl plugin).

Signed-off-by: Frederic Demians <f.demians@tamil.fr>

It works as described, both in authority search result page, and in authority
data entry plugin.

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
2015-04-22 11:54:36 -03:00
Jonathan Druart
a6c9bd0eb5 Bug 9978: Replace license header with the correct license (GPLv3+)
Signed-off-by: Chris Nighswonger <cnighswonger@foundations.edu>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>

http://bugs.koha-community.org/show_bug.cgi?id=9987

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
2015-04-20 09:59:38 -03:00
Jonathan Druart
acff264558 Bug 12861: (follow-up) Noisy warn in error logs
Completely get rid of warns.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
2015-01-21 10:47:47 -03:00
Chris Cormack
b2ce6347e5 Bug 12861 : Noisy warn in the error logs
To test
Hit a url like
cgi-bin/koha/opac-authorities-home.pl?op=do_search&type=opac&operatorc=contains&marclistc=mainentry&and_orc=and&orderby=HeadingAsc&value=FIRECLAY

Notice the warns in the error log
Apply patch
Reload the url
Still works but no warns

(cherry picked from commit c98d805e490a82b2ea9f3d4e0f7278640942ba6d)
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>

Reformatted to fix error M Tompsett spotted

Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Instead of 10 warnings, I now have only 4 warnings.
We're going in the right direction.

Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
2015-01-21 10:47:41 -03:00
Jonathan Druart
df74fe9be8 Bug 11944: FIX Error on editing an authority
Signed-off-by: Paola Rossi <paola.rossi@cineca.it>
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Signed-off-by: Dobrica Pavlinusic <dpavlin@rot13.org>

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
2015-01-13 13:07:11 -03:00
Colin Campbell
61440fab13 Bug 12654 Correct incorrectly quoted regexp
A compile time warning was being generated because \ does not quote
{ Replace the plethora of \ before meta and presumed meta characters
by \Q \E which gets perl to generate all the necessary escapes, we can
assume it knows its regex engine as well or better than the human
programmer

Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Verified change reading the perldoc and also checked that
./misc/link_bibs_to_authorities.pl is no longer showing
the warn and still works.

Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
Works as described, running:
 prove t/db_dependent/AuthoritiesMarc.t
doesn't show the warning anymore.
2014-07-30 11:06:27 -03:00
ad6d367763 Bug 10330 - Rename system preference authoritysep to AuthoritySeperator
Test Plan:
1) Apply patch
2) Run updatedatabase.pl
3) View some records with authorities
4) Note your previously set authority separator should still be in use

Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
No koha-qa errors.

Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2014-05-23 13:26:48 +00:00
Galen Charlton
9fe024667f Bug 7442: (follow-up) restore display of heading type for non-UNIMARC
This patch restores the display of the authority type summary for
MARC21, where at present the heading type (i.e., "Topical Term",
"Personal Names") come over for display in the template.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2014-05-05 01:27:33 +00:00
Fridolyn SOMERS
ebcb64e9fa Bug 7442 (follow-up) ensuring that summary is display when searching on all types of authorities
This patch corrects a small bug :

Authorities search on all types does not show summary because it is computed
with selected type (which is empty) instead of using found authority type.

Test plan :
- Go to intranet autorities search
- Perform a search on all authorities types
- Look at results
  => Without this patch, results rows do not display the autority summary, only
     authorized headings

Signed-off-by: Frédéric Demians <f.demians@tamil.fr>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2014-05-05 01:10:47 +00:00
Fridolyn SOMERS
99dbc5cdf7 Bug 7442: fix crash when selecting an authority with 200$x or 200$y (UNIMARC)
From a biblio record, if one wants to add a 600$a information, a pop-up
appears.  On this new window, on search terms typed and validated, a table
result is displayed, with a column "Get It!" allowing the selection of an
authority.  From here, different cases:

1) If we have a simple authority with 200$a and 200$b subfields, a link
   "choose" is displayed, working correctly.

2) If the authority has different occurences of 200$a/200$b, numeric links (1 2
   and so on) are displayed, one for each occurence.  In the example of my
   screenshot, the line with a "Paul, Korky -- Pauline, Korkette" summary
   possesses two links : "1" will add "Paul, Korky" whereas "2" will add
   "Pauline, Korkette" (couldn't come up with a better name ;)).

3) If the authority has 200$x or 200$y subfields defined, several links are
   also created, when it should not be the case.  In our example, "Niclausse,
   Paul -- Expositions" will create a link "1" for "Niclausse, Paul" and a link
   "2" for "Expositions".  Clicking on the 2nd link leads to the following
   error: Software error: Can't call method "subfields" on an undefined value
   at
   /home/asaurat/workspace/versions/community/authorities/blinddetail-biblio-search.pl
   line 86.  Only the cases 1 and 2 should be handled. The creation of links
   for subfields like 200$x or 200$y should be removed.

This problem is caused by the use of " -- " has separator of authorities with
several headings, but also in some heading between main part and subdivisions.
This patch corrects this by using an array in authorities summary so that
presentation is computed in template. I've choosen to use the pipe separator
between authorities with several headings. This may be changed to be
configurable.

Test plan :

- Edit an authority type summary : for example subject (heading on 250) :
  summary "[250a][ -- 250x]"
- Create an authority A1 with one heading and a subdivision : for example a
  subject : 250$a "History" 250$x "20th century"
- Create an authority A2 with several headings. for example a subject : 250$a
  "History" 250$a "Legends"
- Rebuild Zebra queue
- Go to OPAC and click on "Authority search" and search on "History"
  => You will find A1 and A2 :
    History -- 20th century
    History | Legends
- Go to intranet autorities search and search on "History"
  => You will find A1 and A2 :
    History -- 20th century
    History | Legends
- Edit a record using this autorities type as thesaurus : for example on 606$a
- Click on thesaurus link and search on "History"
  => You will find A1 and A2 :
    History -- 20th century ; 0 times ; choose ; Edit authority
    History | Legends       ; 0 times ; 1 2    ; Edit authority
- Click on link "2" to chosse "Legends"
  => You get "Legends" in heading field : for example 606$a

Signed-off-by: Frédéric Demians <f.demians@tamil.fr>

I can confirm the problem and the solution. I have tested the patch on a large
DB with authorities having multiples headings. There is no regression on bug
4838.

Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Passes all tests and QA script.
Without the patch I couldn't choose between multiple headings
in the authority plugin, but with the patch it works as described.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2014-05-05 01:09:29 +00:00
Galen Charlton
cc7a99ba7f Bug 11797: (follow-up) fix MARC21 branch of bug more idiomatically
This patch puts the MARC21 variant of the bugfix in alignment
with the UNIMARC variant, removing the use of unnecessary
temporary variables.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2014-04-15 16:01:31 +00:00
Jonathan Druart
fa989631f6 Bug 11797: fix odd number of elements in hash (UNIMARC)
I got the same warning for my UNIMARC DB.

Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2014-04-15 16:01:31 +00:00
Mark Tompsett
129bc32811 Bug 11797: fix odd number of elements in hash warning (MARC21)
This was discovered when someone triggered an authority search
on an authority record that was missing what is assumed the
default subfield for a given field.

It, however, also can be triggered in an OPAC authority search
by looking at the record that lacks the default subfield for a
given field.

TEST PLAN
---------
1) Create an authority record with 180$x and NOT 180$v.
   See C4::AuthoritiesMarc::BuildSummary in the 1.. foreach loop
   for known tags and default values. The default subfields are
   the first letter of the $subfields_to_report string.
2) Trigger the bug:
   Method 1: /cgi-bin/koha/opac-authoritiesdetail.pl?authid=#
             Where # is the authority id of your tweaked record.
             The error occurs in Normal view.
   Method 2: Home -> Cataloging -> + New record
                  -> Click the 'Tag Editor' on 100$a
                  Editing of $a to $b and back may be required.
3) Notice there is an error log entry.
4) Apply the patch
5) Attempt to trigger the bug again
6) That specific error log entry is not generated.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Could generate the warning with a missing 151$a with both methods.
No warning anymore after applying this patch.

Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2014-04-15 16:01:31 +00:00
daf2ebc4f5 Bug 11096: support the retrieval of large MARCXML records
This patch makes Koha <-> Zebra use MARCXML for the serialization when
using DOM, and USMARC for GRS-1.

* The following functions are modified to set the Zebra record syntax
according to the current sysprefs and configuration:

- C4::Context->Zconn
- C4::Context-_new_Zconn

* A new function 'new_record_from_zebra' is introduced, which checks the
context we are in, and creates the MARC::Record object using the right
constructor.

The following packages get touched to make use of the new function:
- C4::Search
- C4::AuthoritiesMarc

and the same happens to the UI scripts that make use of them (both in
the OPAC and STAFF interfaces).

* Calls to the unsafe ZOOM::Record->render()[1] method are removed.

Due to this last change the code for building facets was rewritten. And
for performance on the facets creation I pushed higher version
dependencies for MARC::File::XML and MARC::Record (we rely on
MARC::Field->as_string).

* Calls to MARC::Record->new_from_xml and MARC::Record->new_from_usmarc
are wrapped with eval for catching problems [2].

* As of bug 3087, UNIMARC uses the 'unimarc' record syntax. this case is
  correctly handled.
* As of bug 7818 misc/migration_tools/rebuild_zebra.pl behaves like:

- bib_index_mode (defaults to 'grs1' if not specified)
- auth_index_mode (defaults to 'dom')

here we do exactly the same.

To test:
 - prove t/db_dependent/Search.t should pass.
 - Searching should remain functional.
 - Indexing and searching for a big record should work (that's what the
   unit tests do).
 - Test an index scan search (on the staff interface):
    Search > More options > Check "Scan indexes".
 - Enable 'itemBarcodeFallbackSearch' and try to circulate any word, it
   shouldn't break.
 - Searching for a biblio in a new subscription shouldn't break.
 - Running bulkmarcimport.pl shouldn't break.
 - And so on... for the rest of the .pl files.

[1] http://search.cpan.org/~mirk/Net-Z3950-ZOOM/lib/ZOOM.pod#render()
[2] a record that cannot be parsed by MARC::Record is simply skipped (bug 10684)

Sponsored-by: Universidad Nacional de Cordoba
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2014-02-28 19:50:09 +00:00
Galen Charlton
27c312b721 Bug 11533: fix authority searching with no sorting when QueryParser is enabled
This patch fixes an issue where chosing 'None' as the sort order
for an authority search would result in zero hits if QueryParser is
eanbled.

This patch also adds some additional test cases.

To test:

[1] Enable QueryParser.
[2] Perform an authority search in the staff interface that
    uses 'Heading A-Z' as the sort order and returns hits.
[3] Run the same search, but with the sort order set to 'None'.
    No hits are returned.
[4] Apply the patch.
[5] Do step 3 again.  This time, hits should be returned.
[6] Verify that prove -v t/db_dependent/Search.t passes.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2014-01-24 14:02:48 +00:00
5885077fbb Bug 11473 - add 'biblio' and 'item' to cataloguing logging info
This patch adds the words 'biblio' and 'item' to the 'info'
of the cataloguing logs which were missing them (such as biblio
delete, biblio mod, item mod, upload cover image).

This patch also adds 'authority' for authority mod.

_TEST PLAN_

Before applying:
1) Create/view mods for items, biblios, and authorities.
2) Create/view biblio deletion
3) Create/view upload cover image log
4) Note that none of these contain the words 'biblio','item',or
'authority' in their "Info" columns.

Apply patch.

5) Repeat steps 1-3
6) Note that the new logs contain 'biblio','item', and 'authority'
in their "Info" column, while the past ones don't.
7) Note also that 'biblio' and 'item' will have 'Biblio' and 'Item'
appear in their "Object" column for the new logs

Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2014-01-17 05:42:44 +00:00
Jonathan Druart
e5cec32d07 Bug 11131: display authority summary correctly when searching by all authority types
The summary is built using the authtypecode selected from the interface.
So when a search is launch on all auth types, the summary is not
correctly built by the BuildSummary routine.

It should get the authtypecode from the authority (call to
GetAuthTypeCode).

To test:
1/ go to authorities/authorities-home.pl
2/ search <something> by authtype personal name
3/ results are displayed with summary
4/ now select the default entry and search again the
results display but without the summary
5/ apply the patch
6/ search default again, now summary shows

Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Tested with a UNIMARC database, works as described.
All tests and QA script pass.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-12-30 19:11:20 +00:00
Jonathan Druart
e273e11b08 Bug 10602: Set default value for authority fields via the framework
This patch allows to define default values in the authorities framework.

Some code already existed but the feature did not work.

Test plan:
1/ Choose a framework, field and subfields.
2/ Define a default value.
3/ Create a new authority and check that the subfield is
automatically filled with the default value.

Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Work as described. koha-qa reports some tabs, fixed in followup

Test
1) Apply patch, run updatedatabase.pl
2) Edit auth framework, put default value someware, save
3) Add new auth, default value present

Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Passes all tests and QA script.
Verified database update is done correctly.

Controlfields 0xx
- Edited an existing field (001)
- Set a default value for subfield @
- Edited subfield again, checking default was saved correctly
- Verified the default shows up correctly when creating a
  new authority using this authority type

Fields
- Edited an existing field (100)
- Set a default value for subfield e
- Edited subfield again, checking default was saved correctly
- Verified the default shows up correctly when creating a
  new authority using this authority type

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-10-10 23:25:47 +00:00
Fridolyn SOMERS
95c7bb4adf Bug 9282: improve auto-completion for authority record finder
This patch adjusts the auto-completion on the authority record
finder (accessed from the bib editor) so that if you do
start typing in the "Main entry ($a only)" input field, it will
return only the $a of the main heading for matching authority
records.

This fixes a problem where typing "shakes", choosing
"Shakespeare, William, 1564-1616" from the auto-completion
result list, then hitting the search button fails to bring
up results, as the dates come from the $d of the 100 field
(in MARC21).

Signed-off-by: Mathieu Saby <mathieu.saby@univ-rennes2.fr>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Works as advertised.
Tested with an authority where I added my search term in $b.
The modified authority came up in main entry, not in mainmainentry.
That was the desired result.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-10-10 22:23:09 +00:00
Janusz Kaczmarek
68c44093a4 Bug 10335: display translated forms of headings in UNIMARC authorities correctly
To reproduce and test:

To reproduce:
1) Create an authority record with main heading (100) in Latin script
   (e.g. Oppenheimer, Aharon -- subfields $a and $b) and parallel form
   (700) in Hebrew (אופנהיימר, אהרן -- subfields $a and $b).
    Mark it correctly in $8 with freheb (or engheb if you like);
2) Reindex and search;
3) You will see:

Oppenheimer Aharon
freheb: אופנהיימר

Whereas you would rather like to see (mind language and lack of $b above):

Oppenheimer, Aharon
Hebrew: אופנהיימר, אהרן

The patch corrects the issue and should not harm those who (improperly)
put only one triple in $8

Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Work as described. No koha-qa errors.
Same result on OPAC and STAFF

Turns out that test plan is wrong,
you neet to fill tag 200ab, not 100ab, for main heading.
I filled 100a with some example data from UNIMARC auth manual.

Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Took me a bit to figure it out, works according to test plan.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-10-10 19:14:44 +00:00
Stéphane Delaune
cefa7c21e2 Bug 5635: bulkmarcimport new parameters & features
See the script's documentation for more details

New parameters are:
 - authtypes
 - filter
 - insert
 - update
 - all

Signed-off-by: Pascale Nalon <pascale.nalon@gmail.com>
This patch is live in Mines ParisTech since 2012-07-24.
Signing off

Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
 - Moved the sign-off from bugzilla to the commit message.
 - All tests and QA script pass.
 - Amended commit message to list new parameters.
 - Verified this patch works on a UNIMARC installation.
 - Verified normal import still works correct on a MARC21
   installation.
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
2013-03-21 20:21:54 -04:00
Paul Poulain
66b5b75d2c Bug 7440 follow-up removing forgotten NoZebra references
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
2013-03-19 21:17:05 -04:00
4dcee58a4d Bug 7440 - Remove NoZebra vestiges
Removed NoZebra vestiges. This comprises several code blocks that depend on the NoZebra syspref and NZ related functions/methods.

C4::Biblio->
 GetNoZebraIndexes
 _DelBiblioNoZebra
 _AddBiblioNoZebra

C4::Search->
 NZgetRecords
 NZanalyse
 NZoperatorAND
 NZoperatorOR
 NZoperatorNOT
 NZorder

C4::Installer->
 set_indexing_engine

Sponsored-by: Universidad Nacional de Córdoba
Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>

Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
2013-03-19 21:17:04 -04:00
Jared Camins-Esakov
8dcec411a4 Bug 9239 QA follow-up: amend error handling
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
2013-03-16 21:32:35 -04:00
Jared Camins-Esakov
c716c28fa6 Bug 9239 QA follow-up: better error handling
* Fix a long-standing bug in the linker that could crash the linker when
run against odd data.
* Sanitize input to SimpleSearch.
* Correctly handle CCL indexes with QP.

Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
2013-03-16 21:32:34 -04:00
Jared Camins-Esakov
376c55dc4e Bug 9239 QA follow-up: remove stray debug code
Remove a line of debug code from EG, provide better error handling
when presented with weird data in the authority linker, and correct
queryparser configuration to reflect the correct configuration for
Zebra.

Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
2013-03-16 21:32:34 -04:00
Jared Camins-Esakov
144c7f4e4e Bug 9239: Allow the use of QueryParser for all queries
With the inclusion of this patch, all searches will (try) to use
QueryParser for handling queries for both the bibliographic and authority
databases if UseQueryParser is enabled. If QueryParser is unavailable,
UseQueryParser is disabled, or the search uses CCL indexes, the old
search code will be used.

To test:
1) Apply patch.
2) Run the unit test with `prove t/QueryParser.t`
3) Enable the UseQueryParser syspref.
4) Try searches that should return results in the following places:
   * OPAC (simple search)
   * OPAC (advanced search)
   * OPAC (authorities)
   * Staff client (header search)
   * Staff client (advanced search)
   * Staff client (cataloging search)
   * Staff client (authorities)
   * Staff client (importing a batch using a match point)
   * Staff client (searching for an item for adding to a label)
   * Staff client (acquisitions)
   * Staff client (searching for a record to create a serial)
   * ANYWHERE ELSE I HAVE FORGOTTEN
5) Disable the UseQueryParser syspref. Repeat at least some of the
   searches you did above.
6) If all searches worked, sign off.

Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
Signed-off-by: Elliott Davis <elliott@bywatersolions.com>
Searching still works as expected for variuos places.
QueryParser syspref seemed to be enabled by default

Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
2013-03-16 21:32:32 -04:00
Vitor FERNANDES
2b5067b0f9 BUG 9145: Authorities: standard language for UNIMARC
New Authority system preference created UNIMARCAuthorityField100 with default value "afrey50      ba0".
AuthoritiesMarc.pm uses the system preference instead of label "afrey50      ba0".

Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Works as described, all tests and QA script pass.

1) Check system preference is created correctly.
2) Change pref.
3) Catalog a new authority.
4) Change framework to make 100 show in frameworks or check in the
   the database, that now 100 has the new defined value.
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
2013-02-20 08:42:10 -05:00
Fridolyn SOMERS
6e62f58015 Bug 9123: Authorities search ordered by authid does not work
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Tested with Zebra, marc21, grs1.
Discovered that paging through auth search results does no longer work, but that is not related to these changes.

Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Tested with Zebra, marc21, dom.
All tests pass.
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
2012-12-02 16:28:12 -05:00
Jared Camins-Esakov
af02e22ce5 Bug 9070: [SIGNED-OFF] authority searches in auth_finder error out
When using authority records imported into Koha from elsewhere, you
can get an error like:
    Can't use string ("HASH(0xbc6c{30)") as a HASH ref while "strict refs" in use at /usr/share/koha/lib/C4/AuthoritiesMarc.pm line 363.
in authorities/auth_finder.pl. This patch fixes that error.

To test:
1) You will need records imported from elsewhere.
2) Use the authority control plugin in a bib record to search for one of
   those headings.
3) Observe you get a nasty error.
4) Apply patch.
5) Repeat step 2.
6) Observe the error is gone.
7) Sign off.

Signed-off-by: Magnus Enger <magnus@enger.priv.no>
Works as advertised. No warning about "defined(%hash) is deprecated"
under perl v5.10.1.

Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
2012-11-20 14:09:02 -05:00
Jared Camins-Esakov
ae977a9cfd Bug 8914: Blank search that returns invalid authorities fails under Plack
When doing a search with no terms for the default authority type under
Plack, you will get a segfault if it returns any records that do not
have any authority type. As far as I can tell, this cannot be reproduced
under CGI/Apache.

To test:
Confirm that you don't get an error if you click the Submit button in
the authorities sections in the staff client and OPAC without entering
a term or changing the authority type.

You may or may not be able to reproduce the problem prior to applying
the patch, but that is the only situation in which I managed to trigger
it.

Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
2012-10-25 18:41:46 +02:00
Jared Camins-Esakov
cc9b23d3d7 Bug 8905: Error when accessing invalid authority
The GetAuthority shim does not check that authority retrieval was
successful, and therefore can call ->record on an undefined value.

This can be reproduced by trying to load an invalid authority record
using bulkmarcimport. There may be other ways to trigger it, but I'm
not sure what they are.

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
another way to trigger this error is just forging your URL with a wrong
authid !
2012-10-25 15:06:37 +02:00
492dd7dc7f Bug 8870 UNIMARC authorities search doesn't display information properly
- Note field (3xx) are not displayed on search result page. We get
  'HASH()' text.
- Parallel/Other forms (7xx) are not displayed, with language name in front of
  heading, both on result and detail page.
- Note are not displayed.
- On result page, seealso form are displayed, but end with a superfluous '--'
- Style the result page, with condensed block, and space between them.
- Done both on OPAC/staff

To be applied on 3.8.x after bug 8523.

Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
2012-10-22 17:50:06 +02:00
Paul Poulain
af96a8ead3 Bug 8523 follow-up fixing perlcritic 2012-09-21 16:30:36 +02:00
Jared Camins-Esakov
83370c519c Bug 8523: Display auth hierarchies w/all marcflavours
This commit adds support for displaying authority hierarchies for all
flavours of MARC, not just UNIMARC. Display now uses the jQuery
jstree plugin, selected with the help of Owen Leonard, resulting in a
much faster experience for users.

Be aware that the jstree file uses tabs rather than 4-space indentation,
which I left as-is so as to make it easier to integrate upstream
releases in the future.

To test:
1) Enable the AuthDisplayHierarchy syspref
2) Create authority records with a hierarchy of see also fields
   (in MARC21/NORMARC, you'll be using 5xx fields for this, with a
   subfield $w=g for broader terms and subfield $w=h for narrower
   terms)
3) View the authorities in the OPAC, noting the hierarchical view at
   the top of the page.

This initial patch does not create bidirection linkages from
unidirectional links in MARC21 authorities. This means that when moving
up the authority hierarchy, lower levels will disappear. This is
intentional, as the first patch is intended merely to ensure that
AuthDisplayHierarchy functions the same for all marcflavours. A future
patch will add a cron job to generate the bidirectional linkages, once
we are sure that the hierarchy functionality for UNIMARC and
MARC21/NORMARC coexists peaceably.

Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Resolved conflicts in updatedatabase.pl, sysprefs.sql and in one of
the CSS files.

Test plan:

1) Run t/AuthoritiesMarc.t
New tests complete without any errors.

2) Make sure updatedatabase works correctly.
Update works nicely, new system preference is also added to syspref.sql

3) Make sure new terms are translatable.
Created new po files for de-DE and checked for new terms.
All translations appear correctly.

4) Make sure everything works with AuthDisplayHieararchy OFF
- Add authority
- Edit authority
- Delete authority

5) Test feature with AuthDisplayHieararchy ON
- Add authority
- Edit authority
- Delete authority

6) Add a couple of hierarchically linked authorities
Note: links have to be created in both directions

Example:
151 $aGermany
  551 $a Baden-Württemberg $w h

151 $aBaden-Württemberg
  551 $a Konstanz $w h
  551 $a Germany $w g

151 $aKonstanz
  551 $a Baden-Württemberg $w g
  551 $a Fürstenberg $w h
  551 $a Paradies $w h

151 $a Fürstenberg
  551 $a Konstanz $w g

151 $a Paradies
  551 $a Konstanz $w g

Tree shows up nicely above the authority record
- in staff
- in OPAC
  - on the normal view tab
  - on the MARC view tab

7) Checking the logs for warnings
- no Javascript errors or warnings
- no warnings or errors in log files
2012-09-21 14:52:08 +02:00
Jared Camins-Esakov
7ad5e203da Bug 2060: Update command line MARC import scripts
Expose authority import functionality to the command line import
scripts, and rename them from commit_biblios_file.pl and
stage_biblios_file.pl to commit_file.pl and stage_file.pl.

To test (note that these instructions assume you have a MARC21
installation and are using the provided sample file):
1. Find a file of authorities (a sample file with MARC21 authorities
   is attached to bug 7475) and download it to your server
2. Stage the file using the following command (replace <filename> with
   the name of the file you saved in step 1):
   > misc/stage_file.pl --file <filename> --authorities
3. Note the batch number the script assigns to your batch
4. Commit the records using the following command (replace <batchnumber>
   with the batch number you made note of in step 3):
   > misc/commit_file.pl --batch-number <batchnumber>
5. Index the authorities Zebraqueue (or wait)
6. Confirm that the new authorities appear.
7. Create a matching rule with the following settings:
   Code: AUTHTEST
   Description: Personal name main entry
   Match threshold: 999
   Record type: Authority record
   Search index: Heading-main
   Score: 1000
   Tag: 100
   Subfields: a
   Offset: 0
   Length: 0
   (note the ID of this matching rule)
8. Stage the authority file again, this time using the following
   command:
   > misc/stage_file.pl --file <filename> --authorities \
     --match <matchingrule>
7. Revert the import with the following command:
   > misc/commit_file.pl --batch-number <batchnumber> --revert
8. Index the authorities Zebraqueue (or wait)
9. Confirm that the records have been removed
10. Import an authority record with the Stage MARC/Manage staged MARC
    tools in exactly the way you would for a bibliographic record,
    but choose "Authority" instead of "Bibliographic" for the record
    type.

Signed-off-by: Elliott Davis <elliott@bywatersolutions.com>

Testing plan delivers as it should.

Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Rebased on latest master 11 September 2012
2012-09-19 17:16:18 +02:00
Jared Camins-Esakov
6e71b80ca3 Bug 7475: Teach matching rules to handle authorities
* Add the code necessary to handle authorities with matching rules and
  import batches.
* Update all the scripts that use the matcher and import batch code
  to use the new API.
* Add authority records to the matching rules interface in the staff
  client.

http://bugs.koha-community.org/show_bug.cgi?id=2060
Signed-off-by: Elliott Davis <elliott@bywatersolutions.com>

Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Rebased on latest master 11 September 2012
2012-09-19 17:15:56 +02:00
Jared Camins-Esakov
ffc6730552 Bug 8743: ZOOM error when merging authority change to biblios
The merge fails and the following errors appear in the log:
[Sun Sep 02 23:38:10 2012] [error] [client 75.149.175.233] ZOOM error 25
"Specified element set name not valid for specified database" (addinfo:
"F") from diag-set 'Bib-1', referer:
http://****/cgi-bin/koha/authorities/authorities.pl?authid=3010
[Sun Sep 02 23:38:11 2012] [error] [client 75.149.175.233] Premature end of
script headers: authorities.pl, referer:
http://****/cgi-bin/koha/authorities/authorities.pl?authid=3010

Test plan:
1) Set dontmerge to "Do"
2) Find an authority that is linked to a bib record.
3) Edit the main entry of the authority.
4) Save the authority record.
5) Notice that you got a nasty error and the bib record was not updated.
6) Apply patch.
7) Edit the main entry of the authority again (you can just set it back
   to what it was to start with).
8) Save authority record.
9) Notice that there was no error, and the bib record was updated.
10) Sign off.

Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
2012-09-14 15:46:19 +02:00
Fridolyn SOMERS
2ca3663687 Bug 8071: link between bib and authorities with the authid
Do not automatically populate $9 in bibliographic headings when the
$9 is set in the authorized heading field of the authority record.

Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
2012-09-14 13:29:14 +02:00
Jared Camins-Esakov
4f043f3f77 Bug 3462: Links in authorities should be hyperlinks
Make see also links in both the OPAC and authority module search results
into hyperlinks and not just textual strings.

To test:
1. Do a search for an authority that will bring up a heading with a
   see also reference in the staff client and the OPAC.
2. Confirm that the see also references listed in the search results
   are now hyperlinks, which work.

Also quiets an unnecessary warning about an uninitialized value.

Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Rebased 26 July 2012

Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Rebased on master 1 August 2012

Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Rebased on master 6 August 2012

Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Rebased on master 11 September 2012

Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Patch works nicely and is a great improvement.

Tests:
- Checked links for existing and non existing "see alsos" give correct
  search results
- Verified links show up for all "see alsos" in a result list
- Verified links are properly linked with the correct names
- Checked logs don't show errors

Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
2012-09-13 16:53:50 +02:00
Jared Camins-Esakov
bc05b5d163 Bug 7417: Include see from references in bibliographic searches
This patch adds the Koha::Indexer::RecordNormalizer and
Koha::Indexer::MARC::RecordNormalizer::EmbedSeeFromHeadings packages
to enable the inclusion of alternate forms of headings in bibliographic
searches. When the new syspref IncludeSeeFromInSearches is turned on
(default is off) rebuild_zebra.pl will insert see from headings from
authority records into bibliographic records when indexing, so that a
search on an obsolete term will turn up relevant records.

To test:
1) Enable IncludeSeeFromInSearches
2) Add a heading that has an alternate form to a record (for example,
   "Cooking" has the alternate form "Cookery," if you have authority
   records from LC)
3) Index the zebraqueue (or reindex if you haven't indexed your system
   yet)
4) Confirm that if you search for "Cookery" you get the record you
   just modified

Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Rebased on master 5 August 2012
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Rebased on master 11 September 2012

Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>

Also checked:
- Verified database update works correctly
- Checked system preference and its description
- Checked staff/opac detail pages with feature on/off
- Checked staff/opac search facets
- Downloaded and tested records in various formats
- Tried different searches for 'see from' entries of authorities
- Ran all unit tests

No problems found.
2012-09-13 14:19:28 +02:00
Jared Camins-Esakov
8471158160 Bug 8209: "Did you mean?" from authorities
One of the ideas behind authority records is that users who search for
one term should have related terms (according to the authority file)
suggested to them. At the moment, Koha doesn't do that. Adding an
authority searching step to regular searches and displaying any
suggestions in a "Did you mean" bar at the top of the results would be
very useful.

This commit adds a Koha::SuggestionEngine class which is in charge of
getting suggestions from individual suggestion engine plugins, which
much be in the Koha::SuggestionEngine::Plugin::* namespace, and extend
Koha::SuggestionEngine::Base. Suggestions are loaded asynchronously
using AJAX, and a link to a page with suggestions is provided for users
with Javascript turned off.

The AuthorityFile suggestion engine plugin looks up the specified search
terms in the authority file and then suggests searches for records using
matching authorities.

Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Rebased 2 August 2012 and incorporated QA feedback

Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>

Test plan:
- Verified database update added system preference correctly,
  pref defaults to OFF
- Verified search results and detail pages in OPAC and staff
  still worked the same as before
  * for no results
  * with results
- Activated system preference and tested various searches
  * Searches from simple search
  * Searches from advanced search
  * Search links in records
- Deactivated Javascript - verified fallback works correctly

Notes:

- Suggested terms can include autorities with no linked records.
- When combining more than one search option using advanced search
  this results in "no suggestions" more often. Feature works best
  from simple search.

Overall great feature making use of authorities in a user friendly way!

Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Rebased on latest master 2012-09-10
Signed-off-by: wajasu <matted-34813@mypacks.net>
2012-09-13 11:34:28 +02:00
Jared Camins-Esakov
31f41e2c1d Bug 8206: Specify index in OPAC authority search
Adds a dropdown to allow the user to choose to search the 'mainentry,'
and 'match-heading'indexes, in addition to the 'any' index which is
the default.

To test (you will need authority records with see-from and
see-also headings):
1. Go to the browse subjects and authors section in the OPAC
2. Do a search for a fairly generic term that is used as a
   see-also term in a lot of records. Note that your search
   results include both the record that you wanted and all
   the other records that refer to it.
3. Apply the patch.
4. Repeat the search from step 2. Notice there is no change
   to the results.
5. Repeat the search from step 2, but choose "in main entry"
   and observe that you are now only seeing authority records
   with the search entered in their main entry.
6. Repeat the search from step 2, but choose "in any heading"
   and observe that the results are the same as in step 2. This
   is intentional, so that in the future notes in authorities
   can be made searchable.
7. Search for an obsolete form of heading with the "in keyword"
   option selected. Notice what results you get.

Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Works nicely. Hiding the new options can be done with CSS:

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
2012-09-07 15:16:40 +02:00
Fridolyn SOMERS
88e66b674e Bug 8576: Software error on authority edition when using merge
Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
2012-08-28 17:55:33 +02:00
Paul Poulain
98a207ef95 Merge remote-tracking branch 'origin/new/bug_6720' 2012-08-02 15:40:31 +02:00
Jared Camins-Esakov
983012d5b8 Bug 6720 follow-up: fix error when auth has been deleted
Thanks to Paul Poulain for finding the problem and proposing a solution.

Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
2012-08-02 15:40:06 +02:00
Jared Camins-Esakov
41df3e08d3 Bug 8204 follow-up: quiet warn
Also removes unnecessary variable declaration. Thanks to Jonathan Druart
for spotting it.

Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
2012-08-01 14:59:11 +02:00
Jared Camins-Esakov
255e1910ee Bug 8204: fix UNIMARC display
UNIMARC display of authorities in the OPAC wasn't working due to
changes in the BuildSummary function since the code was originally
developed.

Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
2012-07-26 18:13:39 +02:00
Jared Camins-Esakov
569e3823fd Bug 8204: Add user friendly authority view to OPAC
Up until now, the only authority view in the OPAC was a
rather-unfriendly expanded MARC view. This patch adds a user-friendly
view similar to the biblio details view.

Specific features to be aware of:
* Right-to-left text in the MARC21 880 field will show up in the
  appropriate location with the appropriate alignment and wrapping
* There is very little CSS styling. Any suggestions for how to make
  the display more attractive would be gratefully received.

To test:
1) Do a search for an authority in the OPAC.
2) Choose an authority record to view.
3) Observe that the view is more user-friendly and polished.

Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Works very nicely now. The old view is still available, but the
user is presented with a nicer non-MARC view first.
2012-07-26 18:11:44 +02:00
Jared Camins-Esakov
049d13e24d Bug 6720 follow-up: use authtypetext instead of summary
Although auth_types.summary contains a string describing the type of
authority in MARC21, it does not in UNIMARC. auth_types.authtypetext
contains the proper label in both marcflavours.

Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
2012-07-17 15:25:41 +02:00
Jared Camins-Esakov
ca7fa487c6 Bug 6720: Display the authority type in OPAC
Due to a peculiarity in the way SearchAuthorities worked, the authority
search results on the OPAC always listed the authority type that the
user searched for, rather than the authority type of the results.

Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Works nicely, result list for OPAC search now shows the authority type
for each record.

Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Rebased 20 June 2012.

Signed-off-by: Mirko Tietgen <5p4m@gmx.de>
Works as expected.
2012-07-17 15:25:39 +02:00
Jared Camins-Esakov
3aaaf2f2e4 Bug 7943: Authority search results are untranslatable
The HTML for authority search results was previously generated in
C4::AuthoritiesMarc::BuildSummary, which meant that it couldn't be
translated. This patch moves the HTML generation into the templates
by introducing a new authorities-search-results.inc include file for
both the OPAC and the Intranet which contains a Template::Toolkit BLOCK
for rendering the authority results. Fixes the authority autocomplete
by removing the untranslatable strings, and returning only data from
the database.

To test:
1. Apply patch.
2. Test authority searching in the authority module in the staff client
3. Test authority searching in the authority control plugin in the
   cataloguing module (and the plugin for UNIMARC field 210$c, if you
   can figure out how)
4. Test authority searching in the OPAC

Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Tested with MARC21 data.

1) Applies cleanly on current HEAD.

2) Authority search in staff

Patch works wonderfully, only some small notes found while testing that have
not been changed by this patch:

ENH note: Search terms show up nowhere. So if I want to change the sorting, I have to
repeat typing in my search term. Even if the form does not keep the term, it
should be visible somewhere on the screen what I searched for.

ENH note: The pagination on top and at the bottom of the result list are formatted
differently. Maybe some missing CSS?

ENH note: Also the authority type is not shown at all in the result list.

3) Cataloguing and authority plugins

The autocomplete function works nicely.

ENH note: There is only one small enhancement
I could imagine. If I start my search from 100 it will limit the search to
'persons' but the autocomplete will also suggest other authorities. It would
be a bit cleaner, if the autocomplete could limit by the appropriate authority
type too. Very nice feature.

Plugins overall work nicely. Created links include the authority numbers and work
correctly.

4) Authority search in OPAC

Works nicely. Display is consistent, but translatability greatly improved.

ENH note: In staff we use 'Details' in OPAC we use 'View full heading' - I wonder
if maybe 'details' would be better understandable for users?

Note: Code reveals a system preference 'AuthDisplayHierarchy' that is
not available in the system preference editor. I talked to Jared and he
will work on this feature later on. For now it's no regression, as the
preference has never been visible.
2012-06-20 10:07:30 +02:00
Chris Cormack
509d673f10 Bug 7941 : Fix version numbers in modules
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
2012-06-11 17:29:38 +02:00
Fridolyn SOMERS
4db3ffed91 BUG5312 XHTML correction in authority summary
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
2012-06-10 11:17:16 +02:00
Jared Camins-Esakov
937480abe0 Bug 7617: Sort authority results by authid
Add the option of sorting authority search results by authid, and instruct the
FirstMatch and LastMatch linkers to use that sort order rather than the default
search order.

To test:
1. Install new Zebra authorities config
    etc/zebradb/marc_defs/marc21/authorities/authority-koha-indexdefs.xml,
    etc/zebradb/marc_defs/marc21/authorities/authority-zebra-indexdefs.xsl,
    etc/zebradb/marc_defs/marc21/authorities/record.abs, and
    etc/zebradb/marc_defs/unimarc/authorities/record.abs
2. Reindex authorities in Zebra
3. Set LinkerModule to FirstMatch or LastMatch
4. Add two identical authority records, and a bib record with a heading that
   matches them
5. Run misc/link_bibs_to_authorities.pl on that record
6. Confirm that the authid that's been inserted into subfield $9 of that
   heading is the first, if you selected FirstMatch, or last if you selected
   LastMatch

Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
I followed the test plan and checked that for "Last match" and "First match"
the correct authority was selected and linked to the record.

Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
2012-03-29 11:04:58 +02:00
Janusz Kaczmarek
fe37dd8167 Bug 5657: Changes made to sub merge to eliminate errors occuring when updating bibio records in batch (after updating multiple authority records)
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
2012-03-21 11:13:17 +01:00
Jared Camins-Esakov
5207699f98 signed off Bug 7284: Authority matching improvements
Squashed patch incorporating all previous patches (there is no functional
change compared to the previous version of this patch, this patch merely
squashes the original patch and follow-up, and rebases on latest master).

=== TL;DR VERSION ===
*** Installation ***
1. Run installer/data/mysql/atomicupdate/bug_7284_authority_linking_pt1
and installer/data/mysql/atomicupdate/bug_7284_authority_linking_pt2
2. Make sure you copy the following files from kohaclone to koha-dev:
etc/zeradb/authorities/etc/bib1.att,
etc/zebradb/marc_defs/marc21/authorities/authority-koha-indexdefs.xml,
etc/zebradb/marc_defs/marc21/authorities/authority-zebra-indexdefs.xsl,
etc/zebradb/marc_defs/marc21/authorities/koha-indexdefs-to-zebra.xsl, and
etc/zebradb/marc_defs/unimarc/authorities/record.abs
3. Run misc/migration_tools/rebuild_zebra.pl -a -r

*** New sysprefs ***
* AutoCreateAuthorities
* CatalogModuleRelink
* LinkerModule
* LinkerOptions
* LinkerRelink
* LinkerKeepStale

*** Important notes ***
You must have rebuild_zebra processing the zebraqueue for bibs when testing this
patch.

=== DESCRIPTION ===

*** Cataloging module ***
* Added an additional box to the authority finder plugin for "Heading match,"
  which consults not just the main entry but also See-from and See-also-from
  headings.

* With this patch, the automatic authority linking will actually work properly
  in the cataloging module. As Owen pointed out while testing the patch,
  though, longtime users of Koha will not be expecting that. In keeping with
  the principles of least surprise and maximum configurability, a new syspref,
  CatalogModuleRelink makes it possible to disable authority relinking in the
  cataloging module only (i.e. leaving it enabled for future runs of
  link_bibs_to_authorities.pl).  Note that though the default behavior matches
  the current behavior of Koha, it does not match the intended behavior.
  Libraries that want the intended behavior rather than the current behavior
  will need to adjust the CatalogModuleRelink syspref.

*** misc/link_bibs_to_authorities.pl ***
Added the following options to the misc/link_bibs_to_authorities.pl script:
--auth-limit        Only process those headings that match the authorities
                    matching the user-specified WHERE clause.
--bib-limit         Only process those bib records that match the
                    user-specified WHERE clause.
--commit            Commit the results to the database after every N records
                    are processed.
--link-report       Display a report of all the headings that were processed.

Converted misc/link_bibs_to_authorities.pl to use POD.

Added a detailed report of headings that linked, did not link, and linked
in a "fuzzy" fashion (the exact semantics of fuzzy are up to the individual
linker modules) during the run.

*** C4::Linker ***
Implemented new C4::Linker functionality to make it possible to easily add
custom authority linker algorithms. Currently available linker options are:
* Default: retains the current behavior of only creating links when there is
  an exact match to one and only one authority record; if the 'broader_headings'
  option is enabled, it will try to link to headings to authority records for
  broader headings by removing subfields from the end of the heading (NOTE:
  test the results before enabling broader_headings in a production system
  because its usefulness is very much dependent on individual sites' authority
  files)
* First Match: based on Default, creates a link to the *first* authority
  record that matches a given heading, even if there is more than one
  authority record that matches
* Last Match: based on Default, creates a link to the *last* authority
  record that matches a given heading, even if there is more than one record
  that matches

The API for linker modules is very simple. All modules should implement the
following two functions:
<get_link ($field)> - return the authid for the authority that should be
linked to the provided MARC::Field object, and a boolean to indicate whether
the match is "fuzzy" (the semantics of "fuzzy" are up to the individual plugin).
In order to handle authority limits, get_link should always end with:
    return $self->SUPER::_handle_auth_limit($authid), $fuzzy;

<flip_heading ($field)> - return a MARC::Field object with the heading flipped
to the preferred form. At present this routine is not used, and can be a stub.

Made the linking functionality use the SearchAuthorities in C4::AuthoritiesMarc
rather than SimpleSearch in C4::Search. Once C4::Search has been refactored,
SearchAuthorities should be rewritten to simply call into C4::Search. However,
at this time C4::Search cannot handle authority searching. Also fixed numerous
performance issues in SearchAuthorities and the Linker script:
* Correctly destroy ZOOM recordsets in SearchAuthorities when finished. If left
  undestroyed, efficiency appears to approach O(log n^n)
* Add an optional $skipmetadata flag to SearchAuthorities that can be used to
  avoid additional calls into Zebra when all that is wanted are authority
  records and not statistics about their use

*** New sysprefs ***
* AutoCreateAuthorities - When this and BiblioAddsAuthorities are both turned
  on, automatically create authority records for headings that don't have
  any authority link when cataloging. When BiblioAddsAuthorities is on and
  AutoCreateAuthorities is turned off, do not automatically generate authority
  records, but allow the user to enter headings that don't match an existing
  authority. When BiblioAddsAuthorities is off, this has no effect.
* CatalogModuleRelink - when turned on, the automatic linker will relink
  headings when a record is saved in the cataloging module when LinkerRelink
  is turned on, even if the headings were manually linked to a different
  authority by the cataloger. When turned off (the default), the automatic
  linker will not relink any headings that have already been linked when a
  record is saved.
* LinkerModule - Chooses which linker module to use for matching headings
  (current options are as described above in the section on linker options:
  "Default," "FirstMatch," and "LastMatch")
* LinkerOptions - A pipe-separated list of options to set for the authority
  linker (at the moment, the only option available is "broader_headings," which
  is described below)
* LinkerRelink - When turned on, the linker will confirm the links for headings
  that have previously been linked to an authority record when it runs. When
  turned off, any heading with an existing link will be ignored.
* LinkerKeepStale - When turned on, the linker will never *delete* a link to an
  authority record, though, depending on the value of LinkerRelink, it may
  change the link.

*** Other changes ***
* Cleaned up authorities code by removing unused functions and adding
  unimplemented functions and added some unit tests.

* This patch also modifies the authority indexing to remove trailing punctuation
  from Match indexes.

* Replace the old BiblioAddAuthorities subroutines with calls into the new
  C4::Linker routines.

* Add a simple implementation for C4::Heading::UNIMARC. (With thanks to F.
  Demians, 2011.01.09) Correct C4::Heading::UNIMARC class loading. Create
  biblio tag to authority types data structure at initialization rather than
  querying DB.

* Ran perltidy on all changed code.

*** Linker Options ***
Enter "broader_headings" in LinkerOptions. With this option, the linker will
try to match the following heading as follows:
=600  10$aCamins-Esakov, Jared$xCoin collections$vCatalogs$vEarly works to
1800.

First: Camins-Esakov, Jared--Coin collections--Catalogs--Early works to 1800
Next: Camins-Esakov, Jared--Coin collections--Catalogs
Next: Camins-Esakov, Jared--Coin collections
Next: Camins-Esakov, Jared (matches! if a previous attempt had matched, it
would not have tried this)

This is probably relevant only to MARC21 and LCSH, but could potentially be of
great use to libraries that make heavy use of floating subdivisions.

=== TESTING PLAN ===

Note: all of these tests require that you have some authority records,
preferably for headings that actually appear in your bibliographic data. At
least one authority record must contain a "see from" reference (remember which
one contains this, as you'll need it for some of the tests). The number shown
in the "Used in" column in the authority module is populated using Zebra
searches of the bibliographic database, so you *must* have
rebuild_zebra.pl -b -z [-x] running in cron, or manually run it after running
the linker.

*** Testing the Heading match in the cataloging plugin ***
1.  Create a new record, and open the cataloging plugin for an
    authority-controlled field.
2.  Search for an authority by entering the "see from" term in the Heading Match
    box
3.  Confirm that the appropriate heading shows up
4.  Search for an authority by entering the preferred heading into the Main
    entry or Main entry ($a only) box (i.e., repeat the procedure you usually
    use for cataloging, whatever that may be)
5.  Confirm that the appropriate heading shows up

*** Testing the cataloging interface ***
6.  Turn off BiblioAddsAuthorities
7.  Confirm that you cannot enter text directly in an authority-controlled field
8.  Confirm that if you search for a heading using the authority control plugin
    the heading is inserted (note, however, that this patch does not AND IS NOT
    INTENDED TO fix the bugs in the authority plugin with duplicate subfields;
    those are wholly out of scope- this check is for regressions)
9.  Turn on BiblioAddsAuthorities and AutoCreateAuthorities
10. Confirm that you can enter text directly into an authority-controlled field,
    and if you enter a heading that doesn't currently have an authority record,
    an authority record stub is automatically created, and the heading you
    entered linked
11. Confirm that if you enter a heading with only a subfield $a that fully
    *matches* an existing heading (i.e. the existing heading has only
    subfield $a populated), the authid for that heading is inserted into
    subfield $9
12. Confirm that if you enter a heading with multiple subfields that *matches*
    an existing heading, the authid for that heading is inserted into
    subfield $9
13. Turn on BiblioAddsAuthorities and turn off AutoCreateAuthorities
14. Confirm that you can enter text directly into an authority-controlled field,
    and if you enter a heading that doesn't currently have an authority record,
    an authority record stub is *not* created
15. Confirm that if you enter a heading with only a subfield $a that *matches*
    an existing heading, the authid for that heading is inserted into
    subfield $9
16. Confirm that if you enter a heading with multiple subfields that *matches*
    an existing heading, the authid for that heading is inserted into
    subfield $9
17. Create a record and link an authority record to an authorized field using
    the authority plugin.
18. Save the record. Ensure that the heading is linked to the appropriate
    authority.
19. Open the record. Change the heading manually to something else, leaving
    the link. Save the record.
20. Ensure that the heading remains linked to that same authority.
21. Change CatalogModuleRelink to "on."
22. Open the record. Use the authority plugin to link that heading to the
    same authority record you did earlier.
23. Save the record. Ensure that the heading is linked to the appropriate
    authority.
24. Open the record. Change the heading manually to something else, leaving
    the link. Save the record.
25. Ensure that the heading is no longer linked to the old authority record.

*** Testing link_bibs_to_authorities.pl ***
26. Set LinkerModule to "Default," turn on LinkerRelink and
    BiblioAddsAuthorities, and turn AutoCreateAuthorities and
    LinkerKeepStale off
27. Edit one bib record so that an authority controlled field that has already
    been linked (i.e. has data in $9) has a heading that does not match any
    authority record in your database
28. Run misc/link_bibs_to_authorities.pl --link-report --verbose --test (you may
    want to pipe the output into less or a file, as the result is quite a lot of
    information)
29. Look over the report to see if the headings that you have authority records
    for report being matched, that the heading you modified in step 2 is
    reported as "unlinked," and confirm that no changes were actually made to
    the database (to check this, look at the bib record you edited earlier, and
    check that the authid in the field you edited hasn't changed)
30. Run misc/link_bibs_to_authorities.pl --link-report --verbose (you may want
    to pipe the output into less or a file, as the result is quite a lot of
    information)
31. Check that the heading you modified has been unlinked
32. Change the modified heading back to whatever it was, but don't use the
    authority control plugin to populate $9
33. Run misc/link_bibs_to_authorities.pl --link-report --verbose
    --bib-limit="biblionumber=${BIB}" (replacing ${BIB} with the biblionumber
    of the record you've been editing)
34. Confirm that the heading has been linked to the correct authority record
35. Turn LinkerKeepStale on
36. Change that heading to something else
37. Run misc/link_bibs_to_authorities.pl --link-report --verbose
    --bib-limit="biblionumber=${BIB}" (replacing ${BIB} with the biblionumber
    of the record you've been editing)
38. Confirm that the $9 has not changed
39. Turn LinkerKeepStale off
40. Create two authorities with the same heading
41. Run misc/migration_tools/rebuild_zebra.pl -a -z
42. Enter that heading into the bibliographic record you are working with
43. Run misc/link_bibs_to_authorities.pl --link-report --verbose
    --bib-limit="biblionumber=${BIB}" (replacing ${BIB} with the biblionumber
    of the record you've been editing)
44. Confirm that the heading has not been linked
45. Change LinkerModule to "FirstMatch"
46. Run misc/link_bibs_to_authorities.pl --link-report --verbose
    --bib-limit="biblionumber=${BIB}" (replacing ${BIB} with the biblionumber
    of the record you've been editing)
47. Confirm that the heading has been linked to the first authority record it
    matches
48. Change LinkerModule to "LastMatch"
49. Run misc/link_bibs_to_authorities.pl --link-report --verbose
    --bib-limit="biblionumber=${BIB}" (replacing ${BIB} with the biblionumber
    of the record you've been editing)
50. Confirm that the heading has been linked to the second authority record it
    matches
51. Run misc/link_bibs_to_authorities.pl --link-report --verbose
    --auth-limit="authid=${AUTH}" (replacing ${AUTH} with an authid)
52. Confirm that only that heading is displayed in the report, and only those
    bibs with that heading have been changed

If all those things worked, good news! You're ready to sign off on the patch
for bug 7284.

Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Rebased on latest master and squashed follow-up, 16 February 2012
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Rebased on latest master, 21 February 2012

Signed-off-by: schuster <dschust1@gmail.com>
2012-03-07 17:34:11 +01:00
Frédérick Capovilla
6408547cc0 Use MARCAuthorityControlField008 in AddAuthority for empty 008s
The old default value for field 008 didn't have the correct length,
which could potentially cause data corruptions.

Uses the MARCAuthorityControlField008 preference added by BUG4839.

http://bugs.koha-community.org/show_bug.cgi?id=7072
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
2011-12-27 17:57:42 +01:00
05d35b0ae0 6094 Fixing ModAuthority problems
Pref MergeAuthoritiesOnUpdate does not exist; should be dontmerge
(AuthoritiesMarc.pm).

Instead of folder modified_authorities, now introducing a table for this
purpose: need_merge_authorities. This eliminates several permissions and
security issues. This change applies to AuthoritiesMarc.pm and
merge_authority.pl.

POD lines added for ModAuthority. Deprecated parameter $merge removed.

Test this patch by applying the db revision first from the second patch.

August 4, 2011: Rebased.

Signed-off-by: Frédéric Demians <f.demians@tamil.fr>

Thanks Marcel. It works as advertised. Both modes are functionnal
(back):

- Immediate with dontmerge=0: After modifying an authority record, its
  linked biblios are immediately modified. This isn't the case in 3.4.5.
- Delayed with dontmerge=1: After modifying an authority record, its
  linked biblios are not modified. But an entry is added to
  need_merge_authority new table and 'merge_authorities.pl -b' script
  updates biblios.

Comment: need_merge_authority, like zebraqueue, should be cleared
from time to time.

Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
2011-10-20 11:28:53 +13:00
Frédérick Capovilla
94700d65c0 Bug 5737: Modified the merge subroutine in AuthoritiesMarc.pm so it won't destroy the connection to the biblioserver.
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Signed-off-by: Ian Walls <ian.walls@bywatersolutions.com>
2011-08-03 14:01:52 +12:00
Katrin Fischer
4d81858ec6 Bug 4433: Small display problem for topical subjects
This removes the subfield codes shown for topical subject authorities only.

To test:
- Add different authority records and at least one for a topical subject.
- Use search in intranet and opac (turn on opacauthorities) to serch for your subjects.
- All headings will show without the subfield codes (at least $a in front), except the
topical subject.

This patch removes the subfield codes for the topical headings.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Works as advertised in opac and staff. (July 18, 2011)

Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
2011-08-01 13:30:38 +12:00
Colin Campbell
d8b362e0f9 Bug 5415 Let calls of SimpleSearch utilize considtent interface
Remove some unnecessary checks when check of error is
sufficient. Make the order in some cases more logical
Should remove some possibilities of runtime warning noise.
Although some calls belong to the 'Nothing could
ever go wrong' school have added some warnings

Signed-off-by: Christophe Croullebois <christophe.croullebois@biblibre.com>
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
2011-04-08 13:52:57 +12:00
Chris Cormack
7d63594b81 Merge remote branch 'kc/new/bug_3072' into kcmaster 2011-04-08 11:22:57 +12:00
6e58529bdd Bug 3072: Search for Heading-Main in authorities
Includes Heading-Main in authorities bib1 attribute set.
Also includes Any in authorities bib1 (was still missing).
Also removes temporary fix in AuthoritiesMarc.pm.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Rebased after pushed fix for #5924.
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
2011-04-08 11:22:02 +12:00
Paul Poulain
c9def93286 Bug 5915 : C4::AuthoritiesMarc AddAuthority
Fixing 100$a simple error: Sometimes, language is encoded fre50 and not
frey50.
This patch adds check for that problem

Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
2011-04-07 16:33:50 +12:00
Julian Maurice
9dc9f73bfe Bug 5923 : Removed link from variable $summary
Variable $summary contained <a> and <b> tag. They have been deleted.

Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
2011-04-07 16:25:22 +12:00