Some file tidying to pass the QA tests
Signed-off-by: Aleisha Amohia <aleishaamohia@hotmail.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Before this patch if a record had a 751 and a 781 you could have fields repeated.
This patch reduces the scope of the fields to subdivision variable as it is only used in processing 7xx
fields and should not be shared between fields.
I also add unit tests
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Aleisha Amohia <aleishaamohia@hotmail.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Currently when searching for a geographic authority record the search will fail if the record has a heading in a 78X field. The system tries to do a regex match against an undefined variable causing an error. This patch makes that regex match conditional on the variable being defined to allow the search to succeed.
Test plan:
1) Navigate to Authorities
2) In the search bar at the top of the page, click on the dropdown options and in the 'Authority type' field, select 'Geographic Name'
3) Click search
4) You should have a list of authorities
5) Click on any authority record and then click edit and select to edit the record
6) Click on the "7" button
7) Click on the green text next to the '781' field to get the list of fields
8) In field 'v' enter any string you like
9) Click save
10) Repeat steps 1-3, this time it should display an error message for an Unmatched [ in regex
11) Apply patch
12) restart_all
13) Refresh the page, the results should show and the string you entered in the 'v' field should display on the record you edited
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Aleisha Amohia <aleishaamohia@hotmail.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
This patch updates the test files to correctly create authorities and/or correctly
mock the needed data for authorities.
the tests in t/db_dependent/Authority/Merge.t cover the case of 'Default' authorities
which don't have valid headings - so I had to add handling for blanking the heading field
when Default type is used.
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
This patch adds a new heading field containing the display form of the authority record
NOTE: If trying to save an auhority in the 'DEFAULT' framework, you will get an error, you should not be
using DEFAULT for authorities and we should remove from the list on another bug
To test:
1 - Apply patches
2 - Update database
3 - Restart all
4 - Create a new authority, save.
5 - Do this for various types
6 - View the db records:
SELECT * FROM auth_header\G
7 - Note new heading field is populated correctly
8 - Edit your new authorities
9 - Confirm the heading field is updated correctly
10 - Import some authorities and confirm heading generated correctly
11 - Import auth via Z39.50 and confirm heading generated correctly
Signed-off-by: Andrew Fuerste-Henry <andrewfh@dubcolib.org>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
If autolinking is enabled, there are scenarios where the authorities merge process
updates the record, then sends it to ModBiblio which then AutoLinks the record, undoing
the changes in the record (as the 'from' authority is not removed until the process is complete)
This patch simply disables the autolinking process when merging authorities
To test:
Set system preferences
AutoCreateAuthorities = don't generate
RequireChoosingExistingAuthority = don't require
AutoLinkBiblios = Do
CatalogModuleRelink = Do
LinkerKeepStale = Don't
LinkerModule = first match
LinkerRelink = Do
1. Create an authority record
1.1. Go to Authorities
1.2. Click on "New authority" and choose an authority type (I chose Corporate name)*
1.3. Fill in the mandatory fields (000, 003, 005, 008, 040)
1.4. In tab 1, click on the label of the main heading (110 for Corporate name)
1.5. Fill in subfield
1.6. Copy content of subfield
1.7. Click on "Save"
1.8. Note the authority record number
2. Duplicate the authority record
2.1. Click on "Edit" and choose "Edit as new (duplicate)"
2.2. Click on "Save"
2.3. Click on "No: Save as new authority"
2.4. Note the authority record number (should be the number from step 1.8 +1)
3. Link one of the authority records to a bibliographic record
3.1 In another tab, do a catalog search
3.2. Click on "Edit record" under one of the search results
3.3. Go to the relevant tab (for corporate name, I used field 710 in tab 7)
3.3.a. If there are no empty fields for that tag, click the "Repeat this tag" button (two rectangles)
3.4. Paste the text previously copied in subfield
3.5. Click on "Link authorities automatically" at the top of the page
3.6. Note the authority record number in subfield
--> The linked authority record should be the first authority record (step 1.8)
3.7. Click on "Save"
4. Merge the two authority records, keep the one that is not linked to the bibliographic record
4.1. Go back to the tab with the authorities
4.1. Go to Authorities
4.2. Search for the aurhorities previous created
4.2. Click on "Actions" and choose "Merge" for both records
4.3. Select the record number that is NOT linked to the bibliographic record
4.4. Click on "Next"
4.5. Click on "Merge"
5. Check the bibliographic record
5.1. Go back to the tab with the bibliographic record and refresh the page (Ctrl+F5)
5.2. Hover the mouse cursor over the name or term, without clicking
5.3. Note the record number that appears at the bottom of the screen
--> The record number was not changed
6. Apply the patch
6.1. Run prove t/AuthoritiesMarc_MARC21.t
6.2. Repeat Previous steps 1, 2, 3, 4, 5
--> The record number is change
Signed-off-by: Barbara Petritsch <barbara.petritsch@wienmuseum.at>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Test plan:
prove -v t/db_dependent/Authority/Merge.t
Signed-off-by: David Nind <david@davidnind.com>
Amended-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Replace $biblio->id with $biblio->biblionumber
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
We are calling ->field with an undef parameter.
MARC::Record does not like that and says:
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
For now, a fix at our side.
Test plan:
Run SearchAuthorities without passing an authtypecode.
Without this patch, it results in a warning per hit.
With this patch, the noise should be gone.
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
This patch replaces the uses of the static method with the new filter.
It also moves the tests to the relevant place and removes the (now)
unused method.
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
I have some trouble to get the tidyness test on
AuthoritiesMarc.pm passing. There are several outstanding
patches touching this file, so I didn't dare to perltidy
the whole file just yet.
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Instead of hardcoding the separator we take the value from systempreference.
Also adjust the unit test to expect the "equalterm" array.
Test plan same as before.
Signed-off-by: Christian Nelson <christian.nelson@uwasa.fi>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Currently when a MARC21 authority has 7XX field they do not display on
authority search result page. 7XX is equal term of 1XX based on LC rules.
Applying this patch will enable that feature, which is important for non latin
catalogues.
Test plan:
1) Add a new authority with 1XX and 7XX fields.
2) Try to search that authority and check if 7XX values are displayed
3) Apply this patch
4) Try to search that authority and you will see that 7XX values are displayed
like "1XX = 7XX"
Sponsored-by: Keratsini-Drapetsona Municipal Library, Greece
Mentored-by: Andreas Roussos
Signed-off-by: Frank Hansen <frank.hansen@ub.lu.se>
Works for me!
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
This enhancement adds a system preference StripWhitespaceChars which,
when enabled, will strip leading and trailing whitespace characters from
all fields when cataloguing both bibliographic records and authority
records. Whitespace characters that will be stripped are:
- spaces
- newlines
- carriage returns
- tabs
To test:
1. Apply patch and install database updates
2. Go to Administration, system preferences, find the new
StripWhitespaceChars preference. It should be "Don't strip" by default.
Change it to "Strip".
3. Search for a biblio record and edit it. Put some leading or trailing
whitespace characters in input fields and textarea fields and save.
4. Confirm these characters are removed when you save the record.
5. Repeat steps 3 and 4 for authority records.
6. Confirm tests pass t/db_dependent/Biblio/ModBiblioMarc.t
Sponsored-by: Educational Services Australia SCIS
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Bug 30358: (follow-up) Also strip inner newlines
This patch amends the StripWhitespaceChars system preference to also
strip inner newlines (line breaks and carriage returns) when enabled.
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Bug 30358: (follow-up) Inner newlines should be replaced with a space
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Bug 30358: (follow-up) Fixing tests and including for inner newlines
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Bug 30358: (follow-up) Clarify syspref wording about fields affected
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Bug 30358: (follow-up) Consider field has multiple subfields of same key
To test:
1) Click the clone subfield button to make multiple subfields with the
same key, i.e. 500$a$a$a
2) Save the record and confirm that the fields contain the correct data
after whitespaces are stripped.
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Bug 30358: (follow-up) Put multiple subfields fix on auth side
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Bug 30358: (follow-up) stripWhitespaceChars subroutine and tests
To test:
Confirm test plan above still works as expected and tests pass in
t/Koha_MetadataRecord.t
Sponsored-by: Catalyst IT
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Bug 30358: (follow-up) Fixing ModBiblioMarc.t tests
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Bug 30358: (follow-up) Do not strip whitespace from control fields
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Bug 30358: (follow-up) Simplify regex
The regex does the following:
1. Replace newlines and carriage returns with a space
2. Replace leading and trailing whitespace with nothing (strip)
Signed-off-by: Hammat Wele <hammat.wele@inlibro.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
C4::AuthoritiesMarc method GetAuthorizedHeading is not exported thus it is called in other modules :
> git grep GetAuthorizedHeading
C4/AuthoritiesMarc.pm:=head2 GetAuthorizedHeading
C4/AuthoritiesMarc.pm: $heading = &GetAuthorizedHeading({ record => $record, authid => $authid })
C4/AuthoritiesMarc.pm:sub GetAuthorizedHeading {
C4/Breeding.pm: $heading = C4::AuthoritiesMarc::GetAuthorizedHeading({ record => $marcrecord });
C4/ImportBatch.pm: $row->{'authorized_heading'} = C4::AuthoritiesMarc::GetAuthorizedHeading( { authid => $row->{'candidate_match_id'} } );
C4/ImportBatch.pm: my $authorized_heading = C4::AuthoritiesMarc::GetAuthorizedHeading({ record => $marc_record });
This patch adds it to be exported.
For example for use in Koha plugins.
Test plan :
Check import of authorities from a file is OK
Signed-off-by: Phil Ringnalda <phil@chetcolibrary.org>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
This patch joins authority queries together with OR like Elasticsearch.
To the best of the author's knowledge, this code only gets triggered
when using record matching rules, but there may be other situations
where it's used.
Authority search in cataloguing plugins use a different path, and
authority search in authority home doesn't allow multiple queries,
so they'll never trigger this OR joining of queries.
Test plan:
0. Apply patch and koha-plack --restart kohadev
1. Create a record matching rule with the following:
Matching rule code: NID
Description: NormdatenID
Match threshold: 100
Record type: Authority record
Search index: Any
Score: 101
Tag: 035
Subfields: a
Offset: 0
Length: 0
Normalization rule: None
Click "Remove this match check"
2. Stage gnd.xml (from BZ 20596 attachments)
- Choose "MARCXML" for "Format"
- Choose "Authority" for "Record type"
- Choose "NID" for "Record matching rule"
- Click "Stage for import"
3. Note the job gets to "100% Finished"
(Before this patch, it would say "0% Failed")
4. View the batch
5. Import the record
6. Repeat steps 2-4 and note that the record matching rule
worked
7. Go to http://localhost:8081/cgi-bin/koha/authorities/authorities-home.pl
8. Search for "e"
9. Note that the search works
10. Go to http://localhost:8081/cgi-bin/koha/cataloguing/addbiblio.pl?frameworkcode=
11. Click tab 6
12. Click on the tag editor next to "a Personal Name"
13. Type "e" into each box and click "Search"
14. Note that the search works
Signed-off-by: Jan Kissig <jan.kissig@th-wildau.de>
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
The test
$MARCfrom->field('1..', '2..'))[0]->tag ne ($MARCto->field('1..', '2..'))[0]->tag
is not completely consistent with following code in sub merge.
I decided to get the authtype code from the old record that comes
from Koha and should include the type.
Remaining changes refer to indentation/comments.
Test plan:
Run t/db_dependent/Authority/Merge.t
Note: The test actually could be extended a bit for mocking MARC
flavor, but needs some additional framework support to work.
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
In those rare cases when authorities are updated by an external agency (or
even internally, by reviewing and correcting an exported authority file)
when the heading tag will be changed (seems odd but happens:
111 Congress ==> 110 Corporate body.Congress ;
100 Person ==> 110 Corporate body (a company named with person's name ;
151 City--object ==> 150 Object (city) etc.)
and then the authority record in Koha database will be updated with
bulkmarcimport or by calling directly ModAuthority from a custom script,
the merge function "doesn't know" that the change to the authority type
has been made and, consequently, doesn't adequately change the tag in
related fields in biblio records (as it would if two different records
with different authtypecode were merged with Koha interface).
This is because at the moment when merge function is being called
by ModAuthority:
Koha::Authority::Types->find($autfrom->authtypecode)
Koha::Authority::Types->find($authto->authtypecode)
both have the same value (because $mergefrom == $mergeto).
Therefore in case when $mergefrom == $mergeto and the heading tag changes,
$authtypefrom and $authfrom calculated in merge in the ordinar way are
misleading and should not be taken unto consideration.
Test plan:
==========
1. run t/db_dependent/Authority/Merge.t
2. you should see problems in "Test update A with modified heading tag
(changing authtype)"
3. apply the patch
4. run t/db_dependent/Authority/Merge.t again
5. the test should pass
Alternatively:
1. have an authority record used in biblio;
export it to file;
change 1XX heading tag to a different (but reasonable) value
and possibly change also the content of the heading
(one can delete also 942 but it doesn't matter);
make bulkmarcimport.pl -a -update -file <modified_auth_file> and
see that the tag in biblio record has not been changed (whereas
the type of authority record did change);
2. make orders in database (so that the authority type and the tag of
the field in biblio record correspond); apply the patch;
3. repeat the test from 1.
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
This patch replaces the use of $dbh->do by corresponding use of
Koha::Authority object
Test plan:
1. Verify that creating/modifying authority still work
2. prove t/db_dependent/AuthoritiesMarc.t
Signed-off-by: Joonas Kylmälä <joonas.kylmala@iki.fi>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
To test:
0 - Have Koha using Elasticsearch
1 - Set QueryRegexEscapeOptions to 'unescape escaped'
2 - Attempt to add a new 'GENRE/FORM' authority record
3 - On save you get a 500 error
Unable to understand your search query, please rephrase and try again.
at /kohadevbox/koha/Koha/SearchEngine/Elasticsearch/Search.pm line 102
4 - Apply patch, restart all
5 - Attempt to add a new 'GENRE/FORM' authority record
6 - Success!
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
JD Amended patch:
-# FIXME Special case here
- print "Biblio not found\n,";
+ print "Biblio not found\n";
- my $biblio = Koha::Biblio->find($hostbiblionumber);
+ my $biblio = Koha::Biblios->find($hostbiblionumber);
Rebased-by: Joonas Kylmälä <joonas.kylmala@iki.fi>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
This patch adds "complete field" to the authority "starts with"
search so that it uses the untokenized "p" register.
Test plan:
1. Apply the patch
2. koha-plack --restart kohadev
3. Go to http://localhost:8081/cgi-bin/koha/authorities/authorities-home.pl
4. Type in "Espen" into the search box and hit "Submit"
5. Note that there are 3 results
6. Change "contains" to "starts with" and hit "Submit"
7. Note that no results are returned
8. Change the search from "Espen" to "Sandberg" and hit "Submit"
9. Note that 3 results are returned
10. Experiment to your heart's content and rejoice at your new found power
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Same as bug 30465 for authorities
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
Same as bug 30460 for authorities
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
This patch removes a trivial warning caused by a parameter being
compared to a literal but not being defined. Doing a
$ git grep GetTagsLabels
shows the change is safe as the function is always called with a value
of either 0 or 1.
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
Authority heading use is based on authority MARC 008/14-16. This could
be useful to show on authority search results, if new system preference
ShowHeadingUse is enabled.
To test:
1. Apply patches, update database, restart services
2. Go to Koha Administration -> system preferences -> searching tab.
Notice new ShowHeadingUse system preference is disabled. Leave it
disabled for now.
3. Do an authority search. Confirm it works as expected and no new
information is shown since ShowHeadingUse is disabled.
4. Do a biblio search and edit a record. Go to one of the authority
fields (1xx, 6xx or 7xx) and launch the authority finder plugin. Do an
authority search and confirm it works as expected and no new information
is shown since ShowHeadingUse is disabled.
5. Go back to system preferences and enable ShowHeadingUse syspref.
6. Repeat steps 3 and 4 but confirm the 'Heading use' column now shows, and
correctly displays what the heading can be used for based on 008/14-16.
Sponsored-by: Education Services Australia SCIS
Signed-off-by: Jessica <jzairo@bywatersolutions.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
2021-09-28 Updated version
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
On bug 17591 we discovered that there was something weird going on with
the way we export and use subroutines/modules.
This patch tries to standardize our EXPORT to use EXPORT_OK only.
That way we will need to explicitely define the subroutine we want to
use from a module.
This patch is a squashed version of:
Bug 17600: After export.pl
Bug 17600: After perlimport
Bug 17600: Manual changes
Bug 17600: Other manual changes after second perlimports run
Bug 17600: Fix tests
And a lot of other manual changes.
export.pl is a dirty script that can be found on bug 17600.
"perlimport" is:
git clone https://github.com/oalders/App-perlimports.git
cd App-perlimports/
cpanm --installdeps .
export PERL5LIB="$PERL5LIB:/kohadevbox/koha/App-perlimports/lib"
find . \( -name "*.pl" -o -name "*.pm" \) -exec perl App-perlimports/script/perlimports --inplace-edit --no-preserve-unused --filename {} \;
The ideas of this patch are to:
* use EXPORT_OK instead of EXPORT
* perltidy the EXPORT_OK list
* remove '&' before the subroutine names
* remove some uneeded use statements
* explicitely import the subroutines we need within the controllers or
modules
Note that the private subroutines (starting with _) should not be
exported (and not used from outside of the module except from tests).
EXPORT vs EXPORT_OK (from
https://www.thegeekstuff.com/2010/06/perl-exporter-examples/)
"""
Export allows to export the functions and variables of modules to user’s namespace using the standard import method. This way, we don’t need to create the objects for the modules to access it’s members.
@EXPORT and @EXPORT_OK are the two main variables used during export operation.
@EXPORT contains list of symbols (subroutines and variables) of the module to be exported into the caller namespace.
@EXPORT_OK does export of symbols on demand basis.
"""
If this patch caused a conflict with a patch you wrote prior to its
push:
* Make sure you are not reintroducing a "use" statement that has been
removed
* "$subroutine" is not exported by the C4::$MODULE module
means that you need to add the subroutine to the @EXPORT_OK list
* Bareword "$subroutine" not allowed while "strict subs"
means that you didn't imported the subroutine from the module:
- use $MODULE qw( $subroutine list );
You can also use the fully qualified namespace: C4::$MODULE::$subroutine
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
This makes the comparison between bibliographic field and authority field more robust and per subfield. This makes the comparison not consider the same e.g. the following fields:
$a Test User
$a Test $b User
The actual issue cannot be as easily reproduced with the patches for bug 21826 applied, but here's a test plan anyway:
1. Make sure tests pass (especially t/db_dependent/AuthoritiesMarc*)
2. Make sure authority linking still works properly
3. Make sure authority and biblio frameworks allow subfield i
4. Make sure that even if you add subfield i to 700 in biblio, authority link is kept the same
5. Make sure that even if you add subfield i to the authority record, the authority link is kept the same
Signed-off-by: Frank Hansen <frank.hansen@ub.lu.se>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Same patch for the authority frameworks/records
Sponsored-by: Orex Digital
Signed-off-by: Michal Denar <black23@gmail.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Sometimes you know that the merge is useless, since the reporting
tag did not change.
Might be handy in batch processing of authority records.
Test plan:
Pick an authority with a few linked biblio recs (so that it will
immediately merge; lower than MergeLimit).
Open authority record and save it.
Verify that a linked biblio record was merged. Check time in 005.
You proved that this patch does not break the regular process.
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Phil Ringnalda <phil@chetcolibrary.org>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
When batch editing, 2 reindex calls are sent to ES/Zebra.
We can easily avoid that reusing the skip_modzebra_update (renamed skip_record_index)
Additionally we should only send one request for biblio, and we should
only do it if we succeed
As the whole batch mod is in a transaction it is possible to fail in which case
Zebra queue is reset, but ES indexes have already been set
In addition to the skip param this patchset moves Zebra and Elasticsearch calls to
Indexer modules and introduces a generic Koha::SearchEngine::Indexer so that we don't
need to check the engine when calling for index
The new index_records routine takes an array so that we can reduce the calls to
the ES server.
The index_records routine for Zebra loops over ModZebra to avoid affecting current behaviour
Test plan:
General tests, under both search engines:
1 - Add a biblio and confirm it is searchable
2 - Edit the biblio and confirm changes are searchable
3 - Add an item, confirm it is searchable
4 - Delete an item, confirm it is not searchable
5 - Delete a biblio, confirm it is not searchable
6 - Add an authority and confirm it is searchable
7 - Delete an authority and confirm it is not searchable
Batch mod tests, under both search engines
1 - Have a bib with several items, none marked 'not for loan'
2 - Do a staff search that returns this biblio
3 - Items show as available
4 - Click on title to go to details page
5 - Edit->Item in a batch
6 - Set the not for loan status for all items
7 - Repeat your search
8 - Items show as not for loan
9 - Test batch deleting items
a - Test with a list of items, not deleting bibs
b - Test with a list of items, deleting bibs if no items remain where all items are only item on a biblio:
SELECT MAX(barcode) FROM items GROUP BY biblionumber HAVING COUNT(barcode) IN (1)
c - Test with a list of items, deleting bibs if no items remain where some items are the only item on a biblio:
SELECT MAX(barcode) FROM items GROUP BY biblionumber HAVING COUNT(barcode) IN (1,2)
10 - Confirm records are update/deleted as appropriate
Signed-off-by: Bob Bennhoff <bbennhoff@clicweb.org>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
This patch adds a .perlcriticrc (copied from qa-test-tools) and fixes
almost all perlcrictic violations according to this .perlcriticrc
The remaining violations are silenced out by appending a '## no critic'
to the offending lines. They can still be seen by using the --force
option of perlcritic
This patch also modify t/00-testcritic.t to check all Perl files using
the new .perlcriticrc.
I'm not sure if this test script is still useful as it is now equivalent
to `perlcritic --quiet .` and it looks like it is much slower
(approximatively 5 times slower on my machine)
Test plan:
1. Run `perlcritic --quiet .` from the root directory. It should output
nothing
2. Run `perlcritic --quiet --force .`. It should output 7 errors (6
StringyEval, 1 BarewordFileHandles)
3. Run `TEST_QA=1 prove t/00-testcritic.t`
4. Read the patch. Check that all changes make sense and do not
introduce undesired behaviour
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Lines can be moved. Deletion can be done too if skip_merge is not set.
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
At the last development meeting we have voted to remove the
QueryParser-related code
https://wiki.koha-community.org/wiki/Development_IRC_meeting_19_February_2020
Hea tells us that it has not been adopted, and the code/bug tracker that
it is not really usable as it. As nobody is willing to work on it, we
decided to remove it instead.
Test plan:
% prove t/db_dependent/Search.t
must return green
See commits from bug 9239 and confirm that the code is removed in this
patch.
Also play with the search on the UI and confirm that you do not see
obvious regressions
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Note that bug 24267 will fix the issue during Z39 searching for new auth, but the duplicate check at saving will still fail
To test:
0 - Set SearchEngine to Elasticsearch, QueryRegexEscapeOptions to 'Don't Escape'
1 - Browse to Authorities
2 - Select New->New from Z3950
3 - Search for subject 'Short stories'
4 - Auth search explodes
5 - Apply patch
6 - Repeat and confirm the search results return
7 - Import and save the GENRE/FORM record for Short stories
8 - Repeat the search and import the same record
9 - Confirm that upon save you are notified of duplicate, don't save
10 - set QueryRegexEscapeOptions to 'Escape' and confirm Z39 searching works and importing/save notifies of duplicate
11 - set SearchEngine to Zebra and confirm Z39 searching works and importing/save notifies of duplicate
Signed-off-by: Kelly McElligott <kelly@bywatersolutions.com>
Signed-off-by: Myka Kennedy Stephens <mkstephens@lancasterseminary.edu>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Like this one (16.11 line number):
Odd number of elements in anonymous hash at C4/AuthoritiesMarc.pm line 1070.
We need to add a scalar to some MARC::Field::subfield calls. In list context
an empty list returned affects the hash built around it.
Test plan:
Could reproduce this warning easily from OPAC authority search.
opac-authorities-home.pl calling BuildSummary.
Signed-off-by: Bouzid Fergani <bouzid.fergani@inlibro.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
To test:
1 - Set search engine to Zebra
2 - Have some authorities and indexes up to date
3 - Search authorities with option 'search entire record'
4 - No results
5 - Apply patch
6 - Run unit tests, they pass!
7 - Repeat search
8 - Results!
Signed-off-by: Pierre-Marc Thibault <pierre-marc.thibault@inLibro.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Sponsored-by: National Library of Finland
Signed-off-by: Brendan Gallagher <brendan@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
To test:
1 - Define a rule:
Matching rule code: LCAUTH
Match threshold: 100
Record type: Authority record
MATCH POINTS
Search index: LC-card-number
Score: 100
Tag: 010
Subfield: a
2 - Have a record with an LC number in 010
3 - Export it
4 - Edit the record in Koha, move the value from 010 to another field
5 - Import the original record
6 - Use the matching rule above, you expect it won't match (nothing in 010 anymore)
7 - It matches?!?!?!
8 - Enable zebra logging in koha-conf.xml
<zebra_loglevels>none,fatal,warn,request,info</zebra_loglevels>
9 - tail -f /var/log/koha/kohadev/zebra-output.log
10 - On manage marc apply matchign with no rule, then reapply with rule
11 - View the output in the log - the index searched is 'Any'
12 - Apply patch
13 - Reapply matching
14 - Note we search the expected index and no match is found
NOTE: By applying bug 21579, I was able to view the difference, and
see that it didn't match identically, because I had moved the
value from 010 to 155. No log checking needed. clean/delete batch.
Apply patch, no match. clean/delete batch.
Edit authority back, matches.
Signed-off-by: Mark Tompsett <mtompset@hotmail.com>
Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
In sub merge the Koha::Authority object was already created, so it was kind
of silly to recreate it each time with new.
In Koha::Authority, the ControlledIndicators object can be saved in the
object too for reuse during its life time. Similarly, we prevent repeated
lookups for the reporting tag (via authority type).
Test plan:
[1] Run t/db_dependent/Authority/Merge.t
[2] Run t/db_dependent/Koha/Authorities.t
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Limited benchmarking shows me a reduction of 11% on Merge.t and a
reduction of 4% on Authorities.t when comparing with and without this
last patch. The larger impact on Merge.t could be expected.
Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
This patch does:
[1] Adds Koha::Authority->controlled_indicators (with a test).
[2] Adds a call to controlled_indicators in AuthoritiesMarc::merge.
Unit test Merge.t is extended too.
[3] Simplifies the code in authorities/blinddetail-biblio-search.pl by
calling controlled_indicators.
Test plan:
[1] Run t/db_dependent/Koha/Authorities.t
[2] Run t/db_dependent/Authority/Merge.t
[3] Steps 3 to 7 for MARC21:
Create a PERSO_NAME authority with 008/11=r and ind1=3
[4] Edit a biblio and add a 600 linked to the new authority.
[5] Verify that the biblio has ind1==3 and ind2==7 and $2==aat.
(If $2 is not visible, check the metadata in biblio_metadata.)
[6] Edit the PERSO_NAME authority and change 008/11 to '|' (bar).
[7] Verify that merge updated your biblio record: $ind2==4 and $2 gone.
[8] UNIMARC: Follow the pattern from steps 3 to 7.
Create authority, link it in a biblio, check indicators (they should
be copied both). Edit authority, change indicators and verify the
merge results in the biblio record.
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Tested PERSO_NAME and UNIF_TITLE.
For UNIF_TITLE the second authority indicator is copied to ind1 or ind2,
depending on the biblio tag involved.
Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>
Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
While doing a merge, if a subfield(s) precedes the controlled subfields
(like $i before $a in 7XX, which comes before $a -- rare but will
become more and more usual) after merging will be moved to the end.
This is not right.
The patch (with AuthorityMergeMode == loose) make merge consider the
subfields order: all subfields which originally were found before
the first controlled subfield (e.g. $i before $a in 7XX / MARC 21)
will remain in the front, the rest of not controlled subfields that
should remain in the field will come after the subfields copied
from authority rec.
As a bonus, $9 will be placed at the end.
To test:
0) Have AuthorityMergeMode == loose;
1) Have some field in bibliorecord, controlled by an authority, with
extra subfield(s) (i.e. not present in authority rec.) placed at the
beginning of the field;
2) Open (not necessarily edit) and save the connected authority;
3) See that the extra subfieds were moved to the end of the field
(and $9 is in the front);
4) Apply the patch;
5) Reorder subfields in biblio field;
6) Open (not necessarily edit) and save the connected authority;
7) See that the order has been conserved, additionally $9 the last
subfield in the field.
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Amended:
Moved field creation to its original place. Changed $9 handling. Simplified the following add_subfields for loop. Edited comments.
Restored the append_fields_ordered call (see comment6).
With this patch, the Merge.t test now passes.
Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Commit b4392018bc [Bug 12478: make things using SimpleSearch use the new version] changed sub FindDuplicateAuthority and replaced this call (from 3.22.x) in 16.05 (pushed April 2016):
my ($error, $results, $total_hits) = C4::Search::SimpleSearch( $query, 0, 1, [ "authorityserver" ] );
The new call does unfortunately not include the authorityserver:
my ($error, $results, $total_hits) = $searcher->simple_search_compat( $query, 0, 1 );
Simple_search_compat redirects to C4/Search/SimpleSearch and SimpleSearch assumes a biblioserver if no server is passed.
This effectively makes FindDuplicateAuthority useless since we will no longer find duplicates and we could see an error like this in the log:
16:51:42-04/10 zebrasrv(51) [request] Search biblios ERROR 114 1 1+0 RPN @attrset Bib-1 @and @attr 1=authtype PERSO_NAME @attr 1=Heading Moerenhout
The fix is obviously trivial.
Test plan:
[1] Go to Authorities.
[2] Add a new authority PERSO_NAME with 100a Moerenhout.
[3] Repeat step 2. Verify that you get the duplicate authority warning.
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
After the introduction of Koha::Authorities->get_usage_count with bug
9988, we can now replace the remaining occurrences of CountUsage.
At the same time we remove CountUsageChildren. This was an empty sub.
The typo get_count_usage in a subtest title is adjusted.
Test plan:
[1] Run t/db_dependent/Koha/Authorities.t
[2] Perform a search on authorities-home.pl and verify that you see
plausible numbers for 'used in xx records'.
[3] Click on Details for one authority. See the same number?
[4] Do the same as in 2/3 for Authority search on OPAC.
[5] Remember the authid and enter this in the record numbers box on
tools/batch_delete_records.pl. Select Authorities and click
Continue. The next form shows a column "Used in". Do you see
the same count again?
[6] Git grep CountUsage.
You should see just one hit in a comment that can be kept in
Koha/Authorities.pm.
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
