Commit graph

642 commits

Author SHA1 Message Date
0a37e8f7fd
Bug 29697: Remove GetHiddenItemnumbers
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

JD amended patch:
-    my @items        = $biblio->items->filter_by_visible_in_opac({ patron => $logged_in_user })->unblessed;
-    foreach my $item (@items) {
+    my $items        = $biblio->items->filter_by_visible_in_opac({ patron => $logged_in_user })->unblessed;
+    foreach my $item (@$items) {

-    for my $itm (@items) {
+    for my $itm (@$items) {

-    $dat->{ITEM_RESULTS}   = \@items;
+    $dat->{ITEM_RESULTS}   = $items;

-        @items_to_show = Koha::Items->search( { itemnumbers => [ map { $_->{itemnumber} } @all_items ] } )
-                                    ->filter_by_visible_in_opac( { patron => $patron } );
+        @items_to_show = Koha::Items->search( { itemnumber => [ map { $_->{itemnumber} } @all_items ] } )
+                                    ->filter_by_visible_in_opac( { patron => $patron } )->as_list;

-            my @items_to_show = $items->filter_by_visible_in_opac({ opac => 1, patron => $patron });
+            my @items_to_show = $items->filter_by_visible_in_opac({ opac => 1, patron => $patron })->as_list;

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2022-07-22 15:24:11 -03:00
daab31ab38
Bug 30327: Fix tests
Corrected variable name on update to match everywhere else

Added a default value for limit in buildQuery and only append limit if it has content

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2022-06-25 15:25:18 -03:00
ee669178f6
Bug 30813: Update TransformMarcToKoha to accept a hashref
This patch updates all the calls to pass a hasref rather than an array

It also removes the no longer used framework parameter

To test:
prove -v t/Biblio.t t/db_dependent/Biblio/TransformMarcToKoha.t

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Joonas Kylmälä <joonas.kylmala@iki.fi>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2022-06-08 11:40:29 -03:00
2da14cf8c1
Bug 30528: Process limits before handling CCL query
This patch moves the code that handles limit processing before that for CCL queries to ensure that special limit cases (branch, multibranch, etc) are handled before adding the limits to the query string

To test:
1 - Apply unit tests patch only
2 - prove -v t/db_dependent/Search.t
3 - It fails
4 - Apply this patch
5 - It passes!
6 - Test searching and confirm things work as epxected

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Joonas Kylmälä <joonas.kylmala@iki.fi>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2022-06-01 13:51:37 -03:00
Aleisha Amohia
50d917dbfc Bug 30291: Changes to controller scripts
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
2022-05-05 11:17:36 -10:00
0e880cf233 Revert "Bug 28371: Passpreviously fetched branches and itemtypes through and fetch all needed AV at once"
This reverts commit 8e1d5d4b8f.

Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
2022-04-25 20:52:28 -10:00
8e1d5d4b8f Bug 28371: Passpreviously fetched branches and itemtypes through and fetch all needed AV at once
This patch updates the searchResuls code to pass through the pre-constructed branches and itemtype lookups
to XSLTParse4Display to avoid repeating this

It also updates getAuthorisedValues4MARCSubfields to fetch the values for mapped subfields and pass
then through to transforMarc4XSLT

Note that we currently blank invalid branches and itemtypes - I presrve this, we should open another bug
if we want to change this behaviour

Changes are covered by tests

To test:
1 - Perform searches in OPAC and staff client that return many records
2 - Use the 'Network' tab on the browser console (opened with F12 usually) to see the time taken
3 - Note the speed before the patch
4 - Apply patch
5 - restart all the things
6 - Note improvement in speed

**Note: The improvement is more drastic the more items per record, try adding large numbers of items to your search results to test**

Signed-off-by: Owen Leonard <oleonard@myacpl.org>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
2022-04-25 10:00:22 -10:00
bc1c8b4338 Bug 30244: Include lost items in list of hidden items
When building search results for XSLT we generate and pass a list of hidden
itemnumbers.

We do skip the lost items in our parsing, however, we neglect to add the itemnumber
to the hidden list

This patch simply adds the lost itemnumbers to the list

There is more work to be done here to simpliofy this process, however, this patch resolves
the issue and can be backported to stable branches

To test:
1 - Set systempreference hidelostitems to "Don't show"
2 - Edit a record to set one item as lost and one as available
3 - Perform an OPAC search that returns the record above
4 - Note that the lost item shows in availability line
5 - Click on the record - note the lost item does not show on details
6 - Apply patch
7 - Reload search results
8 - Lost itme no longer displays

Signed-off-by: Andrew Fuerste-Henry <andrew@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
2022-03-25 11:01:04 -10:00
6f8b94f49c Bug 30110: Fix concatenation during assignements
There are several occurrences of `my $var += ` or .= in the code.
It should not cause problems but it's confusing.

Test plan:
Read the patch and confirm that the changes make sense.

Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
2022-03-24 14:22:10 -10:00
5bd71b5b7d Bug 22605: (follow-up) Fix operators
- Don't add an operator on first loop
- Only add plus option on last loop
- Fix indentation of first search box
- Remove spaces from operators in query_cgi and add to query and query_desc

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
2022-03-24 14:22:09 -10:00
a0a72c9653 Bug 19532: (RM follow-up) More use of system preference
When system preference is off, call no code related to Koha::Recalls.

Also add some missing module import.

Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
2022-03-14 23:11:12 -10:00
Aleisha Amohia
b800c2e68d Bug 19532: Other objects used in recalls feature
- biblio->recalls
- biblio->can_be_recalled
- item->recall
- item->can_be_recalled
- item->can_set_waiting_recall
- item->check_recalls
- patron->recalls
- Biblio.RecallsCount

and relevant tests
- t/db_dependent/Stats.t
- t/db_dependent/Koha/Item.t
- t/db_dependent/Koha/Biblio.t
- t/db_dependent/Koha/Patron.t
- t/db_dependent/XSLT.t
- t/db_dependent/Search.t
- t/db_dependent/Holds.t
- t/db_dependent/Circulation/transferbook.t
- t/db_dependent/Circulation.t

Signed-off-by: David Nind <david@davidnind.com>

Signed-off-by: David Nind <david@davidnind.com>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
2022-03-14 22:45:51 -10:00
Thomas Klausner
289551282d Bug 30153: Use uppercase AND to join query parts
C4::Search->FindDuplicate joins title and author using 'and'
(lowercase). When this query is passed on to ElasticSearch, it
interprets the lowercase 'and' as a term to search for, because the
operator has to be in uppercases ('AND').

Test plan:
* Reproduce the bug:
- Set SearchEngine to ElasticSearch (and make sure you have the data
  indexed etc)
- Find an existing book, note the title (245a) and the author (100a)
- Create a new book (Cataloging -> New Record)
- Fill in the same title and author using the same data as in an
  existing book (and any other fields that might be required)
- Click "save"
=> A new book will be created, the Duplicate Finder has failed

* Apply the patch

* Check if it's working now:
- Create a new book (Cataloging -> New Record)
- Fill in the same title and author using the same data as in an
  existing book (and any other fields that might be required)
- Click "save"
- The DuplicateFinder should now report the already exising book

Maybe we should also check if Zebra does not have any problems with the
uppercase 'AND'? In that case, repeat the above steps, but set
SearchEngine to Zebra :-)

Sponsored-by: Steiermärkische Landesbibliothek
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
2022-03-01 22:47:46 -10:00
David Gustafsson
7da4912f8e Bug 25616: Uppercase hard coded lower case boolean operators for Elasticsearch
Uppercase occurances of all (hopefully) lowercase "and"
used in ElasticSearch Query String Query contexts

Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
2022-02-24 14:35:36 -10:00
e53667105d Bug 29844: Fix ->search occurrences
and some more...

There are lot of inconsistencies in our ->search calls. We could
simplify some of them, but not in this patch. Here we want to prevent
regressions as much as possible and so don't add unecessary changes.

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
2022-02-09 15:36:23 -10:00
2f4d5954c9 Bug 28365: (Bug 19873 follow-up) Make it possible to search on value 0
Bug 19873 fixed search with value 0.
It works in first page but not in other pages.

This patch fixes query_cgi var.

Test plan:
  - create a st-numeric index in zebra conf related to
    a numeric field
    i.e:
    yourindex 1=yourindex 4=109
  - fill a field with 0 in several biblio records,
  - reindex your biblios,
  - search yourindex=0 and go to second page of results,
  - should not work,
  - apply this patch,
  - test again,
  - should work

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-11-03 15:40:52 +01:00
20991feb62 Bug 29374: Don't crash if search engine returns a deleted record
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-10-29 15:46:17 +02:00
dffbfad156 Bug 20463: (follow-up) Configure short name mrl for Multipart-resource-level index
This adds mrl as appreviation of Multipart-resource-level to
the Zebra configuration and Search.pm so it can be used when
searching.

Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-10-25 14:08:06 +02:00
Hayley Pelham
45fee768e9 Bug 20463: Adding index for Leader 19 Multipart resource record level
Sponsored-by: Bibliotheksservice-Zentrum Baden-Wuerttemberg

Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-10-25 14:08:06 +02:00
1ad0e172cd Bug 3142: Use filter_by_for_holds where holdability is calculated
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-10-08 16:57:29 +02:00
08c03af2ae Bug 18984: Remove NORMARC support
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-10-07 15:36:40 +02:00
bc67449a46 Bug 28826: Add FacetOrder system preference
This bug adds a system preference to control ordering of facets and
adds the control to both Zebra and Elasticsearch

To test:
1 - Have a koha that can use both Zebra and ES
2 - Set 'displayFacetCount' to true
3 - Search in ES and Zebra
4 - Note facets in Zebra sorted alphabetically, ES by usage
5 - Apply patch, updatedatabase
6 - Search in ES and Zebra, facets are alphabetically sorted in both
7 - Find new syspref FacetOrder and set to 'by usage'
8 - Search in both engines, facets sorted by usage

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-09-29 11:34:21 +02:00
87de9fbb8a Bug 28847: Remove dead code and make regex consistent
This updates the regex code to match throughout the routine and removes
code that would no longer be reached.

Note that the code to update library name in the query descruiption was
broken before this patch, so removal does not change behaviour

Signed-off-by: Joonas Kylmälä <joonas.kylmala@iki.fi>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-09-20 14:04:48 +02:00
777c519c5a Bug 28847: Move SearchLimitLibrary code to QueryBuilder Modules
This patch removes the code from the search scripts into QueryBuilder
modules.

To test:
1 - Have a library group defined as a search group for both staff and opac
2 - Search on staff client and opac with that group limit and a single branch limit
3 - Note your results/counts
4 - Note the visuals of the search description
5 - Apply patch
6 - Repeat searches
7 - All should work as before

Signed-off-by: Andrew Fuerste-Henry <andrew@bywatersolutions.com>

Signed-off-by: Joonas Kylmälä <joonas.kylmala@iki.fi>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-09-20 14:04:48 +02:00
fc20c964b0 Bug 28373: Add PassItemMarcToXSLT system preference
Default stylesheets do not reference item fields for XSLT display, however, we
spend time translating the values in the item fields.

This patch adds a system preference, PassItemMarcToXSLT. and unless enabled we remove
item fields before processing

To test:
1 - Perform some search on the staff client and opac
2 - Use the console (F12) to view the time spent on the network tab
3 - Note performance
4 - Apply patch, updatedatabase, restart_all
5 - Repeat searches
6 - Note that display has not changed
7 - Note performance, results should display slightly faster

Signed-off-by: Emmi Takkinen <emmi.takkinen@koha-suomi.fi>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-08-31 08:49:04 +02:00
deece7b76b Bug 28830: Add cni index for 003
This patch adds the cni/Control-number-identifier index to enable
searches to use the 003 field.

Test plan
1/ Apply patch
2/ Re-index using updated configurations
3/ Confirm cni:number searches yield the expected results
4/ Signoff

Split-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Pasi Kallinen <pasi.kallinen@koha-suomi.fi>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-08-30 17:02:07 +02:00
977774f095 Bug 12561: Change prototype of XSLTParse4Display
Use a hash for better readability and reusability.

Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-08-16 09:57:05 +02:00
56c2856e26 Bug 12561: Remove non-XSLT views
Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-08-16 09:57:04 +02:00
9d6d641d1f Bug 17600: Standardize our EXPORT_OK
On bug 17591 we discovered that there was something weird going on with
the way we export and use subroutines/modules.
This patch tries to standardize our EXPORT to use EXPORT_OK only.

That way we will need to explicitely define the subroutine we want to
use from a module.

This patch is a squashed version of:
Bug 17600: After export.pl
Bug 17600: After perlimport
Bug 17600: Manual changes
Bug 17600: Other manual changes after second perlimports run
Bug 17600: Fix tests

And a lot of other manual changes.

export.pl is a dirty script that can be found on bug 17600.

"perlimport" is:
git clone https://github.com/oalders/App-perlimports.git
cd App-perlimports/
cpanm --installdeps .
export PERL5LIB="$PERL5LIB:/kohadevbox/koha/App-perlimports/lib"
find . \( -name "*.pl" -o -name "*.pm" \) -exec perl App-perlimports/script/perlimports --inplace-edit --no-preserve-unused --filename {} \;

The ideas of this patch are to:
* use EXPORT_OK instead of EXPORT
* perltidy the EXPORT_OK list
* remove '&' before the subroutine names
* remove some uneeded use statements
* explicitely import the subroutines we need within the controllers or
modules

Note that the private subroutines (starting with _) should not be
exported (and not used from outside of the module except from tests).

EXPORT vs EXPORT_OK (from
https://www.thegeekstuff.com/2010/06/perl-exporter-examples/)
"""
Export allows to export the functions and variables of modules to user’s namespace using the standard import method. This way, we don’t need to create the objects for the modules to access it’s members.

@EXPORT and @EXPORT_OK are the two main variables used during export operation.

@EXPORT contains list of symbols (subroutines and variables) of the module to be exported into the caller namespace.

@EXPORT_OK does export of symbols on demand basis.
"""

If this patch caused a conflict with a patch you wrote prior to its
push:
* Make sure you are not reintroducing a "use" statement that has been
removed
* "$subroutine" is not exported by the C4::$MODULE module
means that you need to add the subroutine to the @EXPORT_OK list
* Bareword "$subroutine" not allowed while "strict subs"
means that you didn't imported the subroutine from the module:
  - use $MODULE qw( $subroutine list );
You can also use the fully qualified namespace: C4::$MODULE::$subroutine

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-07-16 08:58:47 +02:00
5da5a2bbc5 Bug 28384: Add 'no_items' option to TransformMarcToKoha
This patch adds a new option 'no_items' and uses it in the C4::Search::searchResults
routine. We don't use the item info fetched here, so skipping those lines saves us time.

Additionally, I fix an incorrect FIXME comment, and pass the ISBN returned by the routine above
into GetNormalizedISBN to save another lookup

TO test:
1 - Enable AmazonCoverImages system preference
2 - Search staff client with a term that returns books with covers
3 - Apply patch
4 - prove -v t/db_dependent/Biblio/TransformMarcToKoha.t
5 - Confirm searching works
6 - Confirm Amazon images display (normalized_isbn is used for these)

Signed-off-by: David Nind <david@davidnind.com>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
[EDIT] Amended, replacing a tab character.

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-07-12 15:47:34 +02:00
b8ab792dc4 Bug 28483: Remove warnings from Search.t
There are too many warnings about uninitialized variables in Search.pm.
This patch deals with that, the same way it is dealt accross the file:
by setting an empty string when things are not defined.

To test:
1. Run:
   $ kshell
  k$ prove t/db_dependent/Search.t
=> FAIL: Wow, too many warnings. Some related to this bug, some not.
2. Apply this patch
3. Repeat 1
=> SUCCESS: No more warnings!
4. Sign off :-D

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-07-12 11:44:55 +02:00
3fe837bc74 Bug 28572: Remove C4::Debug
We are using Koha::Logger when it makes sense to keep the info,
otherwise we simply remove it

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Bug 28572: Replace missing occurrence in misc/admin/koha-preferences

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-06-22 12:04:32 +02:00
2a0be09d63 Bug 18989: Allow displaying biblios with all items hidden by OpacHiddenItems
Bug 10584 made Koha hide biblios for which all items match some criteria (in OpacHiddenItems) so they are hidden.
Add syspref OpacHiddenItemsHidesRecord controlling this behaviour.

Test plan :
1)
1.1) Create some biblio records with one item having damaged=1
1.2) Define system preference OpacHiddenItems = damaged: 1
2)
2.1) Set system preference OpacHiddenItemsHidesRecord to 'don't hide'
2.2) At OPAC : perform a search showing those records and some more
2.3) Check you see the all the records
2.4) For a record with hidden item check you don't get HTTP 404 for : normal view, ISBD view, MARC view
2.5) Check you can had tags on this record
2.6) Add record to basket, check you see it in basket
3)
3.1) Set system preference OpacHiddenItemsHidesRecord to 'hide'
3.2) At OPAC : perform a search showing those records and some more
3.3) Check you don't see the records with hidden item
3.4) For a record with hidden item check you get HTTP 404 for : normal view, ISBD view, MARC view
3.5) Show basket, check you see the records with hidden item

Signed-off-by: Kelly McElligott <kelly@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-05-12 14:12:07 +02:00
93a297a7a2 Bug 27928: FindDuplicate should use simple_search_compat
This patch switches from calling SimpleSearch directly to using the compat
method so that Elastic is used when it is the chosen search engine

To test:
 1 - Set SearchEngine syspref to 'Zebra'
 2 - Add a suggestion with an existing title, e.g.:
   'E street shuffle'
 3 - Confirm you get a duplication warning
 4 - Change SearchEngine to ES
 5 - Repeat get same warning
 6 - sudo koha-zebra --stop
    'ps aux | grep zebra'  to confirm they are stoped
    'sudo pkill -9 zebra' if they arent;
 7 - Repeat suggestion - no duplication warning (because zebra returns nothing)
 8 - Delete the suggestion
 9 - Apply patches
10 - Restart all the things, stop zebra
11 - Repeat suggestion
12 - Successful warning from ES
13 - prove -v t/db_dependent/Search.t

Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-04-01 18:03:23 +02:00
Petro Vashchuk
ebf123bf13 Bug 27745: prevent warning for uninitialized value as a hash key
When you edit and save items with an empty "c - Shelving location"
field, it gets stored into the database as "NULL",
which is allowed in the database, and the field is not "required",
so this seems expected.

But when the search page is rendered, it gets value by location key
($shelflocations->{ $item->{location} }) which gives undef warning.

Solved by checking if $item->{location} is perl true.

To reproduce:
    1) Edit the book or create new one with an empty shelving location.
    2) Use the search feature to have that book in the searched list.
    3) Check plack-intranet-error.log to find "Use of uninitialized
value in hash element" error.
    4) Apply the patch.
    5) Repeat the search and check the logs again to ensure that error
didn't appear again.

Signed-off-by: Didier Gautheron <didier.gautheron@biblibre.com>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-03-05 13:48:31 +01:00
46f7239b08 Bug 27673: Replace YAML with YAML::XS
From tht YAML pod:

"""
This module has been released to CPAN as YAML::Old, and soon YAML.pm will be changed to just be a frontend interface module for all the various Perl YAML implementation modules, including YAML::Old.

If you want robust and fast YAML processing using the normal Dump/Load API, please consider switching to YAML::XS. It is by far the best Perl module for YAML at this time. It requires that you have a C compiler, since it is written in C.
"""

See also
https://gitlab.com/koha-community/qa-test-tools/-/merge_requests/35

Test plan:
Try some place where YAML::XS is not used and confirm that it works
correctly

QA note: This patch removes some uses of YAML that were not useful

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Joonas Kylmälä <joonas.kylmala@helsinki.fi>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-02-16 14:54:50 +01:00
06a58fe56d Bug 20410: Remove OpacGroupResults and PazPar2
This feature is not used as far as we know and it's not known to work.
It's preferable to remove it.

Test plan:
Make sure the OpacGroupResults pref code is removed, as well as the
PazPar2 files and code.

Signed-off-by: Owen Leonard <oleonard@myacpl.org>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-02-08 16:12:42 +01:00
a52a9d7875 Bug 26957: Don't remove words that are also operators from search terms
FindDuplicate has a hardcoded regex to remove 'and', 'or', and 'not' from any search fields, however,
is does this with no regard for these terms being embedded in the middle of a word or phrase.

E.g.:
'Coupland' becomes 'Coupl'
'Frog and toad' becomes 'frog toad'
etc.

This patch simplay removes the substitutions as we already quote the terms as well

To test:
1 - Add records to your system like:
    The night circus by Erin Morgenstern
    Frog and toad all year
    Nothing ever happens
2 - Attempt to add purchase suggestions like
    title: the night circus, author:Morgenstern, Erin
    title: For and toad all year
    title:nothing ever happens (case is important)
3 - No warnings about existing biblios
4 - Apply patches
5 - Delete existing suggestions
6 - Repeat step 2
7 - Each attempt should warn you of existing biblio

Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-01-12 16:02:35 +01:00
f6b5dfa4b7 Bug 12430: Use releance ranking without QueryWeightFields
The system preference "QueryWeightFields" is mutually exclusive with
the system preference "QueryAutoTruncate" and the * truncation
modifier, when not using QueryParser.

If you use truncation, relevance won't work anymore. (N.B. Relevance
doesn't work probably when using QueryParser, but for a very different
reason beyond the current scope of this bug.)

This patch adds relevance ranking when using truncation (or basically
when QueryWeightFields is disabled).

_TEST PLAN_

1) Turn on "QueryWeightFields" and set "QueryAutoTruncate" to auto
2) Turn off "UseQueryParser"
3) Do a keyword search in the OPAC
4) Note that the results are ordered by biblionumber ascending

5) Apply patch

6) Do the same keyword search in the OPAC
7) Note that the results are never ordered differently
(presumably in relevance order :P)

If you're a stickler, throw in some warns so that you can see
what CCL query is being sent to Zebra.

Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-12-04 15:26:56 +01:00
Julian Maurice
96cc447045 Bug 25898: Prohibit indirect object notation
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-10-15 12:56:30 +02:00
c96fabb8c4 Bug 17661: Trivial simplification using += operator
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-08-31 16:10:25 +02:00
224ac84aec Bug 17661: (follow-up) Update regex to support Unicode characters
Rather than limiting initials to [A-Z] we should test for a broad
range of uppercase letters.

The ES/Zebra changes are slightly different because of Perl vs Java regex
conventions. POerl may support either, but I found 'Uppercase' to be a bit more explicit

More info here:
https://perldoc.perl.org/perlunicode.html

TO test:
Same plan as before but use Ж. as the ending initial
Confirm the period is preserved and other punctuation removed

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-08-31 16:10:25 +02:00
e34f95a1f5 Bug 17661: Ending punctuation causes duplicate facets
The current code for facets doesn't pull strip ending punctuation from facets
This causes duplicate facets for terms that should be combined

Sometimes series can have different punctuation depending on the field they are in
Author initials punctuation should be preserved

To test:
1 - Do search and pull up some records
2 - Edit some of the records to have authors like:
    Date, C.J.
    Date, C.j.
    Date, C.J .
3 - Edit the records to have some series statments like:
    830 $aDate, C.J. ;$v5
    830 $aDate, C.J. ; $v5
    830 $aDate, C.J.; $v5
4 - Add some 490s to the record with first indicator 1 and series like:
    You wouldn't want to--
    You wouldn't want to
    You wouldn't want to..
5 - Search again and note you have 3 facets each for author and series
6 - Apply patch
7 - Repeat
8 - Now you get 2 facets for author, period not removed when following Upper case immediately, is otherwise
9 - Now you should have a single series facet
10 - Switch search engine to ES (index before applying patch)
11 - Note facets are separate again
12 - Reset mappings and reindex
   perl misc/search_tools/rebuild_elasticsearch -v -r
13 - Repeat search, facets combined as above

Signed-off-by: Sarah Cornell <sbcornell@cityofportsmouth.com>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-08-31 16:10:25 +02:00
Katrin Fischer
ed066caf58 Bug 26015: Terminology: Fix tons more staff clients to staff interfaces
This takes care of more occurences of staff client and changes it to
staff interface, including in code comments.

To test:
- I think in this case careful code review is what we look for.

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-08-31 16:10:25 +02:00
bc1b76be4a Bug 23086: Search for collection is broken
It appears that we are quoting the ccode values deep in the search code.
Under ICU chains this breaks searching by limits

To recreate
 1 - Setup Koha using Zebra and icuchains
 2 - Add ccode to AdvancedSearchTypes
 3 - In koha-conf.xml set zebra debug level to include request
  <zebra_loglevels>none,fatal,warn,request,info</zebra_loglevels>
 4 - Set some items into different ccodes
 5 - On opac perform a search for:
    ccode:NFIC
 6 - It works
 7 - tail -n 50 /var/log/koha/kohadev/zebra-output.log
 8 - Note search request like:
    Search biblios OK 26 1 1+0 RPN @attrset Bib-1 @attr 1=8009 NFIC
 9 - On opac go to advanced search, select Collection, and limit to smae code a s above
10 - No results
11 - Check the zebra-output.log:
    Search biblios OK 0 1 1+0 RPN @attrset Bib-1 @attr 1=8009 'NFIC'
12 - Apply patch
13 - Restart all
14 - Repeat search by collection limit
15 - Success!
16 - Check the zebra-output.log:
    Search biblios OK 0 1 1+0 RPN @attrset Bib-1 @attr 1=8009 NFIC
17 - Add a new ccode value: N)N
18 - Set some items to that ccode
19 - Confirm searching by that ccode works

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Alex Arnaud <alex.arnaud@biblibre.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-07-27 17:24:26 +02:00
Julian Maurice
b168f4a2e9 Bug 21395: Make perlcritic happy
This patch adds a .perlcriticrc (copied from qa-test-tools) and fixes
almost all perlcrictic violations according to this .perlcriticrc
The remaining violations are silenced out by appending a '## no critic'
to the offending lines. They can still be seen by using the --force
option of perlcritic
This patch also modify t/00-testcritic.t to check all Perl files using
the new .perlcriticrc.
I'm not sure if this test script is still useful as it is now equivalent
to `perlcritic --quiet .` and it looks like it is much slower
(approximatively 5 times slower on my machine)

Test plan:
1. Run `perlcritic --quiet .` from the root directory. It should output
   nothing
2. Run `perlcritic --quiet --force .`. It should output 7 errors (6
   StringyEval, 1 BarewordFileHandles)
3. Run `TEST_QA=1 prove t/00-testcritic.t`
4. Read the patch. Check that all changes make sense and do not
   introduce undesired behaviour

Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-06-29 12:37:02 +02:00
35bb41dfc1 Bug 25701: Remove sort on removed field
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-06-15 10:29:53 +02:00
d669694426 Bug 25701: (bug 14419 follow-up) Always display facet groups in the same order
It seems that this regression comes from bug 14419, but I have not found
a logic reason behind that.
This patch restores the behaviour we always had: facet groups must be
displayed in a given order: the Zebra index (au, ccode, holdingbranch,
etc.)

Test plan:
Apply this patch, restart all and confirm that the facets are not moving
up and down when you refresh your search result page.

QA note:
I think the following line must be removed
@facets_loop = sort {defined $a->{expand} && defined $b->{expand} && $a->{expand} cmp $b->{expand}} @facets_loop;

Signed-off-by: David Nind <david@davidnind.com>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-06-15 10:29:53 +02:00
d69ab6ee91
Bug 24458: Use the ViewPolicy filter on search results
This patch implements the use of the ViewPolicy record processor filter
inside C4::Search::searchResults. The idea is that the $record_processor
is instantiated once and reused inside the loop. This leaves options for
further optimizations I will do on a follow-up bug.

The filter is applied to the MARC data before it is passed to the XSLT
processor.

To test:
1. Apply the regression tests patch
2. Run:
   $ kshell
  k$ prove t/db_dependent/Search.t
=> FAIL: This is not implemented, tests fail
3. Apply this patch
4. Repeat 2
5. Feel the joy in your body from a long standing bug being solved
6. Sign off :-D

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-05-15 09:33:22 +01:00
2a216e206c
Bug 25416: Let OPAC XSLTs know if the context is an anonymous session
This patch makes use of the 'variables' parameter in XSLTParse4Display
method in the different places that it is used in the OPAC. It does by
passing this parameter with

    anonymous_session => 1|0

The value will depend on the output from get_template_and_user (i.e. if
there's a returned borrowernumber).

A special case takes place in search results, as the call to
XSLTParse4Display happens in C4::Search::searchResults. So a new
parameter 'xslt_variables' is added to it.

To test:
1. Apply the [DO NOT PUSH] patch
2. Open the OPAC in your browser
3. Try detail pages, search results, tags and lists/shelves pages with
   or without an active session
=> FAIL: It always says (somewhere) 'Anonymous session: Yes'
4. Apply this patch, restart_all
5. Repeat 3
=> SUCCESS: It will tell the Yes/No correctly regarding anonymous
sessions!
6. Sign off :-D

Sponsored-by: Universidad ORT Uruguay
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-05-15 09:33:22 +01:00