This patch updates the Local-Number indexing by adding a zeropad option
to Zebra indexing and adding this to the mapping files
It also updates C4/Search.pm to allow biblionumber as an option
To test:
1 - Apply patches
2 - copy etc/zebradb/marc_defs/marc21/biblios/biblio-zebra-indexdefs.xsl to /etc/koha/zebradb/marc_defs/marc21/biblios/biblio-zebra-indexdefs.xsl
3 - Restart all, reindex zebra
4 - Browse to: http://localhost:8081/cgi-bin/koha/catalogue/search.pl?idx=kw&q=a&sort_by=biblionumber_dsc&count=20
5 - Confirm records sorted correctly
6 - Browse to http://localhost:8081/cgi-bin/koha/catalogue/search.pl?idx=kw&q=a&sort_by=biblionumber_asc&count=20
7 - Confirm records sorted correctly
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit 230bae0b02)
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Corrected variable name on update to match everywhere else
Added a default value for limit in buildQuery and only append limit if it has content
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit daab31ab38)
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
This patch moves the code that handles limit processing before that for CCL queries to ensure that special limit cases (branch, multibranch, etc) are handled before adding the limits to the query string
To test:
1 - Apply unit tests patch only
2 - prove -v t/db_dependent/Search.t
3 - It fails
4 - Apply this patch
5 - It passes!
6 - Test searching and confirm things work as epxected
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Joonas Kylmälä <joonas.kylmala@iki.fi>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit 2da14cf8c1)
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
This patch updates the searchResuls code to pass through the pre-constructed branches and itemtype lookups
to XSLTParse4Display to avoid repeating this
It also updates getAuthorisedValues4MARCSubfields to fetch the values for mapped subfields and pass
then through to transforMarc4XSLT
Note that we currently blank invalid branches and itemtypes - I presrve this, we should open another bug
if we want to change this behaviour
Changes are covered by tests
To test:
1 - Perform searches in OPAC and staff client that return many records
2 - Use the 'Network' tab on the browser console (opened with F12 usually) to see the time taken
3 - Note the speed before the patch
4 - Apply patch
5 - restart all the things
6 - Note improvement in speed
**Note: The improvement is more drastic the more items per record, try adding large numbers of items to your search results to test**
Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
When building search results for XSLT we generate and pass a list of hidden
itemnumbers.
We do skip the lost items in our parsing, however, we neglect to add the itemnumber
to the hidden list
This patch simply adds the lost itemnumbers to the list
There is more work to be done here to simpliofy this process, however, this patch resolves
the issue and can be backported to stable branches
To test:
1 - Set systempreference hidelostitems to "Don't show"
2 - Edit a record to set one item as lost and one as available
3 - Perform an OPAC search that returns the record above
4 - Note that the lost item shows in availability line
5 - Click on the record - note the lost item does not show on details
6 - Apply patch
7 - Reload search results
8 - Lost itme no longer displays
Signed-off-by: Andrew Fuerste-Henry <andrew@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
There are several occurrences of `my $var += ` or .= in the code.
It should not cause problems but it's confusing.
Test plan:
Read the patch and confirm that the changes make sense.
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
- Don't add an operator on first loop
- Only add plus option on last loop
- Fix indentation of first search box
- Remove spaces from operators in query_cgi and add to query and query_desc
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
When system preference is off, call no code related to Koha::Recalls.
Also add some missing module import.
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
C4::Search->FindDuplicate joins title and author using 'and'
(lowercase). When this query is passed on to ElasticSearch, it
interprets the lowercase 'and' as a term to search for, because the
operator has to be in uppercases ('AND').
Test plan:
* Reproduce the bug:
- Set SearchEngine to ElasticSearch (and make sure you have the data
indexed etc)
- Find an existing book, note the title (245a) and the author (100a)
- Create a new book (Cataloging -> New Record)
- Fill in the same title and author using the same data as in an
existing book (and any other fields that might be required)
- Click "save"
=> A new book will be created, the Duplicate Finder has failed
* Apply the patch
* Check if it's working now:
- Create a new book (Cataloging -> New Record)
- Fill in the same title and author using the same data as in an
existing book (and any other fields that might be required)
- Click "save"
- The DuplicateFinder should now report the already exising book
Maybe we should also check if Zebra does not have any problems with the
uppercase 'AND'? In that case, repeat the above steps, but set
SearchEngine to Zebra :-)
Sponsored-by: Steiermärkische Landesbibliothek
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
Uppercase occurances of all (hopefully) lowercase "and"
used in ElasticSearch Query String Query contexts
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
and some more...
There are lot of inconsistencies in our ->search calls. We could
simplify some of them, but not in this patch. Here we want to prevent
regressions as much as possible and so don't add unecessary changes.
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
Bug 19873 fixed search with value 0.
It works in first page but not in other pages.
This patch fixes query_cgi var.
Test plan:
- create a st-numeric index in zebra conf related to
a numeric field
i.e:
yourindex 1=yourindex 4=109
- fill a field with 0 in several biblio records,
- reindex your biblios,
- search yourindex=0 and go to second page of results,
- should not work,
- apply this patch,
- test again,
- should work
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
This adds mrl as appreviation of Multipart-resource-level to
the Zebra configuration and Search.pm so it can be used when
searching.
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Sponsored-by: Bibliotheksservice-Zentrum Baden-Wuerttemberg
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
This bug adds a system preference to control ordering of facets and
adds the control to both Zebra and Elasticsearch
To test:
1 - Have a koha that can use both Zebra and ES
2 - Set 'displayFacetCount' to true
3 - Search in ES and Zebra
4 - Note facets in Zebra sorted alphabetically, ES by usage
5 - Apply patch, updatedatabase
6 - Search in ES and Zebra, facets are alphabetically sorted in both
7 - Find new syspref FacetOrder and set to 'by usage'
8 - Search in both engines, facets sorted by usage
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
This updates the regex code to match throughout the routine and removes
code that would no longer be reached.
Note that the code to update library name in the query descruiption was
broken before this patch, so removal does not change behaviour
Signed-off-by: Joonas Kylmälä <joonas.kylmala@iki.fi>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
This patch removes the code from the search scripts into QueryBuilder
modules.
To test:
1 - Have a library group defined as a search group for both staff and opac
2 - Search on staff client and opac with that group limit and a single branch limit
3 - Note your results/counts
4 - Note the visuals of the search description
5 - Apply patch
6 - Repeat searches
7 - All should work as before
Signed-off-by: Andrew Fuerste-Henry <andrew@bywatersolutions.com>
Signed-off-by: Joonas Kylmälä <joonas.kylmala@iki.fi>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Default stylesheets do not reference item fields for XSLT display, however, we
spend time translating the values in the item fields.
This patch adds a system preference, PassItemMarcToXSLT. and unless enabled we remove
item fields before processing
To test:
1 - Perform some search on the staff client and opac
2 - Use the console (F12) to view the time spent on the network tab
3 - Note performance
4 - Apply patch, updatedatabase, restart_all
5 - Repeat searches
6 - Note that display has not changed
7 - Note performance, results should display slightly faster
Signed-off-by: Emmi Takkinen <emmi.takkinen@koha-suomi.fi>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
This patch adds the cni/Control-number-identifier index to enable
searches to use the 003 field.
Test plan
1/ Apply patch
2/ Re-index using updated configurations
3/ Confirm cni:number searches yield the expected results
4/ Signoff
Split-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Pasi Kallinen <pasi.kallinen@koha-suomi.fi>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Use a hash for better readability and reusability.
Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
On bug 17591 we discovered that there was something weird going on with
the way we export and use subroutines/modules.
This patch tries to standardize our EXPORT to use EXPORT_OK only.
That way we will need to explicitely define the subroutine we want to
use from a module.
This patch is a squashed version of:
Bug 17600: After export.pl
Bug 17600: After perlimport
Bug 17600: Manual changes
Bug 17600: Other manual changes after second perlimports run
Bug 17600: Fix tests
And a lot of other manual changes.
export.pl is a dirty script that can be found on bug 17600.
"perlimport" is:
git clone https://github.com/oalders/App-perlimports.git
cd App-perlimports/
cpanm --installdeps .
export PERL5LIB="$PERL5LIB:/kohadevbox/koha/App-perlimports/lib"
find . \( -name "*.pl" -o -name "*.pm" \) -exec perl App-perlimports/script/perlimports --inplace-edit --no-preserve-unused --filename {} \;
The ideas of this patch are to:
* use EXPORT_OK instead of EXPORT
* perltidy the EXPORT_OK list
* remove '&' before the subroutine names
* remove some uneeded use statements
* explicitely import the subroutines we need within the controllers or
modules
Note that the private subroutines (starting with _) should not be
exported (and not used from outside of the module except from tests).
EXPORT vs EXPORT_OK (from
https://www.thegeekstuff.com/2010/06/perl-exporter-examples/)
"""
Export allows to export the functions and variables of modules to user’s namespace using the standard import method. This way, we don’t need to create the objects for the modules to access it’s members.
@EXPORT and @EXPORT_OK are the two main variables used during export operation.
@EXPORT contains list of symbols (subroutines and variables) of the module to be exported into the caller namespace.
@EXPORT_OK does export of symbols on demand basis.
"""
If this patch caused a conflict with a patch you wrote prior to its
push:
* Make sure you are not reintroducing a "use" statement that has been
removed
* "$subroutine" is not exported by the C4::$MODULE module
means that you need to add the subroutine to the @EXPORT_OK list
* Bareword "$subroutine" not allowed while "strict subs"
means that you didn't imported the subroutine from the module:
- use $MODULE qw( $subroutine list );
You can also use the fully qualified namespace: C4::$MODULE::$subroutine
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
This patch adds a new option 'no_items' and uses it in the C4::Search::searchResults
routine. We don't use the item info fetched here, so skipping those lines saves us time.
Additionally, I fix an incorrect FIXME comment, and pass the ISBN returned by the routine above
into GetNormalizedISBN to save another lookup
TO test:
1 - Enable AmazonCoverImages system preference
2 - Search staff client with a term that returns books with covers
3 - Apply patch
4 - prove -v t/db_dependent/Biblio/TransformMarcToKoha.t
5 - Confirm searching works
6 - Confirm Amazon images display (normalized_isbn is used for these)
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
[EDIT] Amended, replacing a tab character.
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
There are too many warnings about uninitialized variables in Search.pm.
This patch deals with that, the same way it is dealt accross the file:
by setting an empty string when things are not defined.
To test:
1. Run:
$ kshell
k$ prove t/db_dependent/Search.t
=> FAIL: Wow, too many warnings. Some related to this bug, some not.
2. Apply this patch
3. Repeat 1
=> SUCCESS: No more warnings!
4. Sign off :-D
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
We are using Koha::Logger when it makes sense to keep the info,
otherwise we simply remove it
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Bug 28572: Replace missing occurrence in misc/admin/koha-preferences
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Bug 10584 made Koha hide biblios for which all items match some criteria (in OpacHiddenItems) so they are hidden.
Add syspref OpacHiddenItemsHidesRecord controlling this behaviour.
Test plan :
1)
1.1) Create some biblio records with one item having damaged=1
1.2) Define system preference OpacHiddenItems = damaged: 1
2)
2.1) Set system preference OpacHiddenItemsHidesRecord to 'don't hide'
2.2) At OPAC : perform a search showing those records and some more
2.3) Check you see the all the records
2.4) For a record with hidden item check you don't get HTTP 404 for : normal view, ISBD view, MARC view
2.5) Check you can had tags on this record
2.6) Add record to basket, check you see it in basket
3)
3.1) Set system preference OpacHiddenItemsHidesRecord to 'hide'
3.2) At OPAC : perform a search showing those records and some more
3.3) Check you don't see the records with hidden item
3.4) For a record with hidden item check you get HTTP 404 for : normal view, ISBD view, MARC view
3.5) Show basket, check you see the records with hidden item
Signed-off-by: Kelly McElligott <kelly@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
This patch switches from calling SimpleSearch directly to using the compat
method so that Elastic is used when it is the chosen search engine
To test:
1 - Set SearchEngine syspref to 'Zebra'
2 - Add a suggestion with an existing title, e.g.:
'E street shuffle'
3 - Confirm you get a duplication warning
4 - Change SearchEngine to ES
5 - Repeat get same warning
6 - sudo koha-zebra --stop
'ps aux | grep zebra' to confirm they are stoped
'sudo pkill -9 zebra' if they arent;
7 - Repeat suggestion - no duplication warning (because zebra returns nothing)
8 - Delete the suggestion
9 - Apply patches
10 - Restart all the things, stop zebra
11 - Repeat suggestion
12 - Successful warning from ES
13 - prove -v t/db_dependent/Search.t
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
When you edit and save items with an empty "c - Shelving location"
field, it gets stored into the database as "NULL",
which is allowed in the database, and the field is not "required",
so this seems expected.
But when the search page is rendered, it gets value by location key
($shelflocations->{ $item->{location} }) which gives undef warning.
Solved by checking if $item->{location} is perl true.
To reproduce:
1) Edit the book or create new one with an empty shelving location.
2) Use the search feature to have that book in the searched list.
3) Check plack-intranet-error.log to find "Use of uninitialized
value in hash element" error.
4) Apply the patch.
5) Repeat the search and check the logs again to ensure that error
didn't appear again.
Signed-off-by: Didier Gautheron <didier.gautheron@biblibre.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
From tht YAML pod:
"""
This module has been released to CPAN as YAML::Old, and soon YAML.pm will be changed to just be a frontend interface module for all the various Perl YAML implementation modules, including YAML::Old.
If you want robust and fast YAML processing using the normal Dump/Load API, please consider switching to YAML::XS. It is by far the best Perl module for YAML at this time. It requires that you have a C compiler, since it is written in C.
"""
See also
https://gitlab.com/koha-community/qa-test-tools/-/merge_requests/35
Test plan:
Try some place where YAML::XS is not used and confirm that it works
correctly
QA note: This patch removes some uses of YAML that were not useful
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Joonas Kylmälä <joonas.kylmala@helsinki.fi>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
This feature is not used as far as we know and it's not known to work.
It's preferable to remove it.
Test plan:
Make sure the OpacGroupResults pref code is removed, as well as the
PazPar2 files and code.
Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
FindDuplicate has a hardcoded regex to remove 'and', 'or', and 'not' from any search fields, however,
is does this with no regard for these terms being embedded in the middle of a word or phrase.
E.g.:
'Coupland' becomes 'Coupl'
'Frog and toad' becomes 'frog toad'
etc.
This patch simplay removes the substitutions as we already quote the terms as well
To test:
1 - Add records to your system like:
The night circus by Erin Morgenstern
Frog and toad all year
Nothing ever happens
2 - Attempt to add purchase suggestions like
title: the night circus, author:Morgenstern, Erin
title: For and toad all year
title:nothing ever happens (case is important)
3 - No warnings about existing biblios
4 - Apply patches
5 - Delete existing suggestions
6 - Repeat step 2
7 - Each attempt should warn you of existing biblio
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
The system preference "QueryWeightFields" is mutually exclusive with
the system preference "QueryAutoTruncate" and the * truncation
modifier, when not using QueryParser.
If you use truncation, relevance won't work anymore. (N.B. Relevance
doesn't work probably when using QueryParser, but for a very different
reason beyond the current scope of this bug.)
This patch adds relevance ranking when using truncation (or basically
when QueryWeightFields is disabled).
_TEST PLAN_
1) Turn on "QueryWeightFields" and set "QueryAutoTruncate" to auto
2) Turn off "UseQueryParser"
3) Do a keyword search in the OPAC
4) Note that the results are ordered by biblionumber ascending
5) Apply patch
6) Do the same keyword search in the OPAC
7) Note that the results are never ordered differently
(presumably in relevance order :P)
If you're a stickler, throw in some warns so that you can see
what CCL query is being sent to Zebra.
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Rather than limiting initials to [A-Z] we should test for a broad
range of uppercase letters.
The ES/Zebra changes are slightly different because of Perl vs Java regex
conventions. POerl may support either, but I found 'Uppercase' to be a bit more explicit
More info here:
https://perldoc.perl.org/perlunicode.html
TO test:
Same plan as before but use Ж. as the ending initial
Confirm the period is preserved and other punctuation removed
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
The current code for facets doesn't pull strip ending punctuation from facets
This causes duplicate facets for terms that should be combined
Sometimes series can have different punctuation depending on the field they are in
Author initials punctuation should be preserved
To test:
1 - Do search and pull up some records
2 - Edit some of the records to have authors like:
Date, C.J.
Date, C.j.
Date, C.J .
3 - Edit the records to have some series statments like:
830 $aDate, C.J. ;$v5
830 $aDate, C.J. ; $v5
830 $aDate, C.J.; $v5
4 - Add some 490s to the record with first indicator 1 and series like:
You wouldn't want to--
You wouldn't want to
You wouldn't want to..
5 - Search again and note you have 3 facets each for author and series
6 - Apply patch
7 - Repeat
8 - Now you get 2 facets for author, period not removed when following Upper case immediately, is otherwise
9 - Now you should have a single series facet
10 - Switch search engine to ES (index before applying patch)
11 - Note facets are separate again
12 - Reset mappings and reindex
perl misc/search_tools/rebuild_elasticsearch -v -r
13 - Repeat search, facets combined as above
Signed-off-by: Sarah Cornell <sbcornell@cityofportsmouth.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
This takes care of more occurences of staff client and changes it to
staff interface, including in code comments.
To test:
- I think in this case careful code review is what we look for.
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
It appears that we are quoting the ccode values deep in the search code.
Under ICU chains this breaks searching by limits
To recreate
1 - Setup Koha using Zebra and icuchains
2 - Add ccode to AdvancedSearchTypes
3 - In koha-conf.xml set zebra debug level to include request
<zebra_loglevels>none,fatal,warn,request,info</zebra_loglevels>
4 - Set some items into different ccodes
5 - On opac perform a search for:
ccode:NFIC
6 - It works
7 - tail -n 50 /var/log/koha/kohadev/zebra-output.log
8 - Note search request like:
Search biblios OK 26 1 1+0 RPN @attrset Bib-1 @attr 1=8009 NFIC
9 - On opac go to advanced search, select Collection, and limit to smae code a s above
10 - No results
11 - Check the zebra-output.log:
Search biblios OK 0 1 1+0 RPN @attrset Bib-1 @attr 1=8009 'NFIC'
12 - Apply patch
13 - Restart all
14 - Repeat search by collection limit
15 - Success!
16 - Check the zebra-output.log:
Search biblios OK 0 1 1+0 RPN @attrset Bib-1 @attr 1=8009 NFIC
17 - Add a new ccode value: N)N
18 - Set some items to that ccode
19 - Confirm searching by that ccode works
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Alex Arnaud <alex.arnaud@biblibre.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
This patch adds a .perlcriticrc (copied from qa-test-tools) and fixes
almost all perlcrictic violations according to this .perlcriticrc
The remaining violations are silenced out by appending a '## no critic'
to the offending lines. They can still be seen by using the --force
option of perlcritic
This patch also modify t/00-testcritic.t to check all Perl files using
the new .perlcriticrc.
I'm not sure if this test script is still useful as it is now equivalent
to `perlcritic --quiet .` and it looks like it is much slower
(approximatively 5 times slower on my machine)
Test plan:
1. Run `perlcritic --quiet .` from the root directory. It should output
nothing
2. Run `perlcritic --quiet --force .`. It should output 7 errors (6
StringyEval, 1 BarewordFileHandles)
3. Run `TEST_QA=1 prove t/00-testcritic.t`
4. Read the patch. Check that all changes make sense and do not
introduce undesired behaviour
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
It seems that this regression comes from bug 14419, but I have not found
a logic reason behind that.
This patch restores the behaviour we always had: facet groups must be
displayed in a given order: the Zebra index (au, ccode, holdingbranch,
etc.)
Test plan:
Apply this patch, restart all and confirm that the facets are not moving
up and down when you refresh your search result page.
QA note:
I think the following line must be removed
@facets_loop = sort {defined $a->{expand} && defined $b->{expand} && $a->{expand} cmp $b->{expand}} @facets_loop;
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
This patch implements the use of the ViewPolicy record processor filter
inside C4::Search::searchResults. The idea is that the $record_processor
is instantiated once and reused inside the loop. This leaves options for
further optimizations I will do on a follow-up bug.
The filter is applied to the MARC data before it is passed to the XSLT
processor.
To test:
1. Apply the regression tests patch
2. Run:
$ kshell
k$ prove t/db_dependent/Search.t
=> FAIL: This is not implemented, tests fail
3. Apply this patch
4. Repeat 2
5. Feel the joy in your body from a long standing bug being solved
6. Sign off :-D
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
This patch makes use of the 'variables' parameter in XSLTParse4Display
method in the different places that it is used in the OPAC. It does by
passing this parameter with
anonymous_session => 1|0
The value will depend on the output from get_template_and_user (i.e. if
there's a returned borrowernumber).
A special case takes place in search results, as the call to
XSLTParse4Display happens in C4::Search::searchResults. So a new
parameter 'xslt_variables' is added to it.
To test:
1. Apply the [DO NOT PUSH] patch
2. Open the OPAC in your browser
3. Try detail pages, search results, tags and lists/shelves pages with
or without an active session
=> FAIL: It always says (somewhere) 'Anonymous session: Yes'
4. Apply this patch, restart_all
5. Repeat 3
=> SUCCESS: It will tell the Yes/No correctly regarding anonymous
sessions!
6. Sign off :-D
Sponsored-by: Universidad ORT Uruguay
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
When catalog has fields 773 (461 in UNIMARC), those linked items are fetched for display in search results.
Looks like the code could be more performant by replacing item search by $9 with direct call with GetMarcItem().
Test plan :
1) Enable system preference EasyAnalyticalRecords
2) Get a record A with a lot of items, it will be the 'Host record'
3) Click on 'Analytics' on this record
4) On one of its items (say item A1) click on 'Create analytics' to
create record B
5) Enter the same title as record A and all mandatory fields
6) Edit field 773 (461 in UNIMARC)
6) Check that $0 contains the biblionumber of record A
7) Check that $9 contains the itemnumber of item A1
8) Perform a search on record A title, be sure you see record B with
other results
=> Compare execution times with and without patch
Signed-off-by: Hugo Agud <hagud@orex.es>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
