TO test:
0 - Have a koha with ES enabled and some auth records
1 - backup your db
2 - sudo koha-mysql <<instance>>
3 - UPDATE auth_types SET authtypecode="PERSO_NAMB" WHERE
authtypecode="PERSO_NAME" (or adjust as necessary)
4 - Search for some authorities where a PERSO_NAME record will be
returned
5 - KA_BOOM!
6 - Apply patch
7 - Results are returned, they do not display a code, viewing record you
won't see code
Signed-off-by: Nicolas Legrand <nicolas.legrand@bulac.fr>
Signed-off-by: Alex Arnaud <alex.arnaud@biblibre.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
We likely shoudln't pass through an uncoverted sort order for now, but
it does allow us to look ahead to implementing the orders directly so
seems a good option to have.
Either this patch should be used, or the commented out tests should be
removed
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Currently sort order is extracted from sort condition by splitting the
field, instead use regular expression to extract the last part preceded
by underscore.
Signed-off-by: Nicolas Legrand <nicolas.legrand@bulac.fr>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
To test:
1 - Enable Zebra
2 - Perform an auth search
3 - note results
4 - Enable ES
5 - Repeat search, note (likely) diff results
6 - Open a record in cataloging and use the button to launch auth search
7 - Perform same search as above, note results match for eiher engine
selected
8 - NOTE: Disbale sorting for ES search - this will be dealt with in
another report
Signed-off-by: David Bourgault <david.bourgault@inlibro.com>
Signed-off-by: Nicolas Legrand <nicolas.legrand@bulac.fr>
Signed-off-by: Alex Arnaud <alex.arnaud@biblibre.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
To test:
0 - Apply Unit test patch only
1 - prove t/db_dependent/Koha_SearchEngine_Elasticsearch_Search.t
2 - Should fail
3 - Apply this patch
4 - prove t/db_dependent/Koha_SearchEngine_Elasticsearch_Search.t
5 - should pass
6 - search for 'Local-number.raw:"4"' (or a vlid biblionumber)
7 - should get expected result
Signed-off-by: David Bourgault <david.bourgault@inlibro.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
This patch adds a new statuc function to Koha::SearchEngine::ElasticSearch
which is instended to replace most of get_elasticsearch_params. This function
reads the configuration from C4::Context->config('elasticsearch') and raises
relevant exceptions when mandatory entries are missing.
Its behaviour is covered by tests.
To test:
- Run:
$ kshell
k$ prove t/Koha/SearchEngine/Elasticsearch.t
=> SUCCESS: Tests pass!
- Sign off :-D
Signed-off-by: David Bourgault <david.bourgault@inlibro.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
This code seems a bit odd, $self->sort_fields()->{$name} is checking if
there is any mapping in ES for a field, so if one marc field in an index
is marked as sortable this will exists. We need to check the individual
marc field to see if it should be added to the sort index here.
If you apply the first patch, reindex, and view a record in es:
curl -XGET 'localhost:9200/koha_kohadev_biblios/data/19/?pretty' | grep
-A 10 author__sort
You will see that 245$c is included in the record without that
additional check
Apply this, reindex again, and that field should not be included
Signed-off-by: David Bourgault <david.bourgault@inlibro.com>
Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Fix incorrect condition for if clause for generating <field>__sort mappings
for Elasticsearch. Also remove redundant check for same condition when
generating fixer rules.
Test plan:
1. Inspect current mappings for example by viewing: http://<elasticsearch_host>:9200/koha_<koha_instance_name>_biblios/_mapping.
2. If using the default configuraion only "author" has a sort field (author__sort).
4. Appy patch.
5. Reindex using rebuild_elastic_search.pl.
6. All fields except those with sort sort set to "0" should now have sort fields, which in the default configuration is all but "author".
Signed-off-by: David Bourgault <david.bourgault@inlibro.com>
Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Test plan assumes
QueryAutoTruncate = automatically
SearchEngine = Elasticsearch
To test:
0 - Apply Unit test patch only
1 - prove t/db_dependent/Koha_SearchEngine_Elasticsearch_Search.t
2 - Should fail
3 - Apply this patch
4 - prove t/db_dependent/Koha_SearchEngine_Elasticsearch_Search.t
5 - should pass
6 - search for 'Local-number:"4"' (or a vlid biblionumber)
7 - should get expected result
Signed-off-by: David Bourgault <david.bourgault@inlibro.com>
Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
This patch adds unit tests for the introduced changes in
build_query_compat.
It removes a warning too.
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
To test:
1 - Enable suppression
2 - Suppress some records
3 - Apply all the patches
4 - Reindex ES
5 - Search and don't get suppressed records
6 - Disable suppression
7 - Search and get all the records
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
To test:
OPAC: Both SearchEngine "Elasticsearch" and "Zebra" should work with
OpacSuppression set to "yes"
NB: OPAC suppression is not implemented for Elasticsearch
Signed-off-by: David Bourgault <david.bourgault@inlibro.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
By using a different split regex, we can simplify a bit the process of
appending '*' to every word of the query
Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
To test:
1 - prove t/db_dependent/Koha_SearchEngine_Elasticsearch_Search.t
2 - do some searches in staff client and test results
Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
This patchset adds a subroutine '_truncate_terms' to the ES QueryParser.
If QueryAutoTruncate is enabled this function will be called for any
search to add wildcard '*' to all terms
To test:
1 - Enable Elasticsearch and have some records indexed
2 - Search for partial terms
3 - Note they fail unless '*' is appended
4 - Apply patch, leave QueryAutoTruncate disabled
5 - Note partial term searches still fail
6 - Enable QueryAutoTruncate
7 - Note partial term searches succeed
8 - Do some regular and advanced searches to make sure results are as
expected
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
You must install the icu plugin for elasticsearch
https://www.elastic.co/guide/en/elasticsearch/plugins/current/analysis-icu.html
Once installed, apply this patch
Reindex your data, deleting the existing indexes
perl /home/vagrant/kohaclone/misc/search_tools/rebuild_elastic_search.pl
-d
Find (or add) some titles with accented characters
Verify that a search for the exact character or the unaccented version
works
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
I dug into this a little bit, it seems the issue is that we were double
processing the query because of an extra call to
C4::AuthoritiesMarc::SearchAuthorities
I think we can simply remove this call
To test:
1 - Perform an authority search for a value that returns results
2 - Wrap it in quotes and search
3 - 'Internal server error'
4 - Apply patch
5 - Search with quotes succeeds
https://bugs.koha-community.org/show_bug.cgi?id=16796
Signed-off-by: Lee Jamison <ldjamison@marywood.edu>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
It seems we have a syntax mismatch - any vs all
all seems to the be list we expect so updating code that way
To test:
1 - Enabled Elasticsearch
2 - Index some authorities
3 - Perform a 'Search entire record' search
4 - Internal server error (
Invalid marclist field provided: all at
/usr/local/koha/Koha/SearchEngine/Elasticsearch/QueryBuilder.pm
line 433.
)
5 - Run:
$ sudo koha-shell kohadev
k$ cd kohaclone
k$ prove t/db_dependent/Koha/SearchEngine/Elasticsearch/QueryBuilder.t
=> FAIL: Tests fail because 'any is used'
6 - Apply patch
7 - Search should work
8 - Run:
k$ prove t/db_dependent/Koha/SearchEngine/Elasticsearch/QueryBuilder.t
=> SUCCESS: Tests pass!
9 Sign off :-D
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Edited the test plan so it mentions the new tests
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
There was a bug that meant a very large offset in the search params
will cause the search script to run forever (or long enough to crash
the machine)
To test
1/ Get ready with sudo top so you can kill the thread before it causes
your machine to OOM
2/ Hit a page like yourdomain.com/cgi-bin/koha/opac-search.pl?q=1&offset=-9999999999999999999
3/ Notice the process runs for a long time
4/ Kill the process
5/ Apply the patch
6/ Hit the page again, notice the it loads (offset is set to zero)
7/ Do the same to search in the staff client
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
To test:
1 - Index some stuff with multiple fields defined for sorting
i.e. Authorites - make heading sortable - default is 110a and 111a for
heading - a record with 111a empty will make the sort field empty
2 - view the record:
curl http://localhost:9200/koha_kohadev_authorities/data/30?pretty=true
3 - Note the blank field
4 - Apply patch
5 - Reindex
6 - Fields should be correctly populated
Unit tests to follow (once I have the originals working for all)
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
To test:
1 - Make sure you have latest koha deps, catmandu versions should be:
libcatmandu-marc-perl 1.09-1~kohadev1
libcatmandu-perl 1.0304-2~kohadev1
2 - Reindex elastic
3 - Try searching and likely notice odd results
4 - Try:
curl -XGET
'http://localhost:9200/koha_kohadev_biblios/data/792?pretty=true'
with a known biblionumber and notice some null fields
5 - Apply patch
6 - Reindex
7 - Note fields are populated and search works as expected
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
When we will replace the Zebra code in sub merge, we will call SearchEngine
to pass records and we need a routine to extract a biblionumber from
a search result record. A record from Zebra still must be converted to
MARC::Record. This is no longer needed for a ElasticSearch record.
Test plan:
Run t/db_dependent/Koha/SearchEngine/Search.t
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Marc Véron <veron@veron.ch>
Signed-off-by: Jacek Ablewicz <abl@biblos.pk.edu.pl>
Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
To test:
1 - Have ES enabled and biblios and authorities indexed
2 - Perform a catalog search
3 - Try to sort results by author, it will fail
4 - Perform an authority search with sorting enabled, it will fail
5 - Apply patch
6 - Reindex, deleting current indexes
perl /home/vagrant/kohaclone/misc/search_tools/rebuild_elastic_search.pl
-d -v
7 - Repeat 3&4, they should succeed
Signed-off-by: Srdjan <srdjan@catalyst.net.nz>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
This patch allows you to use the "qualifier,qualifier" syntax
in the Record Matching Rules "Search Index" when using the QueryParser.
While QueryParser doesn't support this syntax, it will now fallback
correctly to non-QueryParser functionality. Without the patch, your search
will just fail silently.
This patch also adds a "skip_normalize" option to C4::Search::SimpleSearch(),
and uses the option during C4::Matcher::get_matches. This prevents
the s/:/=/g and s/=/:/g normalization. This normalization is heavy-handed,
and while it is necessary sometimes to generate a valid CCL query or
QueryParser query, C4::Matcher::get_matches() already produces a valid
CCL query, so we don't need to do this normalization.
Additionally, this normalization causes problems when you use a
Zebra register which isn't normalized: namely the "u" register.
Strings are stored "as is", so http://localhost/koha/resource/1 is
stored as is during indexing. When you search, you need to pass
the exact same thing through the query to get a match. Using
http=//localhost/koha/resource/1 in your query will yield zero results.
_TEST PLAN_
0) Apply patch
1) Create a Record Matching Rule in Koha with the following details:
Matching rule code: TEST
Description: Test
Match threshold: 100
Record type: Bibliographic
Match point 1:
Search index: id-other,st-urx
Score: 100
Tag: 024
Subfields: a
Normalization rule: None
2) Create a record in Koha with an indexable URI
2a) Default framework
2b) 024 $a http://koha-community.org/test $2 uri
2c) 040 $c test
2d) 245 $a This is a test record
2e) 942 $c Books
2f) Save (save again if cautioned about missing fields as these should autofill)
3) Do a full re-index of Zebra
4) Download your record from Koha as a .mrc file (ie isomarc, binary marc, etc)
5) Go to "Stage MARC records for import"
5a) Upload your .marc file.
5b) Change your "Record matching rule" to "Test"
5c) Click Stage for import
9) It should say "1 records with at least one match in catalog per matching rule "Test".
NOTE: For completeness, you can go through this process on a clean master branch,
and note that it will say '0 records with at least one match in catalog per matching rule "TEST"'
Signed-off-by: Alex Buckley <alexbuckley@catalyst.net.nz>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
To test:
1 - Apply patch
2 - Backup your db
3 - Drop and create a new db to ensure your mappings are refreshed from
the patch
4 - add some titles with items with collection codes
5 - search and see collection code facets
6 - sign off
Work to be done:
1 - Replace codes with descriptions
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Brendan A Gallagher <brendan@bywatersolutions.com>
The loading of file admin/searchengine/elasticsearch/mappings.yaml
specifies 'type' as empty, which fails with Mysql 5.7+ which is more
stringent.
Also, forcing an empty value into a boolean also fails.
Both issues arise when updatedatabase.pl hit 3.23.00.050.
NOTE: both issues could also be resolved by actually setting values in
the load file. This doesn't make this solution incorrect, though.
To Test/reproduce:
-1) Happens with Mysql 5.7.4. Maybe earlier, but certainly at
that point. Use a Xenial kohadevbox to more easily test.
0) Find a database on 3.22 or earlier, save it.
a) place
sql_mode=IGNORE_SPACE,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION
into the [mysqld] section of /etc/mysql/mysql.conf.d/mysqld.cnf
b) restart you mysql server
c) drop the db and recreate it
d) checkout the 3.22.x branch
e) do a web install
f) remove the added sql_mode
g) restart the mysql server
1) Set your code base to master
2) run updatedatabase.pl
3) See the errors on 3.23.00.050
4) Apply the patch
5) Reload the 3.22 db.
a) repeat steps 0(a)-0(g)
b) don't forget the caching issues
6) succeed with updatedatabase.pl
7) drop the db and recreate it
8) run the web installer
9) notice no issues either.
10) run koha qa test tools
NOTE: This bug only solved the upgrade portion.
I added the kohastructure.sql change as well.
I confirmed that all the code changes were
triggered with this test plan.
Signed-off-by: Mark Tompsett <mtompset@hotmail.com>
Signed-off-by: Mehdi Hamidi <mehdi.hamidi@inlibro.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
This patch makes the 'Locations' facet work as expected (i.e. having the
same behaviour it has for Zebra: picking the 952$c in MARC21 and 995e
for UNIMARC).
It also adds the code to handle holding and home library settings for
facets and makes the facets show the library name instead of the branch
code.
The mappings are updated so the labels match what facets.inc expect to
work properly.
To test:
- On master, do a search that returns biblios with items having
homebranch set.
=> FAIL: Under the 'Locations' label on the facets you will notice
branchcodes are shown.
- Apply the patch
- Restart memcached and plack (just in case, it was tricky)
- Reset your mappings:
http://localhost:8081/cgi-bin/koha/admin/searchengine/elasticsearch/mappings.pl?op=reset&i_know_what_i_am_doing=1
- Restart memcached and plack (again, not sure if needed)
- Make sure this mappings are set:
homebranch => HomeLibrary
holdingbranch => HoldingLibrary
(Note: it might not be set due to the place the yaml file is being picked)
- Reindex your records:
$ sudo koha-shell kohadev
k$ cd kohaclone
k$ perl misc/search_tools/rebuild_elastic_search.pl -d -v
- Repeat the initial search
=> SUCCESS: 'Location' contains the right stuff, 'Home libraries' and
'Holding libraries' too.
- Run
k$ prove t/db_dependent/Koha_SearchEngine_Elasticsearch_Search.t
=> SUCCESS: Tests pass!
- Sign off :-D
Note: play with the 'DisplayLibraryFacets' syspref options.
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Builds on top of commit:
Bug 17255 - Upgrade Elastic Search code to work with version 2.4+ - rebased wip
-Fix data type 'string' to 'keyword' and 'text'
-index: not_analyzed deprecated, replaced with type: keyword which is equivalent
-store: yes was deprecated, use store: true
TODO: Installer bindings to both the debian package install and the raw developer install.
A taster in Buug 17851
ZE TEST PLAN
0. Remove existing ES and reinstall ES 5.1
apt-get purge elasticsearch
Follow instruction here:
https://www.elastic.co/guide/en/elasticsearch/reference/current/deb.html
1. Reset Elasticsearch index since facets are hard coded to dynamic search_marc_mappings.
1a. perl -e 'use Koha::SearchEngine::Elasticsearch; Koha::SearchEngine::Elasticsearch->reset_elasticsearch_mappings();'
1b. If you get trouble, simply DELETE FROM [search_fields|search_marc_to_fields|search_marc_mapping];
and retry 1a.
1c. Destroy elasticsearch index
curl -XDELETE localhost:9200/koha_biblios
so it can be recreated
2. Recreate the index:
perl misc/search_tools/rebuild_elastic_search.pl
2a. Add something to index if your koha.biblio-table is empty
3. Fetch all indexed records and the facet for subject__facet
curl -XGET localhost:9200/koha_biblios/data/_search?pretty -d '{
"aggregations": {
"my_agg": {
"terms": {
"field": "subject__facet"
}
}
}
}'
4. Run the included tests:
perl t/db_dependent/Koha_Elasticsearch_Indexer.t
perl t/db_dependent/Koha_Elasticsearch.t
perl t/db_dependent/Koha_SearchEngine_Elasticsearch_Search.t
Have fun with your new ES 5.1 cluster!
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
-Changed deprecated facets to aggregations
-Fixed boolean datatypes not allowing analyzers to be specified
-Fixed deprecated '_id' to 'es_id'. Now the ES-index has the correct id==biblionumber
ZE TEST PLAN
1. Reset Zebra index since facets are hard coded to dynamic search_marc_mappings.
2. perl misc/search_tools/rebuild_elastic_search.pl
3. Fetch all indexed records and the facet for subject__facet
curl -XGET localhost:9200/koha_biblios/data/_search?pretty -d '{
"aggregations": {
"my_agg": {
"terms": {
"field": "subject__facet"
}
}
}
}'
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Elastic appears to be indexing onloan as a string, but our code assumes
it is a boolean.
Test Plan:
1) Ensure you are set up using Elastic as your search engine
2) Search only for available items from the advanced search
3) Note you get no results
4) Apply this patch
5) Re-run the search
6) You should now get results!
Signed-off-by: Jennifer Schmidt <jschmidt@switchinc.org>
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
There is a bug in Koha::SearchEngine::Elasticsearc::Search->json2marc, it
assumes that the MARC::Field constructor always takes >= 5 parameters.
This assumption is wrong for control fields, to create a control field you
need to call the constructor with:
MARC::Field->new($tag, $value);
Note that I got "Too much data for control field" in the _warning value of my
MARC::Field because too many parameters were passed, and the value was undef.
That broke the result search links (on the staff interface) for DB with
biblio.biblionumber mapped with 001.
Other bugs will certainly be fixed by this patch.
For instance I got:
GetMarcBiblio called with undefined biblionumber at /home/koha/src/opac/opac-search.pl line 664.
GetCOinSBiblio called with undefined record at /home/koha/src/opac/opac-search.pl line 665.
in my logs, with this patch I don't get them anymore.
Test plan:
You can try to recreate the different issues and confirm than this patch
fixes them.
Or just run the tests
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
What we currently have:
Koha/ElasticSearch.pm
Koha/ElasticSearch/Indexer.pm
Koha/SearchEngine/Elasticsearch/QueryBuilder.pm
Koha/SearchEngine/Elasticsearch/Search.pm
What we want:
Koha/SearchEngine/Elasticsearch.pm
Koha/SearchEngine/Elasticsearch/Indexer.pm
Koha/SearchEngine/Elasticsearch/QueryBuilder.pm
Koha/SearchEngine/Elasticsearch/Search.pm
Test plan:
% git grep -i Koha::ElasticSearch
% git grep ElasticSearch|grep -v Catmandu::Store::ElasticSearch
should not return any result
Do a full reindex and search for records
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Brendan Gallagher <brendan@bywatersolutions.com>
Probably a leftover from the original SolR code, there are unused references to Moose in the new Elasticsearch code.
To test:
- Run:
$ git grep Moose
=> FAIL: There are unneeded references to Moose:
Koha/SearchEngine/Zebra/QueryBuilder.pm:use Moose::Role;
Koha/SearchEngine/Zebra/Search.pm:#use Moose::Role;
- Apply the patch
- Run:
$ git grep Moose
=> SUCCESS: No references to Moose
- Run:
$ prove t/Search* t/db_dependent/Search*
=> SUCCESS: Tests pass
- Sign off
Signed-off-by: Jacek Ablewicz <abl@biblos.pk.edu.pl>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Jesse Weaver <jweaver@bywatersolutions.com>
Before this patch it was 24.6%, after it is 64%
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Jesse Weaver <jweaver@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Brendan Gallagher <brendan@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@unc.edu.ar>
Signed-off-by: Chris <chrisc@catalyst.net.nz>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Jesse Weaver <jweaver@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Brendan Gallagher <brendan@bywatersolutions.com>
When Zebra is defined as the search engine, the authority search is
broken (Can't locate object method "simple_search_compat" via package
"Koha::SearchEngine::Zebra::Search" at /home/koha/koha/C4/AuthoritiesMarc.pm line 357.)
simple_search_compat is not defined in
Koha::SearchEngine::Zebra::Search, it should be a wrapper for
C4::Search::SimpleSearch
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Jesse Weaver <jweaver@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Brendan Gallagher <brendan@bywatersolutions.com>
By default ES returns the facet terms ordered by most used, which makes
sense.
This patch removes resort done in the scripts (catalogue/search.pl and
opac/opac-search.pl) and moves it to the module.
For Zebra it's now done in C4::Search::getRecords, and there is no
change to expect (still alphabetically).
On the Elastic search side, we could imagine to let the library define
the order of the facets. The facet terms are now sorted by most used.
To test easily this change, turn on the displayFacetCount pref.
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Jesse Weaver <jweaver@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Brendan Gallagher <brendan@bywatersolutions.com>
This reverts commit cd4905c2969b067476881016d0b03271f0bcc7c8.
This commit caused an error in C4::Search::GetFacets when running in
zebra mode.
Conflicts:
Koha/SearchEngine/Elasticsearch/Search.pm
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Jesse Weaver <jweaver@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Brendan Gallagher <brendan@bywatersolutions.com>
The system preference FacetMaxCount should work as expected with ES.
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Jesse Weaver <jweaver@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Brendan Gallagher <brendan@bywatersolutions.com>
If no limit are passed, the url will contain '&limit=' anyway.
It is not necessary and can be avoided easily
Test plan:
1/ Search for a term in your catalogue
2/ Hover over a link in the facet area
3/ The link is
cgi-bin/koha/opac-search.pl?idx=kw&q=your_term&limit=&limit=[...]
With this patch, the empty limit parameter does not appear.
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Jesse Weaver <jweaver@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Brendan Gallagher <brendan@bywatersolutions.com>
By default ES returns the facet terms ordered by most used, which makes
sense.
This patch removes resort done in the scripts (catalogue/search.pl and
opac/opac-search.pl) and moves it to the module.
For Zebra it's now done in C4::Search::getRecords, and there is no
change to expect (still alphabetically).
On the Elastic search side, we could imagine to let the library define
the order of the facets. The facet terms are now sorted by most used.
To test easily this change, turn on the displayFacetCount pref.
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Jesse Weaver <jweaver@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Brendan Gallagher <brendan@bywatersolutions.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Jesse Weaver <jweaver@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Brendan Gallagher <brendan@bywatersolutions.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Jesse Weaver <jweaver@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Brendan Gallagher <brendan@bywatersolutions.com>
David Gustafsson
Srdjan
Julian Maurice
Blou
Olli-Antti Kivilahti
Jesse Weaver
Chris Cormack
Tomas Cohen Arazi
Robin Sheat