.pm must not have -x
.t must have -x
.pl must have -x
Test plan:
Apply only the first patch, run the tests and confirm that the failures
make sense
Apply this patch and confirm that the test now returns green
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Implement optimized indexing for Elasticsearch
How to test:
1) Time a full elasticsearch re-index without this patch by running the
rebuild_elastic_search.pl with the -d flag:
`koha-shell <instance_name> -c "time rebuild_elastic_search.pl -d"`.
2) Apply this patch.
3) Time a full re-index again, it should be about twice at fast (for a
couple of thousand biblios, with fewer biblios results may be more
unpredictable).
Sponsored-by: Gothenburg University Library
Signed-off-by: Ere Maijala <ere.maijala@helsinki.fi>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
To test:
1 - Apply first patch and this patch
2 - prove -v t/db_dependent/Koha_Elasticsearch.t
3 - New tests will fail
4 - Apply followup patch
5 - Tests shuld pass
Signed-off-by: David Bourgault <david.bourgault@inlibro.com>
Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
The test files do not need to return 1
Patch generated with:
perl -p -i -e "s/^1;\n//xsm" t/**/*.t
Test plan:
git grep '^1;$' t/**/*.t
should not return any results
NOTE: does not fix C4/SIP/t, nor xt tests.
Signed-off-by: Mark Tompsett <mtompset@hotmail.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
This patch tries to introduce exhaustive tests for this class method.
I didn't try to provide a regression test for the current bug per-se, but
cover the current method behaviour as much as I could.
(kidclamp) I added a quick test of _convert_marc_to_json to use the mocking here
and illuminate what the change does, before the patches this should
fail (fields are indexed in place of one another), after it should succeed (new indexed fields are appended).
A minor bug is highlighted by this new tests, I'll provide a followup for it.
To test:
- Run:
$ sudo koha-shell kohadev
k$ de kohaclone
k$ prove t/db_dependent/Koha_Elasticsearch.t
=> FAIL: The returned fixer rules are not the expected ones
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Builds on top of commit:
Bug 17255 - Upgrade Elastic Search code to work with version 2.4+ - rebased wip
-Fix data type 'string' to 'keyword' and 'text'
-index: not_analyzed deprecated, replaced with type: keyword which is equivalent
-store: yes was deprecated, use store: true
TODO: Installer bindings to both the debian package install and the raw developer install.
A taster in Buug 17851
ZE TEST PLAN
0. Remove existing ES and reinstall ES 5.1
apt-get purge elasticsearch
Follow instruction here:
https://www.elastic.co/guide/en/elasticsearch/reference/current/deb.html
1. Reset Elasticsearch index since facets are hard coded to dynamic search_marc_mappings.
1a. perl -e 'use Koha::SearchEngine::Elasticsearch; Koha::SearchEngine::Elasticsearch->reset_elasticsearch_mappings();'
1b. If you get trouble, simply DELETE FROM [search_fields|search_marc_to_fields|search_marc_mapping];
and retry 1a.
1c. Destroy elasticsearch index
curl -XDELETE localhost:9200/koha_biblios
so it can be recreated
2. Recreate the index:
perl misc/search_tools/rebuild_elastic_search.pl
2a. Add something to index if your koha.biblio-table is empty
3. Fetch all indexed records and the facet for subject__facet
curl -XGET localhost:9200/koha_biblios/data/_search?pretty -d '{
"aggregations": {
"my_agg": {
"terms": {
"field": "subject__facet"
}
}
}
}'
4. Run the included tests:
perl t/db_dependent/Koha_Elasticsearch_Indexer.t
perl t/db_dependent/Koha_Elasticsearch.t
perl t/db_dependent/Koha_SearchEngine_Elasticsearch_Search.t
Have fun with your new ES 5.1 cluster!
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
What we currently have:
Koha/ElasticSearch.pm
Koha/ElasticSearch/Indexer.pm
Koha/SearchEngine/Elasticsearch/QueryBuilder.pm
Koha/SearchEngine/Elasticsearch/Search.pm
What we want:
Koha/SearchEngine/Elasticsearch.pm
Koha/SearchEngine/Elasticsearch/Indexer.pm
Koha/SearchEngine/Elasticsearch/QueryBuilder.pm
Koha/SearchEngine/Elasticsearch/Search.pm
Test plan:
% git grep -i Koha::ElasticSearch
% git grep ElasticSearch|grep -v Catmandu::Store::ElasticSearch
should not return any result
Do a full reindex and search for records
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Brendan Gallagher <brendan@bywatersolutions.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Jesse Weaver <jweaver@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Brendan Gallagher <brendan@bywatersolutions.com>
Signed-off-by: Robin Sheat <robin@catalyst.net.nz>
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
Input.t was replaced because it tested a now obsolete function.
Input.pm has that function commented out.
Several files were renamed to match their counterparts or
correct misspellingz.
Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
Jonathan Druart
Nick Clemens
Josef Moravec
Alex Arnaud
David Gustafsson
Tomás Cohen Arazi
Olli-Antti Kivilahti
Chris Cormack
Katrin Fischer
Chris Cormack
Joe Atzberger
rangi