Browse Source
Improvements: 1) Index settings moved from code to etc/searchengine/elasticsearch/index_config.yaml. An alternative can be specified in koha-conf.xml. 2) Field settings moved from code to etc/searchengine/elasticsearch/field_config.yaml. An alternative can be specified in koha-conf.xml. 3) mappings.yaml has been moved from admin/searchengine/elasticsearch to etc/searchengine/elasticsearch. An alternative can be specified in koha-conf.xml. 4) Default settings have been improved to remove punctuation from phrases used for sorting etc. 5) State variables are used for storing configuration to avoid parsing it multiple times. 6) A possibility to reset the fields too has been added to the reset operation of mappings administration. 7) mappings.yaml has been moved from admin/searchengine/elasticsearch to etc/searchengine/elasticsearch. 8) An stdno field type has been added for standard identifiers. To test: 1) Run tests in t/Koha/SearchEngine/Elasticsearch.t 2) Clear tables search_fields and search_marc_map 3) Go to admin/searchengine/elasticsearch/mappings.pl?op=reset&i_know_what_i_am_doing=1 4) Verify that admin/searchengine/elasticsearch/mappings.pl displays the mappings properly, including ISBN and other standard number fields. 5) Index some records using the -d parameter with misc/search_tools/rebuild_elastic_search.pl to recreate the index 6) Verify that you can find the records 7) Put <elasticsearch_index_mappings>non_existent</elasticsearch_index_mappings> to koha-conf.xml 8) Verify that admin/searchengine/elasticsearch/mappings.pl?op=reset&i_know_what_i_am_doing=1 fails because it can't find non_existent. 9) Copy etc/searchengine/elasticsearch/mappings.yaml to a new location and make elasticsearch_index_mappings setting in koha-conf.xml point to it. 10) Make a change in the new mappings.yaml. 11) Clear table search_fields (mappings reset doesn't do it yet, see bug 20248) 12) Go to admin/searchengine/elasticsearch/mappings.pl?op=reset&i_know_what_i_am_doing=1 13) Verify that the changes you made are now visible in the mappings UI Signed-off-by: Nicolas Legrand <nicolas.legrand@bulac.fr> Signed-off-by: Nick Clemens <nick@bywatersolutions.com> Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>18.05.x
Ere Maijala
6 years ago
committed by
Jonathan Druart
10 changed files with 250 additions and 121 deletions
@ -0,0 +1,61 @@ |
|||||
|
--- |
||||
|
# General field configuration |
||||
|
general: |
||||
|
_all: |
||||
|
type: string |
||||
|
analyzer: analyser_standard |
||||
|
properties: |
||||
|
record: |
||||
|
store: true |
||||
|
type: text |
||||
|
# Search fields |
||||
|
search: |
||||
|
boolean: |
||||
|
type: boolean |
||||
|
null_value: false |
||||
|
integer: |
||||
|
type: integer |
||||
|
null_value: 0 |
||||
|
stdno: |
||||
|
type: text |
||||
|
analyzer: analyser_stdno |
||||
|
search_analyzer: analyser_stdno |
||||
|
fields: |
||||
|
phrase: |
||||
|
type: text |
||||
|
analyzer: analyser_stdno |
||||
|
search_analyzer: analyser_stdno |
||||
|
raw: |
||||
|
type: keyword |
||||
|
copy_to: _all |
||||
|
default: |
||||
|
type: text |
||||
|
analyzer: analyser_standard |
||||
|
search_analyzer: analyser_standard |
||||
|
fields: |
||||
|
phrase: |
||||
|
type: text |
||||
|
analyzer: analyser_phrase |
||||
|
search_analyzer: analyser_phrase |
||||
|
raw: |
||||
|
type: keyword |
||||
|
copy_to: _all |
||||
|
# Facets |
||||
|
facet: |
||||
|
default: |
||||
|
type: keyword |
||||
|
# Suggestible |
||||
|
suggestible: |
||||
|
default: |
||||
|
type: completion |
||||
|
analyzer: simple |
||||
|
search_analyzer: simple |
||||
|
# Sort |
||||
|
sort: |
||||
|
default: |
||||
|
type: text |
||||
|
analyzer: analyser_phrase |
||||
|
search_analyzer: analyser_phrase |
||||
|
fields: |
||||
|
phrase: |
||||
|
type: keyword |
@ -0,0 +1,34 @@ |
|||||
|
--- |
||||
|
# Index configuration that defines how different analyzers work. |
||||
|
index: |
||||
|
analysis: |
||||
|
analyzer: |
||||
|
# Phrase analyzer is used for phrases (phrase match, sorting) |
||||
|
analyser_phrase: |
||||
|
tokenizer: keyword |
||||
|
filter: |
||||
|
- icu_folding |
||||
|
char_filter: |
||||
|
- punctuation |
||||
|
analyser_standard: |
||||
|
tokenizer: icu_tokenizer |
||||
|
filter: |
||||
|
- icu_folding |
||||
|
analyser_stdno: |
||||
|
tokenizer: whitespace |
||||
|
filter: |
||||
|
- icu_folding |
||||
|
char_filter: |
||||
|
- punctuation |
||||
|
normalizer: |
||||
|
normalizer_keyword: |
||||
|
type: custom |
||||
|
filter: |
||||
|
- icu_folding |
||||
|
char_filter: |
||||
|
# The punctuation filter is used to remove any punctuation chars in fields that don't use icu_tokenizer. |
||||
|
punctuation: |
||||
|
type: pattern_replace |
||||
|
# The pattern contains all ASCII punctuation characters. |
||||
|
pattern: '([\x00-\x1F,\x21-\x2F,\x3A-\x40,\x5B-\x60,\x7B-\x89,\x8B,\x8D,\x8F,\x90-\x99,\x9B,\x9D,\xA0-\xBF,\xD7,\xF7])' |
||||
|
replacement: '' |
@ -0,0 +1,8 @@ |
|||||
|
$DBversion = 'XXX'; # will be replaced by the RM |
||||
|
if( CheckVersion( $DBversion ) ) { |
||||
|
$dbh->do( "ALTER TABLE search_field CHANGE COLUMN type type ENUM('', 'string', 'date', 'number', 'boolean', 'sum', 'isbn', 'stdno') NOT NULL COMMENT 'what type of data this holds, relevant when storing it in the search engine'" ); |
||||
|
|
||||
|
# Always end with this (adjust the bug info) |
||||
|
SetVersion( $DBversion ); |
||||
|
print "Upgrade to $DBversion done (Bug 20073 - Add new types for Elasticsearch fields)\n"; |
||||
|
} |
Loading…
Reference in new issue