To Test:
Check the following files that all all "biblio" or "biblo record" are
now writen in full as "bibliographic record".
modules/admin/authtypes.tt
modules/admin/biblio_framework.tt
modules/admin/preferences/authorities.pref
modules/admin/preferences/cataloguing.pref
modules/admin/preferences/circulation.pref
modules/admin/preferences/opac.pref
modules/admin/preferences/searching.pref
modules/admin/preferences/serials.pref
modules/admin/preferences/web_services.pref
modules/admin/searchengine/elasticsearch/mappings.tt
Corrected a single capitalization error during signoff
Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
This patch modifies several administration templates to use the
Bootstrap grid instead of YUI.
This patch also removes obsolete "text/javascript" attributes from
<script> tags and "text/css" attributes from <style> tags in the
modified templates.
To test, apply the patch and view the following pages, confirming that
they look correct at various browser widths:
- Navigate directly to /cgi-bin/koha/admin/printers.pl
- View and edit printers
- Administration -> Search engine configuration (or navigate directly
to /cgi-bin/koha/admin/searchengine/elasticsearch/mappings.pl
- Administration -> Circulation and fines rules
- Administration -> SMS cellular providers (enable SMSSendDriver or
navigate directly to /cgi-bin/koha/admin/sms_providers.pl)
- Administration -> Z39.50/SRU servers -> New SRU server -> Modify SRU
search fields mapping
- Administration -> System preferences -> Local use
- View and edit local use preferences
- Administration -> Transport cost matrix
- With UsageStats enabled, go to Administration -> Share your usage
statistics.
Signed-off-by: Claire Gravely <claire.gravely@bsz-bw.de>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
- Remove useless 'action' attribute in forms
- $op should never be equal to 'reset', so remove the test
- Move 'op' parameter inside <button>s to avoid having different HTML
tags for elements that have the same kind of behaviour
Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Improvements:
1) Mappings UI now has button that allows one to reset the mappings.
2) Mappings UI displays the items in alphabetical order.
3) Indexing script drops and recreates the index right away, which
helps prevent ES from autocreating a bad index if someone does something
while the first batch of records is being processed.
4) Indexing script has nicer output.
To test:
1) Change mappings.yaml file
2) Reset mappings in UI in mappings.pl
3) Verify the mappings have been changed in UI
4) The field order is alphabetical
5) Rebuild script has clean output
6) Run test t/db_dependent/Koha_Elasticsearch_Indexer.t
Signed-off-by: Bouzid Fergani <bouzid.fergani@inlibro.com>
Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Here we go, next step then.
As we did not fix the performance issue when autofiltering
the variables (see bug 20975), the only solution we have is to add the
filters explicitely.
This patch has been autogenerated (using add_html_filters.pl, see next
pathces) and add the html filter to all the variables displayed in the
template.
Exceptions are made (using the new 'raw' TT filter) to the variable we
already listed in the previous versions of this patch.
To test:
- Use t/db_dependent/Koha/Patrons.t to populate your DB with autogenerated
data which contain <script> tags
- Remove them from borrower_debarments.comments (there are allowed here)
update borrower_debarments set comment="html tags possible here";
- From the interface hit page and try to catch alert box.
If you find one it means you find a possible XSS.
To know where it comes from:
* note the exact URL where you found it
* note the alert box content
* Dump your DB and search for the string in the dump to identify its
location (for instance table.field)
Next:
* Ideally we would like to use the raw filter when it is not necessary
to HTML escape the variables (in big loop for instance)
* Provide a QA script to catch missing filters (we want html, uri, url
or raw, certainly others that I am forgetting now)
* Replace the html filters with uri when needed (!)
Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Improvements:
1) Index settings moved from code to etc/searchengine/elasticsearch/index_config.yaml. An alternative can be specified in koha-conf.xml.
2) Field settings moved from code to etc/searchengine/elasticsearch/field_config.yaml. An alternative can be specified in koha-conf.xml.
3) mappings.yaml has been moved from admin/searchengine/elasticsearch to etc/searchengine/elasticsearch. An alternative can be specified in koha-conf.xml.
4) Default settings have been improved to remove punctuation from phrases used for sorting etc.
5) State variables are used for storing configuration to avoid parsing it multiple times.
6) A possibility to reset the fields too has been added to the reset operation of mappings administration.
7) mappings.yaml has been moved from admin/searchengine/elasticsearch to etc/searchengine/elasticsearch.
8) An stdno field type has been added for standard identifiers.
To test:
1) Run tests in t/Koha/SearchEngine/Elasticsearch.t
2) Clear tables search_fields and search_marc_map
3) Go to admin/searchengine/elasticsearch/mappings.pl?op=reset&i_know_what_i_am_doing=1
4) Verify that admin/searchengine/elasticsearch/mappings.pl displays the mappings properly, including ISBN and other standard number fields.
5) Index some records using the -d parameter with misc/search_tools/rebuild_elastic_search.pl to recreate the index
6) Verify that you can find the records
7) Put <elasticsearch_index_mappings>non_existent</elasticsearch_index_mappings> to koha-conf.xml
8) Verify that admin/searchengine/elasticsearch/mappings.pl?op=reset&i_know_what_i_am_doing=1 fails because it can't find non_existent.
9) Copy etc/searchengine/elasticsearch/mappings.yaml to a new location and make elasticsearch_index_mappings setting in koha-conf.xml point to it.
10) Make a change in the new mappings.yaml.
11) Clear table search_fields (mappings reset doesn't do it yet, see bug 20248)
12) Go to admin/searchengine/elasticsearch/mappings.pl?op=reset&i_know_what_i_am_doing=1
13) Verify that the changes you made are now visible in the mappings UI
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Bug 20073: Move Elasticsearch yaml files back to admin directory
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
This reverts commit f489d2034b.
This commit breaks the install process when using debian packages.
Reverting as we are very close to the 18.05.00 release
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
This patch adds a link to the Elasticsearch configuration page in the
administration sidebar menu when Elasticsearch is enabled.
This patch also corrects the title of the Elasticsearch page to match
the page heading. "Search engine configuration" instead of
"Elasticsearch mappings."
To test, apply the patch and set the SearchEngine system preference to
"Elasticsearch."
- View any page in the administration module which shows the
left-hand sidebar menu. The link to search engine configuration
should be there.
- View the search engine configuration page and confirm that the page
title is correct.
- Set the SearchEngine system preferenc to "Zebra."
- The search engine configuration link should no longer appear in the
administration sidebar menu.
Signed-off-by: David Bourgault <david.bourgault@inlibro.com>
Signed-off-by: Séverine QUEUNE <severine.queune@bulac.fr>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Improvements:
1) Index settings moved from code to etc/searchengine/elasticsearch/index_config.yaml. An alternative can be specified in koha-conf.xml.
2) Field settings moved from code to etc/searchengine/elasticsearch/field_config.yaml. An alternative can be specified in koha-conf.xml.
3) mappings.yaml has been moved from admin/searchengine/elasticsearch to etc/searchengine/elasticsearch. An alternative can be specified in koha-conf.xml.
4) Default settings have been improved to remove punctuation from phrases used for sorting etc.
5) State variables are used for storing configuration to avoid parsing it multiple times.
6) A possibility to reset the fields too has been added to the reset operation of mappings administration.
7) mappings.yaml has been moved from admin/searchengine/elasticsearch to etc/searchengine/elasticsearch.
8) An stdno field type has been added for standard identifiers.
To test:
1) Run tests in t/Koha/SearchEngine/Elasticsearch.t
2) Clear tables search_fields and search_marc_map
3) Go to admin/searchengine/elasticsearch/mappings.pl?op=reset&i_know_what_i_am_doing=1
4) Verify that admin/searchengine/elasticsearch/mappings.pl displays the mappings properly, including ISBN and other standard number fields.
5) Index some records using the -d parameter with misc/search_tools/rebuild_elastic_search.pl to recreate the index
6) Verify that you can find the records
7) Put <elasticsearch_index_mappings>non_existent</elasticsearch_index_mappings> to koha-conf.xml
8) Verify that admin/searchengine/elasticsearch/mappings.pl?op=reset&i_know_what_i_am_doing=1 fails because it can't find non_existent.
9) Copy etc/searchengine/elasticsearch/mappings.yaml to a new location and make elasticsearch_index_mappings setting in koha-conf.xml point to it.
10) Make a change in the new mappings.yaml.
11) Clear table search_fields (mappings reset doesn't do it yet, see bug 20248)
12) Go to admin/searchengine/elasticsearch/mappings.pl?op=reset&i_know_what_i_am_doing=1
13) Verify that the changes you made are now visible in the mappings UI
Signed-off-by: Nicolas Legrand <nicolas.legrand@bulac.fr>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Having to write [% KOHA_VERSION %] for each url is bad because:
- It's easily forgettable when adding new <script> or <link>
- It prevents grep'ing for the full filename
- It violates the DRY principle
- If at some point we want to change the "force js and css reload"
mechanism, it will be tedious
This patch:
- adds a Template::Toolkit plugin that generates <script> and
<link> tags for JS and CSS files, and inserts automatically the Koha
version in the filename
- use the new plugin to remove all occurences of [% KOHA_VERSION %]
- remove the code that was adding KOHA_VERSION as a template variable
Test plan:
1. Apply patch
2. Go to several different pages in Koha (opac and intranet) while
checking your browser's dev tools (there should be no 404 for JS and
CSS files, and the Koha version should appear in filenames) and the
server logs (there should be no "File not found")
3. `git grep KOHA_VERSION` should return nothing
4. prove t/db_dependent/Koha/Template/Plugin/Asset.t
Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
This patch updates a warning message and error message in the Search
engine configuration for Elasticsearch.
Note: elasticsearch does not need to be active in order to test.
To test warning message:
1. go to: cgi-bin/koha/admin/searchengine/elasticsearch/mappings.pl
2. note the warning message
3. Apply patch
4. Note the change in warning message. Confirm it makes sense.
To test error message:
Apply patch and check the changed message in the template file mappings.tt and confirm it makes sense.
Signed-off-by: Nicolas Legrand <nicolas.legrand@bulac.fr>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
I ran:
$ git grep -l cat-search.inc | grep admin
This means I believe the outstanding ones are
(koha-tmpl/intranet-tmpl/prog/en/modules/):
- admin/auth_subfields_structure.tt
- admin/clone-rules.tt
- admin/marc_subfields_structure.tt
- admin/searchengine/elasticsearch/mappings.tt
One other was recommended by Katrin in comment #9:
- plugins/plugins-home.tt
Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Typo found when translating in:
intranet-tmpl/prog/en/modules/admin/searchengine/elasticsearch/mappings.tt:74
An error occurred when updateing mappings => updateing
Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Also some close tag </selected> should be </select>
Follow previous patch to test
Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Test plan
1) Have SearchEngine syspref set to Elastic
2) Look at source on admin/searchengine/elasticsearch/mappings.pl
-> without patch you'll see select with attribute
data-id="mapping_search_field_name" closing tag written badly as
"/select>"
-> with patch everything looks OK
Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
This new page (admin/searchengine/elasticsearch/mappings.pl) will permit
to manage the ES mappings.
For the biblios and authorities indexes, the different mappings can be
managed from this single page.
The interface let you add, remove and update mappings and search fields.
It's also possible to reorder the mappings, as the order can be important
in the indexation process. Note that the table can be displayed in a
different order that the one it was before saving, but the mappings are grouped
by search field and the order inside the search field is preserved.
Limitations:
- If something went wrong during the insertion/deletion/modification,
the users will loose all these changes.
TODO:
- Add a specific permission (?)
- Add some data checks client side (JS)
- Use checkboxes for facet and suggestible (lazy today...)
- Understand the difference between the 3 values that sortable can have
and improve the value for the options in the select box.
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Brendan Gallagher <brendan@bywatersolutions.com>
