Koha/etc
Nick Clemens 9951e230e4
Bug 35086: Add chunk_size option to elasticsearch configuration
Whne performing batch operations we can send a large numebr of records for reindexing at once.
Currently this can create requetss that are too large for Elasticsearch to process. We need
to break these requests into chunks/

This patch adds a chunk_size configuration to the elasticsearch stanza in koha-conf.xml

If blank we default to 5000.

To test:
0 - Have Koha using Elasticsearch
1 - Create and download a report of all barcodes:
    SELECT barcode FROM items
2 - Batch modify these items
3 - Note a single ESindexing job is created
4 - Create and download a report of all authority ids:
    SELECT auth_header.authid FROM auth_header
5 - Setup a marc modification template, and batch modify all the authorities
6 - Again note a single ES backgorund job is created
7 - Apply patch
8 - Repeat the modifications above - you still get a single job
9 - Edit koha-conf.xml and add <chunk_size>250</chunk_size> to elasticsearch stanza
10 - Repeat modifications - you now get several background ES jobs
11 - prove -v t/db_dependent/Koha/SearchEngine/Elasticsearch/Indexer.t

Signed-off-by: David Nind <david@davidnind.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-01-16 12:06:00 +01:00
..
z3950 Bug 25716: (QA follow-up) Make sure a comment doesn't break tests 2022-10-24 18:52:35 -03:00
zebradb Bug 35455: Remove = in ICU for indexing/searching 2023-12-20 15:13:46 +01:00
apache.conf
koha-conf.xml Bug 35086: Add chunk_size option to elasticsearch configuration 2024-01-16 12:06:00 +01:00
koha-httpd.conf Bug 30708: Add apache RewriteRule 2023-10-18 15:41:42 -03:00
koha-worker.service Bug 32992: Move background_jobs_worker to misc/workers 2023-02-24 17:52:19 -03:00
log4perl.conf Bug 32612: (QA follow-up) Remove superfluous line 2023-02-03 10:30:03 -03:00
nginx.conf Bug 32716: larger proxy_buffer_size in NGINX example config 2023-04-14 11:35:18 -03:00
README.txt
SIPconfig.xml Bug 34868: (QA follow-up) Fix XML 2023-11-10 10:59:39 -03:00

Koha Configuration Files:

The following files specify the base configuration for Koha ZOOM:

 * koha-httpd.conf  
In a debian system, this apache configuration file will be symlinked
from /etc/apache2/sites-enabled
Specify Koha's IP address with NameVirtualHost
Set ServerName, etc

 * koha-production.xml  
 * koha-testing.xml 
These are the production and testing configurations for zebrasrv and for Koha.
The first part of each file specifies Zebra server names, indexing configuration files,
and query language configurations.  Koha configuration directives follow. 

 * zebra-authorities.cfg  
 * zebra-biblios.cfg