Koha/misc
Nick Clemens aba2453ad6
Bug 32594: Add a dedicated Elasticsearch biblio indexing background queue
Currently we generate large numebrs if single record reindex for circulation and other
actions. It can take a long time to process these as we need to load the ES settings for each.

This patch updates the Elasticsearch background jobs to throw records into a new queue
that can be processed by it's own worker and adds a dedicated worker that batches the jobs
every 1 second.

To test:
1 - Apply patches, set SearchEngine system preference to 'Elasticsearch'
2 - perl misc/search_tools/es_indexer_daemon.pl
3 - Leave the running in terminal and perform actions in staff interface:
    - Checking out a bib
    - Returning a bib
    - Editing a single bib
    - Editing a single item
    - Batch editing bibs
    - Batch editing items
4 - Confirm for each action that records are updated in search/search results
5 - Stop the script
6 - set SearchEngine system preference to 'Zebra'
7 - perl misc/search_tools/es_indexer_daemon.pl
8 - Script dies as Elasticsearch not enabled

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Emily Lamancusa <emily.lamancusa@montgomerycountymd.gov>

Bug 32594: (follow-up) Adjust logging per bug 32612

JD amended patch: tidy! There were tabs here...

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-02-24 17:52:20 -03:00
..
admin Bug 31325: Fix koha-preferences get 2022-08-16 09:11:40 -03:00
bin Bug 17600: Fix POSIX imports 2021-07-29 14:37:26 +02:00
cronjobs Bug 32922: Remove space in shebang 2023-02-20 09:44:06 -03:00
devel Bug 17600: Standardize our EXPORT_OK 2021-07-16 08:58:47 +02:00
interface_customization Bug 23148: Replace Bridge icons with transparent PNG files 2020-07-20 16:16:37 +02:00
maintenance Bug 32922: Remove space in shebang 2023-02-20 09:44:06 -03:00
migration_tools Bug 32798: Update parameter passed to Koha::Biblio::Metadata->record 2023-02-21 10:23:34 -03:00
release_notes Add release notes for Koha 22.11.00 2022-11-25 15:20:02 -03:00
search_tools Bug 18984: Remove NORMARC support 2021-10-07 15:36:40 +02:00
translator Bug 32356: Install installer translated files on update 2023-02-09 10:21:17 -03:00
workers Bug 32594: Add a dedicated Elasticsearch biblio indexing background queue 2023-02-24 17:52:20 -03:00
add_date_fields_to_marc_records.pl Bug 29697: Replace GetMarcBiblio occurrences with $biblio->metadata->record 2022-07-22 15:24:11 -03:00
add_statistics_borrowers_categorycode.pl Bug 32922: Remove space in shebang 2023-02-20 09:44:06 -03:00
batchCompareMARCvsFrameworks.pl Bug 28617: Remove kohalib.pl and rely on PERL5LIB 2021-12-07 12:16:28 -10:00
batchdeletebiblios.pl Bug 17600: Standardize our EXPORT_OK 2021-07-16 08:58:47 +02:00
batchDeleteUnusedSubfields.pl Bug 28617: Remove kohalib.pl and rely on PERL5LIB 2021-12-07 12:16:28 -10:00
batchImportMARCWithBiblionumbers.pl Bug 28617: Remove kohalib.pl and rely on PERL5LIB 2021-12-07 12:16:28 -10:00
batchRebuildBiblioTables.pl Bug 30813: Update TransformMarcToKoha to accept a hashref 2022-06-08 11:40:29 -03:00
batchRebuildItemsTables.pl Bug 29697: Use flag embed_items 2022-07-22 15:24:11 -03:00
batchRepairMissingBiblionumbers.pl Bug 29697: Replace GetMarcBiblio occurrences with $biblio->metadata->record 2022-07-22 15:24:11 -03:00
check_sysprefs.pl Bug 17600: Standardize our EXPORT_OK 2021-07-16 08:58:47 +02:00
commit_file.pl Bug 15869: Change framework on overlay 2023-02-07 10:32:22 -03:00
export_borrowers.pl Bug 28327: Unify CSV delimiter special behavior for tabulation 2022-08-19 15:53:12 -03:00
export_records.pl Bug 17600: Standardize our EXPORT_OK 2021-07-16 08:58:47 +02:00
exportauth.pl Bug 28617: Remove kohalib.pl and rely on PERL5LIB 2021-12-07 12:16:28 -10:00
import_patrons.pl Bug 32793: import_patrons.pl typo in usage 2023-02-03 10:30:14 -03:00
koha-install-log Bug 28519: Put CGI::Session::Serialize::yamlxs in lib directory 2021-06-17 10:07:36 +02:00
link_bibs_to_authorities.pl Bug 29697: Replace GetMarcBiblio occurrences with $biblio->metadata->record 2022-07-22 15:24:11 -03:00
load_yaml.pl Bug 29207: Getopt::Long should not ignore cases 2021-10-25 14:08:06 +02:00
mod_zebraqueue.pl Bug 17600: Standardize our EXPORT_OK 2021-07-16 08:58:47 +02:00
perlmodule_ls.pl
perlmodule_rm.pl Bug 21395: Make perlcritic happy 2020-06-29 12:37:02 +02:00
process_ill_updates.pl Bug 30484: Implement support for ILL request updates 2022-09-23 09:18:45 -03:00
recreateIssueStatistics.pl Bug 30275: Rename issues.renewals to issues.renewals_count 2022-07-05 09:45:55 -03:00
sax_parser_print.pl
sax_parser_test.pl Bug 17600: Standardize our EXPORT_OK 2021-07-16 08:58:47 +02:00
sip_cli_emulator.pl Bug 32537: (QA follow-up): Tidy code 2023-02-03 10:30:10 -03:00
stage_file.pl Bug 28617: Remove kohalib.pl and rely on PERL5LIB 2021-12-07 12:16:28 -10:00
z3950_responder.pl Bug 29207: Getopt::Long should not ignore cases 2021-10-25 14:08:06 +02:00