Koha/misc
Kyle M Hall d8fc079eee
Bug 28833: Speed up holds queue builder via parallel processing
The holds queue builder can take a very long time to run on systems with many holds. For example, a partner with 124,784 unfilled ( not found ) holds, is taking about 64 minutes to run. If we run that same number of holds in 5 parallel chunks ( splitting the number of records as evenly as possible, but *not* taking into account the holds per bib ), it takes 21.5 minutes. If we use 10 loops, it takes less then 14 minutes.

Test Plan:
0) Install the Perl library Parallel::ForkManager
1) Generate a huge number of holds ( a few thousand at the minimum )
2) Run the holds queue builder, use the `time` utility to track how much
   time it took to run
3) Set HoldsQueueParallelLoopsCount to 10
4) Repeat step 2, note the improvement in speed
5) Experiment with other values for HoldsQueueParallelLoopsCount
6) prove t/db_dependent/HoldsQueue.t

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-10-25 11:09:53 +02:00
..
admin Bug 37682: Lazy-load modules for setters in koha-preferences tool 2024-08-27 12:14:23 +02:00
bin Bug 37543: (follow-up) Tidy 2024-08-16 17:33:43 +02:00
cronjobs Bug 28833: Speed up holds queue builder via parallel processing 2024-10-25 11:09:53 +02:00
devel Bug 36309: Make create_superlibrarian.pl output more useful 2024-04-22 08:57:35 +02:00
interface_customization Bug 23148: Replace Bridge icons with transparent PNG files 2020-07-20 16:16:37 +02:00
maintenance Bug 31143: (QA follow-up) Tidy 2024-10-23 20:02:21 +02:00
migration_tools Bug 18273: Do not insert authority if there already are duplicates in koha 2024-10-21 15:10:32 +02:00
release_notes 24.05.00: Add release notes 2024-05-27 18:26:45 +02:00
search_tools Bug 35345: Add --where option to rebuild_elasticsearch.pl 2024-04-22 12:39:26 +02:00
translator Bug 38164: Skip fuzzy translations 2024-10-18 17:44:17 +02:00
workers Bug 37787: Remove mention of elastic_index from POD 2024-10-23 19:14:55 +02:00
add_date_fields_to_marc_records.pl Bug 23924: Reset the fields to add 2023-05-05 12:13:53 -03:00
add_statistics_borrowers_categorycode.pl Bug 34478: op =~ ^cud- in pl/pm 2024-03-01 10:56:11 +01:00
batchCompareMARCvsFrameworks.pl Bug 28617: Remove kohalib.pl and rely on PERL5LIB 2021-12-07 12:16:28 -10:00
batchdeletebiblios.pl Bug 17600: Standardize our EXPORT_OK 2021-07-16 08:58:47 +02:00
batchDeleteUnusedSubfields.pl Bug 28617: Remove kohalib.pl and rely on PERL5LIB 2021-12-07 12:16:28 -10:00
batchImportMARCWithBiblionumbers.pl Bug 28617: Remove kohalib.pl and rely on PERL5LIB 2021-12-07 12:16:28 -10:00
batchRebuildBiblioTables.pl Bug 30813: Update TransformMarcToKoha to accept a hashref 2022-06-08 11:40:29 -03:00
batchRebuildItemsTables.pl Bug 29697: Use flag embed_items 2022-07-22 15:24:11 -03:00
batchRepairMissingBiblionumbers.pl Bug 29697: Replace GetMarcBiblio occurrences with $biblio->metadata->record 2022-07-22 15:24:11 -03:00
check_sysprefs.pl Bug 17600: Standardize our EXPORT_OK 2021-07-16 08:58:47 +02:00
commit_file.pl Bug 15869: Change framework on overlay 2023-02-07 10:32:22 -03:00
export_borrowers.pl Bug 28327: Unify CSV delimiter special behavior for tabulation 2022-08-19 15:53:12 -03:00
export_records.pl Bug 36770: (QA follow-up) Tidy export_records.pl 2024-08-09 18:44:54 +02:00
exportauth.pl Bug 28617: Remove kohalib.pl and rely on PERL5LIB 2021-12-07 12:16:28 -10:00
import_patrons.pl Bug 34621: Tidy import_patrons.pl 2024-04-12 14:02:43 +02:00
koha-install-log Bug 28519: Put CGI::Session::Serialize::yamlxs in lib directory 2021-06-17 10:07:36 +02:00
link_bibs_to_authorities.pl Bug 30024: Make link_bibs_to_authorities.pl rely on LinkerRelink 2023-11-03 14:22:45 -03:00
load_yaml.pl Bug 29207: Getopt::Long should not ignore cases 2021-10-25 14:08:06 +02:00
mod_zebraqueue.pl Bug 17600: Standardize our EXPORT_OK 2021-07-16 08:58:47 +02:00
process_ill_updates.pl Bug 35581: Koha::Illrequest -> Koha::ILL::Request 2024-04-22 08:57:45 +02:00
recreateIssueStatistics.pl Bug 34478: op =~ ^cud- in pl/pm - Manual revert changes to recreateIssueStatistics.pl 2024-03-01 10:57:15 +01:00
sax_parser_print.pl
sax_parser_test.pl Bug 17600: Standardize our EXPORT_OK 2021-07-16 08:58:47 +02:00
sip_cli_emulator.pl Bug 34478: op =~ ^cud- in pl/pm - Manual revert changes to sip_cli_emulator.pl 2024-03-01 10:57:14 +01:00
stage_file.pl Bug 28617: Remove kohalib.pl and rely on PERL5LIB 2021-12-07 12:16:28 -10:00
z3950_responder.pl Bug 29207: Getopt::Long should not ignore cases 2021-10-25 14:08:06 +02:00