Nick Clemens
9951e230e4
Whne performing batch operations we can send a large numebr of records for reindexing at once.
Currently this can create requetss that are too large for Elasticsearch to process. We need
to break these requests into chunks/
This patch adds a chunk_size configuration to the elasticsearch stanza in koha-conf.xml
If blank we default to 5000.
To test:
0 - Have Koha using Elasticsearch
1 - Create and download a report of all barcodes:
SELECT barcode FROM items
2 - Batch modify these items
3 - Note a single ESindexing job is created
4 - Create and download a report of all authority ids:
SELECT auth_header.authid FROM auth_header
5 - Setup a marc modification template, and batch modify all the authorities
6 - Again note a single ES backgorund job is created
7 - Apply patch
8 - Repeat the modifications above - you still get a single job
9 - Edit koha-conf.xml and add <chunk_size>250</chunk_size> to elasticsearch stanza
10 - Repeat modifications - you now get several background ES jobs
11 - prove -v t/db_dependent/Koha/SearchEngine/Elasticsearch/Indexer.t
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
|
||
|---|---|---|
| .. | ||
| Acquisition | ||
| Biblio | ||
| Budgets | ||
| Circulation | ||
| cypress | ||
| data | ||
| db_dependent | ||
| edi_testfiles | ||
| etc | ||
| External | ||
| Koha | ||
| lib | ||
| Number | ||
| Search | ||
| Serials | ||
| SIP | ||
| Template | ||
| Test | ||
| 00-check-atomic-updates.t | ||
| 00-deprecated.t | ||
| 00-load.t | ||
| 00-merge-conflict-markers.t | ||
| 00-testcritic.t | ||
| 00-valid-xml.t | ||
| Auth.t | ||
| AuthoritiesMarc_MARC21.t | ||
| AuthoritiesMarc_UNIMARC.t | ||
| Barcodes_annual.t | ||
| Barcodes_EAN13.t | ||
| Barcodes_hbyymmincr.t | ||
| Barcodes_incremental.t | ||
| Biblio.t | ||
| Biblio2.t | ||
| Budgets.t | ||
| Charset.t | ||
| Circulation_barcodedecode.t | ||
| ClassSortRoutine.t | ||
| ClassSortRoutine_Dewey.t | ||
| ClassSortRoutine_Generic.t | ||
| ClassSortRoutine_LCC.t | ||
| ClassSource.t | ||
| Context.t | ||
| Contract.t | ||
| CookieManager.t | ||
| Creators.t | ||
| DateUtils.t | ||
| dummy.t | ||
| Edifact.t | ||
| EdiInvoice.t | ||
| Ediorder.t | ||
| Ediordrsp.t | ||
| Form_MessagingPreferences.t | ||
| HtmlTags.t | ||
| ImportBatch.t | ||
| Installer_PerlModules.t | ||
| ItemCirculationAlertPreference.t | ||
| Koha_MetadataRecord.t | ||
| Koha_SearchEngine_Elasticsearch_Browse.t | ||
| Koha_Template_Plugin_Cache.t | ||
| Koha_Template_Plugin_HtmlId.t | ||
| Koha_Template_Plugin_Koha.t | ||
| Koha_Util_FrameworkPlugin.t | ||
| Koha_Util_MARC.t | ||
| Koha_XSLT_HTTPS.t | ||
| Labels.t | ||
| Labels_split_ccn.t | ||
| Labels_split_ddcn.t | ||
| Labels_split_lccn.t | ||
| Labels_split_Regex.t | ||
| Languages.t | ||
| Logger.t | ||
| Makefile.t | ||
| Members_Messaging.t | ||
| OpenLibrarySearch.t | ||
| Output.t | ||
| Output_JSONStream.t | ||
| Patroncards.t | ||
| Patroncards_Batch.t | ||
| Patroncards_Lib.t | ||
| Patroncards_Patroncard.t | ||
| Patroncards_Profile.t | ||
| Patroncards_Template.t | ||
| RecordProcessor.t | ||
| Ris.t | ||
| Scheduler.t | ||
| Scrubber.t | ||
| Search.t | ||
| Search_PazPar2.t | ||
| SimpleMARC.t | ||
| SMS.t | ||
| SuggestionEngine.t | ||
| SuggestionEngine_AuthorityFile.t | ||
| template_filters.t | ||
| timezones.t | ||
| TmplToken.t | ||
| Token.t | ||