Ere Maijala e74d86a343 Bug 27584: Refactor OAI-PMH paging to improve performance ... Includes the following optimizations: - Use next biblionumber instead of large offset in the queries. - Use unions instead of subqueries - Avoid fetching item timestamps when items are not included. Test plan: 1. Without the patch, try harvesting a Koha database with (and without for good measure) `include_items: 1` in the OAI-PMH configuration file pointed to by preference OAI-PMH:ConfFile and take note of performance. For useful metrics the database must be large enough to not fit in InnoDB buffers or OS file cache. 2. Apply the patch. 3. Run tests: prove -v t/db_dependent/OAI 4. Try again the harvesting from step 1 and compare performance with step 1. Signed-off-by: David Cook <dcook@prosentient.com.au> Signed-off-by: Nick Clemens <nick@bywatersolutions.com> Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>		2021-05-07 14:44:00 +02:00
..
Acquisition	Bug 27486: Rename system preference delimiter to CSVDelimiter	2021-01-29 09:03:45 +01:00
api/v1	Bug 28254: Add missing tests for PUT and POST	2021-05-04 15:05:37 +02:00
Auth	Bug 26384: Fix executable flags	2020-09-11 09:56:56 +02:00
AuthoritiesMarc	Bug 21958: (QA follow-up) Correct indirect object notation	2021-04-01 17:13:56 +02:00
Authority	Bug 26384: Fix executable flags	2020-09-11 09:56:56 +02:00
Biblio	Bug 25306: Remove framework paramter from ModBiblioMarc	2021-01-12 16:13:50 +01:00
Circulation	Bug 27896: Remove DeleteTransfer	2021-04-23 11:58:49 +02:00
CourseReserves	Bug 26384: Fix executable flags	2020-09-11 09:56:56 +02:00
Creators	Bug 25898: Prohibit indirect object notation	2020-10-15 12:56:30 +02:00
cronjobs	Bug 26384: Fix executable flags	2020-09-11 09:56:56 +02:00
data	Bug 27968: Fix framework import if isurl has an incorrect value	2021-04-23 10:03:30 +02:00
Exporter	Bug 26384: Fix executable flags	2020-09-11 09:56:56 +02:00
Holds	Bug 27069: Adjust tests	2021-04-07 16:08:04 +02:00
Illrequest	Bug 26384: Fix executable flags	2020-09-11 09:56:56 +02:00
Items	Bug 4037: Add item type filter to inventory tool	2021-02-09 09:23:51 +01:00
Koha	Bug 28254: (QA follow-up) override -> force	2021-05-04 14:22:08 +02:00
Labels	Bug 26384: Fix executable flags	2020-09-11 09:56:56 +02:00
LDAP	Bug 21395: (QA follow-up) Remove some introduced issues	2020-06-29 12:37:02 +02:00
Letters	Bug 26734: (QA follow-up) Unit tests for C4::Letters	2021-04-27 16:23:49 +02:00
Members	Bug 25898: Prohibit indirect object notation	2020-10-15 12:56:30 +02:00
misc/translator	Bug 25067: Adjust paths in tests	2020-11-06 16:59:40 +01:00
OAI	Bug 27584: Refactor OAI-PMH paging to improve performance	2021-05-07 14:44:00 +02:00
Patron	Bug 26937: Add an optional delay to the CheckPrevCheckout syspref.	2021-03-01 15:14:22 +01:00
Record	Bug 25898: Fix new occurrences	2020-10-15 13:56:41 +02:00
Reports	Bug 27380: FIX Reports/Guided.t	2021-02-12 14:31:01 +01:00
Reserves	Bug 26634: Remove GetHoldRule subroutine in C4::Reserves	2020-12-21 10:07:38 +01:00
Search	Bug 25898: Prohibit indirect object notation	2020-10-15 12:56:30 +02:00
selenium	Bug 26703: Fix selenium tests	2021-04-21 15:25:07 +02:00
Serials	Bug 26384: Fix executable flags	2020-09-11 09:56:56 +02:00
SIP	Bug 25690: Add SIP2 tests for checking out with holds	2021-03-04 16:18:42 +01:00
Template/Plugin	Bug 26984: Make tests pass if AnonymousPatron is set	2020-11-20 08:16:02 +01:00
Utils	Bug 23271: Replace search_limited with search_with_library_limits	2021-04-21 15:25:07 +02:00
www	Bug 26031: Remove the failing tests and the diag	2020-11-20 08:16:02 +01:00
00-strict.t	Bug 25898: Prohibit indirect object notation	2020-10-15 12:56:30 +02:00
01-test_dbic.t	Bug 26384: Fix executable flags	2020-09-11 09:56:56 +02:00
Accounts.t	Bug 27636: (QA follow-up) Unit tests for AutoReconcile	2021-04-30 17:07:31 +02:00
Acquisition.t	Bug 23971: Rename pref with AcquisitionLog	2021-04-23 11:58:49 +02:00
Amazon.t
ArticleRequests.t	Bug 26250: Fix tests when SearchEngine=Elastic	2020-08-31 16:10:26 +02:00
AudioAlerts.t	Bug 22001: Remove the RaiseError occurrences from tests	2020-03-27 08:52:56 +00:00
Auth.t	Bug 21325: Add tests	2021-04-16 14:15:37 +02:00
Auth_with_cas.t	Bug 20854: Add more CAS tests	2021-04-21 10:51:29 +02:00
Auth_with_ldap.t	Bug 25172: Fix Auth_with_ldap.t	2020-04-22 15:43:16 +01:00
AuthorisedValues.t	Bug 23830: Make Koha::AuthorisedValues use Koha::Objects::Limit::Library	2021-02-08 17:09:34 +01:00
AuthoritiesMarc.t	Bug 21958: Fix bibliographic record field comparison with authority	2021-04-01 17:13:56 +02:00
AuthUtils.t	Bug 26384: (QA follow-up) Fix two exceptions	2020-09-11 09:56:56 +02:00
BackgroundJob.t	Bug 25898: Prohibit indirect object notation	2020-10-15 12:56:30 +02:00
Barcodes.t	Bug 26250: Fix tests when SearchEngine=Elastic	2020-08-31 16:10:26 +02:00
Barcodes_ValueBuilder.t	Bug 26384: Fix executable flags	2020-09-11 09:56:56 +02:00
Biblio.t	Bug 27577: Unit test	2021-05-06 15:52:38 +02:00
Breeding.t	Bug 24052: Fix Koha/XSLT/Base.t	2020-03-24 10:42:23 +00:00
Budgets.t	Bug 23195: (follow-up) One text invoice should be open	2021-04-22 14:50:11 +02:00
Cache.t	Bug 16067: Remove FastMmap	2020-12-04 15:40:58 +01:00
Calendar.t	Bug 26384: Fix executable flags	2020-09-11 09:56:56 +02:00
Charset.t	Bug 26384: Fix executable flags	2020-09-11 09:56:56 +02:00
check_kohastructure.t	Bug 26384: Fix executable flags	2020-09-11 09:56:56 +02:00
check_sysprefs.t	Bug 23895: Fix check_sysprefs.t	2020-10-14 15:52:29 +02:00
Circulation.t	Bug 26405: Prevent Circulation.t failing on 'AddRenewal left both fines'	2021-04-27 16:23:49 +02:00
Clubs.t
Context.t	Bug 21395: Make perlcritic happy	2020-06-29 12:37:02 +02:00
Contract.t	Bug 26384: Fix executable flags	2020-09-11 09:56:56 +02:00
CourseReserves.t	Bug 22001: Remove the RaiseError occurrences from tests	2020-03-27 08:52:56 +00:00
db_structure.t	Bug 26384: Fix executable flags	2020-09-11 09:56:56 +02:00
DecreaseLoanHighHolds.t	Bug 14866: Make high holds work with different item types	2020-11-06 15:55:17 +01:00
Filter_MARC_ViewPolicy.t	Bug 26384: Fix executable flags	2020-09-11 09:56:56 +02:00
Fines.t	Bug 26384: Fix executable flags	2020-09-11 09:56:56 +02:00
FrameworkPlugin.t	Bug 25898: Prohibit indirect object notation	2020-10-15 12:56:30 +02:00
Heading.t	Bug 21958: Allow 'e' subfield for authorities	2021-04-06 15:56:30 +02:00
Heading_MARC21.t	Bug 24269: Adjust C4::Heading to generate headings from auth tags	2020-02-19 13:33:32 +00:00
Hold.t	Bug 26498: (QA follow-up) Add handling for update	2021-04-07 13:10:31 +02:00
Holds.t	Bug 16787: 'Too many holds' message appears inappropriately and is missing data	2021-04-22 14:37:14 +02:00
HoldsQueue.t	Bug 27069: Adjust tests	2021-04-07 16:08:04 +02:00
Holidays.t	Bug 25723: Remove tests for removed method	2020-07-20 17:45:31 +02:00
Illcomments.t	Bug 26384: Fix executable flags	2020-09-11 09:56:56 +02:00
Illrequestattributes.t	Bug 26384: Fix executable flags	2020-09-11 09:56:56 +02:00
Illrequests.t	Bug 22818: Fix TestBuilder.t	2020-11-11 18:03:17 +01:00
ILSDI_Services.t	Bug 16787: (follow-up) Add and adjust ILSDI tests	2021-04-22 17:31:03 +02:00
ImportBatch.t	Bug 24469: Move the new queries to a dedicated ImportBatch subroutine	2021-02-18 11:52:59 +01:00
ImportExportFramework.t	Bug 27569: Add tests for ODS export	2021-02-16 10:22:13 +01:00
Installer.t	Bug 26325: Add primary_key_exists check to C4::Installer	2020-10-01 10:32:15 +02:00
Items.t	Bug 24446: (QA follow-up) Update unit test for ModItemTransfer	2021-03-03 15:36:13 +01:00
Koha.t	Bug 26384: Fix executable flags	2020-09-11 09:56:56 +02:00
Koha_Authority.t
Koha_Database.t	Bug 26384: Fix executable flags	2020-09-11 09:56:56 +02:00
Koha_Elasticsearch.t	Bug 26384: Fix executable flags	2020-09-11 09:56:56 +02:00
Koha_ExternalContent_OverDrive.t	Bug 25527: Add logger to Koha::ExternalContent	2020-05-18 15:25:03 +01:00
Koha_ExternalContent_RecordedBooks.t
Koha_Misc_Files.t	Bug 22001: Remove the RaiseError occurrences from tests	2020-03-27 08:52:56 +00:00
Languages.t
Letters.t	Bug 14723: Additional delivery notes to messages	2021-05-07 13:48:55 +02:00
LibraryGroups.t	Bug 26384: Fix executable flags	2020-09-11 09:56:56 +02:00
Linker_Default.t	Bug 24269: Adjust C4::Heading to generate headings from auth tags	2020-02-19 13:33:32 +00:00
Linker_FirstMatch.t	Bug 24269: Adjust C4::Heading to generate headings from auth tags	2020-02-19 13:33:32 +00:00
Log.t	Bug 26384: Fix executable flags	2020-09-11 09:56:56 +02:00
MarcModificationTemplates.t	Bug 26894: Handle subfield 0 and delete empty fields	2021-01-12 16:02:35 +01:00
Members.t	Bug 28001: (bug 27050 follow-up) Fix delete_patrons if no category passed	2021-04-16 12:28:17 +02:00
Message.t	Bug 28230: Regression tests	2021-04-28 11:07:05 +02:00
MungeMarcPrice.t
NewsChannels.t	Bug 26384: Fix executable flags	2020-09-11 09:56:56 +02:00
Overdues.t	Bug 28144: Add a regression test	2021-04-27 16:23:49 +02:00
Passwordrecovery.t	Bug 22343: (follow-up) Remove traces of Mail::Sendmail	2020-10-05 11:15:51 +02:00
Patroncards.t
PatronLists.t
Patrons.t	Bug 27554: Clarify and add tests for update_category_to	2021-02-01 16:21:25 +01:00
Record.t	Bug 25898: Prohibit indirect object notation	2020-10-15 12:56:30 +02:00
RecordProcessor_EmbedSeeFromHeadings.t	Bug 25898: Prohibit indirect object notation	2020-10-15 12:56:30 +02:00
Reports.t
Reserves.t	Bug 27069: Adjust tests	2021-04-07 16:08:04 +02:00
rollingloans.t	Bug 26384: Fix executable flags	2020-09-11 09:56:56 +02:00
RotatingCollections.t	Bug 17202: Unit tests	2021-04-28 11:07:06 +02:00
Search.t	Bug 27928: FindDuplicate should use simple_search_compat	2021-04-01 18:03:23 +02:00
Serials.t	Bug 27842: Add tests	2021-04-28 11:07:06 +02:00
Serials_2.t	Bug 26384: Fix executable flags	2020-09-11 09:56:56 +02:00
Service.t
ShelfBrowser.t	Bug 26384: Fix executable flags	2020-09-11 09:56:56 +02:00
Sitemapper.t	Bug 24840: Replace DateTime->now with dt_from_string	2020-04-08 11:54:23 +01:00
Stats.t	Bug 26384: Fix executable flags	2020-09-11 09:56:56 +02:00
StockRotationItems.t	Bug 22569: Add Unit Tests	2021-03-11 11:51:40 +01:00
StockRotationRotas.t	Bug 26384: Fix executable flags	2020-09-11 09:56:56 +02:00
StockRotationStages.t	Bug 26384: Fix executable flags	2020-09-11 09:56:56 +02:00
SuggestionEngine_ExplodedTerms.t	Bug 25898: Prohibit indirect object notation	2020-10-15 12:56:30 +02:00
Suggestions.t	Bug 26384: Fix executable flags	2020-09-11 09:56:56 +02:00
sysprefs.t	Bug 22880: (follow-up) Loose ends: Better DB update, global inclusion, etc.	2020-02-24 13:24:25 +00:00
TablesSettings.t	Bug 25898: Prohibit indirect object notation	2020-10-15 12:56:30 +02:00
Tags.t
Templates.t
TestBuilder.t	Bug 28234: Regression tests	2021-04-28 10:34:36 +02:00
Upload.t	Bug 28123: Fix expected value in unit test	2021-04-16 12:28:18 +02:00
UsageStats.t	Bug 23971: Fix UsageStats.t	2021-04-23 12:45:41 +02:00
Virtualshelves.t	Bug 26384: Fix executable flags	2020-09-11 09:56:56 +02:00
XISBN.t	Bug 28288: Prevent XSISBN.t to fail if service is returning 500	2021-05-04 16:38:27 +02:00
XSLT.t	Bug 21260: (follow-up) Consolidate preference	2021-04-14 16:05:53 +02:00
yaml.t	Bug 22824: Adjust tests to match final behaviours	2021-03-04 16:18:42 +01:00
zebra_config.pl	Bug 24735: Remove QueryParser-related code	2020-03-02 11:13:03 +00:00