Bug 27584: Refactor OAI-PMH paging to improve performance
authorEre Maijala <ere.maijala@helsinki.fi>
Mon, 1 Feb 2021 11:25:04 +0000 (13:25 +0200)
committerJonathan Druart <jonathan.druart@bugs.koha-community.org>
Fri, 7 May 2021 12:44:00 +0000 (14:44 +0200)
commite74d86a3431558b97d2fcd0922467e265cb749c3
treec770585b47a9fc4a8b5e348bf9f4da3c3e8564ac
parentb3139750d6006ec0cc291285f633d6d0374583fd
Bug 27584: Refactor OAI-PMH paging to improve performance

Includes the following optimizations:
- Use next biblionumber instead of large offset in the queries.
- Use unions instead of subqueries
- Avoid fetching item timestamps when items are not included.

Test plan:

1. Without the patch, try harvesting a Koha database with (and without for good measure) `include_items: 1` in the OAI-PMH configuration file pointed to by preference OAI-PMH:ConfFile and take note of performance. For useful metrics the database must be large enough to not fit in InnoDB buffers or OS file cache.
2. Apply the patch.
3. Run tests: prove -v t/db_dependent/OAI
4. Try again the harvesting from step 1 and compare performance with step 1.

Signed-off-by: David Cook <dcook@prosentient.com.au>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Koha/OAI/Server/ListBase.pm
Koha/OAI/Server/ResumptionToken.pm
t/db_dependent/OAI/Server.t