Koha-community/Koha
13
7
Fork 0
Code Issues Releases Wiki Activity

Bug 23377: use ENV syspref override instead of complete disable

Browse source 
In an attempt to save time, bulkmarcimport temporarily sets CataloguingLog
and AuthoritiesLog to 0.  It does this by disabling syspref caching and saving
the changes to the database (then replacing the original values at completion).

Unfortunately, this disables other key sysprefs from being cached, and results in
a 50% increase in processing time for the script.

This patch instead utilizes the ENV variable override feature of sysprefs, which
preempts the cache in C4::Context->preference().

To test:
1. Perform a bulkmarcimport with a reasonable number of biblios (~1000 will do)
2. Note the time taken to complete
3. Apply patch
4. Revert the biblio load performed
5. Perform another bulkmarcimport with the same biblios and commandline options
6. Note the time taken to complete
7. Compare times.  The time from step 6 should be about 33% less than the time from step 2
8. Check Cataloguing and Authorities Logs to verify imported records were not logged
9. Profit!

Signed-off-by: Michal Denar <black23@gmail.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
This commit is contained in:
Ian Walls 2019-07-31 16:32:23 +00:00 committed by Martin Renvoize
parent 3e93514ac3
commit f036038318
Signed by: martin.renvoize
GPG key ID: 422B469130441A0F
1 changed files with 5 additions and 13 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

18
misc/migration_tools/bulkmarcimport.pl
View file

@ -159,15 +159,8 @@ if ((not defined $sourcesubfield) && (not defined $sourcetag)){
# Disable logging for the biblios and authorities import operation. It would unnecessarily # Disable logging for the biblios and authorities import operation. It would unnecessarily
# slow the import # slow the import
$ENV{SYSPREF_OVERRIDE_CataloguingLog} = 0;
# Disable the syspref cache so we can change logging settings $ENV{SYSPREF_OVERRIDE_AuthoritiesLog} = 0;
C4::Context->disable_syspref_cache();
# Save current CataloguingLog and AuthoritiesLog sysprefs values
my $CataloguingLog = C4::Context->preference( 'CataloguingLog' );
my $AuthoritiesLog = C4::Context->preference( 'AuthoritiesLog' );
# Disable logging for both
C4::Context->set_preference( 'CataloguingLog', 0 );
C4::Context->set_preference( 'AuthoritiesLog', 0 );
if ($fk_off) { if ($fk_off) {
$dbh->do("SET FOREIGN_KEY_CHECKS = 0"); $dbh->do("SET FOREIGN_KEY_CHECKS = 0");
@ -565,10 +558,9 @@ if ($fk_off) {
$dbh->do("SET FOREIGN_KEY_CHECKS = 1"); $dbh->do("SET FOREIGN_KEY_CHECKS = 1");
} }
# Restore CataloguingLog # Restore CataloguingLog and AuthoritiesLog
C4::Context->set_preference( 'CataloguingLog', $CataloguingLog ); delete $ENV{SYSPREF_OVERRIDE_CataloguingLog};
# Restore AuthoritiesLog delete $ENV{SYSPREF_OVERRIDE_AuthoritiesLog};
C4::Context->set_preference( 'AuthoritiesLog', $AuthoritiesLog );
my $timeneeded = gettimeofday - $starttime; my $timeneeded = gettimeofday - $starttime;
print "\n$i MARC records done in $timeneeded seconds\n"; print "\n$i MARC records done in $timeneeded seconds\n";