Koha-community/Koha
13
7
Fork 0
Code Issues Releases Wiki Activity

Bug 24123: Fix import of UTF-8 encoded MARC21 MARCXML using bulkmarcimport (elastic only)

Browse source 
If elastic is used as search engine, the bulkmarcimport.pl will not
handle correctly UTF-8 encoded MARCXML

Koha::SearchEngine::Search->new uses a require statement to load the correct Search module.
This is done l.257 of bulkmarcimport.pl:
  257 my $searcher = Koha::SearchEngine::Search->new

Koha::SearchEngine::Elasticsearch::Search will `use MARC::File::XML`, and so resets the arguments set before:
  216     $MARC::File::XML::_load_args{BinaryEncoding} = 'utf-8';

  220     $MARC::File::XML::_load_args{RecordFormat} = $recordformat;

An easy (but dirty) fix could be to move the declaration of my $searcher before in the script.
The tricky (but correct) fix would be to remove the long standing "ugly hack follows" comment.

This patch is the easy, and dirty, fix

Test plan:
Use the command line tool to import MARXCML records that contains unicode characters into Koha

Something like `misc/migration_tools/bulkmarcimport.pl -biblios -file record.marcxml -m=MARCXML`

Without this patch you will notice that unicode characters will not be displayed correctly

Signed-off-by: Michal Denar <black23@gmail.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
This commit is contained in:
Jonathan Druart 2019-12-10 15:24:36 +01:00 committed by Martin Renvoize
parent 7df88be2a4
commit c121a53dd7
Signed by: martin.renvoize
GPG key ID: 422B469130441A0F
1 changed files with 11 additions and 10 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

21
misc/migration_tools/bulkmarcimport.pl
View file

@ -199,6 +199,17 @@ if ($test_parameter) {
my $marcFlavour = C4::Context->preference('marcflavour') || 'MARC21'; my $marcFlavour = C4::Context->preference('marcflavour') || 'MARC21';
# The definition of $searcher must be before MARC::Batch->new
my $searcher = Koha::SearchEngine::Search->new(
{
index => (
$authorities
? $Koha::SearchEngine::AUTHORITIES_INDEX
: $Koha::SearchEngine::BIBLIOS_INDEX
)
}
);
print "Characteristic MARC flavour: $marcFlavour\n" if $verbose; print "Characteristic MARC flavour: $marcFlavour\n" if $verbose;
my $starttime = gettimeofday; my $starttime = gettimeofday;
my $batch; my $batch;
@ -254,16 +265,6 @@ if ($logfile){
print $loghandle "id;operation;status\n"; print $loghandle "id;operation;status\n";
} }
my $searcher = Koha::SearchEngine::Search->new(
{
index => (
$authorities
? $Koha::SearchEngine::AUTHORITIES_INDEX
: $Koha::SearchEngine::BIBLIOS_INDEX
)
}
);
RECORD: while ( ) { RECORD: while ( ) {
my $record; my $record;
# get records # get records