Koha-community/Koha
13
7
Fork 0
Code Issues Releases Wiki Activity

Bug 27667: Display a warning for records missing in Elasticsearch

Browse source 
On "About Koha" page are displayed the numbers of indexed biblios and authorities with ES.
It could be great if we could have comparison with the number of records in the database to display the number of missing records.
In the "System information" tab with the other "Data problems".

If Searchengine has any issue (index count will be -1), we do not
display missing records.

Test plan :
1) Use Elasticsearch searchengine
2) Rebuild all records : koha-elasticsearch --rebuild -d kohadev
3) Go to "About Koha" page and look at System information tab
=> you see no warning
4) On a biblio record delete the MARCXML in biblio_metadata.metadata
5) Rebuild all records : koha-elasticsearch --rebuild -d kohadev
6) Go to "About Koha" page and look at System information tab
=> you see warning : 1 record(s) missing on a total of 435 in indice koha_kohadev_biblios.
7) On a authority record delete the MARCXML in auth_header.marcxml
8) Rebuild all records : koha-elasticsearch --rebuild -d kohadev
9) Go to "About Koha" page and look at System information tab
=> you see warning : 1 record(s) missing on a total of 1705 in indice koha_kohadev_authorities.

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>

Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit fee0f29792)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
This commit is contained in:
Fridolin Somers 2022-05-11 09:38:23 -10:00 committed by Lucas Gass
parent 35b8967b84
commit 21ff480612
2 changed files with 43 additions and 8 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

31
about.pl
View file

@ -41,8 +41,10 @@ use C4::Installer::PerlModules;
use Koha;
use Koha::DateUtils qw( dt_from_string output_pref );
use Koha::Acquisition::Currencies;
use Koha::Authorities;
use Koha::BackgroundJob;
use Koha::BiblioFrameworks;
use Koha::Biblios;
use Koha::Email;
use Koha::Patron::Categories;
use Koha::Patrons;
@ -354,6 +356,7 @@ if ( C4::Context->preference('SearchEngine') eq 'Elasticsearch' ) {
my $es_status;
my $es_config_error;
my $es_running = 1;
my $es_has_missing = 0;
my $es_conf;
try {
@ -378,15 +381,15 @@ if ( C4::Context->preference('SearchEngine') eq 'Elasticsearch' ) {
my $es_status->{version} = $es->info->{version}->{number};
foreach my $index ( @indexes ) {
my $count;
my $index_count;
try {
$count = $es->indices->stats( index => $index )
$index_count = $es->indices->stats( index => $index )
->{_all}{primaries}{docs}{count};
}
catch {
if ( ref($_) eq 'Search::Elasticsearch::Error::Missing' ) {
push @{ $es_status->{errors} }, "Index not found ($index)";
$count = -1;
$index_count = -1;
}
elsif ( ref($_) eq 'Search::Elasticsearch::Error::NoNodes' ) {
$es_running = 0;
@ -397,15 +400,31 @@ if ( C4::Context->preference('SearchEngine') eq 'Elasticsearch' ) {
}
};
my $db_count = -1;
my $missing_count = 0;
if ( $index eq $biblios_index_name ) {
$db_count = Koha::Biblios->search->count;
} elsif ( $index eq $authorities_index_name ) {
$db_count = Koha::Authorities->search->count;
}
if ( $db_count != -1 && $index_count != -1 ) {
$missing_count = $db_count - $index_count;
$es_has_missing = 1 if $missing_count > 0;
}
push @{ $es_status->{indexes} },
{
index_name => $index,
count => $count
index_name => $index,
index_count => $index_count,
db_count => $db_count,
missing_count => $missing_count,
};
}
$es_status->{running} = $es_running;
$template->param( elasticsearch_status => $es_status );
$template->param(
elasticsearch_status => $es_status,
elasticsearch_has_missing => $es_has_missing,
);
}
}

20
koha-tmpl/intranet-tmpl/prog/en/modules/about.tt
View file

@ -80,7 +80,7 @@
|
Indices:
[% FOREACH index IN elasticsearch_status.indexes %]
[% index.index_name | html %] (count: <emph>[% index.count | html %]</emph>)[% UNLESS loop.last %], [% END %]
[% index.index_name | html %] (count: <emph>[% index.index_count | html %]</emph>)[% UNLESS loop.last %], [% END %]
[% END %]
[% ELSE %]
<span class="bg-warning">not running</span>
@ -216,7 +216,7 @@
</div>
<div role="tabpanel" class="tab-pane" id="sysinfo">
[% IF warnPrefBiblioAddsAuthorities || warnPrefEasyAnalyticalRecords || warnPrefAnonymousPatronOPACPrivacy || warnPrefAnonymousPatronAnonSuggestions || warnPrefAnonymousPatronOPACPrivacy_PatronDoesNotExist || warnPrefAnonymousPatronAnonSuggestions_PatronDoesNotExist || warnPrefKohaAdminEmailAddress || warnPrefOpacHiddenItems || invalid_yesno.count || warnNoActiveCurrency || warnIsRootUser || xml_config_warnings.size || AutoSelfCheckPatronDoesNotHaveSelfCheckPerm || AutoSelfCheckPatronHasTooManyPerm || warnStatisticsFieldsError || warnNoTemplateCaching || warnILLConfiguration || has_ai_issues || oauth2_missing_deps || bad_yaml_prefs || warnRelationships || log4perl_errors || config_bcrypt_settings_no_set || warnHiddenBiblionumbers.size || warnConnectBroker %]
[% IF warnPrefBiblioAddsAuthorities || warnPrefEasyAnalyticalRecords || warnPrefAnonymousPatronOPACPrivacy || warnPrefAnonymousPatronAnonSuggestions || warnPrefAnonymousPatronOPACPrivacy_PatronDoesNotExist || warnPrefAnonymousPatronAnonSuggestions_PatronDoesNotExist || warnPrefKohaAdminEmailAddress || warnPrefOpacHiddenItems || invalid_yesno.count || warnNoActiveCurrency || warnIsRootUser || xml_config_warnings.size || AutoSelfCheckPatronDoesNotHaveSelfCheckPerm || AutoSelfCheckPatronHasTooManyPerm || warnStatisticsFieldsError || warnNoTemplateCaching || warnILLConfiguration || has_ai_issues || oauth2_missing_deps || bad_yaml_prefs || warnRelationships || log4perl_errors || config_bcrypt_settings_no_set || warnHiddenBiblionumbers.size || warnConnectBroker || elasticsearch_has_missing %]
[% IF (warnIsRootUser) %]
<h2>Warning regarding current user</h2>
<p>You are logged in as the database administrative user. This is not recommended because some parts of Koha will not function as expected when using this account.</p>
@ -511,6 +511,22 @@
</table>
[% END %]
[% IF elasticsearch_has_missing %]
<table>
<caption>Records are missing in Elasticsearch search engine</caption>
[% FOREACH index IN elasticsearch_status.indexes %]
[% IF index.missing_count %]
<tr>
<th scope="row"><strong>Warning</strong></th>
<td>
[% index.missing_count | html %] record(s) missing on a total of [% index.index_count | html %] in indice [% index.index_name | html %].
</td>
</tr>
[% END %]
[% END %]
</table>
[% END %]
[% ELSE %]
<p>No warnings.</p>
[% END %]