Rather than performing a symmetric diff and then splitting the results
in a further loop this patch changes to logic to use two asymetric diffs
to get the results more directly.
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
This script uses Array::Utils and adds a dependency
To test:
1 - Have Koha with ES running
2 - Delete some records from ES
curl -X DELETE "es:9200/koha_kohadev_biblios/data/5"
curl -X DELETE "es:9200/koha_kohadev_authorities/data/5"
3 - perl misc/maintenance/compare_es_to_db.pl
4 - Note you are notified of problems in both indexes
5 - perl misc/search_tools/rebuild_elastic_search.pl -a
6 - perl misc/maintenance/compare_es_to_db.pl
7 - Note you are only notified about problems in biblios (assuming you don't have other issues)
8 - perl misc/search_tools/rebuild_elasticsearch.pl -b
9 - perl misc/maintenance/compare_es_to_db.pl
10 - Both counts match, no problems
Signed-off-by: Ere Maijala <ere.maijala@helsinki.fi>
Signed-off-by: Bouzid Fergani <bouzid.fergani@inlibro.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
populate_db.pl from koha-misc4dev must be used instead.
This script existed before the creation of koha-misc4dev, it does no
longer make sense to have it in Koha.
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
SYSPREF_OVERRIDE unfortunately is not supported ;)
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
In an attempt to save time, bulkmarcimport temporarily sets CataloguingLog
and AuthoritiesLog to 0. It does this by disabling syspref caching and saving
the changes to the database (then replacing the original values at completion).
Unfortunately, this disables other key sysprefs from being cached, and results in
a 50% increase in processing time for the script.
This patch instead utilizes the ENV variable override feature of sysprefs, which
preempts the cache in C4::Context->preference().
To test:
1. Perform a bulkmarcimport with a reasonable number of biblios (~1000 will do)
2. Note the time taken to complete
3. Apply patch
4. Revert the biblio load performed
5. Perform another bulkmarcimport with the same biblios and commandline options
6. Note the time taken to complete
7. Compare times. The time from step 6 should be about 33% less than the time from step 2
8. Check Cataloguing and Authorities Logs to verify imported records were not logged
9. Profit!
Signed-off-by: Michal Denar <black23@gmail.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
If elastic is used as search engine, the bulkmarcimport.pl will not
handle correctly UTF-8 encoded MARCXML
Koha::SearchEngine::Search->new uses a require statement to load the correct Search module.
This is done l.257 of bulkmarcimport.pl:
257 my $searcher = Koha::SearchEngine::Search->new
Koha::SearchEngine::Elasticsearch::Search will `use MARC::File::XML`, and so resets the arguments set before:
216 $MARC::File::XML::_load_args{BinaryEncoding} = 'utf-8';
220 $MARC::File::XML::_load_args{RecordFormat} = $recordformat;
An easy (but dirty) fix could be to move the declaration of my $searcher before in the script.
The tricky (but correct) fix would be to remove the long standing "ugly hack follows" comment.
This patch is the easy, and dirty, fix
Test plan:
Use the command line tool to import MARXCML records that contains unicode characters into Koha
Something like `misc/migration_tools/bulkmarcimport.pl -biblios -file record.marcxml -m=MARCXML`
Without this patch you will notice that unicode characters will not be displayed correctly
Signed-off-by: Michal Denar <black23@gmail.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
I am not sure what happened here, but adding strict for TmplTokenizer.pm
breaks a test for translation.
It has been caught by a failing test:
% prove xt/author/translatable-templates.t
will display lines "invalid multibyte sequence"
I did a diff between the generated files and found:
< msgid "Ã rbok"
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
We absolutely need that for modules.
We are enabling strict by using Modern::Perl.
Note that other modules from C4 use strict and warnings, instead of
Modern::Perl
Test plan: git grep 2505 **/*.pm should not return any results
And let's see later what needs to be fixed.
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
This patch removes some warnings when creating new languages,
those related with uft8 and two not related.
Tipical messages are:
>Warning: Can't determine original templates' charset, defaulting to UTF-8
>Warning: Charset Out defaulting to UTF-8
>...
>Removing empty file /../xx-YY-marc-NORMARC.po
>I UTF-8 O UTF-8 at /../kohaclone/misc/translator/tmpl_process3.pl line 365.
>Use of uninitialized value $pref_name in concatenation (.) or string at /.../misc/translator/LangInstaller.pm line 197.
>Created /../misc/translator/po/xx-YY-messages.po.
First four are 'normal' messages
The fifth one correspond to an unquoted preference string
The sixth one is a normal output message from msgint command
We will hide them unless verbose '-v' option is selected,
and fix the error.
To test:
1) Go to misc/translator
2) Create new language files
$ perl translate create xx-YY
3) Check warnings and messages
4) Apply the patch
5) Remove created files, and create them again
Ther must be no messages
6) Repeat 5) with verbose option '-v', check
old messages
7) BONUS: without the patch update a language,
for example de-DE:
$ perl translate update de-DE
Note (git status) a new file, po/de-DE-messages.po~
This is a backup file from msgmerge command
With the patch this backup is removed.
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Liz Rea <wizzyrea@gmail.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
DUEDGST notice required PREDUEDGST to be on, but both messages have to
be generated independently
Test plan:
0) Do not apply the patch and reproduce the bug
- turn OFF sms PREDUEDGST and turn ON sms DUEDGST for a given borrower
- Sms version of DUEDGST is never generated into the message queue.
1) Apply the patch
2) Ensure that the bug is fixed
Signed-off-by: Michal Denar <black23@gmail.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Translate script ignores multiple options in system preferences,
this patch fix that.
To test:
1) Apply the patch
2) Go to misc/translator
cd misc/translator
3) Update your preferred language
perl translate update xx-YY
4) Updated pref file must contain new strings, eg.
egrep "when cataloguing an item" po/xx-YY-pref.po
5) Edit & translate that string
6) Install the updated translation
perl translate install xx-YY
7) Check that the translated pref file has
a) multiple options present
b) the message from 5) is translated
see for example circulation.pref
en: koha-tmpl/intranet-tmpl/prog/en/modules/admin/preferences/circulation.pref
xx: koha-tmpl/intranet-tmpl/prog/xx-YY/modules/admin/preferences/circulation.pref
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
There are two typos in advance_notics.pl that cause DUEDGST messages
not to be sent. See Bugzilla for full details.
If you think the typo is sufficiently obvious, you can just eyeball the
patch and sign off, methinks. Otherwise, testing can be done something
like this:
- Make sure you have enabled enhanced messaging preferences, and a
patron with "Email" and "Digests only" set for "Item due" messages
- Issue an item to this patron, with due date today
- Run something like this to generate advance notices:
$ sudo koha-shell -c "perl \
/home/vagrant/kohaclone/misc/cronjobs/advance_notices.pl -n -c" kohadev
- See that no notices are shown (-n means messages will go to stdout,
instead of into the message queue).
- Apply the patch and run advance_notices.pl again, as before. A DUEDGST
message should now be displayed.
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Bug 20478 introduced a typo where the letter code DUEDGST was changed
to DUEGST. This patch fixes it.
To test:
- Run "grep -r DUEDGST *" on the Koha git repo. Notice that this letter
code is used in e.g. sample notices.
- Run "grep -r DUEGST *" and notice that this code only occurs twice,
in misc/cronjobs/advance_notices.pl.
- Aply this patch.
- Run "grep -r DUEGST *" again, and notice there are no more occurences
of this typo.
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
I do not think we should have the same trick as the intranet, and
display a message. This should be enough.
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
This may be quite long for big catalogue, but I think it is a good one
to have.
Test plan:
Same as first patch, then execute search_for_data_inconsistencies.pl
Notice the error.
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
This patch replaces CSS-based background image icons with Font Awesome
icons. This will eliminate the use of sprite.png altogether.
To test, apply the patch and regenerate the OPAC CSS
(https://wiki.koha-community.org/wiki/Working_with_SCSS_in_the_OPAC_and_staff_client).
Because JavaScript events are often triggered by class, the changes made
in this patch require testing of JavaScript-driven behavior as well as
visual confirmation of the changes.
Enable OPAC features to expose them for testing: Login, Holds, Article
requests, Tags, Cart, Search term highlighting.
- Perform a search in the OPAC
- In the header at the top of the search results, check the
'Unhighlight/Highlight' link.
- Select some search results and verify that the multi-item controls
are enabled: Add to cart/Lists; Place hold; Tag. Check that these
controls work correctly.
- Verify that the following links look correct and work correctly:
'Place hold,' 'Request article,' 'Log in to add tags,' 'Add to
cart.'
- Log in to the OPAC and return to the search results page. Check the
'Add tag' and 'Save to lists' links.
- View the lists page
- Test the 'Edit' and 'Delete' controls.
- View the contents of a list. Test the following:
- Controls at the top of the table of results ('New list,'
'Download list,' 'Send list,' 'Print list,' 'Edit list,' and
'Delete list').
- Multi-selection operations: 'Place hold,' 'Tag,' and 'Remove
from list.'
- Controls for each item on the list: 'Place hold,' 'Add tag,'
'Save to another list,' 'Remove from this list,' and 'Add to
cart.'
- View the bibliographic detail page for any search result
- In the right-hand sidebar menu, verify that these controls look
correct and work correctly: 'Place hold,' 'Print,' 'Request
article,' 'Save to your lists,' 'Add to your cart,'
'Unhighlight,' 'Send to device,' 'Save record,' and 'More
searches.'
- Test the shelf browser by clicking 'Browse shelf' under the call
number for any of the holdings.
- Test the 'Next' and 'Previous' controls.
- Check the tabs for switching between 'Normal,' 'MARC,' and 'ISBD'
views and verify that they look correct on each of those pages.
- Add some items to the Cart and open the Cart
- Test the cart controls at the top: 'More details/Brief display,'
'Send,' 'Download,' 'Empty and close,' 'Hide window,' 'Print.'
- Test the multiple-selection controls: 'Remove,' 'Add to list,'
'Place hold,' and 'Tag.'
- Log in to the OPAC as a user who has items checked out which can be
renewed. On the 'Your summary' page, on the 'Checked out' tab, test
the 'Renew selected' and 'Renew all' links.
Signed-off-by: Liz Rea <wizzyrea@gmail.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
We require next_open_day & prev_open_day to be more flexible. We could
create a separate sub, but that's not very DRY given that they'll do
pretty much the same thing.
So next_open_day becomes next_open_days and prev_open_day becomes
prev_open_days and both functions accept an additional parameter which
determines how many days they add or subtract.
All calls of these two functions have been modified accordingly.
Sponsored-by: Cheshire West and Chester Council
Sponsored-by: Cheshire East Council
Sponsored-by: Newcastle City Council
Sponsored-by: Sefton Council
Signed-off-by: Liz Rea <wizzyrea@gmail.com>
Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
To test:
1 - perl misc/search_tools/rebuild_elastic_search.pl -h
2 - Note it indicates the bn option can be passed to index individual authids
3 - perl misc/search_tools/rebuild_elastic_search.pl -a -bn 92 -v
4 - Note the error
5 - Apply patch
6 - perl misc/search_tools/rebuild_elastic_search.pl -h
7 - Note new option ai|authid for indexing individual authids
8 - Note updated text for bn|biblionumber option
9 - perl misc/search_tools/rebuild_elastic_search.pl -a -bn 92 -v
10 - No errors, but no records indexed
11 - perl misc/search_tools/rebuild_elastic_search.pl -a -ai 92 -v
12 - 1 record indexed
13 - perl misc/search_tools/rebuild_elastic_search.pl -ai 92 -bn 92 -v
14 - 1 authority record and 1 biblio record indexed
Signed-off-by: Michal Denar <black23@gmail.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
This patch adds a new script update_patrons_category.pl which allows for
updating patron categories in a batch upon meeting provided criteria.
This script additionally can replace j2a.pl.
To test:
1 - perl update_patrons_category.pl -h
2 - Ensure help text makes sense and covers all options
3 - Test converting patrons supplying only fromcat and tocat
perl update_patrons_category.pl -f PT -t J -v --confirm
perl update_patrons_category.pl -f J -t PT -v --confirm
4 - All patrons should have been switched to and from Juveniles
5 - Try without --confirm switch
perl update_patrons_category.pl -f PT -t J -v
6 - Should list all patrons but not update
7 - Set the age for juvenile patrons to be outside the range provided in
categories (or set the upper age limit for juveniles to '2')
8 - Test with verbosity and with without --confirm
perl update_patrons_category.pl -f J -a -t PT -v
perl update_patrons_category.pl -f J -a -t PT -v --confirm
9 - Repeat above and verify linked/unlinked guarantors are removed in above scenario
10 - Test various fine and registration limits
11 - Test matching on specific fields i.e. --field surname=acosta
12 - Sign off
Sponsored by: Round Rock Public Library
(https://www.roundrocktexas.gov/departments/library/)
Signed-off-by: Jesse Maseto <jesse@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
* Update C4::Accounts::chargelostitem
* Update C4::Accounts::manualinvoice
* Update C4::Circulation::_FixOverduesOnReturn
* Update C4::Circulation::_FixAccountForLostAndReturned
* Update C4::Overdues::UpdateFine
* Update C4::Overdues::GetFine
* Update C4::Overdues::GetOverduesForBranch
* Update Koha::Account->pay
* Update Koha::Account->add_debit
* Update Koha::Account->non_issues_charges
* Update Koha::Account::Line->apply
* Update Koha::Account::Line->adjust
* Update controller scripts
* Update reports scripts
* Update tests
Test Plan
1) Run the test suit and ensure everything still passes
2) Test reports/cash_register_stats still works
3) Test that adding manual invoices still works
4) Test that making payments still works
5) Test that lost item fee handling still works
6) Test that invoice printing still works
7) Test that the sco still works
Signed-off-by: Séverine QUEUNE <severine.queune@bulac.fr>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
018 - [Reserved for other International Standard Numbers]
033 - Other System Persistent Record Identifier
183 - Coded Data Field: Type of Carrier
203 - Content Form and Media Type
231 - Digital File Characteristics
283 - Carrier Type
338 - Funding Information Note
Signed-off-by: Sonia BOUIS <sonia.bouis@univ-lyon3.fr>
Signed-off-by: Alex Arnaud <alex.arnaud@biblibre.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Test plan:
1. Take a look at files in misc/migration_tools/ifla/data. It
contains all data that will be inserted into Koha. Its content is
based on the previous patches.
2. Run the script misc/migration_tools/ifla/update.pl and verify
that it effectively added the new fields, subfields, authorised
values and authority types.
3. Run the script again and see that it doesn't update existing fields
4. Run with --force and verify that it update existing fields (you can
modify unimarc_ifla.yml to see changes)
5. Run with --force --po-file misc/migration_tools/ifla/language/fr.po
and verify that the labels are now in french
There is a POT file in misc/migration_tools/ifla/language/template.pot,
use it to create PO files for other languages.
Signed-off-by: Sonia BOUIS <sonia.bouis@univ-lyon3.fr>
Signed-off-by: Alex Arnaud <alex.arnaud@biblibre.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
