To test:
1) Verify the error installing translations for any language
2) Apply the patch
3) Repeat 1), check warning is gone
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Without -v the script will no longer display messages unless the error
is major.
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
When 041$a or another subfield is linked to an authorised value list,
Koha will replace the codes with the descriptions from the authorised
value in the MARCXML before passing it to the XSLT.
The XSLT was made to translate codes into descriptions - as there is
no code now, this fails and 'unknown language' is shown.
The patch changes the handling so that when the XSLT is handled something
that doesn't match a known code, it displays the subfield without
processing.
To test:
- Create some records with 041.
Examples: http://www.loc.gov/marc/bibliographic/bd041.html
- Verify this displays nicely in staff and OPAC
- Apply patch
- Verify display is unchanged
- Remove the patch
- Change one of your MARC frameworks (use another framework than
for your first test records)
and link the 041 subfields to the LANG authorised value list
- Go to administration and create some entries for LANG with language
codes and descriptions
- Catalog a 041 using the normal editor (not advanced)
- You can now use a pull down to select the values you want
- Verify the display is now 'Unknown language' for this record
- Apply patch
- Verify all records display the language descriptions now, using
an authorised value or not
- Run misc/maintenance/generate_MARC21Languages.pl
Verify the output at the end reads like:
<xsl:otherwise>
<!-- when 041\$a is mapped to an authorised value list, this will show the description -->
<xsl:value-of select="\$code" />
</xsl:otherwise>
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
To test:
1) Create translation files for a new language
( cd misc/translator; ./translate create xx-YY)
A new language means one that isn't already in Koha, xx-YY=>something you
invent.
2) Verify double encoding
egrep "Aix-Marseille|Jean Prunier|periodika|Bokm" misc/translator/po/xx-YY-*
check strange strings
3) Apply the patch
4) Create po files again
( cd misc/translator; rm -f po/xx-YY*; ./translate create xx-YY)
5) Verify no more double encoding
egrep "Aix-Marseille|Jean Prunier|periodika|Bokm" misc/translator/po/xx-YY-*
check normal string
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Signed-off-by: Didier Gautheron <didier.gautheron@biblibre.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Setup:
1 - Be using Elasticsearch
2 - Reload mappings from the db
Admin->Search configuration
Reset Mappings
3 - Reindex ES and confirm searching is working
To test:
1 - Apply patch
2 - Alter your mappings file for elastic (just change a description for a field)
3 - perl misc/search_tools/rebuild_elasticsearch.pl -r -v
Verbose not necessary, but good for letting you know things are progressing
4 - Confirm the mapping change shows in the interface
5 - Confirm reindex worked and searching is working
Signed-off-by: Andrew Fuerste-Henry <andrew@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
To test:
1 - Use the Koha sample data, or insert a blank 245$b into a record (easiest way is using advanced cataloging editor
2 - Reindex elasticsearch
3 - Check the ES count on the about page
4 - Check the count in the DB (SELECT count(*) FROM biblio)
5 - They don't match!
6 - perl misc/search_tools/rebuild_elastic_search.pl -v -v
7 - No errors indicated
8 - Apply patch
9 - perl misc/search_tools/rebuild_elastic_search.pl -v
10 - You should be notified of an error
11 - perl misc/search_tools/rebuild_elastic_search.pl -v -v
12 - You should be notified of the specific biblio with an error and a (somewhat) readable reason
13 - perl misc/search_tools/rebuild_elastic_search.pl
14 - No output
Signed-off-by: Ere Maijala <ere.maijala@helsinki.fi>
Signed-off-by: Séverine QUEUNE <severine.queune@bulac.fr>
Signed-off-by: Bouzid Fergani <bouzid.fergani@inlibro.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
This patch simply adds new po files for each supported
language.
The are not created on update, so the need to add this files.
To test:
1) Update your preferred language
2) Check missing *installer*po files
3) Create new translation files:
cd /misc/translation
./translate create xx-YY
check *installer*po files
4) Apply patch
5) Repeat 1, verify installer files are updated
see last modification time
Pre-filled translations for some languages
(https://translate.koha-community.org/projects/marc21/)
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
This patch fix the missing xml prolog in translated
files, XML or TT.
Is fixed teaching C4::TTParse not to ignore <?..?> constructs,
then teaching xgettext.pl to ignore those strings. Net result is
that they are copied in the translated file.
To test:
1) Update & install your preferred language,
(cd misc/translator/; perl translate update xx-YY; perl translate install xx-YY )
2) Compare the first lines (head -2) of:
koha-tmpl/opac-tmpl/bootstrap/en/xslt/compact.xsl
koha-tmpl/opac-tmpl/bootstrap/en/modules/opac-opensearch.tt
koha-tmpl/intranet-tmpl/prog/en/xslt/plainMARC.xsl
and
koha-tmpl/opac-tmpl/bootstrap/xx-YY/xslt/compact.xsl
koha-tmpl/opac-tmpl/bootstrap/xx-YY/modules/opac-opensearch.tt
koha-tmpl/intranet-tmpl/prog/xx-YY/xslt/plainMARC.xsl
Check the missing prolog
3) Install this patch, repeat 1 and 2, now the prolog is present
on translated files.
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
We removed UseKohaPlugins in bug 20415 but incidentally added a
reference to it bug 24183. This patch corrects that error.
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
When languages are added or removed when need to flush the caches to
keep the interface consistent
Signed-off-by: Liz Rea <wizzyrea@gmail.com>
Signed-off-by: Bouzid Fergani <bouzid.fergani@inlibro.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
C4::Log::cronlogaction() was never implemented in automatic_item_modification_by_age.pl.
Test Plan:
1) Apply this patch
2) Enable CronjobLog
3) Run automatic_item_modification_by_age.pl succesfully
4) Note the run is noted in the action logs
Signed-off-by: Mark Tompsett <mtompset@hotmail.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
This patch introduces locking in fines.pl. It does so by leveraging on
bug 25109, which introduces a generic locking implementation on
Koha::Script.
The introduced changes:
1. Try to get the lock
2.a. If success, normal execution happens
2.b. If rejected, cronlogaction is called with a meaningful message and
a normal exit happens, so we don't flood the logs in vain. --verbose
will make the script print the same message on STDERR, as already is
the case with this script
To test:
1. Apply this patch
2. In two separate consoles run:
$ kshell
k$ perl misc/cronjobs/fines.pl --verbose
SUCCESS => The first one runs normally, the second one exists really fast
and prints a message about the lock.
3. Sign off :-D
Sponsored-by: Orex Digital
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
In biblio records, a field must be linked to biblio.title.
200$a in UNIMARC, 100$a MARC21 (and other).
If this field is undefined, some pages like checkouts table can fail.
Test plan :
1) Remove biblio title in SQL :
UPDATE biblio SET title='' WHERE biblionumber=XXX
2) Run misc/maintenance/search_for_data_inconsistencies.pl
3) You see the record as an inconsistency
4) Reset a title
UPDATE biblio SET title='50 shades of Grey' WHERE biblionumber=XXX
5) Run misc/maintenance/search_for_data_inconsistencies.pl
6) Record is no longer an inconsistency
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Like in most scripts in misc, add confirm argument
to ensure script is not run without knowing what it does.
Test plan:
1) Run misc/migration_tools/remove_unused_authorities.pl -h
2) You see help line for confirm
3) Run misc/migration_tools/remove_unused_authorities.pl
4) You see help and script does nothing
5) Run misc/migration_tools/remove_unused_authorities.pl -c
6) Script runs like wanted
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
The script misc/migration_tools/remove_unused_authorities.pl directly checks if Zebra search is OK.
This patch changes so that this test is only if Zebra is the search engine.
It also adds a test on the search off any authority number indexed (index 'an').
With Zebra its : an,alwaysmatches=''
With ES its : an:*
This test ensure that biblios records are indexed and that not all autorities will be deleted.
Test plan:
1) On a catalog create a new authority
2) Use Zebra in systempreference SearchEngine
3) Stop Zebra server
4) Run misc/migration_tools/remove_unused_authorities.pl -c
5) The script does nothing and says :
Zebra server seems not to be available. This script needs Zebra runs.
6) Restart Zebra server
7) Delete biblio index base
8) Run misc/migration_tools/remove_unused_authorities.pl -c
9) The script does nothing and says :
Searching authority number in biblio records seems not to be available : an,alwaysmatches=''
10) Use ElasticSearch in systempreference SearchEngine
11) Delete biblio index base
12) Run misc/migration_tools/remove_unused_authorities.pl -c
13) The script does nothing and says :
Searching authority number in biblio records seems not to be available : an:*
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
remove_unused_authorities.pl script can be improved.
This patch changes changes verbosity so than test mode can be used
to know the autorities that are used and those that can be deleted.
It also writes a line in output if limited authority type(s).
This patch also removes the unused vars $thresholdmin and $thresholdmax.
It also changes the query to use SQL with parameters for authority types.
Test plan :
1) On a catalog create a new authority
2) Be sure catalog is well indexed
3) Run misc/migration_tools/remove_unused_authorities.pl -t
4) You will see the line :
*** Testing only, authorities will not be deleted. ***
5) You will see lines of :
authid=x type=y : used X time(s)
6) You will see the line for the authority created in 1) :
authid=x type=y : can be deleted
7) You will see at the end :
x authorities parsed
y can be deleted because unused
z unchanged because used
8) Run misc/migration_tools/remove_unused_authorities.pl
9) You don't see the line :
*** Testing only, authorities will not be deleted. ***
10) You will see lines of :
authid=x type=y : used X time(s)
11) You will see the line for the authority created in 1) :
authid=x type=y : deleted
12) You will see at the end :
x authorities parsed
y deleted because unused
z unchanged because used
13) Run misc/migration_tools/remove_unused_authorities.pl --auth NP --auth CO
14) You see the line :
Restricted to authority type(s) : NP,CO.
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
In order to deal automatically with suggestions that need to be
archived, this cronjob will help to add the "archived" flag following
their age or status.
Test plan:
- Execute the script with the --help parameter and read the documentation.
- Does it make sense?
- Use it without the --confirm flag and try several combination
For instance you will want to archive suggestions that have been
rejected for more than a week:
perl misc/cronjobs/archive_purchase_suggestions.pl
--age-date-field=rejecteddate --age=weeks:1
or even suggestion that have been suggested for more than a year and
have a ORDERED status:
perl misc/cronjobs/archive_purchase_suggestions.pl
--age-date-field=suggesteddate --age=years:1 --status=ORDERED
- Use it with the --confirm flag and make sure the suggestions have been
archived.
Sponsored-by: BULAC - http://www.bulac.fr/
Signed-off-by: Séverine QUEUNE <severine.queune@bulac.fr>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Recent versions of MariaDB changed the output of 'DESCRIBE' for
timestamp columns with defaults from `CURRENT_TIMESTAMP` to
`current_timestamp()`. As such the code inside
DBIx::Class::Schema::Loader which catches such cases and outputs
`\"current_timestamp"` as a sensible cross platform default is missed
and this leads of inconsistent class files and bugs with out default
lookup code in Koha::Objects.
This patch serves as a backport of the code I have submitted upstream
such that out developers can continue to use update_dbix_class_files.pl
to build their schema classes from the database and regardless of their
db server version get a consistently correct output.
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
I believe the error is triggered when borrowernumbers are left empty in
the accountlines table. Not sure why this would happen, but it appears
to be what causes the problem.
Do not apply the first patch if testing this patch.
To test:
1) sudo koha-mysql INSTANCENAME
2) Create a test borrower, add any payment etc to create an accountline,
then delete this borrower
3) ensure the AccountAutoReconcile syspref is disabled
4) Go to another borrower's accounting tab
5) Create a manual credit or debit. Confirm this shows in the 'Make a
payment' tab as an amount that COULD be applied, but isn't automatically
applied
6) in your terminal, run the reconcile_balances.pl script
7) Confirm the error does not show in the logs and the balance for
the borrower is correctly reconciled.
Sponsored-by: Horowhenua District Council
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
This patch corrects a few additional cases where DateTime->now is called
directly instead of via Koha::DateUtils.
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
We should use Koha::DateUtils instead of Date::Time directly
This patch simplay replaces calls to now() with a call to dt_from_string()
which does effectively the same thing.
Probably reading the code and verifying changes is sufficient but...
To test:
1 - confirm the files all compile
2 - confirm all tests pass
3 - confirm Koha still works
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
This patch adds a simple hook to the process_message_queue cronjob
to allow plugins to pre-process messages in the messaging queue before
they are sent.
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
To test:
1 - Have some patrons in your system
2 - Export some of their info via reports
SELECT cardnumber, userid, surname, firstname, password, branchcode, categorycode
3 - Edit the file from above, changing all the password lines
4 - Import the file with overwrite
5 - Confirm passwords have not changed (run the report again and confirm the hashes are the same)
6 - Apply patch
7 - Restart all the things
8 - Check the new box on import screen to overwrite passwrods
9 - Import file again
10 - Confirm passwords have changed
11 - Signin using new password to verify the hash is the password as supplied
12 - Repeat via commandline import supplying --overwrite_passwords option
13 - Verify works as expected
14 - Prove -v t/db_dependent/Koha/Patrons/Import.t
Sponsored-by: ByWater Solutions
Signed-off-by: Ron Marion <ron.marion@goddard.edu>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Use the sub from C4::Installer to avoid dup of code.
Note:
We are going to modify the script and so will do more stuffs.
We may want to rename it, maybe installer_utilities.pl,
misc/installer/yaml_utility.pl, any suggestions?
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
This patch adds a command line utility to load
yaml files.
To test:
1) Apply the patch
2) Try loading a file, e.g. sample notices
a) mysql -e "delete from letter"
b) misc/load_yaml.pl -f installer/data/mysql/en/mandatory/sample_notices.yml
3) Try loading a file with a syntax error,
edit previous file adding ':' at the end of a field without quotes
then load again
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Test plan:
- launch perl misc/search_tools/export_elasticsearch_mappings.pl >
/path/to/my_mappings.yaml
- set koha-conf.elasticsearch_index_mappings to
/path/to/my_mappings.yaml,
- go to admin -> Search engine configuration,
- click on "Reset mappins",
- check that your search fields and mappings are as expected
Signed-off-by: Bouzid Fergani <bouzid.fergani@inlibro.com>
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Owen Leonard 2018-03-16 10:47:47 UTC :
<<
I don't think the system preference adds any security. There are already multiple permissions required for working with plugins:
- Configure plugins
- Manage plugins ( install / uninstall )
- Use report plugins
- Use tool plugins
And even with those permissions your server must be configured to allow the use of plugins.
>>
Test plan :
1) Install kitchen sink plugin https://github.com/bywatersolutions/koha-plugin-kitchen-sink
2) Run misc/devel/install_plugins.pl
3) Set config enable_plugins=1
4) Check all parts of the plugin are working
5) Set config enable_plugins=0
6) Check all parts of the plugin are disabled
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Andrew Fuerste-Henry <andrew@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
To test:
1 - Verify on staff side that patron can be edited to opt in our out of auto renewal
2 - Check out some items to a patron opted in to auto renewal
3 - Ensure the items are checked out and set to autorenew
4 - Login on the opac at the patron
5 - Verify items cannot be renewed as scheduled for auto-renewal
6 - On staff side, opt patron out of auto renewal
7 - Verify on opac items are no longer marked for auto renewal
8 - Run the auto renewal cron job, items are not renewed
9 - Set 'no renewal before' to a setting that would prevent renewal
10 - Verify that opting patron in or out of auto renewal changes only the reason items cannot be renewed
11 - Set 'no renewal before' to a setting that would allow for renewal
12 - Verify that opting patron in/out changes their ability to renew
13 - Verify that when opted out cron does not renew
14 - Verify that when opted in the item is auto renewed
15 - Reset the due date, opt out, verify manual renewal succeeds
Signed-off-by: Andrew Fuerste-Henry <andrew@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Test plan:
0) Apply patch
1) try to run affected scripts and ensure the amounts are corectly
calculated for different patron categories:
installer/data/mysql/fix_unclosed_nonaccruing_fines_bug17135.pl
misc/cronjobs/fines.pl
misc/cronjobs/staticfines.pl
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
When importing with bulkmarcimport, authorities may or may not be updated based
on which authority is newer (005 are compared). This patch allows to keep track
in the result yaml file if an authority has been updated or not:
Signed-off-by: Jon Knight <J.P.Knight@lboro.ac.uk>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Adds the ability to specify a file with the --file flag that
should be a list of borrowernumbers for deletion.
If used without other flags it will delete the list of borrowers,
if used with other flags it will treat the other criteria
as filters for the list.
To Test:
1) Specify a list, e.g. borrowers.txt, of borrowernumbers in the database and put them in a text file with one per line
2) Invoke the script like ./delete_patrons.pl --file borrowers.txt to delete those patrons
3) Populate the list with a mix of borrowers in a category and those not, say categoryfoo
4) Invoke the scirpt as ./delete_patrons.pl --category_code categoryfoo --file borrowers.txt
5) Only the borrowers in the file that have the specified category will be deleted
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
To test:
Have a record with the 000 position 5 set to "deleted" (record status)
Use koha-shell to run the job with the -i flag: perl misc/cronjobs/delete_records_via_leader.pl -c -i -v
* items and the record should be deleted
Mark another record with the 000 position 5 set to deleted (record status)
Use koha-shell to run the job without the -i flag: perl misc/cronjobs/delete_records_via_leader.pl -c -v
* if the record had items, it will not be deleted
* if the record did not have items, it will be deleted.
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Starting to replace the ModItem calls with Koha::Item->store
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
To test:
1 - Have a patron wiht an sms number, but no email
2 - Enable sms by setting SMSDriver syspref to Email
3 - Set Notice triggers to send an overdue via sms and email
4 - Ensure the selected notice is defined only for 'email'
5 - Checkout an overdue to the patron above
You can set a specify a due date in the past
6 - perl misc/cronjobs/overdue_notices.pl -t
7 - Check the patron notices tab, they have a print notice and an SMS notice pending
8 - The SMS notice used the 'email' template
9 - Apply patch
10 - sudo koha-mysql kohadev
11 - DELETE * FROM message_queue WHERE borrowernumber={borrower as above};
12 - Run the cron again
13 - Patron should have a print notice queued, no sms
14 - Define an SMS notice
15 - delete the message queue
16 - Run again
17 - The patron should have a print and an SMS
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
This patch updates the overdue_notices functionality to use the
inbound_email_address method to obtain the correct email address
for a branch.
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
This patch standardises the encoding name used in direct calls
to new_from_xml() to 'UTF-8' instead of 'utf8' or 'utf-8'.
Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
This patch adds handling of null values on translatable
labels.
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
This patch adds the ability to:
1) Create new translation files from yaml installer files
2) Create installer directory for a given language
It will not create a installer directory if it already exists.
New (possible) translation files:
xx-YY-installer.po
xx-YY-installer-MARC21.po
xx-YY-installer-UNIMARC.po
Needs Bug 13897 (for yaml files)
NOTE: updated version adding ability to process multiline
fields, discard small ( < 2) strings, and discard strings
with pure html, TT or punctuation.
To test:
1) Apply patches from Bug 13897
2) Apply this patch
3) Go to misc/translation
4) Create translation files for a NEW language
$ ./translate create xx-YY
check new file 'xx-YY-installer.po'
5) Copy ../../installer/data/mysql/en/optional/auth_val.yml
into ../../installer/data/mysql/en/marcflavour/marc21/mandatory/
and ../../installer/data/mysql/en/marcflavour/unimarc/mandatory/
remove po/xx-YY*, then repeat creation
check new files 'xx-YY-installer.po', 'xx-YY-installer-MARC21.po'
and 'xx-YY-installer-UNIMARC.po'
remove all new files
6) Create for xx-YY again and try update
$ ./translate create xx-YY
edit ../../installer/data/mysql/en/optional/auth_val.yml
and change one char in one of the translatable values,
also edit 'xx-YY-installer.po', translate the same
string (in msgstr).
Do an update
$ ./translate update xx-YY
check in 'xx-YY-installer.po' a fuzzy value for the
translated value and the preservation of the translation
Fix the translation, or add a new one.
7) Create an install dir for xx-YY
./translate install xx-YY
Check new dir '../../installer/data/mysql/xx-YY/'
Check files on it
$ tree ../../installer/data/mysql/xx-YY/
and compare with ../../installer/data/mysql/en/
All installation files must be present
8) Try a new Koha install using this language in the
usual way.
Check in authorised_values table for the translated string.
9) Try create an install dir for an existing language
(eg. es-ES, de-DE or fr-FR ), eg.
./translate install de-DE
Install dir is detected and not changed, a note is printed.
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
At the last development meeting we have voted to remove the
QueryParser-related code
https://wiki.koha-community.org/wiki/Development_IRC_meeting_19_February_2020
Hea tells us that it has not been adopted, and the code/bug tracker that
it is not really usable as it. As nobody is willing to work on it, we
decided to remove it instead.
Test plan:
% prove t/db_dependent/Search.t
must return green
See commits from bug 9239 and confirm that the code is removed in this
patch.
Also play with the search on the UI and confirm that you do not see
obvious regressions
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
If you try to update some language translation files
translate script will complain of missing *-messajes-js.po
file.
This patch adds those missing files
To test:
1) Go to misc translation
2) Update some language, eg- fr-FR
Check error
/usr/bin/msgmerge: error while opening "/.../misc/translator/po/es-ES-messages-js.po" for reading: No such file or directory
3) Apply this patch
4) Repeat 2), no error message
Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Use intrahtdocs and opachtdocs in order to get the correct paths for all
kind of installations
Test plan:
1. With a dev install, run ./translator create/update/install xx-XX and
verify that there is no error message
2. With a standard install, run ./translator create/update/install xx-XX
and verify that there is no error message
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Tested for dev/single/standard installs, no errors,
just a (normal) warn from po2json.
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Bug 9978 should have fixed them all, but some were missing.
We want all the license statements part of Koha to be identical, and
using the GPLv3 statement.
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
This patch adds the ability to compress/uncompress
translations files.
On update/install the files are uncompressed first
The only gain is to use less space.
To test:
1) Apply the patch
2) Go to misc/translator
3) Try it
$ ./translate compress fr-FR (check po/*.gz)
$ ./translate uncompress fr-FR ( check normal files)
4) Try again with verbose mode
$ ./translate compress fr-FR -v (list compressed files)
$ ./translate uncompress fr-FR -v (list uncompressed files)
5) Try update compressed files
$ ./translate compress fr-FR
$ ./translate update fr-FR (result is uncompressed)
6) Try install compressed files
$ ./translate compress fr-FR
$ ./translate install fr-FR
7) Try compress all
$ ./translate compress (add '-v' for verbose output)
8) Try uncompress all
$ ./translate uncompress (add '-v' for verbose output)
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Bouzid Fergani <bouzid.fergani@inlibro.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Since these operations impact on the resultset, the counts should be
saved before.
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Bouzid Fergani <bouzid.fergani@inlibro.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
The warn must be done in the cronjob.
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Followed this test plan (with two follow-ups applied):
[1] Prefs: UnsubscribeReflectionDelay=1, PatronAnonymizeDelay=2, PatronRemovalDelay=3, FailedLoginAttempts was undef
[2] Pick borrower and set expiry to NOW-2, and lock him (login_attempts=-1) Could be achieved too by settings FailedLoginAttempts and trying wrong passwords. Run cleanup job:
Locked 0 patrons
Anonymized 1 patrons
Deleted 0 patrons
[3] Pick borrower, set expiry to NOW-3. Run cleanup job:
Locked 0 patrons
Anonymized 0 patrons
Deleted 1 patrons
Signed-off-by: Bouzid Fergani <bouzid.fergani@inlibro.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
misc/cronjobs/update_totalissues.pl when issuing statistics table, no-incremental, has a problem for biblio with multiple items.
The first SQL query gets the biblios linked to items with issues in statistics :
SELECT
biblio.biblionumber, COUNT(statistics.itemnumber)
FROM
biblio
LEFT JOIN
items
ON
(biblio.biblionumber=items.biblionumber)
LEFT JOIN
statistics
ON
(items.itemnumber=statistics.itemnumber)
WHERE
statistics.type = 'issue'
$limit
GROUP BY
biblio.biblionumber
The second SQL query is :
SELECT
biblio.biblionumber, 0
FROM
biblio
LEFT JOIN
items
ON
(biblio.biblionumber=items.biblionumber)
LEFT JOIN
statistics
ON
(items.itemnumber=statistics.itemnumber)
WHERE
statistics.itemnumber IS NULL
GROUP BY
biblio.biblionumber
The problem is that this second query will set to 0 where ANY item has no entry in statistics table.
So when running it sets 0 to the biblio that had a value from first
query.
I think the best fix is to use "WHERE statistics.type = 'issue'" inside que JOIN :
LEFT JOIN statistics ON (items.itemnumber=statistics.itemnumber AND statistics.type = 'issue')
Test plan :
1) Begin with an empty database
2) Create a biblio 1 with no items
3) Create a biblio 2 with 3 items
4) Create a biblio 3 with 2 items
5) Checkout and checkin all items of biblio 2
6) Checkout and checkin the firt item of biblio 3
7) run misc/cronjobs/update_totalissues.pl --use-stats -v
8) Check biblio 1 has biblioitems.totalissues = 0
9) Check biblio 2 has biblioitems.totalissues = 3
10) Without patch the biblio 3 has biblioitems.totalissues = 0
11) With patch the biblio 3 has biblioitems.totalissues = 1
12) Check misc/cronjobs/update_totalissues.pl --incremental is OK
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
This patch adds an optional maxdays parameter to the fines.pl cronjob.
The new parameter allows system administrators to set a maximum number
of days for the job to process overdues. This can improve performance.
Signed-off-by: Sally Healey <sally.healey@cheshiresharedservices.gov.uk>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
We certainly faced 3 similar bugs due to this syntax: bug 23006, bug
22941 and bug 17526.
To prevent other issues related to this syntax this patch suggests to
replace them all in one go.
Test plan:
Confirm that the 2 syntaxes are similar
Eyeball the patch and confirm that there is no typo!
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
It adds Javascript equivalent of Koha::I18N's exported subroutines, and
they are used the same way.
String extraction is done only on *.js files and require gettext 0.19
(available in Debian jessie, and also in wheezy-backports)
It adds Javascript library Gettext.js for handling translation and a
Perl script po2json to transform PO file into JSON.
Gettext.js and po2json both come from Locale::Simple.
There are several tools named po2json. It's simpler to integrate this
one into Koha than to check if the good one is installed on the system.
Locale::Simple is not needed.
To avoid polluting the global namespace too much, this patch also
introduce a global JS object named Koha and add some stuff in Koha.i18n
Test plan:
1. Add a translatable string in a JS file. For example, add this:
alert(__nx("There is one item", "There are {count} items", 3,
{count: 3}));
to staff-global.js
2. cd misc/translator && ./translate update fr-FR
3. Open misc/translator/po/fr-FR-messages-js.po, verify that your
string is present, and translate it
4. cd misc/translator && ./translate install fr-FR
5. (Optional) Verify that
koha-tmpl/intranet-tmpl/prog/fr-FR/js/locale_data.js exists and
contains your translation
6. Open your browser on the staff main page, change language and verify
that the message is translated
7. Repeat 1-6 on OPAC side
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Works well, translation is OK and test message is displayed correctly.
Current qa-tool error is a false positive.
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Usually commit takes a value
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Andrew Fuerste-Henry <andrew@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
To test:
1 - Apply patch
2 - Have some items marked for auto-renewal
3 - Run the job with no parameters
4 - It should print 'Test mode'
5 - Provide -v
6 - It should print 'Test mode' then a line for each item
7 - Provide -v -c
8 - It should not say test mode, but should provide a line for each item
9 - Provide -c
10 - Shoudl run as expected with no output
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Andrew Fuerste-Henry <andrew@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
It is annoying to have to specify database connection parameters each
time DBIx::Class files need to be updated.
This patch adds a new option --koha-conf that takes an optional <path>
which defaults to the value of KOHA_CONF environment variable, and use
the database connection parameters found in that file.
--db_* options override values from $KOHA_CONF
Test plan:
1. Run the script with the same parameters as before the patch and see
that it still works.
Example:
misc/devel/update_dbix_class_files.pl --db_name koha_dev \
--db_user koha --db_pass koha
2. Verify that KOHA_CONF is set and execute:
misc/devel/update_dbix_class_files.pl --koha-conf
Verify that Koha/Schema files were updated accordingly
3. Execute:
misc/devel/update_dbix_class_files.pl --koha-conf \
/path/to/another/koha-conf.xml
Verify that Koha/Schema files were updated accordingly
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
This patch tries to change the POD as suggested in the QA process.
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
misc/cronjobs/cleanup_database.pl provides some database cleanup.
Yet some tables that could need cleanup are not in this script.
This patch adds cleanup for branchtransfers table.
Test plan :
- Count : select count(*),datearrived from branchtransfers group by datearrived;
- Run cleanup : misc/cronjobs/cleanup_database.pl -v --transfers 30
- Recount
Signed-off-by: Mark Tompsett <mtompset@hotmail.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
misc/cronjobs/cleanup_database.pl provides some database cleanup.
Yet some tables that could need cleanup are not in this script.
This patch adds cleanup for old_reserves table.
Test plan :
- Count : select count(*),date(timestamp) from old_reserves group by date(timestamp);
- Run cleanup : misc/cronjobs/cleanup_database.pl -v --old-reserves 30
- Recount
Signed-off-by: Mark Tompsett <mtompset@hotmail.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
misc/cronjobs/cleanup_database.pl provides some database cleanup.
Yet some tables that could need cleanup are not in this script.
This patch adds cleanup for old_issues table.
Test plan :
- Count : select count(*),date(timestamp) from old_issues group by date(timestamp);
- Run cleanup : misc/cronjobs/cleanup_database.pl -v --old-issues 30
- Recount
Signed-off-by: Mark Tompsett <mtompset@hotmail.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
misc/cronjobs/cleanup_database.pl provides some database cleanup.
Yet some tables that could need cleanup are not in this script.
This patch adds cleanup for deletedborrowers table.
Test plan :
- Count : select count(*),date(updated_on) from deletedborrowers group by date(updated_on);
- Run cleanup : misc/cronjobs/cleanup_database.pl -v --deleted-patrons 30
- Recount
Signed-off-by: Mark Tompsett <mtompset@hotmail.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
misc/cronjobs/cleanup_database.pl provides some database cleanup.
Yet some tables that could need cleanup are not in this script.
This patch adds cleanup for deleted catalog tables.
Note that deletedbiblio_metadata is managed by foreign key on biblionumber.
Test plan :
- Count :
select count(*),year(timestamp) from deleteditems group by year(timestamp);
select count(*),year(timestamp) from deletedbiblio group by year(timestamp);
select count(*),year(timestamp) from deletedbiblioitems group by year(timestamp);
- Run cleanup : misc/cronjobs/cleanup_database.pl -v --deleted-catalog 30
- Recount
Signed-off-by: Mark Tompsett <mtompset@hotmail.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
misc/cronjobs/cleanup_database.pl provides some database cleanup.
Yet some tables that could need cleanup are not in this script.
This patch adds cleanup for statistics table.
Test plan :
- Count statistics : select count(*),date(datetime) from statistics group by date(datetime);
- Run cleanup : misc/cronjobs/cleanup_database.pl -v --statistics 30
- Recount statistics
Signed-off-by: Mark Tompsett <mtompset@hotmail.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
There was a major (and silly) issue in the previous version, only one AV
was kept as we erased the hash value at the end of each iteration:
$invalid_locations_per_framework->{$framework->frameworkcode } =
{ items => $items, av_category => $mss->authorised_value, kohafield => $kohafield };
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
To test:
Run Longoverdue.pl as koha-shell with both 1 and several --itemtypes specified
notice that it has an error
apply this patch
run longoverdue.pl as koha-shell with both 1 and several --itemtypes specified
notice there isn't an error anymore!
Signed-off-by: David Roberts <david.roberts@ptfs-europe.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Rather than performing a symmetric diff and then splitting the results
in a further loop this patch changes to logic to use two asymetric diffs
to get the results more directly.
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
This script uses Array::Utils and adds a dependency
To test:
1 - Have Koha with ES running
2 - Delete some records from ES
curl -X DELETE "es:9200/koha_kohadev_biblios/data/5"
curl -X DELETE "es:9200/koha_kohadev_authorities/data/5"
3 - perl misc/maintenance/compare_es_to_db.pl
4 - Note you are notified of problems in both indexes
5 - perl misc/search_tools/rebuild_elastic_search.pl -a
6 - perl misc/maintenance/compare_es_to_db.pl
7 - Note you are only notified about problems in biblios (assuming you don't have other issues)
8 - perl misc/search_tools/rebuild_elasticsearch.pl -b
9 - perl misc/maintenance/compare_es_to_db.pl
10 - Both counts match, no problems
Signed-off-by: Ere Maijala <ere.maijala@helsinki.fi>
Signed-off-by: Bouzid Fergani <bouzid.fergani@inlibro.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
populate_db.pl from koha-misc4dev must be used instead.
This script existed before the creation of koha-misc4dev, it does no
longer make sense to have it in Koha.
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
SYSPREF_OVERRIDE unfortunately is not supported ;)
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
In an attempt to save time, bulkmarcimport temporarily sets CataloguingLog
and AuthoritiesLog to 0. It does this by disabling syspref caching and saving
the changes to the database (then replacing the original values at completion).
Unfortunately, this disables other key sysprefs from being cached, and results in
a 50% increase in processing time for the script.
This patch instead utilizes the ENV variable override feature of sysprefs, which
preempts the cache in C4::Context->preference().
To test:
1. Perform a bulkmarcimport with a reasonable number of biblios (~1000 will do)
2. Note the time taken to complete
3. Apply patch
4. Revert the biblio load performed
5. Perform another bulkmarcimport with the same biblios and commandline options
6. Note the time taken to complete
7. Compare times. The time from step 6 should be about 33% less than the time from step 2
8. Check Cataloguing and Authorities Logs to verify imported records were not logged
9. Profit!
Signed-off-by: Michal Denar <black23@gmail.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
If elastic is used as search engine, the bulkmarcimport.pl will not
handle correctly UTF-8 encoded MARCXML
Koha::SearchEngine::Search->new uses a require statement to load the correct Search module.
This is done l.257 of bulkmarcimport.pl:
257 my $searcher = Koha::SearchEngine::Search->new
Koha::SearchEngine::Elasticsearch::Search will `use MARC::File::XML`, and so resets the arguments set before:
216 $MARC::File::XML::_load_args{BinaryEncoding} = 'utf-8';
220 $MARC::File::XML::_load_args{RecordFormat} = $recordformat;
An easy (but dirty) fix could be to move the declaration of my $searcher before in the script.
The tricky (but correct) fix would be to remove the long standing "ugly hack follows" comment.
This patch is the easy, and dirty, fix
Test plan:
Use the command line tool to import MARXCML records that contains unicode characters into Koha
Something like `misc/migration_tools/bulkmarcimport.pl -biblios -file record.marcxml -m=MARCXML`
Without this patch you will notice that unicode characters will not be displayed correctly
Signed-off-by: Michal Denar <black23@gmail.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
I am not sure what happened here, but adding strict for TmplTokenizer.pm
breaks a test for translation.
It has been caught by a failing test:
% prove xt/author/translatable-templates.t
will display lines "invalid multibyte sequence"
I did a diff between the generated files and found:
< msgid "Ã rbok"
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
We absolutely need that for modules.
We are enabling strict by using Modern::Perl.
Note that other modules from C4 use strict and warnings, instead of
Modern::Perl
Test plan: git grep 2505 **/*.pm should not return any results
And let's see later what needs to be fixed.
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
This patch removes some warnings when creating new languages,
those related with uft8 and two not related.
Tipical messages are:
>Warning: Can't determine original templates' charset, defaulting to UTF-8
>Warning: Charset Out defaulting to UTF-8
>...
>Removing empty file /../xx-YY-marc-NORMARC.po
>I UTF-8 O UTF-8 at /../kohaclone/misc/translator/tmpl_process3.pl line 365.
>Use of uninitialized value $pref_name in concatenation (.) or string at /.../misc/translator/LangInstaller.pm line 197.
>Created /../misc/translator/po/xx-YY-messages.po.
First four are 'normal' messages
The fifth one correspond to an unquoted preference string
The sixth one is a normal output message from msgint command
We will hide them unless verbose '-v' option is selected,
and fix the error.
To test:
1) Go to misc/translator
2) Create new language files
$ perl translate create xx-YY
3) Check warnings and messages
4) Apply the patch
5) Remove created files, and create them again
Ther must be no messages
6) Repeat 5) with verbose option '-v', check
old messages
7) BONUS: without the patch update a language,
for example de-DE:
$ perl translate update de-DE
Note (git status) a new file, po/de-DE-messages.po~
This is a backup file from msgmerge command
With the patch this backup is removed.
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
