We want to convert a us or metric date formatted to the format defined in
the pref.
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
- installer/data/mysql/updatedatabase.pl
- misc/maintenance/fix_accountlines_date.pl
To test for updatedatabase.pl
- perl installer/data/mysql/updatedatabase.pl
- review code. Difficult to test, because the code affects
$DBversion = '3.03.00.035'
To test for misc/maintenance/fix_accountlines_date.pl
- review code
- The file seems to be a file used for an update, could it be obsolete?
If yes, remove it in a follow-up
Signed-off-by: Frederic Demians <f.demians@tamil.fr>
Code OK. Tested updatedatabase.pl with an old DB backup: OK.
fix_accountlines_date.pl still working after the patch. Could be removed from
master IMO.
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Most of them were found and fixed using codespell.
Signed-off-by: Stefan Weil <sw@weilnetz.de>
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Signed-off-by: Jonathan Druart <jonathan.druart@koha-community.org>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Chris Nighswonger <cnighswonger@foundations.edu>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
http://bugs.koha-community.org/show_bug.cgi?id=9987
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
This patch
- rename _entity_clean as _clean_ampersand
- rename the script to sanitize_records.pl
- add a --fix-ampersand switch (the only one FOR NOW, enabled by
default) so it is obvious what the script does.
- make POD and usage reflect this changes.
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
This patch adds:
- a new maintenance script batch_sanitize_records
- a new subroutine C4::Charset::SanitizeRecord
- new unit tests for the new subroutine
Test plan:
1/ prove t/db_dependent/Charset.t
2/ Create a record containing "&amp;" (could be follow with as many
'amp;' as you want) in one of its fields and the same for the field
linked to biblioitems.url.
The url should not be sanitized, it may contain "&".
3/ Launch the maintenance script with the -h parameter to see how to use
it.
4/ Launch the script using the different parameters:
--filename=FILENAME
--biblionumbers='XXX'
--auto-search
The auto-search permits to sanitize all records containing "&amp;" in
the marcxml field.
Use the verbose flag for testing.
Without the --confirm flag, nothing is done.
5/ Use the --confirm flag and verify in the biblioitems.marcxml field
that the record has been sanitized.
6/ Try the --reindex flag to reindex records which have been modified.
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
If you compare the database with a partial pref file, you may not be
interested in lots of lines saying that a pref is not in the file. You
were already aware of that ;)
If you add the -partial flag, these lines will be skipped.
Test plan:
Do as described above.
Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Works as described, passes tests and QA script.
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
If you use the restore option with a partial pref file not containing a version, you will have this error. Also the count of updated prefs will be one too high.
This is corrected by a better test of the hash value in the CheckVersionPref subroutine.
Test plan:
[1] Create a small pref file and remove the version pref (if present). You
could for instance do a pref backup and remove most lines.
[2] Use the restore option with and without this patch.
So, something like:
maintclone/misc/maintenance/cmp_sysprefs.pl -c r -f 9999crucial.pref
[3] Check on the error message and the pref count.
Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
Prior to this patch, this script would only modify the date
created and modified fields; this patch changes the behavior
so that the fields can be created if they're not already present
in the record.
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Signed-off-by: Mason James <mtj@kohaaloha.com>
Bug 9192: UNIMARC_sync_date_created_with_marc_biblio.pl field creation (followup)
Add tests :
if field < 10 it must not have a subfield
if field > 9 is must have a subfield
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Both patches applied, no koha-qa errors.
Bug 9192: followup fix typo
Fix typo on usage message.
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
This patch adds a new maintenance script: cmp_sysprefs.pl
Test plan:
1 Run with -cmd backup -file zz01. Check that file.
Run with -cmd test -file zz01. Check zz01.sav.
2 Run with -cmd compare -file zz01. No differences expected.
3 Edit zz01: delete two prefs, change two prefs and add two new prefs.
4 Run with -cmd compare -file zz01. Are all six changes reported?
5 Add local use preference zz02 in the staff client (no explanation, options).
6 Run with -cmd backup -file zz02.
7 Delete local use pref zz02 from the staff client.
8 Run with -cmd restore -file zz02. Check if local pref zz02 came back.
9 Delete local use pref zz02 again from the staff client.
10 Run with -cmd compare -file zz02 -add. Check if local pref zz02 came back.
11 Change the value of zz02 in the staff client.
12 Run with -cmd compare -file zz02 -add. Check the value: not updated?
13 Run with -cmd compare -file zz02 -upd. Check the value: updated now?
14 Edit file zz02. Add a comment line and delete the line for pref zz02.
15 Run with -cmd compare -file zz02 -del. Is pref zz02 deleted?
16 Add local use preference zz02 in the staff client (WITH explanation).
17 Run with -cmd compare -file zz02 -del. pref zz02 should not be deleted.
18 Run with -cmd compare -file zz02 -del -ign-opt. zz02 should be deleted now.
Do the next steps only on a restorable test db:
19 Create file zz03. Leave it empty.
Compare with: -cmd compare -file zz03 -del -ign-opt.
All prefs gone except Version?
20 Restore with: -cmd restore -file zz01.sav.
Compare with -cmd compare -file zz01.sav. Nothing reported?
Note: The explanation or options are not recovered. (See also BZ 10199.)
This affects local use preferences only.
If you need them, restore your test db. Remove the zz files.
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Amended patch
Only cosmetic stuff:
Perltidied with xt/perltidyrc.
Replaced most double quotes by single quotes around SQL statements.
Moved the usage into POD for pod2usage.
Passing no file shows help screen too.
Counting the db updates more accurately with return value of do.
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Signed-off by : Mathieu Saby <mathieu.saby@univ-rennes2.fr>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Before this patch:
* misc/maintenance/fix_mysql_constraints.pl FAIL
pod FAIL
*** ERROR: unterminated B<...> in file misc/maintenance/fix_mysql_constraints.pl
forbidden patterns OK
valid OK
critic FAIL
"require" statement with library name as string at line 25, column 12. Use a bareword instead.
+ I add a die on the open statement
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Passed-QA-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Help: fix_mysql_contraints.pl -h
Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
Passed-QA-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
This script will remove these duplicate fines. To use, repeatably run this
script until there are no more duplicates in the database.
(fix_accountlines_rmdupfines_bug8253.pl)
Duplicate fines would happen if you upgraded to a 3.8 version that does not have the
bug8253 patch, and the misc/cronjobs/fines.pl is run. In 3.8 an upgrade to a more
granular date/time was not addressed for pre-existing fine entries and this script
removes the resulting duplicates. It also intelligently preserves the amount outstanding
for payments already applied. If your version already had the bug8253 patch at the time
of the upgrade, duplicate fines should not have been generatd.
Signed-off-by: wajasu <matted-34813@mypacks.net>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
This patch adds the Koha::Indexer::RecordNormalizer and
Koha::Indexer::MARC::RecordNormalizer::EmbedSeeFromHeadings packages
to enable the inclusion of alternate forms of headings in bibliographic
searches. When the new syspref IncludeSeeFromInSearches is turned on
(default is off) rebuild_zebra.pl will insert see from headings from
authority records into bibliographic records when indexing, so that a
search on an obsolete term will turn up relevant records.
To test:
1) Enable IncludeSeeFromInSearches
2) Add a heading that has an alternate form to a record (for example,
"Cooking" has the alternate form "Cookery," if you have authority
records from LC)
3) Index the zebraqueue (or reindex if you haven't indexed your system
yet)
4) Confirm that if you search for "Cookery" you get the record you
just modified
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Rebased on master 5 August 2012
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Rebased on master 11 September 2012
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Also checked:
- Verified database update works correctly
- Checked system preference and its description
- Checked staff/opac detail pages with feature on/off
- Checked staff/opac search facets
- Downloaded and tested records in various formats
- Tried different searches for 'see from' entries of authorities
- Ran all unit tests
No problems found.
misc/maintenance/make_zebra_dom_cfg_from_record_abs:
generate a DOM filter Zebra index config from a GRS-1 config
Given a Zebra record.abs file containing a set of index definitions for
Zebra's GRS-1 filter, write an equivalent DOM filter configuration.
To generate the XSLT that is to be used by Zebra, run something like
the following on the output of this utility:
xsltproc ZEBRA_CFG_DIR/xsl/koha-indexdefs-to-zebra.xsl \
biblio-koha-indexdefs.xml \
> ZEBRA_CFG_DIR/marc_defs/marc21/biblios/biblio-zebra-indexdefs.xsl
The above example assumes that the output of the program was named
biblio-koha-indexdefs.xsl.
This commit also introduces Koha::Indexer::Utils, a new package for
misceallenous routines that support Koha's indexing definitions.
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
This patch adds a progress indicator to remove_items_from_biblioitems.pl, with
the user option of silencing it with --silent.
Signed-off-by: Liz Rea <lrea@nekls.org>
Signed-off-by: Ian Walls <ian.walls@bywatersolutions.com>
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Now that item data is inserted into the MARC bib record
at the point of indexing, sync_items_in_marc_bib.pl is
no longer needed.
Signed-off-by: Galen Charlton <gmcharlt@gmail.com>
Signed-off-by: Claire Hernandez <claire.hernandez@biblibre.com>
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
* add copyright statement
* make --run switch work; this is now required to cause
the update to run
* improve help text
* make remove_items_from_biblioitems.pl executable
Signed-off-by: Galen Charlton <gmcharlt@esilibrary.com>
Signed-off-by: Claire Hernandez <claire.hernandez@biblibre.com>
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
This is a squash of four patches by Henri-Damien Laurent
starting work on removing the copy of item record information
in the 9XX field of bibliographic records. The reason
for doing this is primarily to improve performance, in particular,
the expense of having to add/modify the bib record whenever an
item changes. Now, whenever an item changes, the bib record is
put in the queue to be reindexed; when the bib is indexed, the 9XX
fields are inserted into the version of the bib that Zebra indexes.
Since rebuild_zebra.pl runs in a separate process, the processing of the
bib record will not delay (e.g.) circulation.
As part of upgrading to 3.4, the following batch script should be run:
misc/maintenance/remove_items_from_biblioitems.pl --run
This should be followed by a complete reindexing of the bib records, e.g.,
misc/migration_tools/rebuild_zebra.pl -b -r
Signed-off-by: Galen Charlton <gmcharlt@gmail.com>
Signed-off-by: Claire Hernandez <claire.hernandez@biblibre.com>
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Add scripts that call ModBiblio and ModItem on all or some of the records in a
given catalog. For use when an upgrade changes the behavior of ModBiblio or
ModItem, and the change needs to be retroactively applied to records already in
the system. Usage is as follows:
misc/maintenance/touch_all_[biblios|items].pl [-v] [--where=STRING]
When invoked with a --where argument, the scripts will only modify those biblios
or items that match.
Signed-off-by: Ian Walls <ian.walls@bywatersolutions.com>
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
It is now possible to specify a command line argument --since so that the
borrowers-force-messaging-defaults script only changes patrons created starting
on a certain day. If the optional argument is not specified, the script applies
to all borrowers.
Signed-off-by: Jared Camins-Esakov <jcamins@bywatersolutions.com>
Signed-off-by: Nicole C. Engard <nengard@bywatersolutions.com>
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
If the EnhancedMessagingPreferences syspref is enabled after borrowers
have been created in the DB, those borrowers won't have messaging
transport preferences default values as defined for their borrower
category, even no transport preferences at all. So you would have to
modify each borrower one by one if you would like to send them 'Hold
Filled' notice for example.
I propose this script to create transport preferences for all existing
borrowers and set them to default values defined for the category they
belong to.
[DOC] Should be documented somewhere.
Signed-off-by: Jared Camins-Esakov <jcamins@bywatersolutions.com>
Signed-off-by: Nicole C. Engard <nengard@bywatersolutions.com>
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
More podchecker cleanups to eliminate warnings / errors
Signed-off-by: Andrew Elwell <Andrew.Elwell@gmail.com>
Signed-off-by: Galen Charlton <gmcharlt@gmail.com>
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
This has bearing on bugs 2905, 2665, 2514 and other "wide character" crashes
related to diacritics and Unicode. This should help open the door for reliable
input of diacriticals via acquisitions.
MARC21_utf8_flag_fix.pl diagnoses and fixes existing problems with MARC data
affected by the bug.
Adding SetMarcUnicodeFlag to TransformKohaToMarc prevents the bug from corrupting
further data.
Signed-off-by: Galen Charlton <galen.charlton@liblime.com>
Acquisitions process seems to be adding records with incorrect
representation of the MARC encoding in leader/09. It should be
'a' meaning UTF-8, for all Koha's internalized records, but in
many cases it appears blank (for MARC-8). This script diagnoses
and repairs the value in the leader, depending on runtime options.
The symptom of this problem is that high-value UNICODE characters
in the record will cause Koha to crash whenever it tries to parse
the MARCXML, giving a "Wide character" fatal. While we work on
fixing the input, this script will fix the existing data.
Signed-off-by: Galen Charlton <galen.charlton@liblime.com>
If series title was previously taken as collectiontitle
then you should launch
misc/maintenance/UNIMARC_fix_collectiontitle.pl
misc/batchRebuildBiblioTables.pl
[RM note: applies only to UNIMARC users - MARC21
users should not run the batch jobs.]
Signed-off-by: Galen Charlton <galen.charlton@liblime.com>
Pulled the GetMarcFromKohaField outside the loop, since we only need to
check it once. Same for $item_sth. Added safety checks for success of
delete_field and GetMarcItem, with warnings on failure.
Signed-off-by: Galen Charlton <galen.charlton@liblime.com>
misc/maintenance/fix_accountlines_date.pl runs through the accountlines table and converts every date in the 'description' column from us or metric format (specified on the command line) to the format in the 'dateformat' syspref.
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
Created a new script, sync_items_to_marc_bib.pl,
to replace the item tags embedded in the MARC bib
records with fresh versions taken from the items table.
This script should be run as follows:
maintenance/sync_items_to_marc_bib.pl --run-update
Assuming that you're using Zebra, rebuild_zebra.pl -b -z
or rebuild_zebra.pl -b -r should be run after running
this script.
This script should be run if you have used
link_bibs_to_authorities.pl prior to the first
patch for bug 2258. It can also be used if there
is any reason to suspect that the embedded item tags
do not reflect the items table.
With this script I am creating a maintenance/ subdirectory of
misc/ to hold scripts that are meant to fix problems
in the database but are not (or should not be, anyway) necessary
for regular use.
Documentation: add to documentation for server side scripts
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
