This has bearing on bugs 2905, 2665, 2514 and other "wide character" crashes
related to diacritics and Unicode. This should help open the door for reliable
input of diacriticals via acquisitions.
MARC21_utf8_flag_fix.pl diagnoses and fixes existing problems with MARC data
affected by the bug.
Adding SetMarcUnicodeFlag to TransformKohaToMarc prevents the bug from corrupting
further data.
Signed-off-by: Galen Charlton <galen.charlton@liblime.com>
Acquisitions process seems to be adding records with incorrect
representation of the MARC encoding in leader/09. It should be
'a' meaning UTF-8, for all Koha's internalized records, but in
many cases it appears blank (for MARC-8). This script diagnoses
and repairs the value in the leader, depending on runtime options.
The symptom of this problem is that high-value UNICODE characters
in the record will cause Koha to crash whenever it tries to parse
the MARCXML, giving a "Wide character" fatal. While we work on
fixing the input, this script will fix the existing data.
Signed-off-by: Galen Charlton <galen.charlton@liblime.com>
If series title was previously taken as collectiontitle
then you should launch
misc/maintenance/UNIMARC_fix_collectiontitle.pl
misc/batchRebuildBiblioTables.pl
[RM note: applies only to UNIMARC users - MARC21
users should not run the batch jobs.]
Signed-off-by: Galen Charlton <galen.charlton@liblime.com>
execute_query now refactored, returns reliable results, does
zero presentation-layer crap. Arguments reduced, client scripts
adapted to new API and performance improved. Text::CSV now used
to generate CSV output, ensuring portability, encoding and accuracy.
Replaced tools/runreport.pl with misc/cronjobs/runreport.pl:
~ security fixed
~ documentation improved
~ TODO: finish sendmail option.
Bug 3077 also fixed.
Signed-off-by: Galen Charlton <galen.charlton@liblime.com>
The old location of runreport.pl was under tools, leaving it exposed
to web requests. This is a security flaw since it does NOT check any
Auth and allows the user to request any Saved Report be run. This is
not a problem under misc/crontab/ and it suggests the more appropriate use.
Guided.pm is not fixed here (see bug 3066), but it is prepared to be fixed
and made compatible with runreport as detailed in the perldoc.
Signed-off-by: Galen Charlton <galen.charlton@liblime.com>
Adds three new switches:
-idmap <filename> - optional output file of
map of source record ID numbers
to Koha biblionumber
-x - if idmap is supplied, MARC tag
to get source record ID from
-y - if idmap is supplied, MARC subfield
to get source record ID from
Signed-off-by: Galen Charlton <galen.charlton@liblime.com>
Pulled the GetMarcFromKohaField outside the loop, since we only need to
check it once. Same for $item_sth. Added safety checks for success of
delete_field and GetMarcItem, with warnings on failure.
Signed-off-by: Galen Charlton <galen.charlton@liblime.com>
Removed cronjob, which was written to work around
a bug in 2.2 that no longer applies and is specific
to a single library in any event.
Signed-off-by: Galen Charlton <galen.charlton@liblime.com>
This patch just fix the script which export to csv the overdue, and field the missing fields
Signed-off-by: Galen Charlton <galen.charlton@liblime.com>
This does not fix all problems recorded in 2883 (see all the FIXME's), but
it does improve the script's basic feedback to an intelligible level.
It also adjusts the documentation and examples to correct bogus usage
instructions.
Signed-off-by: Galen Charlton <galen.charlton@liblime.com>
* removed ersataz YAML::XS dependency
* use 'return' instead of 'return undef'
* minor language changes
Signed-off-by: Galen Charlton <galen.charlton@liblime.com>
Improve URLs checker script in the way (half way) pointed out by Galen:
- A C4::URL::Checker class handle URL checking. This class is not yet
in a separate file in C4 directory. This class would be easily
extended to accomodate authorities URLs checking.
- Script output can now be formatted in CSV or HTML. HTML version
link directly to MARC biblio record editor.
Signed-off-by: Galen Charlton <galen.charlton@liblime.com>
Item type was not retrieved in a query, leading to a case
where an item could be selected by build_holds_queue.pl
to fill a hold request even where forbidden by the
library and item type-level policy.
Signed-off-by: Galen Charlton <galen.charlton@liblime.com>
This includes part of a patch from Henri-Damien Laurent
that could not be applied because Chris and Joe patches
happened to win the race.
Signed-off-by: Galen Charlton <galen.charlton@liblime.com>
* use item branch instead of patron's branch to
look up the applicable hold policies - this makes
requesting in the OPAC consistent with the intranet.
* when generating pick list using build_holds_queue.pl, only match items
to patrons if request is allowed.
Signed-off-by: Galen Charlton <galen.charlton@liblime.com>
Add "use warnings", remove unused variables and unnecessary finish/disconnect
at the end. This script could be improved to run only on tables that need to
be altered instead of touching all of them. It should also probably contain
warnings to the effect that it does not rescue your DATA that was forced into
whatever encoding the table used previously.
Signed-off-by: Galen Charlton <galen.charlton@liblime.com>
This reverts commit def09f5a21.
As I emailed to the patches list Oct 06, 2008:
I suggest we need to revert Josh' commit def09f5a21.
The effect on the crontab example is to invalidate the lines being executed. The lines were apparently copied in from a cron source, not crontab, despite the header describing it NOT being an example for cron. It also runs longoverdue twice, instead of fines.pl.
Signed-off-by: Galen Charlton <galen.charlton@liblime.com>
The problem was that the TmplTokenizer create global symbols to store file
handlers so the objects aren't destroyed by the garbage collector.
Signed-off-by: Galen Charlton <galen.charlton@liblime.com>
As rss.pl is not a CGI script, moved it to join the
other cronjobs. Full documentation of the script
is in misc/cronjobs/rss/rss.pl, but to summarize:
[1] rss.pl is run on the command line to produce
an RSS XML document. The output should be
placed in a directory accessible to the OPAC
(or staff) web interface so that users can download
the RSS feed. An example of usage:
misc/cronjobs/rss.pl lastAcquired.conf
Normally rss.pl should be run periodically (e.g., daily)
to keep the feed up-to-date.
[2] The configuration file (e.g., lastAcquired.conf) lists
* name of the template file to use
* path of output file
* SQL query
rss.pl runs the SQL query, then feeds the output of the
query through the template to produce the output file.
[3] The template file (e.g., lastAcquired.tmpl) uses
HTML::Template syntax like any of the HTML
templates for the web interface.
Signed-off-by: Galen Charlton <galen.charlton@liblime.com>
Add the phrase 'if ( $verbose_logging )' to the two print statements
concerning the skipping of biblio or authority records.
I recently had to split biblio and authority index updating in my cron
script ( had some really big records so had to add the -x switch which
should only be used on biblios accourding to the help ). So I noticed
that rebuild_zebra.pl printed messages that it was skipping biblios or
authorities.
This patch is to conditionalize those prints based on the verbose
logging switch.
Signed-off-by: Galen Charlton <galen.charlton@liblime.com>
Reduce the size of the SQLite database produced by
create_koc_db.pl by emiting only the columns
actually used by Kyle Hall's offline circ client.
Signed-off-by: Galen Charlton <galen.charlton@liblime.com>
This commit adds two options to the overdue_notices.pl script.
The notices are now generated in one of two modes: default or triggered.
In default mode, every time the script runs, a notice is generated if an item
is overdue within a notice date range. This mode is suitable for a weekly cron
(or manual run), but the notice triggers should be syncronized with the cron script such that
the script is run once per notice date range. Otherwise, duplicate messages will
be generated ( or no notice generated if a notice date range passes without the script being run).
Triggered mode is suitable for a nightly cron job: a notice is generated only on those
days that an overdue item enters a notice date range. This mode is susceptible to message
loss when the cron script fails to run, or when notice triggers are edited.
A second flag is added: --list-all, which causes items.content to list all currently overdue items.
Without this flag, only items that are overdue within the given notice's date range are listed on
that notice.
Signed-off-by: Galen Charlton <galen.charlton@liblime.com>
Restoring the functionality of these sysprefs as part of the
rework of build_holds_queue.pl:
* StaticHoldsQueueWeight is an ordered list of library codes.
The order of the libraries in this list determines the order
of libraries in which available items will be chosen to be
targeted to holds. Note that if this list does not include
a library, items from that library will never be targeted to
holds and staff will never be asked to retrieve items for
title-level holds from those libraries.
* RandomizeHoldsQueueWeight is a Boolean parameter, indicating
whether to randomize the libraries in the StaticHoldsQueueWeight
list.
If neither RandomizeHoldsQueueWeight nor StaticHoldsQueueWeight is
set, then the libraries will be used to target items to holds in
alphabetical order by library code.
Signed-off-by: Galen Charlton <galen.charlton@liblime.com>
This reduces the output of the script and zebraidx, and creates a -v
command line switch which will increase the logging to their former
states.
Signed-off-by: Galen Charlton <galen.charlton@liblime.com>
Most Perl scripts (as opposed to modules) do
not need to require Exporter.
No user-visible or documentation changes.
Signed-off-by: Galen Charlton <galen.charlton@liblime.com>
The offline circulation tool can use all of the patron information, not just their ID
numbers. This patch populates the offline circulation database with patron information.
Signed-off-by: Galen Charlton <galen.charlton@liblime.com>
CalcFine returned values that mismatched expectations in fines.pl.
fines.pl refactored: added debugging, prevent needless recreation of
Calendar objects by storing them in hash by branch.
Still outstanding problems with fines, including the output of a field
that has no other references in Koha (so is always undef) and the
incorrect description of FinesMode.
Calendar exported "new" erroneously. I also cleaned up the queries to
avoid needlessly compiling additional statement handles.
Please test and consider application to 3.0 maintenance.
Signed-off-by: Galen Charlton <galen.charlton@liblime.com>
This is the script that generates the patron and holdings database for the offline circ module. I
touched up a few parts of it:
* fixed the database syntax to squash a bug or two
* extracted some methods for readability and reuse
* added some documentation
* added a --file and --force command line argument
* made it die if DBD::SQLite is not present. That module is required for this feature.
Here's an example crontab line that will generate the borrowers.db file each midnight.
0 0 * * * create_koc_db.pl --file /tmp/borrowers.db
Signed-off-by: Galen Charlton <galen.charlton@liblime.com>
This script generates a sqlite database of patron and holdings data for the offline circulation tool.
It's a cronjob, so it should go into misc/cronjobs
Signed-off-by: Galen Charlton <galen.charlton@liblime.com>
SQL fix to properly ensure that if an item is on
the hold shelf, it will not be used for request
targeting, which applies only to items that are
not already on the hold shelf.
Prior to this fix, when checking out an item on the
hold shelf that fills patron A's hold request, it
was possible for that request to not be marked as
filled if another patron had an item-level request
on the item; the second patron's request was
incorrectly targeted by that item.
Signed-off-by: Galen Charlton <galen.charlton@liblime.com>
This batch job now automatically populates the
holds request targeting table hold_fill_targets.
This patch is essentially a revamp of the job,
and includes fixes for the following bugs:
2281 (holds queue report including unavailable items)
2331 (holds queue report not working with item-level
holds)
2332 (holds queue script should attempt to fill
many requests as possible).
Several functions in this batch job are candidates
for being moved to C4::Reserves:
GetBibsWithPendingHoldRequests()
GetPendingHoldRequestsForBib()
GetItemsAvailableToFillHoldRequestsForBib()
MapItemsToHoldRequests()
Signed-off-by: Galen Charlton <galen.charlton@liblime.com>
This patch deprecates the NoReturnSetLost system preference, which, as it
turns out, was not implemented anyway. New longoverdue script allows one to
specify on the commandline system-wide delays for changing items to different
lost statuses, and optionally charge for the item.
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
merge works on the fly now.
But for an obscure reason, merge_authority.pl fails to update database when lanched on command line.
Adding one table to LOCK for noZebra UPDATE in Biblio.pm
You should remove C4::Search from merg_authority.pl
Signed-off-by: Galen Charlton <galen.charlton@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
Correct bug in output of overdue notices that has
items.content displaying only the number of columns
in each items.content structure (currently, 4).
Patch inspired by patch submitted by Paul Poulain.
[LL bug 31]
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
This patch changes all $foo eq undef's to !defined($foo). It also makes misc/spellcheck_suggest/build_spellcheck_suggest.pl have proper syntax.
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
The lines for each item in the overdue notices were not separated by newlines. This cause them to
all be smashed together. I'm putting a newline between them.
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
misc/maintenance/fix_accountlines_date.pl runs through the accountlines table and converts every date in the 'description' column from us or metric format (specified on the command line) to the format in the 'dateformat' syspref.
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
These three programs have been replaced by misc/cronjobs/overdue_notices.pl:
overduenotices-30.pl
overduenotices-csv.pl
overduenotices.pl
This patch deletes them.
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
This patch adds the misc/cronjobs/overdue_notices.pl script that is intended to replace
overduenotices.pl, overduenotices-30.pl and overduenotices-csv.pl. It adds messages to
the message_queue to be sent later (by process_message_queue.pl). It also marks borrowers
as debarred if their issues become too overdue.
It is intended to be run from cron nightly with usage something like:
0 2 * * * misc/cronjobs/overdue_notices.pl
C4::Members:
- improved documentation on ModMember
- made ModMember return a useful value (the return value of the database call)
- added a DebarMember method
- adding t/lib/KohaTest/Members/DebarMember.pm to test ModMember
misc/cronjobs/overdue_notices.pl
- designed to replace overduenotices.pl, overduenotices-30.pl, and overduenotice-csv
Changes to C4::Letters:
- EnqueueLetter now lets you pass in to_address and from_address which can override defaults
- _send_message_by_email pays attention to these defaults.
- now handles attachments with MIME::Lite
C4::Overdues
- added GetBranchcodesWithOverdueRules
- added t/lib/KohaTest/Overdues/GerBranchcodesWithOverdueRules.pm to test that.
circ/overdue.pl
- replaced call to obsolete overduenotices-csv.pl with call to overdue_notices.pl
KohaTest:
- added three helper methods: random_phone, random_email, random_ip
- these can be used to populate example records
- you can now pass an optional lengh to random_string
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
